{"id":23002,"date":"2024-11-19T18:40:57","date_gmt":"2024-11-19T18:40:57","guid":{"rendered":"https:\/\/socialmedialab.ca\/web\/?p=23002"},"modified":"2025-06-17T17:01:48","modified_gmt":"2025-06-17T17:01:48","slug":"introducing-the-new-3d-topic-analyzer-module-in-communalytic","status":"publish","type":"post","link":"https:\/\/socialmedialab.ca\/web\/2024\/11\/19\/introducing-the-new-3d-topic-analyzer-module-in-communalytic\/","title":{"rendered":"Introducing the New Topic Analyzer\u00a0in Communalytic"},"content":{"rendered":"\n
\"\"<\/figure>
\n

Discovering Latent Topics <\/strong><\/h3>\n\n\n\n

We are thrilled to announce the release of the new Topic Analyzer\u00a0in Communalytic<\/a>, a computational social science research tool designed to study online communities and public discourse on social media. Communalytic\u00a0is entirely web-based and requires no special programming or coding experience.<\/p>\n\n\n\n

The Topic Analyzer can automatically group social media posts that are semantically similar<\/a> <\/strong>(i.e., similar in their meaning). The tool is designed to expedite the analysis of a large corpus of text data without the need to read and review every post.<\/p>\n\n\n\n

Researchers can use the Analyzer to discover latent topics in a dataset, i.e., abstract topics that may not be directly observable from reading the posts alone. It can also be used to discover communities of users who share a similar interest in a topic but do not necessarily communicate with each other.<\/p>\n<\/div><\/div>\n\n\n\n

Creating Embeddings with Social Media Posts<\/strong><\/h3>\n\n\n\n

The\u00a0Topic Analyzer\u00a0utilizes sentence-transformer models to convert social media posts into computer-readable\u00a0vector embeddings<\/a>, enabling the capture of the semantic meaning of these<\/span> posts. For more information on embedding, see\u00a0here<\/a>\u00a0and\u00a0here<\/a>.<\/p>\n\n\n\n

Communalytic generates embeddings from social media posts using a multilingual text embedding model from VoyageAI<\/a>: Voyage-3-lite<\/strong> (512 dimensions) in Communalytic EDU<\/strong> and Voyage-3<\/strong> (1024 dimensions) in Communalytic PRO<\/strong>. These general-purpose models are optimized for multilingual retrieval, making them ideal for identifying semantic similarities between sentences in a dataset. These models were selected because they outperform<\/a> similar models for creating embeddings from texts in 27 different languages<\/strong>: Arabic, Bengali, Czech, Danish, Dutch, English, French, Georgian, German, Greek, Hungarian, Italian, Japanese, Korean, Kurdish, Norwegian, Persian, Polish, Portuguese, Russian, Slovak, Spanish, Swedish, Thai, Turkish, Urdu, Vietnamese<\/em>.<\/p>\n\n\n\n

Auto-clustering of Semantically Similar Posts<\/strong><\/h3>\n\n\n\n

After transforming posts into embeddings, Communalytic uses a dimension reduction technique called UMAP<\/a><\/span><\/span><\/span><\/strong><\/strong> to compress the embeddings from 512 or 1024 dimensions down to three dimensions. These reduced embeddings are then visualized with Communalytic’s interactive 3D Semantic Similarity Map<\/strong>, where each dot represents a post. Semantically similar posts are automatically grouped into clusters using one of three clustering algorithms: HDBScan<\/strong>, KMeans<\/strong>, or Gaussian Mixture<\/strong>. Each cluster represents a distinct latent topic. Dots within the same cluster are displayed in the same colour, making it easy to distinguish between topics.<\/p>\n\n\n\n

For instance, consider a dataset containing three posts:<\/p>\n\n\n\n

    \n
  1. “I like apples.”<\/li>\n\n\n\n
  2. “I hate oranges.”<\/li>\n\n\n\n
  3. “I need a car.”<\/li>\n<\/ol>\n\n\n\n

    The Topic Analyzer<\/strong> would cluster the first two posts into a single group because they share semantic similarity\u2014they both discuss preferences related to fruits (despite expressing opposite sentiments). The third post, however, would form its own cluster, as it focuses on an unrelated topic (a need for a car). This demonstrates how the Topic Analyzer attempts to group content based on meaning, rather than just surface-level keywords.<\/em><\/p>\n\n\n\n

    Auto-Labelling of Semantically Similar Posts<\/strong><\/h3>\n\n\n\n

    After the posts are clustered and visualized with the 3D Semantic Similarity Map<\/strong>, researchers can review each cluster manually and assign a descriptive label. Alternatively, they can use one of the available LLMs (e.g., llama-3.1 or mistral-7b) to suggest a topic label for each cluster automatically.<\/p>\n\n\n\n

    A preview of the 3D Semantic Similarity Map<\/strong><\/h3>\n\n\n\n

    Below is a video demo of the Topic Analyzer<\/strong> and the companion 3D Semantic Similarity Map<\/strong>.<\/p>\n\n\n\n

    <\/div>\n\n\n\n
    <\/div>\n\n\n\n
    \n\n\n\n
    \"\"<\/a><\/figure>\n\n\n\n

    About Communalytic<\/strong><\/h3>\n\n\n\n

    Communalytic <\/a>is a no-code computational social science research tool for studying online communities and public discourse on social media. It is designed to provide researchers, journalists, and students with essential resources and infrastructure for conducting independent, public-interest research. It has a full suite of easy-to-use social media data collectors and analyzers \u2013 no coding is required. <\/p>\n\n\n\n

    Users can\u00a0bring their own data<\/a>\u00a0or use one of Communalytic\u2019s various\u00a0social media data collectors<\/a> to collect data from platforms such as Bluesky, Mastodon, Reddit, Telegram, X (formerly Twitter), and YouTube.<\/p>\n\n\n\n

    There are two versions of Communalytic. Each is designed for different purposes and different sets of users:<\/p>\n\n\n\n