Network visualization has long been an important tool for researchers to better understand network structures, form hypotheses, and communicate their results. However, traditional force-directed network visualization is a computationally expensive task. Sizable graphs containing millions and even billions of nodes and edges, such as those that are now available from online social networking APIs, are impractically slow to visualize in this manner.
Our lab has been working on an algorithm (based on Hadoop and Apache Giraph) to overcome this limitation. Our solution fully leverages the power of distributed computing and the scale-free properties of social networks. Once fully tested and implemented, this new technique will significantly reduce the time and computing resources needed to visualize very large social networks with millions of nodes. Our goal is to eventually release the algorithm and software on github and share it with the research community.
Below are the slides of our recent presentation about this project at the 2015 International Sunbelt Social Network Conference in Brighton, UK. In the presentation, we described our algorithm and system design from a network perspective and present some preliminary findings and methodological challenges of this project.