http://bactra.org/notebooks Cosma's Notebooks en http://bactra.org/notebooks/2009/11/14#community-discovery <P><em>Given</em>: a network, especially a large one, directed or not, weighted or not. <em>Desired</em>: a sensible decomposition of the graph into sub-graphs, where in some reasonable sense the nodes in each sub-graph have more to do with each other than with outsiders, i.e., form communities. This is also called "module detection". <P>This seems like a really useful idea to apply to problems I'm interested in, in <a href="neuro-synch.html">neural synchronization</a>; also a place where there could stand to be more interchange between <a href="statistics.html">statistics</a> and <a href="complex-networks.html">complex-network-wallahs</a>. <P>Some of the methods in this area remind me of stuff <a href="christopher-alexander.html">Christopher Alexander</a> did in his 1964 book <cite>Notes on the Synthesis of Form</cite>, but it's been a long time since I read that, so my memory may be faulty. <P>See also: <a href="ecology.html">Ecology</a>; <a href="neuroscience.html">Neuroscience</a>; <a href="signal-transduction.html">Signal Transduction, Gene Regulation and Control of Metabolism</a>; <a href="social-networks.html">Social Networks</a>; <a href="sociology-of-science.html">Sociology of Science</a>; <a href="stat-mech.html">Statistical Mechanics</a>; <a href="synchronization.html">Synchronization</a> <ul>Recommended: <li>Aaron Clauset, "Finding local community structure in networks", <a href="http://arxiv.org/abs/physics/0503036">physics/0503036</a> = <a href="http://dx.doi.org/10.1103/PhysRevE.72.026132"><cite>Physical Review E</cite> <strong>72</strong> (2005): 026132</a> [Clever; but then, Aaron is clever.] <li>Aaron Clauset, M. E. J. Newman and Cristopher Moore, "Finding Community Structure in Very Large Networks", <a href="http://arxiv.org/abs/cond-mat/0408187">cond-mat/0408187</a> = <cite>Physical Review E</cite> <strong>70</strong> (2004): 066111 <LI>J.-J. Daudin, F. Picard and S. Robin, "A Mixture Model for Random Graphs", <a href="http://dx.doi.org/10.1007/s11222-007-9046-7"><cite>Statistics and Computing</cite> <strong>18</strong> (2008): 173--183</a> <li>Michelle Girvan and M. E. J. Newman, "Community structure in social and biological networks," <a href="http://arxiv.org/abs/cond-mat/0112110">cond-mat/0112110</a> = <cite>Proceedings of the National Academy of Sciences</cite> (USA) <strong>99</strong> (2002): 7821--7826 <li>Roger Guimera, Marta Sales-Pardo and Luis A. N. Amaral, "Modularity from Fluctuations in Random Graphs", <a href="http://arxiv.org/abs/cond-mat/0403660">cond-mat/0403660</a> = <cite>Physical Review E</cite> <strong>70</strong> (2004): 025101 <li>Jake M. Hofman, Chris H. Wiggins, "A Bayesian Approach to Network Modularity", <a href="http://arxiv.org/abs/0709.3512">arxiv:0709.3512</a> [For "Bayesian", read "smoothed maximum likelihood". But nonetheless: cool.] <li>Andrea Lancichinetti, Santo Fortunato, Janos Kertesz, "Detecting the overlapping and hierarchical community structure of complex networks", <a href="http://arxiv.org/abs/0802.1218">arxiv:0802.1218</a> [An interesting approach, but not quite as novel as they claim --- cf. Reichardt and Bornholdt --- and I'd really like to see more evidence of superior accuracy and/or robustness] <li>E. A. Leicht, M. E. J. Newman, "Community structure in directed networks", <a href="http://arxiv.org/abs/0709.4500">arxiv:0709.4500</a> <li>M. E. J. Newman <ul> <li>"Modularity and community structure in networks", <a href="http://arxiv.org/abs/physics/0602124">physics/0602124</a> = <cite>Proceedings of the National Academy of Sciences</cite> (USA) <strong>103</strong> (2006): 87577--8582 <li>"Finding community structure in networks using the eigenvectors of matrices", <a href="http://dx.doi.org/10.1103/PhysRevE.74.036104"><cite>Physical Review E</cite> <strong>74</strong> (2006): 036104</a> = <a href="http://arxiv.org/abs/physics/0605087">physics/0605087</a> </ul> <li>M. E. J. Newman and Michelle Girvan <ul> <li>"Mixing patterns and community structure in networks", <a href="http://arxiv.org/abs/cond-mat/0210146">cond-mat/0210146</a> <li>"Finding and evaluating community structure in networks", <cite>Physical Review E</cite> <strong>69</strong> (2003): 026113 = <a href="http://arxiv.org/abs/cond-mat/0308217">cond-mat/0308217</a> </ul> <li>J&ouml;rg Reichardt and Stefan Bornholdt [Code is available by e-mail from Reichardt, who was very helpful to me when I needed to implement their algorithm.] <ul> <li>"Detecting Fuzzy Community Structures in Complex Networks with a Potts Model", <a href="http://dx.doi.org/10.1103/PhysRevLett.93.218701"><cite>Physical Review Letters</cite> <strong>93</strong> (2004): 218701 = <a href="http://arxiv.org/abs/cond-mat/0402349">cond-mat/0402349</a> <li>"Statistical Mechanics of Community Detection", <a href="http://arxiv.org/abs/cond-mat/0603718">cond-mat/0603718</a> = <cite>Physical Review E</cite> <strong>74</strong> (2006): 016110 <li>"Clustering of sparse data via network communities &mdash; a prototype study of a large online market", <a href="http://dx.doi.org/10.1088/1742-5468/2007/06/P06016"><cite>Journal of Statistical Mechanics: Theory and Experiment</cite> (2007): P06016</a> </ul> <li>J&ouml;rg Reichardt and Douglas R. White, "Role models for complex networks", <a href="http://arxiv.org/abs/0708.0958">arxiv:0708.0958</a> [<a href="http://bactra.org/weblog/512.html">Discussion</a>] <li>M. Sales-Pardo, R. Guimera, A. Moreira, L. Amaral, "Extracting the hierarchical organization of complex systems", <a href="http://arxiv.org/abs/0705.1679">arxiv:0705.1679</a> </ul> <ul>Modesty forbids me to recommend: <li>CRS, <a href="http://physics.usfca.edu/marcelo/">Marcelo F. Camperi</a> and <a href="http://www.stat.cmu.edu/~klinkner/">Kristina Lisa Klinkner</a>, "Discovering Functional Communities in Dynamical Networks", <a href="http://arxiv.org/abs/q-bio.NC/0609008">q-bio.NC/0609008</a> </ul> <ul>To read: <li>Edoardo M. Airoldi, David M. Blei, Stephen E. Fienberg and Eric P. Xing, "Mixed membership stochastic blockmodels", <a href="http://arxiv.org/abs/0705.4485">arxiv:0705.4485</a> <li>Nelson Augusto Alves, "Unveiling community structures in weighted networks", <a href="http://arxiv.org/abs/physics/0703087">physics/0703087</a> <li>Leonardo Angelini, Stefano Boccaletti, Daniele Marinazzo, Mario Pellicoro, and Sebastiano Stramaglia, "Fast identification of network modules by optimization of ratio association", <a href="http://arxiv.org/abs/cond-mat/0610182">cond-mat/0610182</a> <li>L. Angelini, D. Marinazzo, M. Pellicoro and S. Stramaglia, "Natural clustering: the modularity approach", <a href="http://arxiv.org/abs/cond-mat/0607643">cond-mat/0607643</a> <li>A. Arenas, J. Duch, A. Fernandez, S. Gomez, "Size reduction of complex networks preserving modularity", <a href="http://arxiv.org/abs/physics/0702015">physics/0702015</a> [Do you really need all those links? Wouldn't your life be simpler if you could just ignore some of them?] <li>Alex Arenas, Alberto Fernandez, Sergio Gomez, "Multiple resolution of the modular structure of complex networks", <a href="http://arxiv.org/abs/physics/0703218">physics/0703218</a> <li>Alex Arenas, Alberto Fernandez, Santo Fortunato, Sergio Gomez, "Motif-based communities in complex networks", <a href="http://arxiv.org/abs/0710.0059">arxiv:0710.0059</a> <li>Jim Bagrow and Erik Bollt, "A Local Method for Detecting Communities", <a href="http://arxiv.org/abs/cond-mat/0412482">cond-mat/0412482</a> <li>James Bagrow, Erik Bollt, Luciano da F. Costa, "Network Structure Revealed by Short Cycles", <a href="http://arxiv.org/abs/cond-mat/0612502">cond-mat/0612502</a> <li>S. Boccaletti, M. Ivanchenko, V. Latora, A. Pluchino and A. Rapisarda, "Dynamical clustering methods to find community structures", <a href="http://arxiv.org/abs/physics/0607179">physics/0607179</a> <li>Michael James Bommarito II, Daniel Martin Katz, Jon Zelner, "On the Stability of Community Detection Algorithms on Longitudinal Citation Data", <a href="http://arxiv.org/abs/0908.0449">arxiv:0908.0449</a> <li>U. Brandes, D. Delling, M. Gaertler, R. Goerke, M. Hoefer, Z. Nikoloski, and D. Wagner, "Maximizing Modularity is hard", <a href="http://arxiv.org/abs/physics/0608255">physics/0608255</a> [i.e., maximizing Newman's Q is NP hard. I haven't read beyond the abstract yet, so I don't know if they address the question of what makes it hard in the hard cases, and whether those are properties we should expect to see in real-world networks. Conceivably, actual social networks are, on average, easy to modularize...] <li>Andrea Capocci, Vito D. P. Servedio, Guido Caldarelli, Francesca Colaiori, "Detecting communities in large networks", <a href="http://arxiv.org/abs/cond-mat/0402499">cond-mat/0402499</a> <li>Horacio Castellini and Lilia Romanelli, "Social network from communities of electronic mail", <a href="http://arxiv.org/abs/nlin.CD/0509021">nlin.CD/0509021</a> <li>Leon Danon, Albert D&iacute;az-Guilera, and Alex Arenas, "The effect of size heterogeneity on community identification in complex networks", <a href="http://dx.doi.org/10.1088/1742-5468/2006/11/P11010"><cite>Journal of Statistical Mechanics: Theory and Experiment</cite> (2006): P11010</a> = <a href="http://arxiv.org/abs/physics/0601144">physics/0601144</a> <li>Leon Danon, Albert D&iacute;az-Guilera, Jordi Duch and Alex Arenas, "Comparing community structure identification", <a href="http://dx.doi.org/10.1088/1742-5468/2005/09/P09008"><cite>Journal of Statistical Mechanics: Theory and Experiment</cite> (2005): P09008</a> = <a href="http://arxiv.org/abs/cond-mat/0505245">cond-mat/0505245</a> <li>Jordi Duch and Alex Arenas, "Community detection in complex networks using extremal optimization", <a href="http://dx.doi.org/10.1103/PhysRevE.72.027104"><cite>Physical Review E</cite> <strong>72</strong> (2005): 027104</a> <li>Illes J. Farkas, Daniel Abel, Gergely Palla, Tamas Vicsek, "Weighted network modules", <a href="http://arxiv.org/abs/cond-mat/0703706">cond-mat/0703706</a> <li>S. Feldt, J. Waddell, V. L. Hetrick, J. D. Berke, and M. Zochowski, "Functional clustering algorithm for the analysis of dynamic network data", <a href="http://dx.doi.org/10.1103/PhysRevE.79.056104"><cite>Physical Review E</cite> <strong>79</strong> (2009): 056104</a> <li>Daniel J. Fenn, Mason A. Porter, Mark McDonald, Stacy Williams, Neil F. Johnson, Nick S. Jones, "Dynamic communities in multichannel data: An application to the foreign exchange market during the 2007--2008 credit crisis", <a href="http://arxiv.org/abs/0811.3988">arxiv:0811.3988</a> <li>Daniel J. Fenn, Mason A. Porter, Peter J. Mucha, Mark McDonald, Stacy Williams, Neil F. Johnson, Nick S. Jones, "Dynamical Clustering of Exchange Rates", <a href="http://arxiv.org/abs/0905.4912">arxiv:0905.4912</a> <li>Sam Field, Kenneth A. Frank, Kathryn Schiller, Catherine Riegle-Crumb and Chandra Muller, "Identifying positions from affiliation networks: Preserving the duality of people and events", <a href="http://dx.doi.org/10.1016/j.socnet.2005.04.005"><cite>Social Networks</cite> <strong>28</strong> (2006): 97--123</a> <li>G. W. Flake, S. R. Lawrence, C. L. Giles and F. M. Coetzee, "Self-organization and identification of Web communities", <cite>IEEE Computer</cite> <strong>36</strong> (2002): 66--71 <li>Santo Fortunato, "Community detection in graphs", <a href="http://arxiv.org/abs/0906.0612">arxiv:0906.0612</a> <li>Santo Fortunato and Marc Bath&eacute;lemy, "Resolution limit in community detection", <a href="http://arxiv.org/abs/physics/0607100">physics/0607100</a> = <a href="http://dx.doi.org/10.1073/pnas.0605965104">cite>Proceedings of the National Academy of Sciences</cite> (USA) <strong>104</strong> (2007): 36--41</a> <li>Santo Fortunato and Claudio Castellano, "Community Structure in Graphs", <a href="http://arxiv.org/abs/0712.2716">arxiv:0712.2716</a> [Review paper; thanks to <a href="http://vielmetti.typepad.com/vacuum">Ed Vielmetti</a> for the pointer] <li>Santo Fortunato, Vito Latora and Massimo Marchiori, "A Method to Find Community Structures Based on Information Centrality", <a href="http://arxiv.org/abs/cond-mat/0402522">cond-mat/0402522</a> <li>Kenneth A. Frank, "Identifying Cohesive Subgroups", <cite>Social Networks</cite> <strong>17</strong> (1995): 27--56 <li>David Gfeller, Jean-C&eacute;dric Chappelier, and Paolo De Los Rios, "Finding instabilities in the community structure of complex networks", <a href="http://dx.doi.org/10.1103/PhysRevE.72.056135"><cite>Physical Review E</cite> <strong>72</strong> (2005): 056135</a> <li>Rumi Ghosh, Kristina Lerman, "Structure of Heterogeneous Networks", <a href="http://arxiv.org/abs/0906.2212">arxiv:0906.2212</a> <li>V. Gol'dshtein and G. A. Koganov, "An indicator for community structure", <a href="http://arxiv.org/abs/physics/0607159">physics/0607159</a> <li>Benjamin H. Good, Yves-Alexandre de Montjoye, Aaron Clauset, "The performance of modularity maximization in practical contexts", <a href="http://arxiv.org/abs/0910.0165">arxiv:0910.0165</a> <li><a href="">Mark S. Handcock</a>, Adrian E. Raftery and Jeremy Tantrum, "Model-Based Clustering for Social Networks" <cite>Journal of the Royal Statistical Society A</cite> <strong>170</strong> (2007): 301--354 [<a href="http://www.csss.washington.edu/Papers/wp46.pdf">PDF preprint</a>] <li>M. B. Hastings, "Community detection as an inference problem", <a href="http://dx.doi.org/10.1103/PhysRevE.74.035102"><cite>Physical Review E</cite> <strong>74</strong> (2006): 035102</a> = <a href="http://arxiv.org/abs/cond-mat/0604429">cond-mat/0604429</a> <li>Erik Holmstr&ouml;m, Nicolas Bock and Joan Br&auml;nnlund, "Density Analysis of Network Community Divisions", <a href="http://arxiv.org/abs/cond-mat/0608612">cond-mat/0608612</a> <li>I. Ispolatov, I. Mazo, A. Yuryev, "Finding mesoscopic communities in sparse networks", <a href="http://arxiv.org/abs/q-bio.MN/0512038">q-bio.MN/0512038</a> = <cite>Journal of Statistical Mechanics</cite> (2006): P09014 <li>Brian Karrer, Elizaveta Levina, M. E. J. Newman, "Robustness of community structure in networks", <a href="http://arxiv.org/abs/0709.2108">arxiv:0709.2108</a> <li>Jussi M. Kumpula, Jari Saramaki, Kimmo Kaski, and Janos Kertesz, "Resolution limit in complex network community detection with Potts model approach",<a href="http://arxiv.org/abs/cond-mat/0610370">cond-mat/0610370</a> <li>Andrea Lancichinetti, Santo Fortunato, "Benchmarks for testing community detection algorithms on directed and weighted graphs with overlapping communities", <a href="http://arxiv.org/abs/0904.3940">arxiv:0904.3940</a> <li>Pierre Latouche, Etienne Birmel&eacute;, Christophe Ambroise, "Overlapping Stochastic Block Models", <a href="http://arxiv.org/abs/0910.2098">arxiv:0910.2098</a> <li>Sune Lehmann, Martin Schwartz, Lars Kai Hansen, "Bi-clique Communities", <a href="http://arxiv.org/abs/0710.4867">arxiv:0710.4867</a> <li>Michele Leone, Sumedha, Martin Weigt, "Clustering by soft-constraint affinity propagation: Applications to gene-expression data", <a href="http://arxiv.org/abs/0705.2646">arxiv:0705.2646</a> <li>Jure Leskovec, Kevin J. Lang, Anirban Dasgupta and Michael W. Mahoney, "Community Structure in Large Networks: Natural Cluster Sizes and the Absence of Large Well-Defined Clusters", <a href="http://arxiv.org/abs/0810.1355">arxiv:0810.1355</a> <li>Ian X.Y. Leung, Pan Hui, Pietro Lio', Jon Crowcroft, "Towards Real Time Community Detection in Large Networks", <a href="http://arxiv.org/abs/0808.2633">arxiv:0808.2633</a> <li>Claire P. Massen, Jonathan P. K. Doye, "Thermodynamics of Community Structure", <a href="http://arxiv.org/abs/cond-mat/0610077">cond-mat/0610077</a> <li>A. D. Medus and C. O. Dorso, "Alternative approach to community detection in networks", <a href="http://dx.doi.org/10.1103/PhysRevE.79.066111"><cite>Physical Review E</cite> <strong>79</strong> (2009): 066111</a> <li>Peter J. Mucha, Thomas Richardson, Kevin Macon, Mason A. Porter, Jukka-Pekka Onnela, "Community Structure in Time-Dependent, Multiscale, and Multiplex Networks", <a href="http://arxiv.org/abs/0911.1824">arxiv:0911.1824</a> <li>Stefanie Muff, Francesco Rao, and Amedeo Caflisch, "Local modularity measure for network clusterizations", <a href="http://dx.doi.org/10.1103/PhysRevE.72.056107"><cite>Physical Review E</cite> <strong>72</strong> (2005): 056107</a> <li>Andreas Noack, "Modularity clustering is force-directed layout", <a href="http://arxiv.org/abs/0807.4052">arxiv:0807.4052</a> <li>Gergely Palla, Imre Derenyi, Illes Farkas and Tamas Vicsek, "Uncovering the overlapping community structure of complex networks in nature and society", <a href="http://dx.doi.org/10.1038/nature03607"><cite>Nature</cite> <strong>435</strong> (2005): 814--818</a> = <a href="http://arxiv.org/abs/physics/0506133">physics/0506133</a> <li>Gergely Palla, Illes J. Farkas, Peter Pollner, Imre Derenyi, Tamas Vicsek, "Directed network modules", <a href="http://arxiv.org/abs/physics/0703248">physics/0703248</a> <li>Nicolas Pissard and Houssem Assadi, "Detecting overlapping communities in linear time with P&amp;A algorithm", <a href="http://arxiv.org/abs/physics/0509254">physics/0509254</a> <li>Pascal Pons, "Post-Processing Hierarchical Community Structures: Quality Improvements and Multi-scale View", <a href="http://arxiv.org/abs/cs.DS/0608050">cs.DS/0608050</a> <li>Mason A. Porter, Jukka-Pekka Onnela, Peter J. Mucha, "Communities in Networks", <a href="http://arxiv.org/abs/0902.3788">arxiv:0902.3788</a> <li>Josep M. Pujol, Javier B&eacute;jar, and Jordi Delgado, "Clustering algorithm for determining community structure in large networks", <a href="http://dx.doi.org/10.1103/PhysRevE.74.016107"><cite>Physical Review E</citE> <strong>74</strong> (2006): 016107</a> <li>Francisco A. Rodrigues, Gonzalo Travieso, Luciano da F. Costa, "Fast Community Identification by Hierarchical Growth", <a href="http://arxiv.org/abs/physics/0602144">physics/0602144</a> <li>Huaijun Qiu and Edwin R. Hancock, "Graph matching and clustering using spectral partitions", <a hrf="http://dx.doi.org/10.1016/j.patcog.2005.06.014"><cite>Pattern Recognition</cite> <strong>39</strong> (2006): 22--34</a> [In this context, for the ideas on hierarchical decomposition, which sounds like it might work for community discovery, if in fact it's not equivalent to some existing community-discovery algorithm.] <li>Usha Nandini Raghavan, Reka Albert, Soundar Kumara, "Near linear time algorithm to detect community structures in large-scale networks", <a href="http://arxiv.org/abs/0709.2938">arxiv:0709.2938</a> ["every node is initialized with a unique label and at every step each node adopts the label that most of its neighbors currently have"] <li>J&ouml;rg Reichardt and Stefan Bornholdt, "When are networks truly modular?", <a href="http://arxiv.org/abs/cond-mat/0606220">cond-mat/0606220</a> <li>J&ouml;rg Reichardt and Michele Leone, "(Un)detectable cluster structure in sparse networks", <a href="http://arxiv.org/abs/0711.1452">arxiv:0711.1452</a> <li>Martin Rosvall and Carl T. Bergstrom <ul> <li>"An information-theoretic framework for resolving community structure in complex networks", <a href="http://arxiv.org/abs/physics/0612035">physics/0612035</a> [Or, <a href="mdl.html">MDL</a> to the rescue!] <li>"Maps of random walks on complex networks reveal community structure", <a href="http://dx.doi.org/10.1073/pnas.0706851105"><cite>Proceedings of the National Academy of Sciences</cite> (USA) <strong>105</strong> (2008): 1118--1123</a> </ul> <li>Erin N. Sawardecker, Marta Sales-Pardo, Lu&iacute;s A. Nunes Amaral, "Detection of node group membership in networks with group overlap", <a href="http://arxiv.org/abs/0812.1243">arxiv:0812.1243</a> <li>Chayant Tantipathananandh, <a href="http://www.cs.uic.edu/~tanyabw/">Tanya Berger-Wolf</a> and David Kempe, "A Framework For Community Identification in Dynamic Social Networks" [<a href="http://www.cs.uic.edu/~tanyabw/research/pubs/TantipathananandhEtal_NetworkCommunities07.pdf">PDF</a>] <li>Joshua R. Tyler, Dennis M. Wilkinson and Bernardo A. Huberman, "Email as Spectroscopy: Automated Discovery of Community Structure within Organizations," <a href="http://arxiv.org/abs/cond-mat/0303264">cond-mat/0303264</a> <li>I. Vragovic and E. Louis, "Network community structure and loop coefficient method", <a href="http://dx.doi.org/10.1103/PhysRevE.74.016105"><cite>Physical Review E</cite> <strong>74</strong> (2006): 016105</a> <li>Matthew L. Wallace, Yves Gingras, Russell Duhon, "A new approach for detecting scientific specialties from raw cocitation networks", <a href="http://arxiv.org/abs/0807.4903">arxiv:0807.4903</a> <li>Huijie Yang, Wenxu Wang, Tao Zhou, Binghong ang and Fangcui Zhao, "Reconstruct the Hierarchical Structure in a Complex Network", <a href="http://arxiv.org/abs/physics/0508026">physics/0508026</a> ["Based upon the eigenvector centrality (EC) measure, a method is proposed to reconstruct the hierarchical structure of a complex network. It is tested on the Santa Fe Institute collaboration network, whose structure is well known."] <li>Hugo Zanghi, Franck Picard, Vincent Miele, Christophe Ambroise, "Strategies for Online Inference of Model-Based Clustering in large Networks", <a href="http://arxiv.org/abs/0910.2034">arxiv:0910.2034</a> <li>Haijun Zhou <ul> <li>"Distance, dissimilarity index, and network community structure," <a href="http://arxiv.org/abs/physics/0302032">physics/0302032</a> <li>"Network Landscape from a Brownian Particle's Perspective," <a href="http://arxiv.org/abs/physics/0302030">physics/0302030</a> </ul> <li>Etay Ziv, Manuel Middendorf and Chris Wiggins, "An Information-Theoretic Approach to Network Modularity", <a href="http://arxiv.org/abs/q-bio.QM/0411033">q-bio.QM/0411033</a> </ul> <ul>To finish writing: <li>"Functional Community Discovery II" </ul>