The little engine(s) that could: Scaling online social networks

IEEE/ACM Transactions on Networking

Volumn 20, Issue 4, 2012, Pages 1162-1175

  Pujol, Josep M ^a,b   Erramilli, Vijay ^a   Siganos, Georgos ^a   Yang, Xiaoyuan ^a   Laoutaris, Nikolaos ^a   Chhabra, Parminder ^a   Rodriguez, Pablo ^a  

^a TELEFONICA RESEARCH   (Spain)

^b 3scale ^*  (Spain)

Author keywords

Algorithms; distributed systems; online social networks (OSNs); scaling

Indexed keywords

APPLICATION DEVELOPERS; APPLICATION LOGIC; CO-LOCATED; COMPLETE SYSTEM; DATA SETS; DATABASE LAYER; DESIGN CHALLENGES; DISTRIBUTED HASH TABLES; DISTRIBUTED SYSTEMS; FACEBOOK; INTER-SERVER COMMUNICATIONS; LOW MEMORY; MULTIPLE SERVERS; ONLINE SOCIAL NETWORKS; SCALING; SOCIAL GRAPHS; VERTICAL SCALING;

ALGORITHMS; MIDDLEWARE; SEMANTICS; SYSTEMS ANALYSIS;

SOCIAL NETWORKING (ONLINE);

EID: 84865325520     PISSN: 10636692     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNET.2012.2188815     Document Type: Article

References (36)

1. High Scalability, "Facebook's memcached multiget hole: More machines != More capacity," 2009 [Online]. Available: http://highscalability.com/ blog/2009/10/26/facebooks-memcachedmultiget-hole-more-machines-more-capacit.html
2. High Scalability, "Friendster lost lead due to failure to scale," 2007 [Online]. Available: http://highscalability.com/blog/2007/11/ 13/friendster-lost-lead-because-of-a-failure-to-scale.html
3. V. Agunati, "Notes From Scaling MySQL-Up or Out," 2008 [Online]. Available: http://venublog.com/2008/04/16/notes-from-scalingmysql-up- or-out/
4. RightScale, "RightScale," 2010 [Online]. Available: http://www. rightscale.com
5. StatusNet, "Status Net," 2010 [Online]. Available: http://status.net
6. N. Niclausse, "Tsung: Distributed load testing tool," 2010 [Online]. Available: http://tsung.erlang-projects.org/
7. High Scalability, "Scaling Twitter: Making Twitter 10000 percent faster," 2009 [Online]. Available: http://highscalability.com/ scalingtwitter-making-twitter-10000-percent-faster
8. A. Adya, W. J. Bolosky, M. Castro, G. Cermak, R. Chaiken, J. R. Douceur, Jon, J. Howell, J. R. Lorch, M. Theimer, and R. P. Wattenhofer, "Farsite: Federated, available, and reliable storage for an incompletely trusted environment," in Proc. OSDI, 2002, pp. 1-14.
9. M. Armbrust, A. Fox, R. Griffith, A. D. Joseph, R. H. Katz, A. Konwinski, G. Lee, D. A. Patterson, A. Rabkin, I. Stoica, and M. Zaharia, "Above the clouds: A Berkeley view of cloud computing," Tech. Rep. UCB/EECS-2009-28, 2009.
10. S. Arora, S. Rao, and U. Vazirani, "Expander flows, geometric embeddings and graph partitioning," J. ACM, vol. 56, no. 2, pp. 1-37, 2009.
11. F. Benevenuto, T.Rodrigues, M. Cha, andV.Almeida, "Characterizing user behavior in online social networks," in Proc. ACM SIGCOMM IMC, 2009, pp. 49-62.
12. V. D. Blondel, J.-L. Guillaume, R. Lambiotte, and E. Lefebvre, "Fast unfolding of communities in large networks," J. Statist.Mech., vol. 10, p. P10008, 2008.
13. G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, andW. Vogels, "Dynamo: Amazon's highly available key-value store," Oper. Syst. Rev., vol. 41, no. 6, pp. 205-220, 2007. (Pubitemid 351429377)
14. J. Fakcharoenphol, S. Rao, and K. Talwar, "Approximating metrics by tree metrics," SIGACT News, vol. 35, no. 2, pp. 60-70, 2004.
15. U. Feige and R. Krauthgamer, "A polylogarithmic approximation of the minimum bisection," SIAM J. Comput., vol. 31, no. 4, pp. 1090-1118, Apr. 2002.
16. M. R. Garey and D. S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness. New York: Freeman, 1979.
17. J. Gray, P. Helland, P. O'Neil, and D. Shasha, "The dangers of replication and a solution," in Proc. ACM SIGMOD, 1996, pp. 173-182. (Pubitemid 126440766)
18. R. G. Guy, J. S. Heidemann, W. Mak, T. W. Page Jr., G. J. Popek, and D. Rothmeier, "Implementation of the Ficus replicated file system," in Proc. USENIX Conf., 1990, pp. 63-72.
19. J. Hamilton, "Geo-replication at Facebook," 2008 [Online]. Available: http://perspectives.mvdirona.com/2008/08/21/GeoReplicationAtFacebook. aspx
20. J. Hamilton, "Scaling LinkedIn," 2008 [Online]. Available: http://perspectives.mvdirona.com/2008/06/08/ScalingLinkedIn.aspx
21. High Scalability, "Why are Facebook, Digg and Twitter so hard to scale?," 2009 [Online]. Available: http://highscalability.com/blog/2009/10/ 13/why-are-facebook-digg-and-twitter-so-hard-to-scale.html
22. J. Rothschild, "High performance at massive scale-Lessons learned at Facebook," 2009 [Online]. Available: http://cns.ucsd. edu/ lecturearchive09.shtml#Roth
23. G. Karypis and V. Kumar, "A fast and high quality multilevel scheme for partitioning irregular graphs," SIAM J. Sci. Comput., vol. 20, no. 1, pp. 359-392, 1998. (Pubitemid 128689516)
24. H. Kwak, Y. Choi, Y.-H. Eom, H. Jeong, and S. Moon, "Mining communities in networks: A solution for consistency and its evaluation," in Proc. ACM SIGCOMM IMC, 2009, pp. 301-314.
25. H. Kwak, C. Lee, H. Park, and S. Moon, "What is Twitter, a social network or a news media," 2010.
26. J. Leskovec, J. Kleinberg, and C. Faloutsos, "Graph evolution: Densification and shrinking diameters," Trans. Knowl. Data Discovery, vol. 1, no. 1, 2007, Article no. 2.
27. J. Leskovec, K. J. Lang, A. Dasgupta, and M. W. Mahoney, "Community structure in large networks: Natural cluster sizes and the absence of large well-defined clusters," in Proc. CORR, 2008, Abs/0810.1355.
28. M.McGiboney, "Twitter's sweet smell of success,"Nielsen, NewYork, NY, 2009 [Online]. Available: http://blog.nielsen.com/nielsenwire/ online-mobile/twitters-tweet-smell-of-success/
29. A. Mislove, M. Marcon, K. P. Gummadi, P. Druschel, and B. Bhattacharjee, "Measurement and analysis of online social networks," in Proc. ACM SIGCOMM IMC, 2007, pp. 29-42.
30. M. Newman and J. Park, "Why social networks are different from other types of networks," Phys. Rev. E, vol. 68, p. 036122, 2003.
31. M. E. J. Newman, "Modularity and community structure in networks," Proc. Nat. Acad. Sci., vol. 103, p. 8577, 2006.
32. J. M. Pujol, V. Erramilli, and P. Rodriguez, "Divide and conquer: Partitioning online social networks," 2009 [Online]. Available: http://www. citebase.org/abstract?id=oai:arXiv.org:0905.4918
33. M. Satyanarayanan, "Coda: A highly available file system for a distributed workstation environment," IEEE Trans. Comput., vol. 39, no. 4, pp. 447-459, Apr. 1990. (Pubitemid 20702255)
34. F. Schneider, A. Feldmann, B. Krishnamurthy, andW.Willinger, "Understanding online social network usage from a network perspective," in Proc. ACM SIGCOMM IMC, 2009, pp. 35-48.
35. D. B. Terry, M. M. Theimer, K. Petersen, A. J. Demers, M. J. Spreitzer, and C. H. Hauser, "Managing update conflicts in Bayou, a weakly connected replicated storage system," in Proc. ACM SOSP, 1995, pp. 172-182.
36. B. Viswanath, A. Mislove, M. Cha, and K. P. Gummadi, "On the evolution of user interaction in Facebook," in Proc. ACM WOSN, 2009, pp. 37-42.