Gossip learning with linear models on fully distributed data

Concurrency and Computation: Practice and Experience

Volumn 25, Issue 4, 2013, Pages 556-571

  Ormándi, Rõbert ^a   Hegedus, István ^a   Jelasity, Márk ^a,b  

^a UNIVERSITY OF SZEGED   (Hungary)

^b INSTITUTE OF EXPERIMENTAL MEDICINE   (Hungary)

Author keywords

bagging; gossip; online learning; P2P; random walk; stochastic gradient descent

Indexed keywords

E-LEARNING; FAULT TOLERANT COMPUTER SYSTEMS; GRADIENT METHODS; LEARNING SYSTEMS; PEER TO PEER NETWORKS; RANDOM PROCESSES; STOCHASTIC SYSTEMS;

BAGGING; GOSSIP; ONLINE LEARNING; RANDOM WALK; STOCHASTIC GRADIENT DESCENT;

LEARNING ALGORITHMS;

EID: 84874111805     PISSN: 15320626     EISSN: 15320634     Source Type: Journal    
DOI: 10.1002/cpe.2858     Document Type: Conference Paper

References (47)

1. Pouwelse JA, Garbacki P, Wang J, Bakker A, Yang J, Iosup A, Epema DHJ, Reinders M, van Steen MR, Sips HJ,. TRIBLER: a social-based peer-to-peer system. Concurrency and Computation: Practice and Experience 2008; 20 (2): 127-138. DOI: 10.1002/cpe.1189.
2. Bai X, Bertier M, Guerraoui R, Kermarrec AM, Leroy V,. Gossiping personalized queries. Proceedings of the 13th International Conference on Extending Database Technology (EBDT'10), Lausanne, 2010.
3. Buchegger S, Schiöberg D, Vu LH, Datta A,. PeerSoN: P2P social networking: early experiences and insights. In Proceedings of the Second ACM EuroSys Workshop on Social Network Systems (SNS'09). ACM: New York, NY, USA, 2009; 46-52, DOI: 10.1145/1578002.1578010.
4. Cheetancheri SG, Agosta JM, Dash DH, Levitt KN, Rowe J, Schooler EM,. A distributed host-based worm detection system. In Proceedings of the 2006 SIGCOMM Workshop on Large-Scale Attack Defense (LSAD'06). ACM: New York, NY, USA, 2006; 107-113, DOI: 10.1145/1162666.1162668. (Pubitemid 46740680)
5. Ormándi R, Hegedus I, Jelasity M,. Asynchronous peer-to-peer data mining with stochastic gradient descent. In 17th International European Conference on Parallel and Distributed Computing (Euro-Par 2011), Vol. 6852, Lecture Notes in Computer Science. Springer-Verlag: Berlin Heidelberg, 2011; 528-540.
6. van Renesse R, Birman KP, Vogels W,. Astrolabe: a robust and scalable technology for distributed system monitoring, management, and data mining. ACM Transactions on Computer Systems 2003; 21 (2): 164-206. DOI: 10.1145/762483.762485. (Pubitemid 44159304)
7. Jelasity M, Montresor A, Babaoglu O,. Gossip-based aggregation in large dynamic networks. ACM Transactions on Computer Systems 2005; 23 (3): 219-252. DOI: 10.1145/1082469.1082470.
8. Boyd S, Ghosh A, Prabhakar B, Shah D,. Randomized gossip algorithms. IEEE Transactions on Information Theory 2006; 52 (6): 2508-2530. (Pubitemid 43898906)
9. Pentland AS,. Society's nervous system: building effective government, energy, and public health systems. Computer 2012; 45 (1): 31-38. DOI: 10.1109/MC.2011.299.
10. Abdelzaher T, Anokwa Y, Boda P, Burke J, Estrin D, Guibas L, Kansal A, Madden S, Reich J,. Mobiscopes for human spaces. Pervasive Computing, IEEE 2007; 6 (2): 20-29. DOI: 10.1109/MPRV.2007.38. (Pubitemid 46645069)
11. Lane N, Miluzzo E, Lu H, Peebles D, Choudhury T, Campbell A,. A survey of mobile phone sensing. Communications Magazine, IEEE 2010; 48 (9): 140-150. DOI: 10.1109/MCOM.2010.5560598.
12. Diaspora. https://joindiaspora.com/.
13. Ma J, Saul LK, Savage S, Voelker GM,. Identifying suspicious URLs: an application of large-scale online learning. In Proceedings of the 26th Annual International Conference on Machine Learning (ICML '09). ACM: New York, NY, USA, 2009; 681-688, DOI: 10.1145/1553374.1553462.
14. Bottou L,. The tradeoffs of large-scale learning, 2007. Tutorial at the 21st Annual Conference on Neural Information Processing Systems (NIPS), http://leon.bottou.org/talks/largescale.
15. Bottou L, LeCun Y,. Large scale online learning. In Advances in Neural Information Processing Systems 16, Thrun S, Saul L, Schölkopf B, (eds). MIT Press: Cambridge, MA, 2004.
16. Duda RO, Hart PE, Stork DG,. Pattern Classification, (2nd edn). Wiley-Interscience: New York, 2000.
17. Cristianini N, Shawe-Taylor J,. An Introduction to Support Vector Machines and Other Kernel-based Learning Methods. Cambridge University Press: Cambridge, 2000.
18. Shalev-Shwartz S, Singer Y, Srebro N, Cotter A,. Pegasos: primal estimated sub-gradient solver for SVM. Mathematical Programming B 2010. DOI: 10.1007/s10107-010-0420-4.
19. Chapelle O,. Training a support vector machine in the primal. Neural Computation 2007; 19: 1155-1178. DOI: 10.1162/neco.2007.19.5.1155.
20. Rokach L,. Ensemble-based classifiers. Artificial Intelligence Review 2010; 33 (1): 1-39. DOI: 10.1007/s10462-009-9124-7.
21. Breiman L,. Bagging predictors. Machine Learning 1996; 24 (2): 123-140. DOI: 10.1007/BF00058655. (Pubitemid 126724382)
22. Breiman L,. Pasting small votes for classification in large databases and on-line. Machine Learning 1999; 36 (1-2): 85-103. DOI: 10.1023/A:1007563306331.
23. King V, Saia J,. Choosing a random peer. In Proceedings of the 23rd Annual ACM Symposium on Principles of Distributed Computing (PODC'04). ACM Press: New York, 2004; 125-130, DOI: 10.1145/1011767.1011786.
24. Stutzbach D, Rejaie R, Duffield N, Sen S, Willinger W,. On unbiased sampling for unstructured peer-to-peer networks. IEEE/ACM Transactions on Networking 2009; 17 (2): 377-390. DOI: 10.1109/TNET.2008.2001730.
25. Hall C, Carzaniga A,. Uniform sampling for directed P2P networks. In Euro-Par 2009, Vol. 5704, Sips H, Epema D, Lin HX, (eds), Lecture Notes in Computer Science. Springer: Berlin Heidelberg, 2009; 511-522, DOI: 10.1007/978-3-642-03869-3-49.
26. Jelasity M, Voulgaris S, Guerraoui R, Kermarrec AM, van Steen M,. Gossip-based peer sampling. ACM Transactions on Computer Systems 2007; 25 (3): 8. DOI: 10.1145/1275517.1275520. (Pubitemid 47330747)
27. Kowalczyk W, Vlassis N,. Newscast EM. In 17th Advances in Neural Information Processing Systems (NIPS)., Saul LK, Weiss Y, Bottou L, (eds). MIT Press: Cambridge, MA, 2005; 713-720.
28. Siersdorfer S, Sizov S,. Automatic document organization in a p2p environment. In Advances in Information Retrieval, Vol. 3936, Lalmas M, et al. (eds), LNCS. Springer: Berlin Heidelberg, 2006; 265-276. (Pubitemid 44029470)
29. Ormándi R, Hegedus I, Jelasity M,. Overlay management for fully distributed user-based collaborative filtering. In Euro-Par 2010, Vol. 6271, D'Ambra P, Guarracino M, Talia D, (eds), Lecture Notes in Computer Science. Springer-Verlag: Berlin Heidelberg, 2010; 446-457, DOI: 10.1007/978-3-642-15277- 1-43.
30. Bakker A, Ogston E, van Steen M,. Collaborative filtering using random neighbours in peer-to-peer networks. In Proceeding of the 1st ACM International Workshop on Complex Networks Meet Information and Knowledge Management (CNIKM '09). ACM: New York, NY, USA, 2009; 67-75, DOI: 10.1145/1651274.1651288.
31. Han P, Xie B, Yang F, Wang J, Shen R,. A novel distributed collaborative filtering algorithm and its implementation on P2P overlay network. In Advances in Knowledge Discovery and Data Mining, Vol. 3056, Dai H, Srikant R, Zhang C, (eds), LNCS. Springer: Berlin Heidelberg, 2004; 106-115. (Pubitemid 38824888)
32. Tveit A,. Peer-to-peer based recommendations for mobile commerce. In Proceedings 1st Intl. Workshop on Mobile Commerce (WMC '01). ACM: New York, 2001; 26-29.
33. Luo P, Xiong H, Lü K, Shi Z,. Distributed classification in peer-to-peer networks. In Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'07). ACM: New York, NY, USA, 2007; 968-976, DOI: 10.1145/1281192.1281296. (Pubitemid 350229282)
34. Ang H, Gopalkrishnan V, Ng W, Hoi S,. Communication-efficient classification in P2P networks. In Machine Learning and Knowledge Discovery in Databases (ECML PKDD), Vol. 5781, Buntine W, Grobelnik M, Mladenic D, Shawe-Ta ylor J, (eds), Lecture Notes in Computer Science. Springer: Berlin Heidelberg, 2009; 83-98, DOI: 10.1007/978-3-642-04180-8-23.
35. Ang H, Gopalkrishnan V, Ng W, Hoi S,. On classifying drifting concepts in P2P networks. In Machine Learning and Knowledge Discovery in Databases (ECML PKDD), Vol. 6321, Balcázar J, Bonchi F, Gionis A, Sebag M, (eds), Lecture Notes in Computer Science. Springer: Berlin Heidelberg, 2010; 24-39, DOI: 10.1007/978-3-642-15880-3-8.
36. Ang H, Gopalkrishnan V, Hoi S, Ng W,. Cascade RSVM in peer-to-peer networks. In Machine Learning and Knowledge Discovery in Databases (ECML PKDD), Vol. 5211, Daelemans W, Goethals B, Morik K, (eds), Lecture Notes in Computer Science. Springer: Berlin Heidelberg, 2008; 55-70, DOI: 10.1007/978-3-540-87479- 9-22.
37. Datta S, Bhaduri K, Giannella C, Wolff R, Kargupta H,. Distributed data mining in peer-to-peer networks. IEEE Internet Computing 2006; 10 (4): 18-26. DOI: 10.1109/MIC.2006.74. (Pubitemid 46444990)
38. Siersdorfer S, Sizov S,. Automatic document organization in a P2P environment. In Advances in Information Retrieval, Vol. 3936, Lalmas M, MacFarlane A, Rüger S, Tombros A, Tsikrika T, Yavlinsky A, (eds), Lecture Notes in Computer Science. Springer: Berlin Heidelberg, 2006; 265-276, DOI: 10.1007/11735106-24. (Pubitemid 44029470)
39. Hensel C, Dutta H,. GADGET SVM: a gossip-based sub-gradient svm solver.International Conference on Machine Learning (ICML), Numerical Mathematics in Machine Learning Workshop, Montreal, 2009.
40. Widrow B, Hoff ME,. Adaptive switching circuits. In 1960 IRE WESCON Convention Record, Part 4. IRE: New York, 1960; 96-104.
41. Bauer E, Kohavi R,. An empirical comparison of voting classification algorithms: bagging, boosting, and variants. Machine Learning 1999; 36 (1): 105-139. DOI: 10.1023/A:1007515423169.
42. PeerSim. http://peersim.sourceforge.net/.
43. Guyon I, Hur AB, Gunn S, Dror G,. Result analysis of the nips 2003 feature selection challenge. In Advances in Neural Information Processing Systems 17. MIT Press: Cambridge, 2004; 545-552.
44. Frank A, Asuncion A,. UCI machine learning repository, 2010.
45. Stutzbach D, Rejaie R,. Understanding churn in peer-to-peer networks. In Proceedings 6th ACM Conference on Internet Measurement (IMC'06). ACM: New York, 2006; 189-202, DOI: 10.1145/1177080.1177105. (Pubitemid 47165601)
46. Roozenburg J,. Secure decentralized swarm discovery in Tribler. Master's Thesis, Parallel and Distributed Systems Group, Delft University of Technology, 2006.
47. McGrew DA, Fluhrer SR,. Multiple forgery attacks against message authentication codes. IACR Cryptology ePrint Archive 2005; 2005: 161.