Convergence of distributed asynchronous learning vector quantization algorithms

Journal of Machine Learning Research

Volumn 12, Issue , 2011, Pages 3431-3466

  Patra, Benoît ^a,b  

^a SORBONNE UNIVERSITÉ   (France)

^b SAS   (France)

Author keywords

Asynchronous; Distributed; Distributed consensus; K means; Scalability; Stochastic optimization; Vector quantization

Indexed keywords

ASYNCHRONOUS; DISTRIBUTED; DISTRIBUTED CONSENSUS; K-MEANS; STOCHASTIC OPTIMIZATIONS;

COMPUTER ARCHITECTURE; CONVERGENCE OF NUMERICAL METHODS; LEARNING ALGORITHMS; OPTIMIZATION; PARALLEL ARCHITECTURES; SCALABILITY; VECTOR QUANTIZATION; VECTORS;

CLUSTERING ALGORITHMS;

EID: 84855999469     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (42)

1. E. A. Abaya and G. L. Wise. Convergence of vector quantizers with applications to optimal quantization. SIAM Journal on Applied Mathematics, 44(1):183-189, 1984.
2. A. Antos. Improved minimax bounds on the test and training distortion of empirically designed vector quantizers. IEEE Transactions on Information Theory, 51(11):4022-4032, 2005. (Pubitemid 41622599)
3. A. Antos, L. Györfi, and A. György. Individual convergence rates in empirical vector quantizer design. IEEE Transactions on Information Theory, 51(11):4013-4022, 2005. (Pubitemid 41622598)
4. P. L. Bartlett, T. Linder, and G. Lugosi. The minimax distortion redundancy in empirical quantizer design. IEEE Transactions on Information Theory, 44(5):1802-1813, 1998. (Pubitemid 128738552)
5. A. Benveniste, M. Métivier, and P. Priouret. Adaptive Algorithms and Stochastic Approximations. Springer-Verlag, 1990.
6. D. P. Bertsekas and J. N. Tsitsiklis. Parallel and Distributed Computation: Numerical Methods. Prentice-Hall, Inc., 1989.
7. G. Biau, L. Devroye, and G. Lugosi. On the performance of clustering in hilbert spaces. IEEE Transactions on Information Theory, 54(2):781-790, 2008. (Pubitemid 351314665)
8. V. D. Blondel, J. M. Hendrickx, A. Olshevsky, and J. N. Tsitsiklis. Convergence in multiagent coordination, consensus, and flocking. In Decision and Control, 2005 and 2005 European Control Conference., 2005.
9. L. Bottou. Stochastic gradient learning in neural networks. In Proceedings of Neuro-Nîmes 91, 1991.
10. L. Bottou. Online algorithms and stochastic approximations. In Online Learning and Neural Networks. Cambridge University Press, 1998.
11. L. Bottou and Y. Bengio. Convergence properties of the k-means algorithm. In Advances in Neural Information Processing Systems. MIT Press, 1995.
12. L. Bottou and Y. Le Cun. Large scale online learning. In Advances in Neural Information Processing Systems 16. MIT Press, 2004.
13. F. Bullo, J. Cortés, and S. Martínez. Distributed Control of Robotic Networks. Princeton University Press, 2009.
14. G. Choquet. Topology. Academic Press, 1966.
15. p (1/n). In In Proceedings of IEEE International Symposium on Information Theory, 1994.
16. J. W. Durham, R. Carli, P. Frasca, and F. Bullo. Discrete partitioning and coverage control with gossip communication. In ASME Conference Proceedings. ASME, 2009.
17. R. Durrett. Probability: Theory and Examples. Duxbury Press, 1990.
18. J. C. Fort and G. Pagès. On the a.s. convergence of the kohonen algorithm with a general neighborhood function. The Annals of Applied Probability, 5(4):1177-1216, 1995. (Pubitemid 126089078)
19. J. C. Fort and G. Pagès. Convergence of stochastic algorithms: from the kushner-clark theorem to the lyapounov functional method. Advances in Applied Probability, 28(4):1072-1094, 1996. (Pubitemid 126513385)
20. P. Frasca, R. Carli, and F. Bullo. Multiagent coverage algorithms with gossip communication: Control systems on the space of partitions. In American Control Conference, 2009.
21. A. Gersho and R. M. Gray. Vector Quantization and Signal Compression. Kluwer, 1992.
22. S. Graf and H. Luschgy. Foundations of Quantization for Probability Distributions. Springer-Verlag, 2000.
23. M. Inaba, N. Katoh, and H. Imai. Applications of weighted voronoi diagrams and randomization to variance-based k-clustering: (extended abstract). In SCG'94: Proceedings of the Tenth Annual Symposium on Computational Geometry, 1994.
24. D. Jungnickel. Graphs, Networks and Algorithms. Springer-Verlag, 1999.
25. T. Kohonen. Analysis of a simple self-organizing process. Biological Cybernetics, 44(2):135-140, 1982. (Pubitemid 12045888)
26. H. J. Kushner and D. S. Clark. Stochastic Approximation for Constrained and Unconstrained Systems. Springer-Verlag, 1978.
27. T. Linder. On the training distortion of vector quantizers. IEEE Transactions on Information Theory, 46(4):1617-1623, 2000.
28. T. Linder. Learning-theoretic methods in vector quantization. In Lecture Notes for the Advanced School on the Principles of Nonparametric Learning, 2001.
29. T. Linder, G. Lugosi, and K. Zeger. Rates of convergence in the source coding theorem, in empirical quantizer design, and in universal lossy source coding. IEEE Transactions on Information Theory, 40(6):1728-1740, 1994.
30. S. Lloyd. Least squares quantization in pcm. IEEE Transactions on Information Theory, 28(2):129-137, 2003.
31. J. B. MacQueen. Some methods of classification and analysis of multivariate observations. In Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, 1967.
32. R. McDonald, K. Hall, and G. Mann. Distributed training strategies for the structured perceptron. In HLT 2010: Human Language Technologies, 2010.
33. N. Murata. A statistical study of on-line learning. In Online Learning and Neural Networks, 1998.
34. G. Pagès. A space vector quantization for numerical integration. Journal of Applied and Computational Mathematics, 89(1):1-38, 1997.
35. D. Pollard. Stong consistency of k-means clustering. The Annals of Statistics, 9(1):135-140, 1981.
36. D. Pollard. Quantization and the method of k-means. IEEE Transactions on Information Theory, 28(2):199-205, 1982a.
37. D. Pollard. A central limit theorem for k-means clustering. The Annals of Probability, 28(4):199-205, 1982b.
38. H. Robbins and S. Monro. A stochastic approximation method. The Annals of Mathematical Statistics, 22(3):400-407, 1951.
39. M. J. Sabin and R. M. Gray. Global convergence and empirical consistency of the generalized lloyd algorithm. IEEE Transactions on Information Theory, 32(2):148-155, 1986.
40. J. Tsitsiklis. Problems in Decentralized Decision Making and Computation. PhD thesis, Department of EECS, MIT, Cambridge, USA, 1984.
41. J. Tsitsiklis, D. Bertsekas, and M. Athans. Distributed asynchronous deterministic and stochastic gradient optimization algorithms. IEEE Transactions on Automatic Control, 31(9):803-812, 1986. (Pubitemid 16629805)
42. M. Zinkevich, A. Smola, and J. Langford. Slow learners are fast. In Advances in Neural Information Processing Systems 22. Curran Associates, 2009.