"Short-Dot": Computing large linear transforms distributedly using coded short dot products

Advances in Neural Information Processing Systems

Volumn , Issue , 2016, Pages 2100-2108

  Dutta, Sanghamitra ^a   Cadambe, Viveck ^b   Grover, Pulkit ^a  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b PENNSYLVANIA STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BOTTLES; COMPUTATION THEORY; COMPUTER PROGRAMMING; ECONOMIC AND SOCIAL EFFECTS; LEARNING ALGORITHMS; LEARNING SYSTEMS; MATHEMATICAL TRANSFORMATIONS; MULTIPROCESSING SYSTEMS; REDUNDANCY;

COMPUTATION TIME; COMPUTING CLUSTERS; LINEAR TRANSFORM; NOVEL TECHNIQUES; PARALLEL AND DISTRIBUTED COMPUTING; PROBABILISTIC ANALYSIS; REDUNDANT COMPUTATION; SYSTEM DESIGNERS;

DISTRIBUTED COMPUTER SYSTEMS;

EID: 85019226266     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (20)

1. Kangwook Lee, Maximilian Lam, Ramtin Pedarsani, Dimitris Papailiopoulos, and Kannan Ramchandran. Speeding Up Distributed Machine Learning Using Codes. NIPS Workshop on Learning Systems, 2015.
2. Da Wang, Gauri Joshi, and Gregory Wornell. Using straggler replication to reduce latency in large-scale parallel computing. In ACM SIGMETRICS Performance Evaluation Review, Volume 43, pages 7-11, 2015.
3. Da Wang, Gauri Joshi, and Gregory Wornell. Efficient Task Replication for Fast Response Times in Parallel Computation. In ACM SIGMETRICS Performance Evaluation Review, Volume 42, pages 599-600, 2014.
4. Gauri Joshi, Yanpei Liu, and Emina Soljanin. On the delay-storage trade-off in content download from coded distributed storage systems. IEEE Journal on Selected Areas in Communications, 32(5):989-997, 2014.
5. Longbo Huang, Sameer Pawar, Hao Zhang, and Kannan Ramchandran. Codes can reduce queueing delay in data centers. In Proceedings IEEE International Symposium on Information Theory (ISIT), pages 2766-2770, 2012.
6. William Dally. High-performance hardware for machine learning. NIPS Tutorial, 2015.
7. Vipin Kumar, Ananth Grama, Gupta Anshul, and George Karypis. Introduction to Parallel Computing: Design and Analysis of Algorithms. The Benjamin/Cummings Publishing Company, Inc., Redwood City, 1994.
8. Geoffrey C Fox, Steve W Otto, and Anthony JG Hey. Matrix algorithms on a hypercube I: Matrix multiplication. Parallel computing, 4(1):17-31, 1987.
9. Volker Strassen. Gaussian elimination is not optimal. Numerische Mathematik, 13(4):354-356, 1969.
10. Grey Ballard, James Demmel, Olga Holtz, and Oded Schwartz. Communication costs of strassen's matrix multiplication. Communications of the ACM, 57(2):107-114, 2014.
11. Jeffrey Dean and Luiz André Barroso. The tail at scale. Communications of the ACM, 56(2):74-80, 2013.
12. Konstantin Shvachko, Hairong Kuang, Sanjay Radia, and Robert Chansler. The Hadoop Distributed File System. In Proceedings IEEE Symposium on Mass Storage Systems and Technologies (MSST), pages 1-10, 2010.
13. Kuang-Hua Huang and Jacob A. Abraham. Algorithm-based fault tolerance for matrix operations. IEEE transactions on computers, 100(6):518-528, 1984.
14. Thomas Herault and Yves Robert. Fault-Tolerance Techniques for High Performance Computing. Springer, 2015.
15. William Ryan and Shu Lin. Channel codes: Classical and Modern. Cambridge University Press, 2009.
16. Songze Li, Mohammad Ali Maddah-Ali, and A Salman Avestimehr. A unified coding framework for distributed computing with straggling servers. arXiv:1609.01690v1 [cs.IT], 2016.
17. Ihab Nahlus, Eric P Kim, Naresh R Shanbhag, and David Blaauw. Energy-efficient Dot-Product Computation using a Switched Analog Circuit Architecture. In International Symposium on Low Power Electronics and Design (ISLPED), pages 315-318, 2014.
18. Ning C Wang, Sujan K Gonugondla, Ihab Nahlus, Naresh Shanbhag, and Eric Pop. GDOT: a Graphene-Based Nanofunction for Dot-Product Computation. In IEEE Symposium on VLSI Technology, 2016.
19. HTCondor. https://research.cs.wisc.edu/htcondor/.
20. Yann LeCun, Corinna Cortes, and Christopher JC Burges. The MNIST database of handwritten digits. http://yann.lecun.com/exdb/mnist, 1998.