Resource-efficient machine learning in 2 KB RAM for the Internet of Things

34th International Conference on Machine Learning, ICML 2017

Volumn 4, Issue , 2017, Pages 3062-3071

  Kumar, Ashish ^a   Goyal, Saurabh ^b   Varma, Manik ^a  

^a MICROSOFT RESEARCH   (United States)

^b INDIAN INSTITUTE OF TECHNOLOGY DELHI   (India)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; DIGITAL ARITHMETIC; FORECASTING; LEARNING SYSTEMS; MICROCONTROLLERS; RANDOM ACCESS STORAGE; TREES (MATHEMATICS);

BATTERY CONSUMPTION; BENCHMARK DATASETS; EFFICIENT PREDICTIONS; LOW-DIMENSIONAL SPACES; PREDICTION ACCURACY; RESOURCE-EFFICIENT; STATE-OF-THE-ART METHODS; TREE-BASED ALGORITHMS;

INTERNET OF THINGS;

EID: 85048452922     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (54)

1. The right whale dataset. https://www. kaggle. com/c/whale-detection-challenge/data.
2. Ba, J. and Caruana, R. Do deep nets really need to be deep? In NIPS, 2014.
3. Blumensath, T. and Davies, M. E. Iterative thresholding for sparse approximations. Journal of Fourier Analysis and Applications, 14(5-6), 2008.
4. BonsaiCode. Code for Bonsai. http://www.manikvarma.org/code/Bonsai/download.html.
5. Breiman, L. Random forests. ML, 2001.
6. Brciman, L., Friedman, J., Stone, C. J., and Olshcn, R. A. Classification and regression trees. In CRC press, 1984.
7. Bucilua, C, Caruana, R., and Niculescu-Mizil, A. Model compression. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, 2006.
8. Burges, C. J. From ranknet to lambdarank to lambdamart: An overview. Learning, 11(23-581), 2010.
9. Campos, T. E., Babu, B. R., and Varma, M. Character recognition in natural images. In VISAPP(2), 2009.
10. Dekel, O., Jacobbs, C, and Xiao, L. Pruning decision forests. In Personal Communications, 2016.
11. Denton, E. L., Zaremba, W., Bruna, J., LeCun, Y, and Fergus, R. Exploiting linear structure within convolutional networks for efficient evaluation. In NIPS, 2014.
12. Duda, R. O., Hart, P. E., and Stork, D. G. Pattern Classification. John Wiley and Sons, 2nd edition, 2002.
13. Han, S., Mao, H., and Dally, W. J. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. In ICLR, 2016.
14. Hsieh, C. J., Si, S., and Dhillon, I. Fast prediction for large-scale kernel machines. In NIPS, 2014.
15. Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R., and Bengio, Y Quantized neural networks: Training neural networks with low precision weights and activations. CoRR, abs/1609.07061, 2016.
16. Hull, J. J. A database for handwritten text recognition research. IEEE PAMI, 16, 1994.
17. Iandola, F. N., Moskewicz, M. W, Ashraf, K., Han, S., Dally, W. J., and Keutzer, K. Squeezenet: Alexnet-level accuracy with 50x fewer parameters and 1MB model size. CoRR, abs/1602.07360, 2016.
18. Ioannou, Y, Robertson, D., Cipolla, R., and Criminisi, A. Deep roots: Improving enn efficiency with hierarchical filter groups. arXiv preprint arXiv: 1605.06489, 2016a.
19. Ioannou, Y, Robertson, D., Kontschieder, D. Zikicand P., Shotton, J., Brown, M., and Criminisi, A. Decision forests, convolutional networks and the models in-between. arXiv preprint arXiv: 1603.01250, 2016b.
20. Jain, P., Tewari, A., and Kar, P. On iterative hard thresholding methods for high-dimensional m-estimation. In NIPS, 2014.
21. Joly, A., Schnitzler, F., Geurts, P., and Wehenkel, L. Ll-based compression of random forest models. In ESANN, 2012.
22. Jose, C, Goyal, P., Aggrwal, P., and Varma, M. Local deep kernel learning for efficient non-linear svm prediction. In ICML, 2013. https://manikvarma.github.io/code/LDKL/download.html.
23. Kasprowski, P. and Ober, J. Eye movement in biometrics. In eccv, 2004.
24. Kontschieder, P., Fiterau, M., Criminisi, A., and Bulo, S. R. Deep neural decision forests. In ICCV, 2015.
25. Krizhevsky, A. Learning multiple layers of features from tiny images. Technical report, 2009.
26. Kulkami, V. Y. and Sinha, P. K. Pruning of random forest classifiers: A survey and future directions. In ICDSE, 2012.
27. Kusner, M. J., Chen, W., Zhou, Q., Xu, Z. E., Weinberger, K. Q., and Chen, Y. Feature-cost sensitive learning with submodular trees of classifiers. In AAAI, 2014a.
28. Kusner, M. J., Tyree, S., Weinberger, K. Q., and Agrawal, K. Stochastic neighbor compression. In ICML, 2014b. http://mkusner.github.io/#code.
29. Le, Q., Sarlós, T., and Smola, A. Fastfood-approximating kernel expansions in loglinear time. In ICML, 2013.
30. LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11): 2278-2324, 1998.
31. Li, S. Z. Markov Random Field Modeling in Image Analysis. Springer-Verlag, 2001.
32. Li, Z., Yang, T., Zhang, L., and Jin, R. Fast and accurate refined nyström-based kernel svm. In AAAI, 2016.
33. Meunier, F., Wood, A., Weiss, K., Huberty, K., Flannery, S., Moore, J., Hettenbach, C, and Lu, B. The internet of things is now connecting the real economy. Technical report, Morgan Stanley, 2014.
34. Murthy, S. K., Kasif, S., and Salzberg, S. A system for induction of oblique decision trees. JAIR, 2, 1994.
35. Nan, F., Wang, J., and Saligrama, V. Feature-budgeted random forest. In ICML, 2015. http://sites.bu.edu/data/code-4/.
36. Nan, F., Wang, J., and Saligrama, V. Pruning random forests for prediction on a budget. In NIPS, 2016.
37. Norouzi, M., Collins, M., Johnson, M. A., Fleet, D. J., and Kohli, P. Efficient non-greedy optimization of decision trees. In NIPS, 2015.
38. Pohlen, T. LibJungle - Decision Jungle Library, https://bitbucket.org/geekStack/libjungle.
39. Rastegari, M., Ordonez, V., Redmon, J., and Farhadi, A. Xnor-net: Imagenet classification using binary convolutional neural networks. In ECCV, 2016.
40. Rokach, L. and Maimon, O. Data mining with decision trees: theory and applications. World scientific, 2014.
41. Shankar, S., Robertson, D., Ioannou, Y., Criminisi, A., and Cipolla, R. Refining architectures of deep convolutional neural networks. In CVPR, 2016.
42. Sherali, H. D, Hobeika, A. G., and Jeenanunta, C. An optimal constrained pruning strategy for decision trees. INFORMS Journal on Computing, 21(1), 2009.
43. Shotton, J., Sharp, T., Kohli, P., Nowozin, S., Winn, J., and Criminisi, A. Decision jungles: Compact and rich models for classification. In NIPS, 2013.
44. Utgoff, P. E. Perceptron trees: A case study in hybrid concept representations. Connection Science, 1(4), 1989
45. Varma, M. and Zisserman, A. A statistical approach to texture classification from single images. IJCV, 62(1- 2): 61-81, 2005.
46. Viola, P. and Jones, M. J. Robust real-time face detection. IJCV, 57(2), 2004.
47. Wang, J., Trapeznikov, K., and Saligrama, V. Efficient learning by directed acyclic graph for resource constrained prediction. In NIPS, 2015.
48. Wang, W, Chen, C, Chen, W., Rai, P., and Carin, L. Deep distance metric learning with data summarization. In ECMLPKDD, 2016.
49. Wu, J., Leng, C, Wang, Y., Hu, Q., and Cheng, J. Quantized convolutional neural networks for mobile devices. In CVPR, 2016.
50. Xu, Z., Weinberger, K. Q., and Chapelle, O. The greedy miser: Learning under test-time budgets. In ICML, 2012.
51. Xu, Z. E., Kusner, M. J., Weinberger, K. Q., and Chen, M. Cost-sensitive tree of classifiers. In ICML, 2013.
52. Yang, J., Li, Y., Tian, Y, Duan, L., and Gao, W. Group-sensitive multiple kernel learning for object categorization. In ICCV, 2009.
53. Yang, Z., Moczulski, M., Denil, M., Freitas, N., Smola, A. J., Song, L., and Wang, Z. Deep fried convnets. ICCV, 2015.
54. Zhang, Y. and Huci-chucn, H. Decision tree pruning via integer programming. Technical report, 2005.