Monotonic calibrated interpolated look-up tables

Journal of Machine Learning Research

Volumn 17, Issue , 2016, Pages 1-47

  Gupta, Maya ^a   Cotter, Andrew ^a   Pfeifer, Jan ^a   Voevodski, Konstantin ^a   Canini, Kevin ^a   Mangylov, Alexander ^a   Moczydlowski, Wojciech ^a   Van Esbroeck, Alexander ^a  

^a GOOGLE INC   (United States)

Author keywords

Interpolation; Interpretability; Look up tables; Monotonicity

Indexed keywords

ARTIFICIAL INTELLIGENCE; CALIBRATION; CONSTRAINT THEORY; INTERPOLATION; LEARNING SYSTEMS;

INTERPRETABILITY; LARGE-SCALE LEARNING; LINEAR INEQUALITY CONSTRAINTS; LOOK UP TABLE; MACHINE LEARNING APPLICATIONS; MACHINE LEARNING PROBLEM; MONOTONICITY; STRUCTURAL RISK MINIMIZATION;

TABLE LOOKUP;

EID: 84989181117     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (75)

1. Y. S. Abu-Mostafa. A method for learning from hints. In Advances in Neural Information Processing Systems, pages 73-80, 1993.
2. N. P. Archer and S. Wang. Application of the back propagation neural network algorithm with monotonicity constraints for two-group classification problems. Decision Sciences, 24(1):60-75, 1993.
3. F. Bach. Learning with submodular functions: A convex optimization perspective. Foundations and Trends in Machine Learning, 6(2), 2013.
4. R. E. Barlow, D. J. Bartholomew, J. M. Bremner, and H. D. Brunk. Statistical inference under order restrictions; the theory and application of isotonic regression. Wiley, New York, USA, 1972.
5. A. Ben-David. Automatic generation of symbolic multiattribute ordinal knowledge based DSS: methodology and applications. Decision Sciences, pages 1357-1372, 1992.
6. A. Ben-David. Monotonicity maintenance in information-theoretic machine learning algorithms. Machine Learning, 21:35-50, 1995.
7. A. Ben-David, L. Sterling, and Y. H. Pao. Learning and classification of monotonic ordinal concepts. Computational Intelligence, 5(1):45-49, 1989.
8. S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, 3(1), 2010.
9. L. Breiman. Random forests. Machine Learning, 45(1):5-32, 2001.
10. J. Casillas, O. Cordon, F. Herrera, and L. Magdalena (Eds.). Trade-off between accuracy and interpretability in fuzzy rule-based modelling. Physica-Verlag, 2002.
11. R. Chandrasekaran, Y. U. Ryu, V. S. Jacob, and S. Hong. Isotonic separation. INFORMS Journal on Computing, 17(4):462-474, 2005.
12. A. Cotter, M. R. Gupta, and J. Pfeifer. A Light Touch for Heavily Constrained SGD. arXiv preprint, 2015. URL http://arxiv.org/abs/1512.04960.
13. N. Dalvi, M. Olteanu, M. Raghavan, and P. Bohannon. Deduplicating a places database. Proc. ACM WWW Conf., 2014.
14. H. Daniels and M. Velikova. Monotone and partially monotone neural networks. IEEE Trans. Neural Networks, 21(6):906-917, 2010.
15. O. Dekel, R. Gilad-Bachrach, O. Shamir, and L. Xiao. Optimal distributed online prediction using mini-batches. Journal Machine Learning Research, 13(1):165-202, January 2012.
16. J. Duchi, E. Hazan, and Y. Singer. Adaptive subgradient methods for online learning and stochastic optimization. Journal Machine Learning Research, 12:2121-2159, 2011.
17. C. Dugas, Y. Bengio, F. Bélisle, C. Nadeau, and R. Garcia. Incorporating second-order functional knowledge for better option pricing. In Advances in Neural Information Processing Systems (NIPS), 2000.
18. C. Dugas, Y. Bengio, F. Bélisle, C. Nadeau, and R. Garcia. Incorporating functional knowledge in neural networks. Journal Machine Learning Research, 2009.
19. W. Duivesteijn and A. Feelders. Nearest neighbour classification with monotonicity constraints. Proc. European Conf. Machine Learning, pages 301-316, 2008.
20. A. Feelders. Monotone relabeling in ordinal classification. Proc. IEEE Conf. Data Mining, pages 803-808, 2010.
21. M. Fernandez-Delgado, E. Cernadas, S. Barro, and D. Amorim. Do we need hundreds of classifiers to solve real world classification problems? Journal Machine Learning Research, 2014.
22. E. K. Garcia and M. R. Gupta. Lattice regression. In Advances in Neural Information Processing Systems (NIPS), 2009.
23. E. K. Garcia, S. Feldman, M. R. Gupta, and S. Srivastava. Completely lazy learning. IEEE Trans. Knowledge and Data Engineering, 22(9):1274-1285, Sept. 2010.
24. E. K. Garcia, R. Arora, and M. R. Gupta. Optimized regression for efficient function evaluation. IEEE Trans. Image Processing, 21(9):4128-4140, Sept. 2012.
25. S. Garcia, A. Fernandez, J. Luengo, and F. Herrera. A study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability. Soft Computing, 13:959-977, 2009.
26. I. J. Good. The Estimation of Probabilities: An Essay on Modern Bayesian Methods. MIT Press, 1965.
27. H. Gruber, M. Holzer, and O. Ruepp. Sorting the slow way: an analysis of perversely awful randomized sorting algorithms. In Fun with Algorithms, pages 183-197. Springer, 2007.
28. M. Gupta, S. Bengio, and J. Weston. Training highly multiclass classifiers. Journal Machine Learning Research, 2014.
29. M. R. Gupta, R. M. Gray, and R. A. Olshen. Nonparametric supervised learning by linear interpolation with maximum entropy. IEEE Trans. Pattern Analysis and Machine Intelligence, 28(5):766-781, 2006.
30. T. Hastie and R. Tibshirani. Generalized Additive Models. Chapman Hall, New York, 1990.
31. T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning. Springer-Verlag, New York, 2001.
32. C. C. Holmes and N. A. Heard. Generalized monotonic regression using random change points. Statistics in Medicine, 22:623-638, 2003.
33. A. Howard and T. Jebara. Learning monotonic transformations for classification. In Advances in Neural Information Processing Systems, 2007.
34. H. Ishibuchi and Y. Nojima. Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning. International Journal of Approximate Reasoning, 44:4-31, 2007.
35. T. Joachims, L. Granka, B. Pan, H. Hembrooke, and G. Gay. Accurately interpreting clickthrough data as implicit feedback. Proc. SIGIR, 2005.
36. H. R. Kang. Comparison of three-dimensional interpolation techniques by simulations. SPIE Vol. 2414, 1995.
37. H. R. Kang. Color Technology for Electronic Imaging Devices. SPIE Press, USA, 1997.
38. J. Kasson, W. Plouffe, and S. Nin. A tetrahedral interpolation technique for color space conversion. SPIE Vol. 1909, 1993.
39. H. Kay and L. H. Ungar. Estimating monotonic functions and their bounds. AIChE Journal, 46(12):2426-2434, 2000.
40. R. E. Knop. A note on hypercube partitions. Journal of Combinatorial Theory, Ser. A, 15 (3):338-342, 1973.
41. W. Kotlowski and R. Slowinski. Rule learning with monotonicity constraints. In Proceedings International Conference on Machine Learning, 2009.
42. F. Lauer and G. Bloch. Incorporating prior knowledge in support vector regression. Machine Learning, 70(1):89-118, 2008.
43. X. Liao, H. Li, and L. Carin. Quadratically gated mixture of experts for incomplete data classification. Proc. ICML, 2007.
44. T.-Y. Liu. Learning to Rank for Information Retrieval. Springer, 2011.
45. Malik Magdon-Ismail and J. Sill. A linear fit gets the correct monotonicity directions. Machine Learning, pages 21-43, 2008.
46. G. Mann, R. McDonald, M. Mohri, N. Silberman, and D. D. Walker. Efficient large-scale distributed training of conditional maximum entropy models. Advances in Neural Information Processing Systems (NIPS), 2009.
47. D. G. Mead. Dissection of the hypercube into simplexes.Proc. Amer. Math. Soc., 76:302-304, 1979.
48. A. Minin, M. Velikova, B. Lang, and H. Daniels. Comparison of universal approximators incorporating partial monotonicity by structure. Neural Networks, 23(4):471-475, 2010.
49. H. Mukarjee and S. Stern. Feasible nonparametric estimation of multiargument monotone functions. Journal of the American Statistical Association, 89(425):77-80, 1994.
50. B. Neelon and D. B. Dunson. Bayesian isotonic regression and trend analysis. Biometrics, 60:398-406, 2004.
51. A. Nemirovski, A. Juditsky, G. Lan, and A. Shapiro. Robust stochastic approximation approach to stochastic programming. SIAM Journal on Optimization, 19(4):1574-1609, January 2009.
52. K. Neumann, M. Rolf, and J. J. Steil. Reliable integration of continuous constraints into extreme learning machines. International Journal of Uncertainty, Fuzziness and Knowledge- Based Systems, 21(supp02):35-50, 2013.
53. R. Nock. Inducing interpretable voting classifiers without trading accuracy for simplicity: Theoretical results, approximation algorithms, and experiments. Journal Artificial Intelligence Research, 17:137-170, 2002.
54. K.-M. Osei-Bryson. Post-pruning in decision tree induction using multiple performance measures. Computers and Operations Research, 34:3331-3345, 2007.
55. R. Potharst and A. J. Feelders. Classification trees for problems with monotonicity constraints. ACM SIGKDD Explorations, pages 1-10, 2002a.
56. R. Potharst and A. J. Feelders. Pruning for monotone classification trees. Springer Lecture Notes on Computer Science, 2810:1-12, 2002b.
57. Y.-J. Qu and B.-G. Hu. Generalized constraint neural network regression model subject to linear priors. IEEE Trans. on Neural Networks, 22(11):2447-2459, 2011.
58. J. O. Ramsay. Estimating smooth monotone functions. Journal of the Royal Statistical Society, Series B, 60:365-375, 1998.
59. G. Rätsch, S. Sonnenburg, and C. Schäfer. Learning interpretable SVMs for biological sequence classification. BMC Bioinformatics, 7, 2006.
60. J. Riihimäki and A. Vehtari. Gaussian processes with monotonicity information. In International Conference on Artificial Intelligence and Statistics, pages 645-652, 2010.
61. R. Rovatti, M. Borgatti, and R. Guerrieri. A geometric approach to maximum-speed n-dimensional continuous linear interpolation in rectangular grids. IEEE Trans. on Computers, 47(8):894-899, 1998.
62. F. Schimdt and R. Simon. Some geometric probability problems involving the Eulerian numbers. Electronic Journal of Combinatorics, 4(2), 2007.
63. G. Sharma and R. Bala. Digital Color Imaging Handbook. CRC Press, New York, 2002.
64. T. S. Shively, T. W. Sager, and S. G. Walker. A Bayesian approach to non-parametric monotone function estimation. Journal of the Royal Statistical Society, Series B, 71(1):159-175, 2009.
65. P. K. Shukla and S. P. Tripathi. A review on the interpretability-accuracy trade-off in evolutionary multi-objective fuzzy systems (EMOFS). Information, 2012.
66. J. Sill. Monotonic networks. Advances in Neural Information Processing Systems (NIPS), 1998.
67. J. Sill and Y. S. Abu-Mostafa. Monotonicity hints. Advances in Neural Information Processing Systems (NIPS), pages 634-640, 1997.
68. D. J. Spiegelhalter and R. P. Knill-Jones. Statistical and knowledge-based approaches to clinical decision support systems, with an application in gastroenterology. Journal of the Royal Statistical Society A, 147:35-77, 1984.
69. J. Spouge, H. Wan, and W. J. Wilbur. Least squares isotonic regression in two dimensions. Journal of Optimization Theory and Applications, 117(3):585-605, 2003.
70. C. Strannegaard. Transparent neural networks. Proc. Artificial General Intelligence, 2012.
71. B. Sun and S. Zhou. Study on the 3D interpolation models used in color conversion. IACSIT Intl. Journal Engineering and Technology, 4(1), 2012.
72. A. van Esbroeck, S. Singh, I. Rubinfeld, and Z. Syed. Evaluating trauma patients: Addressing missing covariates with joint optimization. Proc. AAAI, 2014.
73. M. Villalobos and G. Wahba. Inequality-constrained multivariate smoothing splines with application to the estimation of posterior probabilities. Journal of the American Statistical Association, 82(397):239-248, 1987.
74. S. Wang. A neural network method of density estimation for univariate unimodal data. Neural Computing & Applications, 2(3):160-167, 1994.
75. L. Yu and J. Xiao. Trade-off between accuracy and interpretability: experience-oriented fuzzy modeling via reduced-set vectors. Computers and Mathematics with Applications, 57:885-895, 2012.