How to infer the informational energy from small datasets

Proceedings of the International Conference on Optimisation of Electrical and Electronic Equipment, OPTIM

Volumn , Issue , 2012, Pages 1065-1070

  Caţaron, Angel ^a   Andonie, Rǎzvan ^a,b  

^a TRANSILVANIA UNIVERSITY OF BRASOV   (Romania)

^b CENTRAL WASHINGTON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

FIXED POSITIVE INTEGERS; NEAREST NEIGHBOR DISTANCE; NON-PARAMETRIC; SAMPLE POINT; SMALL DATA SET; STANDARD DISTRIBUTIONS;

OPTIMIZATION;

EID: 84864667521     PISSN: 18420133     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/OPTIM.2012.6231921     Document Type: Conference Paper

References (35)

1. R. Andonie, "Extreme data mining: Inference from small datasets," International Journal of Computers, Communications and Control, vol. 5, pp. 280-291, 2010.
2. V. Vapnik, Statistical Learning Theory. New York: Wiley, 2000.
3. J. L. Balcázar and R. V. Book, "Sets with small generalized Kolmogorov complexity," Acta Inf., vol. 23, no. 6, pp. 679-688, 1986.
4. A. Ambainis, "Application of Kolmogorov complexity to inductive inference with limited memory," in ALT '95: Proceedings of the 6th International Conference on Algorithmic Learning Theory. London, UK: Springer-Verlag, 1995, pp. 313-318.
5. A. Ambainis, K. Apsitis, C. Calude, R. Freivalds, M. Karpinski, T. Lar-feldt, I. Sala, and J. Smotrovs, "Effects of Kolmogorov complexity present in inductive inference as well," in ALT '97: Proceedings of the 8th International Conference on Algorithmic Learning Theory. London, UK: Springer-Verlag, 1997, pp. 244-259.
6. J.-L. Yuan and T. Fine, "Neural-network design for small training sets of high dimension," IEEE Tnansactions on Neural Networks, vol. 9, pp. 266-280, 1998.
7. J.-L. Yuan, "Bootstrapping nonparametric feature selection algorithms for mining small data sets," in Proceedings of the International Joint Conference on Neural Networks (IJCNN), 1999, pp. 2526-2529.
8. C. Huang and C. Moraga, "A diffusion-neural-network for learning from small samples," International Journal of Approximate Reasoning, vol. 35, pp. 137-161, 2004.
9. R. Mao, H. Zhu, L. Zhang, and A. Chen, "A new method to assist small data set neural network learning," in Proceedings of the Sixth International Conference on Intelligent Systems Design and Applications (ISDA06), 2006, pp. 17-22.
10. D.-C. Li, C.-S. Wu, T. T.-I., and L. Y.-S., "Using mega-trend-diffusion and artificial samples in small data set learning for early flexible manufacturing system scheduling knowledge," Computers and Operations Research, vol. 34, pp. 966-982, 2007.
11. D.-C. Li, C.-W. Yeh, T.-I. Tsai, Y.-H. Fang, and S. Hu, "Acquiring knowledge with limited experience," Expert Systems, vol. 24, pp. 162-170, 2007.
12. D.-C. Li, C.-S. Wu, T.-I. Tsai, and F. M. Chang, "Using mega-fuzzification and data trend estimation in small data set learning for early FMS scheduling knowledge," Comput. Oper. Res., vol. 33, no. 6, pp. 1857-1869, 2006.
13. T.-I. Tsai and D.-C. Li, "Approximate modeling for high order nonlinear functions using small sample sets," Expert Syst. Appl., vol. 34, no. 1, pp. 564-569, 2008.
14. D.-C. Li and C.-W. Yeh, "A non-parametric learning algorithm for small manufacturing data sets," Expert Syst. Appl., vol. 34, no. 1, pp. 391-398, 2008.
15. D.-C. Li and C.-W. Liu, "A neural network weight determination model designed uniquely for small data set learning," Expert Syst. Appl., vol. 36, no. 6, pp. 9853-9858, 2009.
16. H. Lohr, Sampling: Design and Analysis. Duxbury Press, 1999.
17. J. Gamez, F. Modave, and O. Kosheleva, "Selecting the most representative sample is NP-hard: Need for expert (fuzzy) knowledge," in Fuzzy Systems, 2008. FUZZ-IEEE 2008. (IEEE World Congress on Computational Intelligence). IEEE International Conference on, June 2008, pp. 1069-1074.
18. B. Silverman, Density Estimation for Statistics and Data Analysis (Chapman & Hall/CRC Monographs on Statistics & Applied Probability). Chapman and Hall/CRC, 1986.
19. L. Paninski, "Estimation of entropy and mutual information," Neural Comput., vol. 15, pp. 1191-1253, June 2003.
20. J. Walters-Williams and Y. Li, "Estimation of mutual information: A survey," in Proceedings of the 4th International Conference on Rough Sets and Knowledge Technology. Berlin, Heidelberg: Springer-Verlag, 2009, pp. 389-396.
21. A. Kraskov, H. Stögbauer, and P. Grassberger, "Estimating mutual information," Phys. Rev. E, vol. 69, pp. 1-16, Jun 2004.
22. J. A. Bonachela, H. Hinrichsen, and M. A. Munoz, "Entropy estimates of small data sets," MATH. THEOR., vol. 41, p. 202001, 2008.
23. J. C. Principe, D. Xu, and J. W. F. III., "Information-theoretic learning," in Unsupervised Adaptive Filtering, S. Haykin, Ed., Wiley, New York, 2000.
24. R. Andonie and A. Caţaron, "An informational energy LVQ approach for feature ranking," in European Symposium on Artificial Neural Networks 2004, pages In d-side publications, 2004, pp. 471-476.
25. A. Caţaron and R. Andonie, "Energy generalized lvq with relevance factors," in Neural Networks, 2004. Proceedings. 2004 IEEE International Joint Conference on, vol. 2, july 2004, pp. 1421-1426 vol.2.
26. - -, "Informational energy kernel for lvq," in Proceedings of the 15th international conference on Artificial neural networks: formal models and their applications - Volume Part II, ser. ICANN'05. Berlin, Heidelberg: Springer-Verlag, 2005, pp. 601-606.
27. R. Andonie, "How to learn from small training sets," Dalle Molle Institute for Artificial Intelligence (IDSIA), Manno-Lugano, Switzerland, invited talk, September 2009.
28. A. Caţaron and R. Andonie, "Energy supervised relevance neural gas for feature ranking," Neural Processing Letters, vol. 32, no. 1, pp. 59-73, 2010.
29. R. Andonie and F. Petrescu., "Interacting systems and informational energy," Foundation of Control Engineering, no. 11, pp. 53-59, 1986.
30. O. Onicescu, "Theorie de l'information. energie informationelle, " C. R. Acad. Sci. Paris, Ser. A-B, no. 263, pp. 841-842, 1966.
31. S. Guiasu, Information theory with applications. McGraw Hill New York, 1977.
32. L. F. Kozachenko and N. N. Leonenko, "Sample estimate of the entropy of a random vector," Probl. Peredachi Inf., vol. 23, no. 2, pp. 9-16, 1987.
33. H. Singh, N. Misra, V. Hnizdo, A. Fedorowicz, and E. Demchuk, "Nearest neightboor estimates of entropy," American Journal of Mathematical and Management Sciences, vol. 23, pp. 301-321, 2003.
34. Q. Wang, S. R. Kulkarni, and S. Verdu, "A nearest-neighbor approach to estimating divergence between continuous random vectors," in Proc. of the IEEE International Symposium on Information Theory, Seattle, WA, 2006.
35. L. Faivishevsky and J. Goldberger, "Ica based on a smooth estimation of the differential entropy," in NIPS, 2008.