Learning from data: A tutorial with emphasis on modern pattern recognition methods

IEEE Sensors Journal

Volumn 2, Issue 3, 2002, Pages 203-217

  Pardo, Matteo ^a   Sberveglieri, Giorgio ^a  

^a UNIVERSITY OF BRESCIA   (Italy)

Author keywords

Boosting; Data analysis; Electronic nose; Ensemble methods; Learning; Pattern recognition

Indexed keywords

BOUNDARY CONDITIONS; DATA ACQUISITION; DATA REDUCTION; LEARNING SYSTEMS; PROBLEM SOLVING; STATISTICAL METHODS; THEOREM PROVING;

BAYES THEOREM; CLASSIFICATION ACCURACY; COMPLEXITY CONTROL; ELECTRONIC NOSE (EN); ENSEMBLE METHODS;

PATTERN RECOGNITION;

EID: 0346649878     PISSN: 1530437X     EISSN: None     Source Type: Journal    
DOI: 10.1109/JSEN.2002.800686     Document Type: Article

References (54)

1. V. N. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995.
2. _, Statistical Learning Theory. New York: Wiley, 1998.
3. M. Pardo, E. Dalcanale, and G. Sberveglieri, "A general framework for learning from data and an application to three electronic nose datasets," in 7th Int. Symp. Olfaction Electron. Nose, Brighton, U.K., July 2000.
4. M. Pardo, G. Sberveglieri, D. Della Casa, G. Valentini, and F. Masulli, "Boosting electronic noses," in Proc. 8th Int. Symp. Olfaction Electron. Nose, Washington, DC, Mar. 2001, to be published.
5. V. Cherkassky and F. Mulier, Learning From Data. New York: Wiley, 1998.
6. A. Webb, Statistical Pattern Recognition. London, U.K.: Arnold, 1999.
7. C. Chatfield, "An initial examination of the data (with discussion)," J. Roy. Statist. Soc. A, vol. 148, pp. 214-253, 1985.
8. J. Tukey, Exploratory Data Analysis. Reading, MA: Addison-Wesley, 1977.
9. K. Fukunaga, Introduction to Statistical Pattern Recognition. New York: Academic, 1990.
10. A. Jain, R. Duin, and J. Mao, "Statistical pattern recognition: A review," IEEE Trans. Pattern Anal. Machine Intell., vol. 22, pp. 4-37, Jan. 2000.
11. C. M. Bishop, Neural Networks for Pattern Recognition. Oxford, U.K.: Oxford Univ. Press, 1995.
12. C. Bishop, "Neural networks and their applications," Rev. Sci. Instrum., vol. 65, no. 6, pp. 1803-1832, 1994.
13. _, "Neural networks: A pattern recognition perspective," in Handbook of Neural Computation, E. Fiesler and R. Beale, Eds. New York: Oxford Univ. Press, 1996. [Online]. Available: http://www.ncrg.aston.ac.uk/.
14. B. D. Ripley, Pattern Recognition and Neural Networks. Cambridge, MA: Cambridge Univ. Press, 1996.
15. S. Haykin, Neural Networks: A Comprehensive Foundation, 2nd ed. Englewood Cliffs, NJ: Prentice-Hall, 1999.
16. W. S. Sarle, "Stopped training and other remedies for overfilling," in Proc. 27th Symp. on Interface Comput. Sci. Statist., 1995, [Online]. Available: ftp://ftp.sas.com/pub/neural/inter95.ps.Z, pp. 352-360.
17. D. J. C. MacKay, "Bayesian interpolation," Neural Comput., vol. 4, pp. 415-447, 1992.
18. F. D. Foresee and M. T. Hagan, "Gauss-Newton approximation to Bayesian regularization," in Proc. 1997 Int. Joint Canf. Neural Networks, 1997, pp. 1930-1935.
19. T. Kohonen, Self-Organizing Maps. New York: Springer-Verlag, 1997.
20. T. G. Dietterich. (1997) Fundamental experimental research in machine learning. [Online]. Available: http://www.cs.orst.edu/~tgd
21. R. O. Duda, P.E. Hart, and D. Stork, Pattern Classification and Scene Analysis, 2nd ed. New York: Wiley, 2000, pt. 1.
22. L. Breiman, G. H. Friedman, R. A. Olshen, and C. J. Stone, Classification and Regression Trees. Belmont, CA: Wadsworth, 1984.
23. J. R. Quinlan, C4.5 Programs for Machine Learning. San Mateo, CA: Morgan Kaufmann, 1993.
24. C. J. Merz and P. M. Murphy. (1998) UCI repository of machine learning databases. [Online], Available: www.ics.uci.edu/mlearn/ML-Repository.html.
25. D. Michie, D. J. Spiegelhalter, and C. C. Taylor, Machine Learning, Neural and Statistical Classification. New York: Ellis Horwood, 1997.
26. R. E. Schapire, Y. Freund, P. Bartlett, and W. Lee, "A new explanation for the effectiveness of voting methods," Ann. Statist., vol. 26, no. 3, pp. 1651-1686, 1998.
27. L. Breiman, "Arcing classifiers," Annals Statist., vol. 26, no. 3, pp. 801-849, 1998.
28. T. G. Dietterich, "Machine learning research: Four current directions," Artificial Intell. Mag., vol. 18, no. 4, pp. 97-136, 1997.
29. M. P. Perrone and L. N. Cooper, "When networks disagree: Ensemble methods for hybrid neural networks," in Artificial Neural Networks for Speech and Vision, R. J. Mammone, Ed. London, U.K.: Chapman & Hall, 1993, pp. 126-142.
30. S. Hashem, "Optimal linear combinations of neural networks," Neural Comput., vol. 10, pp. 599-614, 1997.
31. R.E. Schapire, "A brief introduction to boosting," in 16th Int. Joint Conf. Artificial Intell., 1999, pp. 1401-1406.
32. Y. Freund and R. E. Schapire, "Experiments with a new boosting algorithm," in Proc. 13th Int. Conf. Mach. Learn., 1996, pp. 148-156.
33. L. Breiman, "Bagging predictors," Machine Learning, vol. 24, no. 2, pp. 123-140, 1996.
34. M. I. Jordan and R. A. Jacobs, "Hierarchical mixture of experts and the EM algorithm," Neural Comput., vol. 6, pp. 181-214, 1994.
35. R. A. Jacobs, "Methods for combining experts probability assessment," Neural Comput., vol. 7, pp. 867-888, 1995.
36. K. J. Cherkauker, "Human expert-level performance on a scientific image analysis task by a system using combined artificial neural networks," in Working Notes AAAI Workshop Integrating Multiple Learned Models, P. Chan, Ed., 1996, pp. 15-21.
37. L. Hansen and P. Salamon, "Neural network ensembles," IEEE Trans. Pattern Anal. Machine. Intell., vol. 12, pp. 993-1001, 1990.
38. T. G. Dietterich, "Ensemble methods in machine learning," in Multiple Classifier Syst. First Int. Workshop, MCS2000. Cagliari, Italy: Springer-Verlag, 2000, pp. 1-15.
39. R. E. Schapire, "The strength of weak learnability," Mach. Learn., vol. 5, no. 2, pp. 197-227, 1990.
40. Y. Freund, "Boosting a weak learning algorithm by majority," Inform. Comput., vol. 121, no. 2, pp. 256-285, 1995.
41. Y. Freund and R. E. Schapire, "A decision-theoretic generalization of on-line learning and an application to boosting," J. Comput. Syst. Sci., vol. 55, no. 1, pp. 119-139, 1997.
42. Y. Freund. Slides on boosting. [Online]. Available: http://www.research. att.com/~yoav.
43. J. R. Quinlan, "Bagging, boosting and C4.5," in Proc. Thirteenth Nat. Conf. Artificial Intell., 1996, pp. 725-730.
44. R. E. Schapire, "Using output codes to boost multiclass learning problems," in Proc. Fourteenth Int. Conf. Machine Learning. San Mateo, CA: Morgan Kaufmann, 1997, pp. 313-321.
45. D. Opitz and R. Maclin, "Popular ensemble methods: An empirical study," J. Artif. Intell., vol. 11, pp. 169-198, 1999.
46. T. Dietterich, "An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting and randomization," Mach. Learn., vol. 40, pp. 139-157, 2000.
47. Y. Freund, "An adaptive version of the boost by majority algorithm," in Proc. Twelfth Ann. Conf. Comput. Learning Theory, 1999.
48. R. Anand, G. Mehrotra, C. K. Mohan, and S. Ranka, "Efficient classification for multiclass problems using modular neural networks," IEEE Trans. Neural Networks, vol. 6, pp. 117-124, 1995.
49. T. Dietterich and O. Bakiri, "Solving multiclass learning problems via error-correcting output codes," J. Artif. Intell. Res., no. 2, pp. 263-286, 1995.
50. M. Moreira and E. Mayoraz, "Improved pairwise coupling classifiers with correcting classifiers," in Tenth Eur. Conf. Machine Learning, Chemnitz, Germany, Apr. 1998.
51. M. Pardo, G. Sberveglieri, F. Masulli, and G. Valentini, "Decompositive classification models for electronic noses," Anal. Chim. Acta, vol. 446, pp. 223-232, 2001.
52. F. Masulli and G. Valentini, "Effectiveness of error correcting output codes in multiclass learning problems," in Multiple Classifier Syst. First Int. Workshop, MCS 2000. Cagliari, Italy: Springer-Verlag, 2000, pp. 107-116.
53. _, "Parallel non linear dichotomizers," in IJCNN2000, IEEE-INNS-ENNS Int. Joint Conf. Neural Networks, Como, Italy, 2000.
54. H. Demuth and M. Beale, Manual of the Neural Network Toolbox: The MathWorks, Inc., 1998, version 3.