Challenges and opportunities in applied machine learning

AI Magazine

Volumn 33, Issue 1, 2012, Pages 11-24

  Brodley, Carla E ^a   Rebbapragada, Umaa ^b   Small, Kevin ^c   Wallace, Byron C ^c  

^a Tufts University   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c TUFTS MEDICAL CENTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; CLASSIFICATION (OF INFORMATION);

APPLIED MACHINE LEARNING; CLASSIFICATION MODELS; IN-VITRO; INTERDISCIPLINARY COLLABORATIONS; ITS APPLICATIONS; MACHINE LEARNING PROBLEM; MACHINE LEARNING RESEARCH; REAL-WORLD PROBLEM;

LEARNING SYSTEMS;

EID: 84861416833     PISSN: 07384602     EISSN: None     Source Type: Journal    
DOI: 10.1609/aimag.v33i1.2367     Document Type: Article

References (57)

1. Aisen, A. M.; Broderick, L. S.; Winer-Muram, H.; Brodley, C. E.; Kak, A. C.; Pavlopoulou, C.; Dy, J.; and Marchiori, A. 2003. Automated Storage and Retrieval of Medical Images to Assist Diagnosis: Implementation and Preliminary Assessment. Radiology 228(1): 265-270. (Pubitemid 36736102)
2. Attenberg, J., and Provost, F. 2010. Why Label When You Can Search?: Alternatives to Active Learning for Applying Human Resources to Build Classification Models Under Extreme Class Imbalance. In Proceedings of the 16th ACM SIGKDD International cConference on Knowledge Discovery and Data Mining, 423-432. New York: Association for Computing Machinery.
3. Attenberg, J.; Melville, P.; and Provost, F. 2010. A Unified Approach to Active Dual Supervision for Labeling Features and Examples. In Proceedings of the 2010 European Conference on Machine Learning and Knowledge Discovery in Databases, 40-55. Berlin: Springer.
4. Brodley, C. E., and Friedl, M. A. 1996. Identifying and Eliminating Mislabeled Training Instances. In Proceedings of the Thirteenth National Conference on Artificial Intelligence, 799-805. Palo Alto, CA: AAAI Press.
5. Brodley, C. E., and Friedl, M. A. 1999. Identifying Mislabeled Training Data. Journal of Artificial Intelligence Research 11: 131-167. (Pubitemid 129628762)
6. Brodley, C. E., and Smyth, P. J. 1997. Applying Classification Algorithms in Practice. Statistics and Computing 7(1): 45-56.
7. Brodley, C. E.; Kak, A. C.; Dy, J. G.; Shyu, C. R.; Aisen, A.; and Broderick, L. 1999. Content-Based Retrieval from Medical Image Databases: A Synergy of Human Interaction, Machine Learning, and Computer Vision. In Proceedings of the Sixteenth National Conference on Artificial Intelligence, 760-767. Palo Alto, CA: AAAI Press.
8. Donmez, P., and Carbonell, J. G. 2008. Proactive Learning: Cost-Sensitive Active Learning with Multiple Imperfect Oracles. In Proceedings of the 17th ACM Conference on Information and Knowledge Management, 619-628. New York: Association for Computing Machinery.
9. Draper, B. A.; Brodley, C. E.; and Utgoff, P. E. 1994. Goal-Directed Classification Using Linear Machine Decision Trees. IEEE Transactions on Pattern Analysis and Machine Intelligence 16(9): 888-893.
10. Drummond, C.; Holte, R.; et al. 2005. Severe Class Imbalance: Why Better Algorithms Aren't the Answer. In Proceedings of the 16th European Conference of Machine Learning and Knowledge Discovery in Databases, 539-546. Berlin: Springer.
11. Dy, J., and Brodley, C. E. 2000. Visualization and Interactive Feature Selection for Unsupervised Data. In Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 360-364. New York: Association for Computing Machinery.
12. Dy, J.; Brodley, C.; Kak, A.; Pavlopoulo, C.; Aisen, A.; and Broderick, L. 2003. The Customized-Queries Approach to Image Retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence 25(3): 373-378.
13. Early, J. P.; Brodley, C. E.; and Rosenberg, K. 2003. Behavioral Authentication of Server Flows. In Proceedings of the 19th Annual IEEE Computer Security Applications Confer-ence, 49-55. Piscataway, NJ: Institute of Electrical and Electronics Engineers.
14. Fern, X. Z.; Brodley, C. E.; and Friedl, M. A. 2005. Correlation Clustering for Learning Mixtures of Canonical Correlation Models. In Proceedings of the Fifth SIAM International Conference on Data Mining, 439-448. Philadelphia, PA: Society for Industrial and Applied Mathematics.
15. Friedl, M. A.; Brodley, C. E.; and Stahler, A. 1999. Maximizing Land Cover Classification Accuracies Produced by Decision Trees at Continental to Global Scales. IEEE Transactions on Geoscience and Remote Sensing 37(2): 969-977.
16. Friedl, M. A.; Sulla-Menashe, D.; Tan, B.; Schneider, A.; Ramankutty, N.; Sibley, A.; and Huang, X. 2010. MODIS Collection 5 Global Land Cover: Algorithm Refinements and Characterization of New Datasets. Remote Sensing of Environment 114(1): 168-182.
17. Gamberger, D.; Lavrac, N.; and Dzeroski, S. 1996. Noise Elimination in Inductive Concept Learning: A Case Study in Medical Diagnosis. In Algorithmic Learning Theory: Seventh International Workshop, Lecture Notes in Computer Science 1160, 199-212. Berlin: Springer. (Pubitemid 126143229)
18. Gamberger, D.; Lavrac, C.; Groselj, C. 1999. Experiments with Noise Filtering in a Medical Domain. In Proceedings of the Sixteenth International Conference on Machine Learning, 143-151. San Francisco: Morgan Kaufmann Publishers.
19. He, H., and Garcia, E. 2009. Learning from Imbalanced Data. IEEE Transactions on Knowledge and Data Engineering 21(9): 1263-1284.
20. Jaakkola, T. S. 2000. Tutorial on Variational Approximation Methods. In Advanced Mean Field Methods: Theory and Practice, 129-159. Cambridge, MA: The MIT Press.
21. Kapadia, N.; Fortes, J.; and Brodley, C. 1999. Predictive Application-Performance Modeling in a Computational Grid Environment. In Proceedings of the 8th IEEE International Symposium on High Performance Distributed Computing (HPDC 99), 47-54. Piscataway, NJ: Institute of Electrical and Electronics Engineers. (Pubitemid 30502688)
22. Lallich, S.; Muhlenbach, F.; and Zighed, D. A. 2002. Improving Classification by Removing or Relabeling Mislabeled Instances. In Proceedings of the Thirteenth International Symposium on the Foundations of Intelligent Systems, 5-15. Berlin: Springer-Verlag.
23. Lane, T., and Brodley, C. E. 1999. Temporal Sequence Learning and Data Reduction for Anomaly Detection. ACM Transactions on Computer Security 2(3): 295-331.
24. Lomasky, R.; Brodley, C.; Aerneke, M.; Walt, D.; and Friedl, M. 2007. Active Class Selection. In Proceedings of the Eighteenth European Conference on Machine Learning and Knowledge Discovery in Databases, 640-647. Berlin: Springer.
25. Lu, Z., and Leen, T. K. 2007. Penalized Probabilistic Clustering. Neural Computation 19(6): 1528-1567.
26. MacArthur, S.; Brodley, C. E.; and Broderick, L. 2002. Interactive Content-Based Image Retrieval Using Relevance Feedback. Computer Vision and Image Understanding 88(3): 55-75.
27. McCallum, A., and Nigam, K. 1998. Employing EM and Pool-Based Active Learning for Text Classification. In Proceedings of the Fifteenth International Conference on Machine Learning (ICML), 350-358. San Francisco: Morgan Kaufmann.
28. Moss, E.; Utgoff, P.; Cavazos, J.; Precup, D.; Stefanovic, D.; Brodley, C.; and Scheeff, D. T. 1997. Learning to Schedule Straight-Line Code. In Advances in Neural Information Processing Systems 9, 929-935. Cambridge, MA: The MIT Press.
29. Muchoney, D.; Strahler, A.; Hodges, J.; and LoCastro, J. 1999. The IGBP Discover Confidence Sites and the System for Terrestrial Ecosystem Parameterization: Tools for Validating Global Land Cover Data. Photogrammetric Engineering and Remote Sensing 65(9): 1061-1067. (Pubitemid 29432608)
30. Muhlenbach, F.; Lallich, S.; and Zighed, D. A. 2004. Identifying and Handling Mislabelled Instances. Journal of Intelligent Information Systems 22(1): 89-109.
31. Muslea, I.; Minton, S.; and Knoblock, C. A. 2006. Active Learning with Multiple Views. Journal of Artificial Intelligence Research 27: 203-233. (Pubitemid 44681377)
32. Preston, D.; Brodley, C. E.; Khardon, R.; Sulla-Menashe, D.; and Friedl, M. 2010. Redefining Class Definitions Using Constraint-Based Clustering. In Proceedings of the Sixteenth ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 823-832. New York: Association for Computing Machinery.
33. Provost, F. 2000. Machine Learning from Imbalanced Data Sets 101. In Learning from Imbalanced Data Sets: Papers from the AAAI Workshop. AAAI Technical Report WS-00-05. Palo Alto, CA: AAAI Press.
34. Pusara, M., and Brodley, C. E. 2004. User ReAuthentication Via Mouse Movements. In Proceedings of the 2004 ACM Workshop on Visualization and Data Mining for Computer Security. New York: Association for Computing Machinery.
35. Rebbapragada, U. 2010. Strategic Targeting of Outliers for Expert Review. Ph.D. Dissertation, Tufts University, Boston, MA.
36. Rebbapragada, U., and Brodley, C. E. 2007. Class Noise Mitigation Through Instance Weighting. In Proceedings of the 18th European Conference on Machine Learning (ECML) and the 11th European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD), Lecture Notes in Computer Science Volume 4701, 708-715. Berlin: Springer.
37. Rebbapragada, U.; Lomasky, R.; Brodley, C. E.; and Friedl, M. 2008a. Generating High-Quality Training Data for Automated Land-Cover Mapping. In Proceedings of the 2008 IEEE International Geoscience and Remote Science Symposium, 546-548. Piscataway, NJ: Institute of Electrical and Electronics Engineers.
38. Rebbapragada, U.; Mandrake, L.; Wagstaff, K.; Gleeson, D.; Castao, R.; Chien, S.; and Brodley, C. E. 2009. Improving Onboard Analysis of Hyperion Images by Filtering Mislabeled Training Data Examples. In Proceedings of the 2009 IEEE Aerospace Conference. Piscataway, NJ: Institute of Electrical and Electronics Engineers.
39. Rebbapragada, U.; Protopapas, P.; Brodley, C. E.; and Alcock, C. 2008b. Finding Anomalous Periodic Time Series: An Application to Catalogs of Periodic Variable Stars. Machine Learning 74(3): 281.
40. Sackett, D.; Rosenberg, W.; Gray, J.; Haynes, R.; and Richardson, W. 1996. Evidence Based Medicine: What It Is and What It Isn't. BMJ Journal 312(7023): 71.
41. Settles, B. 2009. Active Learning Literature Survey. Technical Report 1648, Department of Computer Science, University of Wisconsin-Madison, Madison, WI.
42. Sheng, V. S.; Provost, F.; and Ipeirotis, P. G. 2008. Get Another Label? Improving Data Quality and Data Mining Using Multiple, Noisy Labelers. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 614-622. New York: Association for Computing Machinery.
43. Shyu, C.; Brodley, C. E.; Kak, A.; Kosaka, A.; Aisen, A.; and Broderick, L. 1999. Assert: A Physician-in-the-Loop Content-Based Image Retrieval System for HRCT Image Databases. Computer Vision and Image Understanding 75(1-2): 111-132.
44. Stough, T., and Brodley, C. E. 2001. Focusing Attention on Objects of Interest Using Multiple Matched Filters. IEEE Transactions on Image Processing 10(3): 419-426. (Pubitemid 32286646)
45. Tong, S., and Koller, D. 2000. Support Vector Machine Active Learning with Applications to Text Classification. In Journal of Machine Learning Research 2: 999-1006.
46. Valizadegan, H., and Tan, P.-N. 2007. Kernel-Based Detection of Mislabeled Training Examples. In Proceedings of the Seventh SIAM International Conference on Data Mining, 309-319. Philadelphia, PA: Society for Industrial and Applied Mathematics.
47. Vapnik, V. N. 1995. The Nature of Statistical Learning Theory. Berlin: Springer.
48. Venkataraman, S.; Metaxas, D.; Fradkin, D.; Kulikowski, C.; and Muchnik, I. 2004. Distinguishing Mislabeled Data from Correctly Labeled Data in Classifier Design. In Proceedings of the 16th IEEE International Conference on Tools with Artificial Intelligence, 668-672. Piscataway, NJ: Institute of Electrical and Electronics Engineers. (Pubitemid 40455078)
49. Verbaeten, S. 2002. Identifying Mislabeled Training Examples in ILP Classification Problems. In Proceedings of the Twelfth Belgian-Dutch Conference on Machine Learning. Utrecht, The Netherlands: University of Utrecht, Faculty of Mathematics and Computer Science.
50. Verbaeten, S., and Assche, A. V. 2003. Ensemble Methods for Noise Elimination in Classification Problems. In Multiple Classifier Systems, 4th International Workshop, Volume 4. Berlin: Springer.
51. Vickers, A. J., and Elkin, E. B. 2006. Decision Curve Analysis: A Novel Method for Evaluating Prediction Models. Medical Decision Making 26(6): 565-574. (Pubitemid 44728772)
52. Wagstaff, K.; Kocurek, M.; Mazzoni, D.; and Tang, B. 2010. Progressive Refinement for Support Vector Machines. Data Mining and Knowledge Discovery 20(1): 53-69.
53. Wallace, B. C.; Small, K.; Brodley, C. E.; and Trikalinos, T. A. 2010. Active Learning for Biomedical Citation Screening. In Proceedings of the Seventeenth ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 173-182. New York: Association for Computing Machinery.
54. Wallace, B. C.; Small, K.; Brodley, C. E.; and Trikalinos, T. A. 2011. Who Should Label What? Instance Allocation in Multiple Expert Active Learning. In Proceedings of the Eleventh SIAM International Conference on Data Mining (SDM), 176-187. Philadelphia, PA: Society for Industrial and Applied Mathematics.
55. Zeng, X., and Martinez, T. 2001. An Algorithm for Correcting Mislabeled Data. Intelligent Data Analysis 5(1): 491-502.
56. Zeng, X., and Martinez, T. R. 2003. A Noise Filtering Method Using Neural Networks. In Proceedings of the 2003 IEEE International Workshop of Soft Computing Techniques in Instrumentation, Measurement, and Related Applications. Piscataway, NJ: Institute of Electrical and Electronics Engineers.
57. Zhu, X.; Wu, X.; and Chen, S. 2003. Eliminating Class Noise in Large Datasets. In Proceedings of the TwentiethInternational Conference on Machine Learning, 920-927. Palo Alto, CA: AAAI Press.