Identifying noisy features with the Pairwise Attribute Noise Detection Algorithm

Intelligent Data Analysis

Volumn 9, Issue 6, 2005, Pages 589-602

  Khoshgoftaar, Taghi M ^a   Van Hulse, Jason ^a  

^a Florida Atlantic University   (United States)

Author keywords

data cleaning; Noise detection; software measurement; software quality

Indexed keywords

COMPUTER SOFTWARE SELECTION AND EVALUATION; DATA MINING; SIGNAL DETECTION;

DATA CLEANING; DATA MINING AND KNOWLEDGE DISCOVERY; EMPIRICAL STUDIES; LIMITED ATTENTIONS; MULTIPLE-CASE STUDY; NOISE DETECTION; SOFTWARE MEASUREMENT; SOFTWARE QUALITY;

FEATURE EXTRACTION;

EID: 33746913215     PISSN: 1088467X     EISSN: 15714128     Source Type: Journal    
DOI: 10.3233/ida-2005-9606     Document Type: Article

References (29)

1. C.E. Brodley and M.A. Friedl, Identifying mislabeled training data, Journal of Artificial Intelligence Research 11 1999), 131-167.
2. P. Clark and T. Niblett, Rule induction with CN2: Some recent improvements, In Proceedings of the 5th EuropeanWorking Session on Learning, 1991, 151-163.
3. W.J. Conover, Practical Nonparametric Statistics, John Wiley and Sons, New York, NY, USA, 1971.
4. R. Cooley, J. Srivastava and B. Mobasher, Web mining: Information and pattern discovery on the world wide web, In Proceedings of the 9th IEEE International Conference on Tools with Artificial Intelligence (ICTAI'97), November 1997.
5. N.E. Fenton and S.L. Pfleeger, Software Metrics: A Rigorous and Practical Approach, (2nd ed.), PWS Publishing Company: ITP, Boston, MA, 1997.
6. H. Galhardas, D. Florescu, D. Shasha and E. Simon, An extensible framework for data cleaning, In Proceedings of 18th International Conference on Data Engineering, San Jose, CA, February 2000. IEEE Computer Society.
7. D. Gamberger, N. Lavrac and S. Dzeroski, Noise elimination in inductive concept learning: A case study in medical diagnosis, In Proceedings of the 7th International Workshop on Algorithmic Learning Theory, Springer, 1999, 199-212.
8. D. Gamberger, N. Lavrac and C. Groselj, Experiments with noise filtering in a medical domain, In Proceedings of the 16th International Conference on Machine Learning, Morgan Kaufmann, 1999, 143-153.
9. M.A. Hernandez and S.J. Stolfo, Real-world data is dirty: Data cleansing and the merge/purge problem, Data Mining and Knowledge Discovery 2(1) (1998), 9-37.
10. E.S. Keeping, Introduction to Statistical Inference, Dover Publications, 1995.
11. T.M. Khoshgoftaar and E.B. Allen, Logistic regression modeling of software quality, International Journal of Reliability, Quality, and Safety Engineering 6(4) (December 1999), 303-317.
12. T.M. Khoshgoftaar, L.A. Bullar and K. Gao, Detecting outliers using rule-based modeling for improving CBR-based software quality classification models, in: Proceedings of the 16th International Conference on Case-Based Reasoning, (Vol. 1689), K.D. Ashley and D.G. Bridge, eds, Springer-Verlag LNAI, 2003, pp. 216-230.
13. T.M. Khoshgoftaar and P. Rebours, Generarting multiple noise elimination filters with the ensemble-partitioning filter, In Proceedings of the IEEE International Conference on Information Reuse and Integration, pages 369-375, Las Vegas, NV, USA, November 2004. IEEE Systems, Man and Cybernetics Society.
14. T.M. Khoshgoftaar and N. Seliya, Analogy-based practical classification rules for software quality estimation, Empirical Software Engineering Journal 8(4) (December 2003), 325-350.
15. T.M. Khoshgoftaar and N. Seliya, The necessity of assuring quality in software measurement data, In Proceedings of 10th International Software Metrics Symposium, pages 119-130, Chicago, IL, September 2004. IEEE Computer Society.
16. T.M. Khoshgoftaar, N. Seliya and K. Gao, Detecting noisy instances with the rule-based classification model, Intelligent Data Analysis: An International Journal 9(4) (2005) 347-364.
17. T.M. Khoshgoftaar, S. Zhong and V. Joshi, Noise elimination with ensemble-classifier filtering for software quality estimation, Intelligent Data Analysis: An International Journal 9(1) (2005) 3-27.
18. M. Lee, H. Lu, T. Ling and Y. Ko, Cleansing data for mining and warehousing, In Proceedings of 10th International Conference on Database and Expert Systems Applications (DEXA), August 1999, 751-760.
19. W. Lee, S.J. Stolfo and K.W. Mok, A data mining framework for building intrusion detection models, Proceedings of IEEE Symposium on Security and Privacy, May 1999, 120-132.
20. E. Rahm and H. Do, Data cleaning: Problems and current approaches, Bulletin of the Technical Committee on Data Engineering 23(4) (2000), 3-13.
21. V. Raman and J.M. Hellerstein, Potter's wheel: An interactive data cleaning system, In The Very Large Databases (VLDB) Journal, 2001, 381-390.
22. S. Ramasway, R. Rastogi and K. Shim, Efficient algorithms for mining outliers from large data sets, In Proceedings of ACM SIGMOD Conference on Management of Data, ACM, 2000, 427-438.
23. SAS Institute, SAS/STAT User's Guide, SAS Institute Inc., 1992.
24. D. Strong, Y. Lee and R. Wang, Data quality in context, Communications of the ACM 40(5) (1997), 102-110.
25. C.M. Teng, Correcting noisy data, In Proceedings of 6th International Conference Machine Learning (ICML 99), Morgan Kaufmann, 1999, 239-248.
26. J. Van Hulse, T.M. Khoshgoftaar and H. Huang, The pairwise attribute noise detection algorithm, Technical report, Florida Atlantic University, Boca Raton, FL, November 2004.
27. Y. Yang, X. Wu and X. Zhu, Dealing with predictive-but-unpredictable attributes in noisy data sources, In Proceedings of 8th European Conference on Principles and Practice of Knowledge Discovery in Databases, Pisa, Italy, 2004.
28. S. Zhong, T.M. Khoshgoftaar and N. Seliya, Analyzing software measurement data with clustering techniques, IEEE Intelligent Systems, March 2004, 22-29.
29. X. Zhu and X. Wu, Class noise vs attribute noise: A quantitative study of their impacts, Artificial Intelligence Review (2004), in press.