The pairwise attribute noise detection algorithm

Knowledge and Information Systems

Volumn 11, Issue 2, 2007, Pages 171-190

  Hulse, Jason D ^a   Khoshgoftaar, Taghi M ^a   Huang, Haiying ^a  

^a Florida Atlantic Univ   (United States)

Author keywords

Data cleaning; Data quality; Noise detection; PANDA

Indexed keywords

ENGINEERING RESEARCH; SIGNAL DETECTION; SOFTWARE ENGINEERING;

DATA CLEANING; DATA MINING MODELS; DATA QUALITY; DISTANCE BASED OUTLIER DETECTION; NEAREST NEIGHBORS; NOISE DETECTION; PANDA; SOFTWARE MEASUREMENT DATA;

DATA MINING;

EID: 33947404760     PISSN: 02191377     EISSN: 02193116     Source Type: Journal    
DOI: 10.1007/s10115-006-0022-x     Document Type: Article

References (30)

1. Aggarwal C, Yu P (2001) Outlier detection for high dimensional data. In: Proceedings of ACM SIGMOD conference on management of data, ACM Press, Dallas, TX
2. Bobrowski M, Marre M, Yankelevich D. A software engineering view of data quality. Available at www.citeseer.ist.psu.edu/277636.html
3. Brodley CE, Friedl MA (1999) Identifying mislabeled training data. J Artif Intell Res 11: 131-167
4. Clark P, Niblett T (1991) Rule induction with CN2: Some recent improvements. In: Proceedings of the 5th European working session on learning, pp 151-163
5. Dunagan JD (2002). A geometic theory of outliers and perturbation. Ph.D. Dissertation. Available at http://research.microsoft.com/~jdunagan/ thesis.pdf
6. Fenton NE, Pfleeger SL (1997) Software metrics: A rigorous and practical approach, 2nd edn. PWS Publishing Company: ITP, Boston, MA
7. Galhardas H, Florescu D, Shasha D, Simon E (2000) An extensible framework for data cleaning. In: Proceedings of 18th international conference on data engineering, IEEE Computer Society, San Jose, CA
8. Gamberger D, Lavrac N, Dzeroski S (1999) Noise elimination in inductive concept learning: A case study in medical diagnosis. In: Proceedings of the 7th international workshop on algorithmic learning theory, Springer, Berlin Heidelberg Ney York, pp 199-212
9. Gamberger D, Lavrac N, Groselj C (1999) Experiments with noise filtering in a medical domain. In: Proceedings of the 16th international conference on machine learning. Morgan Kaufmann, San Mateo, California, pp 143-153
10. Hernandez MA, Stolfo SJ (1995) The merge/purge problem for large databases. In: Proceedings of ACM SIGMOD conference on management of data, ACM, pp 127-138. citeseer.ist.psu.edu/stolfo95mergepurge.html
11. Hernandez MA, Stolfo, SJ (1998) Real-world data is dirty: Data cleansing and the merge/purge problem. Data Min Knowl Discov 2 (1): 9-37
12. Khoshgoftaar TM, Allen EB (1998) Classifcation of fault-prone software modules: Prior probabilities, costs and model evaluation. Empiric Software Eng 3: 275-298
13. Khoshgoftaar TM, Bullard LA, Gao K (2003) Detecting outliers using rule-based modeling for improving CBR-based software quality classification models. In: Ashley KD, Bridge DG (eds) Proceedings of the 16th international conference on case-based reasoning. LNAI, vol 1689. Springer-Verlag, Berlin Heidelberg New York, pp 216-230
14. Khoshgoftaar TM, Rebours P (2004) Generarting multiple noise elimination filters with the ensemble-partitioning filter. In: Proceedings of the IEEE international conference on information reuse and integration, IEEE Systems, Man and Cybernetics Society, Las Vegas, NV, USA, pp 369-375
15. Khoshgoftaar TM, Seliya N (2004) The necessity of assuring quality in software measurement data. In: Proceedings of 10th international software metrics symposium, IEEE Computer Society, Chicago, IL, pp 119-130
16. Khoshgoftaar TM, Seliya N, Gao K (2005) Detecting noisy instances with the rule-based classification model. Intell Data Anal 9 (4):347-364
17. Khoshgoftaar TM, Zhong S, Joshi V (2005). Noise elimination with ensemble-classifier filtering for software quality estimation. Intell Data Anal 9 (1): 3-27
18. Knorr E, Ng R (1997) A unified notion of outliers: Properties and computation. In Proceedings of knowledge discovery and data mining. American Association for Artificial Intelligence, Newport Beach, CA, pp 219-222
19. Knorr E, Ng R (1998) Algorithms for mining distance-based outliers in large datasets. In: Proceedings of 24th international conference on very large databases, New York, NY, pp 392-403
20. Marcus A, Maletic J, Lin K-I (2001) Ordinal association rules for error identification in datasets. In: Proceedings of 10th international conference on information and knowledge management. ACM Press, Atlanta, GA, pp 589-591
21. Murphy, PM, Aha DW (1998) UCI repository of machine learning databases. University of California, Irvine, Department of Information and Computer Science. http://www.ics.uci.edu/~mlearn/MLRepository.html
22. Quinlan JR (1993) C4.5: Programs for machine learning. Morgan Kaufmann, San Mateo, California
23. Ramasway S, Rastogi R, Shim K (2000) Efficient algorithms for mining outliers from large datasets. In: Proceedings of ACM SIGMOD conference on management of data, ACM, pp 427-438
24. SAS Institute (2004) SAS/STAT user's guide. SAS Institute Inc
25. Shekhar S, Lu C, Zhang P (2002) Detecting graph-based spatial outliers. Intell Data Anal 6: 451-458
26. Strong D, Lee Y, Wang R (1997) Data quality in context. Commun ACM 40 (5): 103-110
27. Teng CM (1999) Correcting noisy data. In: Proceedings of 6th international conference machine learning (ICML 99). Morgan Kaufmann, San Mateo, California, pp 239-248
28. Yang Y, Wu X, Zhu X (2004) Dealing with predictive-but-unpredictable attributes in noisy data sources. In: Proceedings of 8th European conference on principles and practice of knowledge discovery in databases, Pisa, Italy
29. Zhong S, Khoshgoftaar TM, Seliya N (2004) Analyzing software measurement data with clustering techniques. IEEE Intell Syst, pp 22-29
30. Zhu X, Wu X (2004) Class noise vs attribute noise: A quantitative study of their impacts. Artif Intell Rev 22 (3-4): 177-210