Using permutations instead of student's distribution for p-values in paired-difference algorithm comparisons

IEEE International Conference on Neural Networks - Conference Proceedings

Volumn 2, Issue , 2004, Pages 1331-1335

  Menke, Joshua ^a   Martinez, Tony R ^a  

^a 3361 TMCB   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BINARY NUMBERS; DATA SETS; PERMUTATION TESTS; STANDARD ERRORS;

APPROXIMATION THEORY; CONFORMAL MAPPING; DECISION MAKING; ERROR ANALYSIS; ITERATIVE METHODS; LEARNING ALGORITHMS; MATHEMATICAL MODELS; NEURAL NETWORKS; OPTIMIZATION; PROBABILITY; RANDOM PROCESSES; STATISTICAL METHODS;

LEARNING SYSTEMS;

EID: 10944259095     PISSN: 10987576     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2004.1380138     Document Type: Conference Paper

References (11)

1. T. G. Dietterich, "Approximate statistical test for comparing supervised classification learning algorithms," Neural Computation, vol. 10, no. 7, pp. 1895-1923, 1998. [Online]. Available: citeseer.nj.nec.com/ dietterich98approximate.html
2. Y. Reich and S. Barai. "Evaluating machine learning models for engineering problems," 1999. [Online]. Available: citeseer.nj.nec.com/ reich99evaluating.html
3. S. Salzberg, "On comparing classifi ers: Pitfalls to avoid and a recommended approach," Data Mining and Knowledge Discovery, vol. 1, no. 3, pp. 317-328, 1997. [Online]. Available: citeseer.nj.nec.com/salzberg97comparing. html
4. R. Kohavi, "A study of cross-validation and bootstrap for accuracy estimation and model selection," 1995. [Online]. Available: gunther.smeal.psu.edu/kohavi95study.html
5. R. Micheals and T. Boult, "Efficient evaluation of classifi cation and recognition systems," 2001. [Online]. Available: citeseer.nj.nec.com/ article/micheals01efficient.html
6. J. H. Piater, P. R. Cohen, X. Zhang, and M. Atighetchi, "A randomized ANOVA procedure for comparing performance curves," in Proc. 15th International Conf. on Machine Learning. Morgan Kaufmann, San Francisco, CA, 1998, pp. 430-438. [Online]. Available: citeseer.nj.nec.com/67061.html
7. F. Provost, T. Fawcett, and R. Kohavi, "The case against accuracy estimation for comparing induction algorithms," in Proc. 15th International Conf. on Machine Learning. Morgan Kaufmann, San Francisco, CA, 1998, pp. 445-453. [Online]. Available: citeseer.nj.nec.com/article/provost98case.html
8. A. Bradley, "The use of the area under the roc curve in the evaluation of machine learning algorithms," PR, vol. 30, no. 7, pp. 1145-1159, July 1997.
9. K. Woods and K. Bowyer, "Generating roc curves for artifi cial neural networks," IEEE Transactions on Medical Imaging.
10. M. Maloof, "On machine learning, ROC analysis, and statistical tests of signifi cance," in Proceedings of the Sixteenth International Conference on Pattern Recognition. Los Alamitos, CA: IEEE Press, 2002, pp. 204-207.
11. K. B. D. Dorfman and C. Metz, "Receiver operator characteristics rating analysis: Generalization to the populations of readers and patients with the jackknife method." Investigative Radiology.