A clarification of the terms used in comparing semi-automated particle selection algorithms in Cryo-EM

Journal of Structural Biology

Volumn 175, Issue 3, 2011, Pages 348-352

  Langlois, Robert ^a   Frank, Joachim ^a  

^a Och Spine at New York Presbyterian Hospitals   (United States)

Author keywords

Cryo EM; False positive rate; Machine learning; Particle selection

Indexed keywords

ALGORITHM; ARTICLE; AUTOMATION; CONFORMATION; CRYOELECTRON MICROSCOPY; INFORMATION RETRIEVAL; MACHINE LEARNING; METRIC SYSTEM; NOMENCLATURE; PRIORITY JOURNAL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; CRYOELECTRON MICROSCOPY; PATTERN RECOGNITION, AUTOMATED;

EID: 79960928380     PISSN: 10478477     EISSN: 10958657     Source Type: Journal    
DOI: 10.1016/j.jsb.2011.03.009     Document Type: Article

References (41)

1. Adiga U., Baxter W.T., Hall R.J., Rockel B., Rath B.K., et al. Particle picking by segmentation: A comparative study with SPIDER-based manual particle picking. Journal of Structural Biology 2005, 152:211-220.
2. Benjamini Y., Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological) 1995, 57:289-300.
3. Brodersen, K.H., C.S. Ong, K.E. Stephan, J.M. Buhmann, The Binormal Assumption on Precision-Recall Curves, Pattern Recognition (ICPR), 2010 20th International Conference on, pp. 4263-4266.
4. Caruana R., Niculescu-Mizil A. An empirical comparison of supervised learning algorithms, Proceedings of the 23rd international conference on Machine learning 2006, ACM, Pittsburgh, Pennsylvania.
5. Chen J.Z., Grigorieff N. SIGNATURE: a single-particle selection system for molecular electron microscopy. Journal of Structural Biology 2007, 157:168-173.
6. Davis J., Goadrich M. The relationship between Precision-Recall and ROC curves, Proceedings of the 23rd international conference on Machine learning 2006, ACM, Pittsburgh, Pennsylvania.
7. Drummond C., Holte R. Cost curves: an improved method for visualizing classifier performance. Machine Learning 2006, 65:95-130.
8. Fawcett, T., 2004. ROC graphs: Notes and practical considerations for researchers. Machine Learning 31, HPL-2003-4, 1-38.
9. Glaeser R.M. Historical background: why is it important to improve automated particle selection methods?. Journal of Structural Biology 2004, 145:15-18.
10. Hall R.J., Patwardhan A. A two step approach for semi-automated particle selection from low contrast cryo-electron micrographs. Journal of Structural Biology 2004, 145:19-28.
11. Hand D.J., Till R.J. A simple generalisation of the area under the ROC curve for multiple class classification problems. Machine Learning 2001, 45:171-186.
12. Harauz G., Fong-Lochovsky A. Automatic selection of macromolecules from electron micrographs by component labelling and symbolic processing. Ultramicroscopy 1989, 31:333-344.
13. Huang Z., Penczek P.A. Application of template matching technique to particle detection in electron micrographs. Journal of Structural Biology 2004, 145:29-40.
14. Kohavi, F.P.a.T.F.a.R., 1997. The case against accuracy estimation for comparing induction algorithms, In: Proceedings of the Fifteenth International Conference on Machine Learning, Morgan Kaufmann, Vol. 15, pp. 445-453.
15. Kumar V., Heikkonen J., Engelhardt P., Kaski K. Robust filtering and particle picking in micrograph images towards 3D reconstruction of purified proteins with cryo-electron microscopy. Journal of Structural Biology 2004, 145:41-51.
16. Mallick S.P., Zhu Y., Kriegman D. Detecting particles in cryo-EM micrographs using learned features. Journal of Structural Biology 2004, 145:52-62.
17. Ogura T., Sato C. Automatic particle pickup method using a neural network has high accuracy by applying an initial weight derived from eigenimages: a new reference free method for single-particle analysis. Journal of Structural Biology 2004, 145:63-75.
18. Ogura T., Sato C. Auto-accumulation method using simulated annealing enables fully automatic particle pickup completely free from a matching template or learning data. Journal of Structural Biology 2004, 146:344-358.
19. Pascual A., Bárcena M., Merelo J.J., Carazo J.-M. Mapping and fuzzy classification of macromolecular images using self-organizing neural networks. Ultramicroscopy 2000, 84:85-99.
20. Pascual-Montano A., Donate L.E., Valle M., Bárcena M., Pascual-Marqui R.D., et al. A novel neural network technique for analysis and classification of EM single-particle images. Journal of Structural Biology 2001, 133:233-245.
21. Rath B.K., Frank J. Fast automatic particle picking from cryo-electron micrographs using a locally normalized cross-correlation function: a case study. Journal of Structural Biology 2004, 145:84-90.
22. Roseman A.M. Particle finding in electron micrographs using a fast local correlation algorithm. Ultramicroscopy 2003, 94:225-236.
23. Roseman A.M. FindEM-a fast, efficient program for automatic selection of particles from electron micrographs. Journal of Structural Biology 2004, 145:91-99.
24. Sattar, A., B.-H. Kang, M. Sokolova, N. Japkowicz, S. Szpakowicz, 2006. Beyond Accuracy, F-Score and ROC: A Family of Discriminant Measures for Performance Evaluation, AI 2006: advances in Artificial Intelligence, Springer Berlin/Heidelberg, Vol. 4304, pp. 1015-1021.
25. Schapire R.E., Singer Y. Improved boosting algorithms using confidence-rated predictions. Mach. Learn. 1999, 37:297-336.
26. Shaikh T.R., Trujillo R., LeBarron J.S., Baxter W.T., Frank J. Particle-verification for single-particle, reference-based reconstruction using multivariate data analysis and classification. Journal of Structural Biology 2008, 164:41-48.
27. Short J.M. SLEUTH - a fast computer program for automatically detecting particles in electron microscope images. Journal of Structural Biology 2004, 145:100-110.
28. Sigworth F.J. Classical detection theory and the cryo-EM particle selection problem. Journal of Structural Biology 2004, 145:111-122.
29. Singh V., Marinescu D.C., Baker T.S. Image segmentation for automatic particle identification in electron micrographs based on hidden Markov random field models and expectation maximization. Journal of Structural Biology 2004, 145:123-141.
30. Sorzano C.O.S., Recarte E., Alcorlo M., Bilbao-Castro J.R., San-Martín C., et al. Automatic particle selection from electron micrographs using machine learning techniques. Journal of Structural Biology 2009, 167:252-260.
31. Umesh Adiga P.S., Malladi R., Baxter W., Glaeser R.M. A binary segmentation approach for boxing ribosome particles in cryo EM micrographs. Journal of Structural Biology 2004, 145:142-151.
32. Volkmann N. An approach to automated particle picking from electron micrographs based on reduced representation templates. Journal of Structural Biology 2004, 145:152-156.
33. Voss N.R., Yoshioka C.K., Radermacher M., Potter C.S., Carragher B. DoG Picker and TiltPicker: Software tools to facilitate particle selection in single particle electron microscopy. Journal of Structural Biology 2009, 166:205-213.
34. Weng C.G., Poon J. A new evaluation measure for imbalanced datasets. Seventh Australasian Data Mining Conference (AusDM 2008) 2008, Vol. 87:27-32. ACS, Glenelg, South Australia.
35. Witten I.H., Frank E. Data mining: practical machine learning tools and techniques 2005, Morgan Kaufmann.
36. Wong H.C., Chen J., Mouche F., Rouiller I., Bern M. Model-based particle picking for cryo-electron microscopy. Journal of Structural Biology 2004, 145:157-167.
37. Woolford D., Hankamer B., Ericksson G. The Laplacian of Gaussian and arbitrary z-crossings approach applied to automated single particle reconstruction. Journal of Structural Biology 2007, 159:122-134.
38. Woolford D., Ericksson G., Rothnagel R., Muller D., Landsberg M.J., et al. SwarmPS: rapid, semi-automated single particle selection software. Journal of Structural Biology 2007, 157:174-188.
39. Yu Z., Bajaj C. Detecting circular and rectangular particles based on geometric feature detection in electron micrographs. Journal of Structural Biology 2004, 145:168-180.
40. Zhu Y., Carragher B., Mouche F., Potter C.S. Automatic particle detection through efficient Hough transforms. Medical Imaging, IEEE Transactions 2003, 22:1053-1062.
41. Zhu Y., Carragher B., Glaeser R.M., Fellmann D., Bajaj C., et al. Automatic particle selection: results of a comparative study. Journal of Structural Biology 2004, 145:3-14.