Quotation Accuracy Matters: An Examination of How an Influential Meta-Analysis on Active Learning Has Been Cited

Review of Educational Research

Volumn 91, Issue 2, 2021, Pages 272-308

  Martella, Amedee Marchand ^a   Yatcilla, Jane Kinkus ^a   Martella, Ronald C ^a   Marchand Martella, Nancy E ^a   Ozen, Zafer ^a   Karatas, Tugce ^a   Park, Helen H ^a   Simpson, Alexandra ^a   Karpicke, Jeffrey D ^a  

^a PURDUE UNIVERSITY   (United States)

Author keywords

active learning; citation accuracy; citation context analysis; quotation accuracy

Indexed keywords

EID: 85100520537     PISSN: 00346543     EISSN: 19351046     Source Type: Journal    
DOI: 10.3102/0034654321991228     Document Type: Article

References (84)

1. Allison D. B., Brown A. W., George B. J., Kaiser K. A., (2016). Reproducibility: A tragedy of errors. Nature, 530(7588), 27–29. https://doi.org/10.1038/530027a
2. American Psychological Association. (2020). Publication manual of the American Psychological Association (7th ed.). https://doi.org/10.1037/0000165-000
3. Association of American Universities. (2017). Progress toward achieving systemic change: A five-year status report on the AAU undergraduate STEM education initiative. https://www.aau.edu/sites/default/files/AAU-Files/STEM-Education-Initiative/STEM-Status-Report.pdf
4. Awrey J., Inaba K., Barmparas G., Recinos G., Teixeira P. G. R., Chan L. S., Talving P., Demetriades D., (2011). Reference accuracy in the general surgery literature. World Journal of Surgery, 35(3), 475–479. https://doi.org/10.1007/s00268-010-0912-7
5. Bajak A., (2014, May 12). Lectures aren’t just boring, they’re ineffective, too, study finds. Science. https://www.sciencemag.org/news/2014/05/lectures-arent-just-boring-theyre-ineffective-too-study-finds
6. Bareket R., Schonberg M. A., Schonmann Y., (2020). Make quotations great again: A proposal to reduce false knowledge. BMJ Evidence-Based Medicine, 25(1), 12–14. https://doi.org/10.1136/bmjebm-2018-111109
7. *Bayer A., Rouse C. E., (2016). Diversity in the economics profession: A new attack on an old problem. Journal of Economic Perspectives, 30(4), 221–242. https://doi.org/10.1257/jep.30.4.221
8. *Becker B. K., Schiller A. M., Zucker I. H., Eager E. A., Bronner L. P., Godfrey M., (2017). A day of immersive physiology experiments increases knowledge and excitement towards physiology and scientific careers in Native American students. Advances in Physiology Education, 41(1), 137–144. https://doi.org/10.1152/advan.00165.2016
9. Bernstein D. A., (2018). Does active learning work? A good question, but not the right one. Scholarship of Teaching and Learning in Psychology, 4(4), 290–307. https://doi.org/10.1037/stl0000124
10. Bonwell C. C., Eison J. A., (1991). Active learning: Creating excitement in the classroom [Monograph]. http://files.eric.ed.gov/fulltext/ED336049.pdf
11. Bramer W. M., Giustini D., de Jonge G. B., Holland L., Bekhuis T., (2016). De-duplication of database search results for systematic reviews in EndNote. Journal of the Medical Library Association, 104(3), 240–243. https://doi.org/10.3163/1536-5050.104.3.014
12. Brown A. W., Kaiser K. A., Allison D. B., (2018). Issues with data and analyses: Errors, underlying themes, and potential solutions. Proceedings of the National Academy of Sciences, 115(11), 2563–2570. www.pnas.org/cgi/doi/10.1073/pnas.1708279115
13. Bryson D., (2012). Using research papers: Citations, referencing and plagiarism. Journal of Visual Communication in Medicine, 35(2), 82–84. https://doi.org/10.3109/17453054.2012.690128
14. *Bubou G. M., Offor I. T., Bappa A. S., (2017). Why research-informed teaching in engineering education? A review of the evidence. European Journal of Engineering Education, 42(3), 323–335. https://doi.org/10.1080/03043797.2016.1158793
15. Chasteen S., (2016). TRESTLE mini seed grant proposals: Transforming education, supporting teaching and learning excellence. Center for STEM Learning, University of Colorado Boulder. https://www.colorado.edu/csl/sites/default/files/attached-files/trestle_rfp-_minigrant_0.pdf
16. *Clyne A. M., Billiar K. L., (2016). Problem-based learning in biomechanics: Advantages, challenges, and implementation strategies. Journal of Biomechanical Engineering, 138(7), Article 070804. https://doi.org/10.1115/1.4033671
17. Cooper M. M., Stowe R. L., (2018). Chemistry education research: From personal empiricism to evidence, theory, and informed practice. Chemical Reviews, 118(12), 6053–6087. https://doi.org/10.1021/acs.chemrev.8b00020
18. Davids J. R., Weigl D. M., Edmonds J. P., Blackhurst D. W., (2010). Reference accuracy in peer-reviewed pediatric orthopaedic literature. Journal of Bone & Joint Surgery, 92(5), 1155–1161. https://doi.org/10.2106/JBJS.I.00063
19. *Davis D. A., McMahon G. T., (2018). Translating evidence into practice: Lessons for CPD. Medical Teacher, 40(9), 892–895. https://doi.org/10.1080/0142159X.2018.1481285
20. de Lacey G., Record C., Wade J. (1985). How accurate are quotations and references in medical journals? British Medical Journal (Clinical Research Ed.), 291(6499), 884–886. https://doi.org/10.1136/bmj.291.6499.884
21. Deslauriers L., McCarty L. S., Miller K., Callaghan K., Kestin G., (2019). Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proceedings of the National Academy of Sciences of the United States of America, 116(39), 19251–19257. www.pnas.org/cgi/doi/10.1073/pnas.1821936116
22. Drake D. C., Maritz B., Jacobs S. M., Crous C. J., Engelbrecht A., Etale A., Fourie M. J., Furniss D. G., Scott S. L., Parusnath S., Tye D. R., (2013). The propagation and dispersal of misinformation in ecology: Is there a relationship between citation accuracy and journal impact factor? Hydrobiologia, 702(1), 1–4. https://doi.org/10.1177/0309133313482135
23. Fenton J. E., Brazier H., de Souza A., Hughes J. P., McShane D. P., (2000). The accuracy of citation and quotation in otolaryngology/head and neck surgery journals. Clinical Otolaryngology, 25(1), 40–44. https://doi.org/10.1046/j.1365-2273.2000.00322.x
24. Freeman S., Eddy S. L., McDonough M., Smith M. K., Okoroafor N., Jordt H., Wenderoth M. P., (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences of the United States of America, 111(23), 8410–8415. https://doi.org/10.1073/pnas.1319030111
25. French S., Kennedy G., (2017). Reassessing the value of university lectures. Teaching in Higher Education, 22(6), 639–654. https://doi.org/10.1080/13562517.2016.1273213
26. Gasparyan A. Y., Yessirkepov M., Voronov A. A., Gerasimov A. N., Kostyukova E. I., Kitas G. D., (2015). Preserving the integrity of citations and references by all stakeholders of science communication. Journal of Korean Medical Science, 30(11), 1545–1552. https://doi.org/10.3346/jkms.2015.30.11.1545
27. Goodman I. F., Cunningham C. M., Lachapelle C., Thompson M., Bittinger K., Brennan R. T., Delci M., (2002). Final report of the Women’s Experiences in College Engineering (WECE) project. National Science Foundation and Alfred P. Sloan Foundation. http://www.grginc.com/documents/WECE_FINAL_REPORT.pdf
28. Gosling C. M., Cameron M., Gibbons P. F., (2004). Referencing and quotation accuracy in four manual therapy journals. Manual Therapy, 9(1), 36–40. https://doi.org/10.1016/S1356-689X(03)00056-0
29. Gough D., Oliver S., Thomas J., (2013). Learning from research: Systematic reviews for informing policy decisions: A quick guide. Alliance for Useful Evidence. https://www.alliance4usefulevidence.org/assets/Alliance-FUE-reviews-booklet-3.pdf
30. *Grissom S., McCauley R., Murphy L., (2017). How student centered is the computer science classroom? A survey of college faculty. ACM Transactions on Computing Education, 18(1), 1–27. https://doi.org/10.1145/3143200
31. Haak D. C., HilleRisLambers J., Pitre E., Freeman S., (2011). Increased structure and active learning reduce the achievement gap in introductory biology. Science, 332(6034), 1213–1216. https://doi.org/10.1126/science.1204820
32. Harzing A. W., (1995). The persistent myth of high expatriate failure rates. International Journal of Human Resource Management, 6(2), 457–474. https://doi.org/10.1080/09585199500000028
33. Harzing A. W., (2002). Are our referencing errors undermining our scholarship and credibility? The case of expatriate failure rates. Journal of Organizational Behavior, 23(1), 127–148. https://doi.org/10.1002/job.125
34. Haussmann N. S., McIntyre T., Bumby A. J., Loubser M. J., (2013). Referencing practices in physical geography: How well do we cite what we write? Progress in Physical Geography, 37(4), 543–549. https://doi.org/10.1177/0309133313482135
35. Hew K. F., Lo C. K., (2018). Flipped classroom improves student learning in health professions education: A meta-analysis. BMC Medical Education, 18(1), 1–12. https://doi.org/10.1186/s12909-018-1144-z
36. Hinchcliff K. W., Bruce N. J., Powers J. D., Kipp M. L., (1993). Accuracy of references and quotations in veterinary journals. Journal of the American Veterinary Medical Association, 202(3), 397–400.
37. Jergas H., Baethge C., (2015). Quotation accuracy in medical journal articles: A systematic review and meta-analysis. PeerJ: Life & Environment, 3, 1–20. https://doi.org/10.7717/peerj.1364
38. Klahr D., Nigam M., (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. Psychological Science, 15(10), 661–667. https://doi.org/10.1111/j.0956-7976.2004.00737.x
39. Korpela K. M., (2010). How long does it take for the scientific literature to purge itself of fraudulent material? The Breuning case revisited. Current Medical Research and Opinion, 26(4), 843–847. https://doi.org/10.1185/03007991003603804
40. LaBoskey V. K., (1998). The danger of false dichotomies. Teacher Education Quarterly, 25(4), 39–43. https://www.jstor.org/stable/23478155
41. Landis J. R., Koch G. G., (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174. https://doi.org/10.2307/2529310
42. Leatham K. R., (2015). Observations on citation practices in mathematics education research. Journal for Research in Mathematics Education, 46(3), 253–269. https://doi.org/10.5951/jresematheduc.46.3.0253
43. Lederman D., (2014, May 13). A boost for active learning. Inside Higher Education. https://www.insidehighered.com/news/2014/05/13/stem-students-fare-better-when-professors-dont-just-lecture-study-finds
44. Lopresti R., (2010). Citation accuracy in environmental science journals. Scientometrics, 85(3), 647–655. https://doi.org/10.1007/s11192-010-0293-6
45. Lorenzo M., Crouch C. H., Mazur E., (2006). Reducing the gender gap in the physics classroom. American Journal of Physics, 74(2), 118–122. https://doi.org/10.1119/1.2162549
46. Lukić I. K., Lukić A., Glunčić V., Katavić V., Vučenik V., Marušić A., (2004). Citation and quotation accuracy in three anatomy journals. Clinical Anatomy, 17(7), 534–539. https://doi.org/10.1002/ca.10255
47. Luo M., Li C. L., Molina D., Andersen C. R., Panchbhavi V. K., (2013). Accuracy of citation and quotation in foot and ankle surgery journals. Foot & Ankle International, 34(7), 949–955. https://doi.org/10.1177/1071100713475354
48. Martella A. M., Demmig-Adams B., (2018). Combining effective instructional approaches in a large introductory biology classroom: A research review and illustrative case study. Journal on Excellence in College Teaching, 29(2), 121–146.
49. Martella A. M., Klahr D., Li W., (2020). The relative effectiveness of different active learning implementations in teaching elementary school students how to design simple experiments. Journal of Educational Psychology, 112(8), 1582–1596. https://doi.org/10.1037/edu0000449
50. Martella A. M., Lovett M., Ramsay L., (in press). Implementing active learning: A critical examination of sources of variation in active learning science courses. Journal on Excellence in College Teaching.
51. Masic I., (2013). The importance of proper citation of references in biomedical articles. ACTA Informatica Medica, 21(3), 148–155. https://doi.org/10.5455/aim.2013.21.148-155
52. *McCollum B. M., Fleming C. L., Plotnikoff K. M., Skagen D. N., (2017). Relationships in the flipped classroom. Canadian Journal for the Scholarship of Teaching and Learning, 8(3), 1–19. https://doi.org/10.5206/cjsotl-rcacea.2017.3.8
53. Mogull S. A., (2017). Accuracy of cited “facts” in medical research articles: A review of study methodology and recalculation of quotation error rate. PLoS One, 12(9), Article e0184727. https://doi.org/10.1371/journal.pone.0184727
54. Murphy P. K., Knight S. L., Dowd A. C., (2017). Familiar paths and new directions: Inaugural call for manuscripts. Review of Educational Research, 87(1), 3–6. https://doi.org/10.3102/0034654317691764
55. National Science Foundation. (2014, May 12). Enough with the lecturing. https://www.nsf.gov/news/news_summ.jsp?cntn_id=131403
56. Neihouse P. F., Priske S. C., (1989). Quotation accuracy in review articles. DICP: The Annals of Pharmacotherapy, 23(7–8), 594–596. https://doi.org/10.1177/1060028089023007-813
57. Onwuegbuzie A. J., Combs J. P., Frels R. K., Slate J. R., (2011). Editorial: Citation errors revisited: The case for Educational Researcher. Research in the Schools, 18(1), i–xxxv.
58. Parse R. R., (1996). Hear ye, hear ye, novice and seasoned authors! Nursing Science Quarterly, 9(1), 1. https://doi.org/10.1177/089431849600900101
59. *Peters A. M. F., Costello J., Crane D., (2018). Deviating from the traditional instructional tools: Integrating Twitter in a sociology of deviance course. Canadian Journal of Learning and Technology, 44(3), 1–19. https://doi.org/10.21432/cjlt27792
60. Pickles M., (2016, November 23). Shouldn’t lectures be obsolete by now? BBC News. http://www.bbc.com/news/business-38058477
61. *Pienta N. J., (2015). Innocents abroad: Part II. A glimpse at chemical education in India. Journal of Chemical Education, 92(3), 399–400. https://doi.org/10.1021/acs.jchemed.5b00101
62. Santini A., (2018). The importance of referencing. Journal of Critical Care Medicine, 4(1), 3–4. https://doi.org/10.2478/jccm-2018-0002
63. Sanz-Martín M., Pitt K. A., Condon R. H., Lucas C. H., Novaes de, Santana C., Duarte C. M., (2016). Flawed citation practices facilitate the unsubstantiated perception of a global trend toward increased jellyfish blooms. Global Ecology and Biogeogrpahy, 25(9), 1039–1049. https://doi.org/10.1111/geb.12474
64. Schmitt-Harsh M., Harsh J. A., (2017). Engaging nonscience majors in urban ecology: Recommendations for course design. Journal of Environmental Studies and Sciences, 7(4), 550–561. https://doi.org/10.1007/s13412-016-0410-9
65. *Schönwälder J., Friedman T., Pras A., (2017). Using networks to teach about networks. ACM SIGCOMM Computer Communication Review, 47(3), 1–5. https://ccronline.sigcomm.org/wp-content/uploads/2017/08/sigcomm-ccr-final119.pdf
66. Schulmeister L., (1998). Quotation and reference accuracy of three nursing journals. Journal of Nursing Scholarship, 30(2), 143–146. https://doi.org/10.1111/j.1547-5069.1998.tb01269.x
67. Seymour E., Hewitt N. M., (1997). Talking about leaving: Why undergraduates leave the sciences. Westview Press. http://www-personal.umich.edu/~glarose/dept/teaching/resources/talkingAboutLeavingSeymourHewittOverview.pdf
68. Singer S. R., Nielsen N. R., Schweingruber H. A., (Eds.). (2012). Discipline-based education research: Understanding and improving learning in undergraduate science and engineering. National Academies Press. http://eduinfo.cchem.berkeley.edu/pdf/NAPunderstandingandimprovinglearning.pdf
69. Small H., (1982). Citation context analysis. In Dervin B., Voigt M. J., (Eds.), Progress in communication sciences (Vol. 3, pp. 287–310). Ablex.
70. Stains M., Harshman J., Barker M. K., Chasteen S. V., Cole R., DeChenne-Peters S. E., Eagan M. K., Esson J. M., Knight J. K., Laski F. A., Levis-Fitzgerald M., Lee C. J., Lo S. M., McDonnell L. M., McKay T. A., Michelotti N., Musgrove A., Palmer M. S., Plank K. M., …. Young A. M., (2018). Anatomy of STEM teaching in North American universities: Lecture is prominent, but practices vary. Science, 359(6383), 1468–1470. http://chemistry.as.virginia.edu/sites/chemistry.as.virginia.edu/files/2018-Stains%20et%20al-Science-%20COPUS%20profiles.pdf
71. Strauss V., (2017, July 12). It puts kids to sleep—but teachers keep lecturing anyway: Here’s what to do about it. The Washington Post. https://www.washingtonpost.com/news/answer-sheet/wp/2017/07/11/it-puts-kids-to-sleep-but-teachers-keep-lecturing-anyway-heres-what-to-do-about-it/
72. Teixeira M. C., Thomaz S. M., Michelan T. S., Mormul R. P., Meurer T., Fasolli J. V. B., Silveira M. J., (2013). Incorrect citations give unfair credit to review authors in ecology journals. PLoS One, 8(12), Article e81871. https://doi.org/10.1371/journal.pone.0081871
73. Todd P. A., Guest J. R., Lu J., Chou L. M., (2010). One in four citations in marine biology papers is inappropriate. Marine Ecology Progress Series, 408(June), 299–303. https://www.jstor.org/stable/10.2307/24873970
74. Todd P. A., Yeo D. C. J., Li D., Ladle R. J., (2007). Citing practices in ecology: Can we believe our own words? Oikos, 116(9), 1599–1601. https://doi.org/10.1111/j.2007.0030-1299.15992.x
75. Watkins J., Mazur E., (2013). Retaining students in science, technology, engineering, and mathematics (STEM) majors. Journal of College Science Teaching, 42(5), 36–41. https://www.jstor.org/stable/43631580
76. *Watson R. M., Willford J. D., Pfeifer M. A., (2018). A cultured learning environment: Implementing a problem- and service-based microbiology capstone course to assess process- and skill-based learning objectives. Interdisciplinary Journal of Problem-Based Learning, 12(1), 1–22. https://doi.org/10.7771/1541-5015.1694
77. *Wenzel T. J., (2014). Active learning materials for molecular and atomic spectroscopy. Analytical and Bioanalytical Chemistry, 406(22), 5245–5248. https://doi.org/10.1007/s00216-014-7991-5
78. Wieman C. E., (2014). Large-scale comparison of science teaching methods sends clear message. Proceedings of the National Academy of Sciences of the United States of America, 111(23), 8319–8320. https://doi.org/10.1073/pnas.1407304111
79. *Wilcox R. L., Adem P. V., Afshinnekoo E., Atkinson J. B., Burke L. W., Cheung H., Dasgupta S., DeLaGarza J., Joseph L., LeGallo R., Lew M., Lockwood C. M., Meiss A., Norman J., Markwood P., Rizvi H., Shane-Carson K. P., Sobel M. E., Suarez E., … Haspel R. L., (2018). The undergraduate training in genomics (UTRIG) initiative: Early and active training for physicians in the genomic medicine era. Personalized Medicine, 15(3), 199–208. https://doi.org/10.2217/pme-2017-0077
80. Willingham D., (2014, May 20). “Active learning” in college STEM courses: Meta-analysis. RealClearEducation.com. http://www.danielwillingham.com/daniel-willingham-science-and-education-blog/active-learning-in-college-stem-courses-meta-analysis
81. Wilson M. L., (2015). The persistence of misinformation. American Journal of Clinical Pathology, 144(3), 359–360. https://doi.org/10.1309/AJCPMBVI0SW4QHGU
82. *Ylostalo J. H., (2019). Engaging students into their own learning of foundational genetics concepts through the 5E learning cycle and interleaving teaching techniques. Journal of Biological Education, 54(5), 514–520. https://doi.org/10.1080/00219266.2019.1620311
83. Zakrajsek T., (2018). Reframing the lecture versus active learning debate: Suggestions for a new way forward. Education in the Health Professions, 1(1), 1–3. https://doi.org/10.4103/EHP.EHP_14_18
84. Ziman J. M., (1969). Information, communication, knowledge. Nature, 224, 318–324. https://doi.org/10.1038/224318a0