Outcomes assessment and legal research pedagogy

Legal Reference Services Quarterly

Volumn 31, Issue 2, 2012, Pages 184-203

  Feliú, Vicenç ^a   Frazer, Helen ^b  

^a VILLANOVA UNIVERSITY   (United States)

^b UNIVERSITY OF THE DISTRICT OF COLUMBIA   (United States)

Author keywords

Assessment; Formative; Legal research; Summative; Taxonomy

Indexed keywords

EID: 84862672972     PISSN: 0270319X     EISSN: 1540949X     Source Type: Journal    
DOI: 10.1080/0270319X.2012.682938     Document Type: Article

References (116)

1. The ABA is the accrediting agency for law schools.
2. ABA Sec. Leg. Educ. & Admis. to the Bar, Accreditation, available at http://www.americanbar.org/groups/legal_education/resources/accreditation.html
3. ABA Sec. Leg. Educ. & Admis. to the Bar Standards Review Committee, Draft after November 2011 Meeting, Chapter 3, Program of Legal Education, Standard 302, Learning Outcomes. Available at http://www.americanbar.org/content/dam/aba/migrated/2011_build/legal_education/committees/standards_review_documents/jan2012/20111222_standards_chapters_ 1_to_7_post_nov11.authcheckdam.pdf
4. A law school shall identify, define, and disseminate each of the learning outcomes it seeks for its graduating students and for its program of legal education.(2) the professional skills of: (i) legal analysis and reasoning, critical thinking, legal research, problem solving, written and oral communication in a legal context; and (ii) the exercise of professional judgment consistent with the values of the legal profession and professional duties to society, including recognizing and resolving ethical and other professional dilemmas.Note.
5. ABA, supra n. 2, at Chapter 3, Program of Legal Education, Standard 301, Objectives. A law school shall maintain a rigorous educational program that prepares its students for admission to the bar and effective, ethical and responsible participation in the legal profession.
6. ABA, supra n. 2, at Standard 305, Assessment of Student Learning. See also ABA, supra n. 2, at Standard 304, Curriculum, which requires law schools to offer a curriculum that "that is designed to produce graduates who have attained competency in the learning outcomes identified in Standard 302.".
7. ABA Sec. Leg. Educ. & Admis. to the Bar, Accreditation, available at http://www.americanbar.org/groups/legal_education/resources/accreditation.html
8. Formative assessments relate to developing teaching and learning objectives during the course.
9. A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives 245 (Lorin W. Anderson & David R. Krathwohl et al. eds., 2001) (hereinafter the Revised Taxonomy) (Longman 2001).
10. Summative assessments relate to the grading process, which sums up student learning for the whole course.
11. Id. at 247.
12. ABA, supra n. 2, at Standard 305, Assessment of Student Learning. "A law school shall apply a variety of formative and summative assessment methods across the curriculum to provide meaningful feedback to students." Interpretation 305-1:
13. Formative assessment methods are measurements at different points during a particular course or over the span of a student's education that provide meaningful feedback to improve student learning. Summative assessments methods are measurements at the culmination of a particular course or the culmination of any part of a student's legal education that measures the degree of student learning.
14. William G. Spady, Organizing for Results: The Basis of Authentic Restructuring and Reform, 46 Educational Leadership 4-8 (1988);
15. Gwennis McNeir, Outcome-Based Education: Tools for Restructuring, Oregon School Study Council Bulletin (Oregon School Study Council 1993).
16. William G. Spady, Outcome-Based Education: Critical Issues and Answers (Am. Ass'n Sch Administrators 1994). Revised Taxonomy, supra n. 5, at 295.
17. The Boulder Statement on Legal Research Education (2009), available at http://www.colorado.edu/law/events/legalResearchEducation.pdf (hereinafter Boulder Statement);
18. The Boulder Statement on Legal Research Education: Signature Pedagogy Statement (2010), available at http://www1.law.umkc.edu/faculty/callister/Pubs/BB-2010_Signature_Pedagogy_Statement.pdf (hereinafter Boulder Signature Pedagogy).
19. Librarians are familiar with the use of taxonomies in classifying and cataloging information materials. During the last several decades, taxonomies have been introduced as a useful tool in education pedagogy.
20. See generally Taxonomy of Educational Objectives (Benjamin S. Bloom et al. eds., David McKay Co. 1956) (hereinafter Bloom's Taxonomy).
21. Popularly known as The Carnegie Report, co-authored by William M. Sullivan, Anne Colby, Judith Welch Wegner, Lloyd Bond, Lee S. Shulman, and published by Jossey-Bass.
22. ABA Sec. Leg. Educ. & Admis. to the Bar, Report and Recommendations of the Task Force on Lawyer Competency: The Role of the Law Schools (ABA 1979), cited in the Carnegie Report, supra n. 13, at 173.
23. ABA, Legal Education and Professional Development-An Educational Continuum (ABA 1992) (hereinafter MacCrate Report).
24. Id. at 124.
25. The MacCrate Report provides an elaborate explanation of the knowledge, ability, and understanding of legal analysis and "the nature of legal rules and institutions" required for using the tools of legal research to formulate issues for research in an effective research plan.
26. The MacCrate Report requires a law school graduate to have.
27. Knowledge of the Nature of Legal Rules and Institutions;
28. Knowledge of and Ability to Use the Most Fundamental Tools of Legal Research;
29. Understanding of the Process of Devising and Implementing a Coherent and Effective Research Design.
30. Gregory S. Munro, Outcomes Assessment for Law Schools 21-29 (Inst. for L. Sch. Teaching 2000).
31. In the meantime, conferences and a steady stream of articles have focused on outcomes assessment, especially since the ABA began to discuss the topic and then started to draft the outcomes assessment standards.
32. Mary A. Crossley & Lu-in Wang, Learning by Doing: An Experience with Outcomes Assessment, available at http://papers.ssrn.com/s013/papers.cfm?abstract_id=1663327;
33. Andrea A. Curcio, Moving in the Direction of Best Practices and the Carnegie Report: Reflections on Using Multiple Assessments in a Large-Section Doctrinal Course, 19 Widener L.J. 159 (2009);
34. Andrea A. Curcio, Gregory Todd Jones & Tanya M. Washington, Does Practice Make Perfect? An Empirical Examination of the Impact of Practice Essays on Essay Exam Performance, 35 Fla. St. U. L. Rev. 2711 (2008);
35. Andrea A. Curcio, Gregory Todd Jones & Tanya M. Washington, Developing an Empirical Model to Test Whether Required Writing Exercises or Other Changes in Large Section Law Class Teaching Methodologies Result in Improved Exam Performance, 57 J. Leg. Educ. 195 (2007);
36. Susan Hanley Duncan, The New Accreditation Standards Are Coming to a Law School Near You-What You Need to Know about Learning Outcomes and Assessment, 16 Leg. Writing 605 (2010);
37. Janet W. Fisher, Putting Students at the Center of Legal Education: How an Emphasis on Outcome Measures in the ABA Standards for Approval of Law Schools Might Transform the Educational Experience of Law Students, 35 S. Ill. U. L.J. 225 (2011);
38. Kristin B. Gerdy, Teacher, Coach, Cheerleader, and Judge: Promoting Learning through Learner-Centered Assessment, 94 L. Lib. J. 59 (2002);
39. Carolyn Grose, Outcomes-Based Education One Course at a Time: My Experiment with Estates and Trusts, available at http://ssrn.com/abstract+1663327;
40. Dennis R. Honabach, Precision Teaching in Law School: An Essay in Support of Student-Centered Teaching and Assessment, 34 U. Toledo L. Rev. 95 (2002);
41. Beverly Petersen Jennison, Saving the LRW Professor: Using Rubrics in the Teaching of Legal Writing to Assist in Grading Writing Assignments by Section and Provide More Effective Assessment in Less Time, 80 UMKC L. Rev. 353 (2011);
42. Kristina L. Niedringhaus, Teaching Better Research Skills by Teaching Metacognitive Ability, 18 Persps. 113, 117 (2010);
43. Anthony S. Niedwiecki, Teaching for Lifelong Learning: Improving the Metacognitive Skills of Students through More Effective Formative Assessment Techniques (2011), ExpressO, available at http://works.bepress.com/anthony_niedwiecki/1/;
44. Charles A. Rees, The 'Non-Assessment' Assessment Project, 57 J. Leg. Educ. 521 (2007);
45. Lori A. Roberts, Assessing Ourselves: Confirming Assumptions and Improving Student Learning by Efficiently and Fearlessly Assessing Student Learning Outcomes, 3 Drexel L. Rev. 457 (2011);
46. Greg Sergienko, New Modes of Assessment, 38 S.D. L. Rev. 463 (2001);
47. Victoria L. VanZandt, Creating Assessment Plans for Introductory Legal Research and Writing Courses, 16 Leg. Writing 313 (2010).
48. Munro, supra n. 19, at 17.
49. The Carnie Report, supra n. 13.
50. Roy Stuckey et al., Best Practices for Legal Education: A Vision and a Road Map (Clinical Legal Educ. Ass'n 2007).
51. ABA Sec. Leg. Educ. & Admis. to the Bar, Report of the Outcome Measures Committee (July 27, 2008), available at http://apps.americanbar.org/legaled/committees/subcomm/Outcome%20Measures%20Final%20Report.pdf
52. ABA Standards Review Committee, Statement of Accreditation and Statement of Principles and Fundamental Goals of a Sound Program of Legal Education (May 6, 2009), available at http://apps.americanbar.org/legaled/committees/Standards%20Review%20documents/Principles%20and% 20Goals%20Accreditation%205%206%2009.pdf
53. ABA Standards Review Committee, Student Learning Outcomes Subcommittee, Draft after Nov. 2011 Meeting, Chapter 3 Program of Legal Education, Standard 306, Institutional Effectiveness, available at http://www.americanbar.org/groups/legal_education/committees/standards_review.html
54. In measuring its institutional effectiveness pursuant to Standards 202 and the rigor of its education program pursuant to Standard 301, the dean and faculty of a law school shall.
55. Conduct regular, ongoing assessment of whether its learning outcomes, curriculum and delivery, assessment methods and the degree of student attainment of competency in the learning outcomes are sufficient to ensure that its students are prepared to participate effectively, ethically, and responsibly as entry level practitioners in the legal profession;Note.
56. Use the results of this review to improve its curriculum and its delivery with the goal that all students attain competency in the learning outcomes.
57. See supra n. 2 (Emphasis supplied.)
58. ABA, supra n. 4:
59. A law school shall offer a curriculum that is designed to produce graduates who have attained competency in the learning outcomes identified in Standard 302.
60. ABA, supra n. 4:
61. A law school shall apply a variety of formative and summative assessment methods across the curriculum to provide meaningful feedback to students (Emphasis supplied).
62. ABA, supra n. 27.
63. ABA, supra n. 2.
64. ABA, supra n. 2, at Standard 302, Interpretation 302-4.
65. ABA supra n. 2, at Standard 304(a)(2-3), Curriculum, requiring at least one rigorous skills course of three semester hours after the first year, including simulation courses, live client clinics, or field placement.Note.
66. Revised Taxonomy, supra n. 5, at 245.
67. University of the District of Columbia David A. Clarke School of Law Clinics' Competency Principles for Legal Research (copy on file with the authors).
68. Boulder Statement, supra n. 11; Boulder Signature Pedagogy, id.; American Association of Law Libraries Legal Research Competency Principles, infra n. 83.
69. Revised Taxonomy, supra n. 5, at 250.
70. Paul Callister, Time to Blossom: An Inquiry into Bloom's Taxonomy as a Means to Ordered Legal Research Skills, 102 L. Lib. J. 191 (2009).
71. Bloom's Taxonomy, supra n. 12.
72. Revised Taxonomy, supra n. 5.
73. As noted by Professor Callister and others.
74. Callister, supra n. 43.
75. Callister credits Maureen Fitzgerald for identifying Bloom's taxonomy as useful for grounding legal research educational pedagogy.
76. Maureen F. Fitzgerald, What's Wrong with Legal Research and Writing? Problems and Solutions, 88 L. Lib. J. 247, 271 (1996).
77. See generally id.
78. See infra Table 2.
79. Callister, supra n. 43, at 200-202.
80. In using verb forms, Callister is following the Revised Taxonomy, supra n. 5, which labels categories with verbs rather than nouns.
81. Callister, supra n. 43, at 199.
82. Paul D. Callister, Working the Problem, 2003 Ill. B.J., at 43.
83. Callister, supra n. 43, at 206.
84. Kristina L. Niedringhaus, Teaching Better Research Skills by Teaching Metacognitive Ability, 18 Persps. 113, 117 (2010) ("Students who understand what they know and understand how to control their own learning are more likely to understand how to apply what they have learned previously to new situations.")
85. Examples are from Callister, supra n. 43.
86. These examples are from Callister, supra n. 43, at 216.
87. Model Rules of Professional Conduct Annotated (ABA 1996) (hereinafter Model Rules Ann.).
88. Geoffrey C. Hazard, & W. William Hodes, The Law of Lawyering: A Handbook on the Model Rules of Professional Conduct (2d ed., Prentice Hall Law & Bus. 1990).
89. Lawrence Duncan MacLachlan, Gandy Dancers on the Web: How the Internet Has Raised the Bar on Lawyers' Professional Responsibility to Research and Know the Law, 13 Geo. J. Leg. Ethics 607 (1999-2000).
90. For a leading malpractice case that explicates the standard of care, see Smith v. Lewis, 530 P.2d 589, 593 (Cal. 975), overruled on other grounds by In re Marriage of Brown, 544 P.2d 561 (Cal. 1976) (cited and quoted in MacLachlan, supra n. 69, at 616-617).
91. Model Rule 1.1, Competence, supra n. 69. This rule is enforced by a disciplinary rule for "[f]ailing to act competently." Disciplinary Rule 6-101, Failing to Act Competently ("a lawyer shall not handle a legal matter without preparation adequate in the circumstances," id.
92. Model Rules Ann. Rule 1.1 (citing Baird v. Pace, 752 P.2d 505, 509 [Ariz. 1987]), supra n. 69.
93. Smith v. Lewis, supra n. 70.
94. MacLachlan, supra n. 69, at 617.
95. See Boulder Statement, supra n. 11.
96. Boulder Signature Pedagogy, supra n. 11.
97. Smith v. Lewis, 13 Cal. 3d 349, 118 Cal. Rptr. 621, 530 P.2d 589, 78 A.L.R.3d 231 (1975).
98. Fed. R. Civ. P. 11(b)(2).
99. David Shumaker & Mary Talley, Models of Embedded Librarianship: Final Report (2009), available at http://www.sla.org/PDFs/SLA2009/ModelsofEmbeddedLibrarianshipJune162009.pdf
100. Vicenç Feliú & Helen Frazer, Embedded Librarians: Teaching Legal Research as a Lawyering Skill (2011), 61 J. Legal Educ. 540.
101. MacCrate Report, supra n. 15.
102. A Report by the American Association of Law Libraries Research Instruction Caucus (Ellen M. Callinan ed., 1997), available at www.aallnet.org/sis/ripssis/PDFs/core.pdf
103. The Feb. 28, 2011, draft is available at http://researchcompetency.wordpress.com/2011/02 The charge to the Task Force is available at http://www.aallnet.org/Archived/Leadership-Governance/committees/Past-Committees/res-stds.html
104. Callister, supra n. 43, at 191.
105. Boulder Statement, Boulder Signature Pedagogy, supra n. 11.
106. Barbara Bintliff, Legal Research: MacCrate's 'Fundamental Lawyering Skill' Missing in Action, 28 Leg. Ref. Servs. Q. (2009);
107. Callister, supra n. 43;
108. Sarah Valentine, Legal Research as a Fundamental Skill: A Lifeboat for Students and Law Schools, 290 Balt. L. Rev. 173 (2010).
109. The Carnegie Report, supra n. 13, at 191.
110. The Carnegie Report, supra n. 13, at Chapter 5, Assessment and How to Make It Work, 162-184.
111. The Carnegie Report, supra n. 13, at 160; Boulder Statement, supra n. 11.
112. Callister, supra n. 43, at 200.
113. Boulder Statement, Boulder Signature Pedagogy, supra n. 11. (Emphasis supplied.)
114. Boulder Statement, Boulder Signature Pedagogy, supra n. 11.
115. The Carnegie Report, supra n. 13, at 50.
116. Boulder Statement, Boulder Signature Pedagogy, supra n. 11.