Legal research assessment

Legal Reference Services Quarterly

Volumn 28, Issue 3-4, 2009, Pages 201-217

  Canick, Simon ^a  

^a William Mitchell College of Law   (United States)

Author keywords

Aassessment; Examinations; Instruction; Legal research

Indexed keywords

EID: 70949098408     PISSN: 0270319X     EISSN: 1540949X     Source Type: Journal    
DOI: 10.1080/02703190902961569     Document Type: Article

References (93)

1. See Association of Legal Writing Directors/Legal Writing Institute, 2002 Survey Results, available at http://www.alwd.org/surveys/2002.html.
2. See Association of Legal Writing Directors/Legal Writing Institute, 2000 Survey Results, available at http://www.alwd.org/surveys/2000.html.
3. See also Ann Hemmens, Advanced Legal Research Courses: A Survey of ABA-Accredited Law Schools, 94 L. Lib. J. 209, 220 (2002) (finding a 700% increase in the number of law schools offering ALR classes between 1983 and 2000).
4. Cf. Maureen Fitzgerald, What's Wrong with Legal Research and Writing? Problems and Solutions, 88 L. Lib. J. 247, 252 (1996) (noting that although "most law professors recognize the importanc of research and writing, they seem unable as a group to find room for it in the curriculum.")
5. See id. at 271.
6. ABA, Section of Legal Education and Admissions to the Bar, Legal Education and Professional Development: An Educational Continuum: Report of the Task Force on Law Schools and the Profession: Narrowing the Gap, Statement of Fundamental Lawyering Skills and Professional Values (1992) [hereinafter MacCrate Report].
7. William M. Sullivan et al., Educating Lawyers: Preparation For The Profession Of Law 188 (Jossey-Bass/Wiley 2007) [hereinafter Carnegie Report]. The failure of law schools in this regard appears to be unique in professional education.
8. With some important exceptions, the underdeveloped area of legal pedagogy is clinical training, which typically is not a required part of the curriculum and is taught by instructors who are themselves not regular members of the faculty. Compared with the centrality of supervised practice, with mentoring and feedback, in the education of physicians and nurses or the importance of supervised practice in the preparation of teachers or social workers, the relative marginality of clinical training in law schools is striking.
9. Id. at 24.
10. See MacCrate Report, supra n. 5, at 267 (concluding that "the statement should be an essential reference in the accreditation process").
11. Roy Stuckey et al., Best Practices for Legal Education: A Vision and a Road Map 17 (Clinical Legal Education Association 2007) (arguing that "[w]ithout clearer guidance from the accrediation [sic] standards and without any significant internal or external motivators to change the status quo, law schools have been slow to consider the implications of the ABA's mandate to prepare students for effective and responsible participation in the legal profession.")
12. See Joan S. Howland & Nancy J. Lewis, Effectiveness of Law School Legal Research Training Programs, 40 J. Leg. Educ. 381 (1990).
13. See also Paul D. Callister, Beyond Training: Law Librarianship's Quest for the Pedagogy of Legal Research Education, 95 L. Lib. J. 7, 9-11 (2003).
14. Librarians at many law schools have attempted to supplement formal course offerings with brown bag sessions, optional lectures, and "bridge the gap" programs specifically designed to assist summer associates and graduating students.
15. MacCrate Report, supra n. 5, at 157.
16. Specifically, MacCrate advocates an understanding of: "various sources of legal rules and the processes by which these rules are made," sources providing "controlling principles for resolution of various kinds of issues." and "[t]he variety of legal remedies available in any given situation." Id. at 157-158.
17. With respect to fundamental research tools, lawyers "should be generally familiar with the nature of the tool, its likely location in a law library, and the ways in which the tool is used." Id. at 159.
18. Last, lawyers should understand how to formulate issues for research, identify "the full range of search strategies that could be used to research the issues." evaluate the search alternatives before settling on a "research design," and implement that design. Id. At 160-162.
19. Kristin Gerdy, Teacher, Coach, Cheerleader, and Judge: Promoting Learning Through Learner- Centered Assessment, 94 L. Lib. J. 59, 69 (2002).
20. Stuckey, supra n. 7, at 236 (stating that the "primary reason to administer assessments is to find out whether our students are learning what we want them to learn.")
21. Stuckey, supra n. 7, at 238-239.
22. "In sum, except perhaps in legal writing and research courses, the current assessment practices used by most law teachers are abominable." Id. at 239.
23. See generally Joyce M. Janto & Lucinda D. Harrison-Cox, Teaching Legal Research: Past and Present, 84 L. Lib. J. 281 (1992).
24. See e.g. Stuckey, supra n. 7, at 236-238 (concluding that "the end-of-the semester essay exam is neither valid, nor realizable, nor fair."
25. See generally Callister, supra n. 8.
26. Christopher G. Wren & Jill R. Wren, The Teaching of Legal Research, 80 L. Lib. J. 7, 10 (1988).
27. Robert C. Berring & Kathleen Vanden Heuvel, Legal Research: Should Students Learn It or Wing It? 81 L. Lib. J. 431 (1989).
28. Christopher G. Wren & Jill R. Wren, Reviving Legal Research: A Reply to Berring and Vanden Heuvel, 82 L. Lib. J. 463 (1990).
29. Robert C. Berring & Kathleen Vanden Heuvel, Legal Research: A Final Response, 82 L. Lib. J. 495 (1990).
30. Berring recently remarked that "[a]lmost 20 years later one might wonder what the fuss was all about." Robert C. (Bob) Berring Jr., Twenty Years On: The Debate Over Legal Research Instruction, 17 Persps. Teaching Leg. Research & Writing 1, 3 (2008). Available at http://west.thomson.com/pdf/perspec/2008-Fall/2008-Fall-1.pdf.
31. Fitzgerald, supra n. 3, at 255.
32. See e.g. Douglas Miller, Using Examinations in First-Year Legal Research, Writing, and Reasoning Courses, 3 Leg. Writing 217 (1997). I hasten to add that I have oversimplified the concepts here in order to set an easy distinction. I am not trying to belittle the bibliographic approach generally or the use of examinations specifically. On the contrary, I find many of Mr. Miller's arguments persuasive.
33. Janto & Harrison-Cox, supra n. 13, at 289.
34. Baylor Law School has handled this problem by untangling research and writing. In this system, students take a quarter-long, separately graded research class. See Matthew C. Cordon, Beyond Mere Competency: Advanced Legal Research in a Practice-Oriented Curriculum, 55 Baylor L. Rev. 1, 8 (2003).
35. Similarly, Kory Staheli argues that making legal research into a separately graded course "provide[s] needed motivation and incentive for students to learn the material and to master necessary skills." Kory D. Staheli, Motivating Law Students to Develop Competent Legal Research Skills: Combating the Negative Findings of the Howland and Lewis Survey, 14 Legal Ref. Servs. Q. no. 1-2, at 195, 200-201 (1994).
36. See Carnegie Report, supra n. 6, at 173.
37. But see id. at 290 ("We stress that the students are being taught research techniques to insure basic competence, and that the students are themselves responsible for building upon this framework during their three years in law school.")
38. See Greg Sergienko, New Modes of Assessment, 38 S.D. L. Rev. 463, 486 (2001) (concluding that quick, low-stake examinations "encourage students to keep up with work and provide them with frequent feedback on their progress.")
39. But see Miller, supra n. 22, at 233 (admitting that "this reduction in grading time does come at the expense of additional preparation time. Good multiple-choice questions don't drop out of the sky: they can be devilishly difficult to construct and fine tune.")
40. Sergienko, supra n. 28, at 486.
41. See Gerdy, supra n. 11, at 67.
42. "After all, when was the last time an attorney did research that had been 'sanitized' by a law librarian in advance to ensure that the desired resources and results were available?" Id. See also Callister, supra n. 8, at 27.
43. The neat order presented by the casebook method is illusory, a pedagogical nicety but an oversimplification of the world they will face in practicing law. To prepare for independence and to be weaned from their sterile law school environment of neatly tied case studies, students will need to learn to build their own paradigms. No subject offers a better opportunity to practice making order out of chaos than legal research.
44. See Janto & Harrison-Cox, supra n. 13, at 290-91.
45. See Gerdy, supra n. 11, at 72.
46. See id. at 67 (noting the distinction between well-defined and ill-defined problems).
47. See Fitzgerald, supra n. 3, at 265-266.
48. See David J. Boud & Allen Tyree, Self and Peer Assessment in Professional Education: A Preliminary Study in Law, 15 J. Socy. Pub. Teachers L. 65, 65 (stating that lawyers' work is rarely judged formally, except when "something goes terribly wrong. The lack of effective external assessment directing his day-to-day work means that his own ability to define goals and to self-assess his progress toward those goals must be better than those of most non-professionals.")
49. See Sergienko, supra n. 28, at 479 (2001) (noting that commentators believe "self reflection is a critical component of successful professional work.")
50. Cf. Gerdy, supra n. 11, at 78 (noting that first-year students are eager for feedback of the type that they get in none of their other first-year courses.)
51. Brian Huddleston, Trial by Fire. Creating a Practical Application Research Exam, 7 Persps. Teaching Leg. Research & Writing 99, 99 (1999). Available at http://west.thomson.com/pdf/perspec/trl0899.pdf. The response might be that, unlike concert pianists who are trained to perform before audiences, legal research is generally a solitary endeavor. There is little precedent for having a person standing over your shoulder watching you work.
52. Professors tell students that they will not be penalized for wrong turns, but it defies logic to imagine that all of them take that advice to heart. In a similar situation, I would be tempted to clean up my work.
53. Of course, this does not take into account the time necessary to develop the research problems in the first place. Matthew Cordon suggests delegating the task to students. "The difficulty with developing a practical problem has been resolved to a degree by requiring students to draft research problems of their own creation." Cordon, supra n. 24, at 34.
54. Gerdy, supra n. 11, at 80.
55. Id. Gerdy's article includes numerous interesting ideas. Another is the "minute paper," through which students identifying the points they found most confusing in her lectures.
56. At the end of a class period, students are told to write for a minute or two on this question: "If you had to explain the most important concepts you learned today to a class member who was absent-and you are her only source of information-what would you tell her?" The papers, which are submitted anonymously, allow the teacher to assess how much the students have learned and to identify any misunderstandings or common questions.
57. Id. at 82.
58. See Callister, supra n. 8, at 33 (arguing that the "fundamental use" of the Socratic method is "to force an examination of one's own view of the world, including the mental frameworks one develops for understanding and solving problems.")
59. As an adjunct at Fordham, I taught ALR in a smart computer lab. A control panel allowed me to select any computer in the room and broadcast that student's image through the projector. As a result, if someone had a problem-say, a process was not working correctly-we could all look at his or her work to discover what had gone wrong.
60. http://www.adobe.com/products/captivate.
61. http://www.techsmith.com/camtasia.asp.
62. http://camstudio.org. MediaNotes is another potentially useful tool.
63. http://w.cali.org/medianotes. Distributed by Computer-Assisted Legal Instruction, MediaNotes allows teachers to annotate videos (created by Camtasia, CamStudio, or another program). Teachers can use it to point out specific points in the video where students made mistakes or could have followed a more straightforward approach.
64. The potential benefit of time saving for the professor is overwhelmed by the controversy it would cause-students in legal education want their professors to grade them, not other students.
65. See Shirley Rawson & Alan L. Tyree, Self and Peer Assessment in Legal Education, 1 Leg. Educ. Rev. 135, 140 (finding "great student hostility to the inclusion of self and peer assessment marks as part of the formal assessment process.").
66. Sergienko, supra n. 28, at 485
67. See Sergienko, supra n. 28, at 485 ("The work on incompetence in assessment... raises questions about the ability of students to grade exam answers that do not conform well to a model answer.")
68. Ngar-Fun Liu & David Carless, Peer Feedback: The Learning Element of Peer Assessment, 11 Teaching Higher Educ. 279, 280 (2006).
69. See Boud & Tyree, supra n. 38, at 67.
70. Liu & Carless, supra n. 53, at 280.
71. See Boud & Tyree, supra n. 38, at 65.
72. Rawson & Tyree, supra n. 51, at 136.
73. See Liu & Carless supra n. 53, at 281.
74. In 2000, more than two-thirds (sixty-nine percent) of all ALR classes featured pathfinder assignments. See Hemmens, supra n. 2, at 234.
75. See e.g. Peter C. Schanck, Mandatory Advanced Legal Research: A Viable Program for Law Schools, 92 L. Lib. J. 295, 298 (2000).
76. Cordon, supra n. 24, at 34.
77. See also Wren & Wren, supra n. 18, at 487 (identifying pathfinder projects as "training in legal reference, not legal research.")
78. Berring & Vanden Heuvel, supra n. 17, at 446.
79. Id. at 445.
80. Easy to grade, though quite hard to write. See Miller, supra n. 22, at 233.
81. Staheli, supra n. 24, at 201.
82. See e.g. Fitzgerald, supra n. 3, at 267 (complaining that quizzes "test students' ability to find a source but do not assess whether a student knows when to find a source.")
83. Judith Rosenbaum, Why I Don't Give A Research Exam, 11 Persps. Teaching Leg. Research & Writing 1, 5 (2002). Available at http://west.thomson.com/pdf/perspec/Fall&202002/Fall021.pdf.
84. Miller, supra n. 22, at 219.
85. See generally Huddleston, supra n. 41.
86. Huddleston describes the process of administering a practical examination. It was not designed to test "their ability to perform broad, open-ended research that integrated multiple resources to provide a full answer to a complex legal question."
87. Id. at 100. Rather, it sought to assess whether students could "correctly use specific legal bibliographic resources to find material based on given criteria."
88. Id. at 101.
89. Stuckey, supra n. 7, at 133.
90. See Carnegie Report, supra n. 6, at 173 (noting that "[i]n law schools, the means of assessing the skills of practice mirror the wide array of pedagogical practices represented in lawyering and clinical courses. This diversity of assessment procedures can be seen as a strength, as reliance on a single format... would be both impractical and ill-advised.")
91. Gerdy, supra n. 11, at 75.
92. See infra n. 49-50, and accompanying text.
93. Students may also enjoy the variety. And because different students will excel at different types of tasks, using a variety of approaches is equitable. See Gerdy, supra n. 11, at 75.