Direct Assessment of Information Literacy using Writing Portfolios

Journal of Academic Librarianship

Volumn 33, Issue 4, 2007, Pages 462-477

  Scharf, Davida ^a   Elliot, Norbert ^a   Huey, Heather A ^a   Briller, Vladimir ^b   Joshi, Kamal ^a  

^a NEW JERSEY INSTITUTE OF TECHNOLOGY   (United States)

^b PRATT INSTITUTE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 34250027811     PISSN: 00991333     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.acalib.2007.03.005     Document Type: Article

References (44)

1. Lindauer B.G., Arp L., and Woodard B. "The Three Arenas of Information Literacy Assessment". Reference & User Services Quarterly 44 (Winter 2004) 122-129
2. A Similar Orientation is Found in James Elmborg, "Critical Information Literacy: Implications for Instructional Practice". The Journal of Academic Librarianship 32 (March 2006) 192-199
3. Brian Huot, (Re)Articulating Writing Assessment for Teaching and Learning (Utah: Utah State University Press, 2002). For more on the assessment of writing ability, see Patricia Lynne, Coming to Terms: A Theory of Writing Assessment (Utah: Utah State University Press, 2004), Norbert Elliot, On a Scale: A Social History of Writing Assessment in America (New York: Peter Lang, 2005), and Edward M. White, "The Scoring of Writing Portfolios: Phase 2." College Composition and Communication, 56 (June 2005): 581-600. The use of rubrics described by Elizabeth Choinski, Amy E. Mark, and Missy Murphey is similar to the approach proposed by White: "Assessment with Rubrics: An Efficient and Objective Means of Assessing Student Outcomes in an Information Resources Class," portal: Libraries and the Academy 3 (October 2003): 563-575. In a 1984 review article, Peter Cooper wrote that "the study of writing assessment will continue to focus on the comparative strengths and limitations of direct and indirect measures..." Peter L. Cooper, The Assessment of Writing Ability: A Review of the Research (Princeton, NJ: Educational Testing Service, 1984).
4. Among the most Eloquent to Address the Need for Authentic Assessment is Grant Wiggins, Assessing Student Performance: Exploring the Purpose and Limits of Testing (1993), Jossey-Bass, San Francisco
5. For more about teaching in the contact zone, see Mary Louise Pratt, "Arts of the Contact Zone," Profession 91 (1991) 33-40; See also Professing Theory in the Contact Zone, edited by Janice M. Wolff (Urbana, IL: NCTE, 2002).
6. Presidential Committee on Information Literacy: Final Report. American Library Association (1989). Available: at http://www.ala.org/acrl/legalis.html (April 10, 2007).
7. Association for College & Research Libraries, Information Literacy Competency Standards for Higher Education (Chicago, IL: ACRL 2000). Available: http://www.ala.org/ala/acrl/acrlstandards/standards.pdf (April 10, 2007).
8. Association for College & Research Libraries (ACRL). "Research Agenda for Library Instruction and Information Literacy". College & Research Libraries News 64 (February 2003) 108-113
9. Association for College & Research Libraries (ACRL). Characteristics of Programs of Information Literacy that Illustrate Best Practices: A Guideline: Best Practices Initiative. Institute for Information Literacy (2003). Available: http://www.ala.org/ala/acrl/acrlstandards/characteristics.htm (April 10, 2007).
10. Middle States Commission. Characteristics of Excellence in Higher Education: Eligibility Requirements and Standards for Education. Philadelphia, PA: Middle States (2002). Available: http://www.msache.org/msache/content/pdf_files/characteristicsbook.pdf (April 10, 2007).
11. Lazerson M., Wagener U., and Shumanis N. "What makes a Revolution? Teaching and Learning in Higher Education 1980-2000". CHANGE, The Magazine of Higher Learning (2000) 1-6
12. Middle States Commission on Higher Education. Assessing Student Learning and Institutional Effectiveness: Understanding Middle States Expectations (2005), Middle States Commission, Philadelphia, PA
13. Matthews J. "Measure by Measure: A New Effort to Determine How Well Schools Teach". Atlantic Monthly 294 (October 2004) 134
14. Meulemans Y.N. "Assessment City: The Past, Present, and Future State of Information Literacy Assessment". College & Undergraduate Libraries (2002) 61-74
15. Peter Hernon, Robert Dugan & Candy Schwartz, Revisiting Outcomes Assessment in Higher Education. (Westport, Connecticut: Libraries Unlimited, 2006).
16. Peter Ewell, Grading Learning: Projects and Prospects in Measuring Up 2006. Available: http://measuringup.highereducation.org/commentary/gradinglearning.cfm (April 10, 2007).
17. Lois M. Pausch & Mary P. Popp, "Assessment of Information Literacy: Lessons from the Higher Education Movement," Paper presented at the 9th National Conference of the Association of College & Research Libraries. Detroit, MI. 1999.
18. Rutz C., and Lauer-Glebov J. "Assessment and Innovation: One Darn Thing Leads to Another". Assessing Writing 10 (2005) 80-99
19. Snavely L.L., and Wright C.A. "Research Portfolio Use in Undergraduate Honors Education: Assessment Tool and Model for Future Work". The Journal of Academic Librarianship 29 (September 2003) 298-303
20. Valentine B. "Undergraduate Research Behavior: Using Focus Groups to Generate Theory". The Journal of Academic Librarianship 19 (September 1993) 300-304
21. See LibQual. Available: http://www.libqual.org/ (April 10, 2007) for widely used measures of library quality.
22. Singh A.B. "A Report on Faculty Perceptions of Students' Information Literacy Competencies in Journalism and Mass Communication Programs: The ACEJMC Survey". College & Research Libraries 66 (July 2005) 294-310
23. O'Connor L.G., Radcliff C.J., and Gedeon J.A. "Applying Systems Design and Item Response Theory to the Problem of Measuring Information Literacy Skills". College & Research Libraries 63 (November 2002) 528-543
24. Kent State University Libraries and Media Services, Project SAILS: Standardized Assessment of Information Literacy Skills (Kent, OH). Available: https://www.projectsails.org/ (April 10, 2007).
25. Patricia Senn Breivik and Gordon E. Gee, Information Literacy: Revolution in the Library (Phoenix, AZ: American Council on Education and Oryx Press, 1989). Also: Ilene Rockman (ed.). Integrating information literacy into the higher education curriculum 1st ed. (San Francisco: Jossey-Bass, 2004).
26. Dunn K. "Assessing Information Literacy Skills in the California State University: A Progress Report". The Journal of Academic Librarianship 28 (January-February 2002) 26-35
27. Educational Testing Service, ICT Literacy Assessment: An Issue Paper from ETS (Princeton: ETS, 2004). Available: http://www.ets.org/Media/Tests/Information_and_Communication_Technology_Literacy/0202heapaper.pdf (June 28, 2006). www.ets.org/ictliteracy (April 10, 2007).
28. James Madison University Center for Assessment and Research Studies, Information Literacy Test (ILT). Available: http://www.jmu.edu/assessment/resources/Instruments_ILT.htm (April 10, 2007).
29. Collegiate Learning Assessment (CLA) Project. Council for Aid to Education (CAE). Available: http://www.cae.org/content/pro_collegiate.htm (April 10, 2007).
30. Cooney M., and Hiris L. Integrating Information Literacy and its Assessment into a Graduate Business Course: A Collaborative Framework. Research Strategies 19 (2003) 213-232
31. In: Haswell R.H. (Ed). Beyond Outcomes: Assessment and Instruction within a University Writing Program (2001), Ablex, Westport, CT
32. Edward M. White, 582.
33. Portfolio research conducted by the Department of Humanities can be found in recent studies describing longitudinal assessment efforts (Nancy W. Coppola, "Setting the Discourse Community: Tasks and Assessment for the New Technical Communication Service Course." Technical Communication Quarterly 8 (Summer 1999): 249-267, studies of on-line portfolios (Carol S. Johnson, The Analytic Assessment of Online Portfolios in Undergraduate Technical Communication: A Model, Journal of Engineering Education 95.4 (October 2006): 279-287.) and of graduate student writing (Nancy Coppola and Norbert Elliot, ''Assessment of Graduate Programs in Technical Communication: A Model,'' in J Allen and M Hundleby (Eds.), in Assessment in Professional and Technical Communication (New York: State University of New York Press, forthcoming). Research leading to the present study of information literacy can be found in Norbert Elliot, Vladimir Briller, and Kamal Joshi, ''Portfolio Assessment: Quantification and Community,'' Journal of Writing Assessment 3.1 (2006): 5-30. The NJIT Department of Humanities, in addition, maintains a site describing its range of assessment activities. College Writing Assessment. Available: http://cwa.njit.edu/ (April 10, 2007).
34. Lindauer, "The Three Arenas of Information Literacy Assessment," 122-129.
35. Carlson investigated bibliographies across six departments: Art, Classics, English, History, General Humanities, and Religion. In the present study, courses addressing each of these areas are offered within the unit. While Carlson found variance in citation behavior in academic discipline and level of course, such infrastructure issues (as they impact assessment and subsequent instruction based on the assessment results) are lessened in the Department of Humanities, a unit in which the majority of instructors teach courses from the first through the senior years. See Jake Carlson, "An Examination of Undergraduate Student Citation Behavior," The Journal of Academic Librarianship 32 (January 2006): 14-22.
36. This definition of community is informed by George S. Wood and Juan C. Judikis, Conversations on Community Theory (West Lafayette, IN: Purdue University Press, 2002), 12-17. As well, the commitment to civil inquiry - "overcoming the fragmentation of consciousness without illegitimately distorting or suppressing any of its modes" (24) - is taken from Glenn Tinder, Community: Reflections on a Tragic Ideal (Baton Rouge, LA: LSU Press, 1980).
37. In assessing the portfolios of these senior-level students (n = 80), we found that the scores on critical thinking (M = 7.82, SD = 1.55) and drafting (M = 7.08, SD = 2.13) met the cut score of 7. (That is, as two readers independently award a score from 6 (high) to 1 (low), a score on any variable below 7 suggests below average work and is cause for concern.) Indeed, the overall portfolio score (the dependent variable), a holistically oriented reading by the instructors, was also acceptable (M = 8.10, SD = 1.70). The citation variable, however, received scores that were unacceptably low (M = 6.37, SD = 2.32).
38. Elliot, Briller & Joshi, "Portfolio Assessment: Quantification and Community" 5-30.
39. 1), a two-tailed p value was used for the later measure.
40. Pamela Moss, "Can There Be Validity without Reliability?" Educational Researcher 23.2 (1994): 5-12; See also Michael Williamson, "The Worship of Efficiency: Untangling Theoretical and Practical Considerations in Writing Assessment," Assessing Writing 1 (1994): 147-173.
41. Wang R. "The Lasting Impact of a Library Credit Course". portal: Libraries and the Academy 6 (January 2006) 79-92
42. Holliday W., and Fagerheim B. "Integrating Information Literacy with a Sequenced English Composition Curriculum". portal: Libraries and the Academy 6 (April 2006) 169-184
43. ACRL. "Research Agenda for Library Instruction and Information Literacy". College and Research Libraries News 64 (March 2003) 108-113
44. Tushman M.L., and Scanlan T.J. "Boundary Spanning Individuals: Their Role in Information Transfer and their Antecedents". The Academy of Management Journal 24 (June 1981) 289-305