The question of education science: experimentism versus experimentalism

Educational Theory

Volumn 55, Issue 3, 2005, Pages 307-321

  Howe, Kenneth R ^a  

^a UNIVERSITY OF COLORADO   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 33745647490     PISSN: 00132004     EISSN: 17415446     Source Type: Journal    
DOI: None     Document Type: Article

References (47)

1. It is more than a little ironic that, as the renewed push for scientific educational research has gained momentum, its subordination to partisan political ends has reached an all-time high. See Thomas Schwandt's "A Diagnostic Reading of Scientifically Based Research for Education," included in this symposium. Also see Ernest House, "Bush's Neo-fundamentalism and the New Politics of Evaluation," Studies in Educational Policy and Educational Philosophy 2 (2003), available at http://www.upi.artisan.se; and Kenneth Howe, "Free Market Free for All," EducationWeek 21, no. 30 (2002): 32, 34-35.
2. These critiques have almost certainly contributed to backlash and retrenchment on the part of experimentists, which is illustrated by Robert Boruch and Thomas Cook's gratuitous swipes at anyone who would criticize experimentism.
3. See, for example, Robert Boruch, "The Virtues of Randomness," Education Next 2, no. 3 (2002): 36-42;
4. and Thomas Cook, "Randomized Experiments in Educational Policy Research: A Critical Examination of the Reasons the Educational Evaluation Community Has Offered for Not Doing Them," Educational Evaluation and Policy Analysis 24, no. 3 (2002): 175-199. A further illustration is the posturing toward postmodernism by the National Research Council's Scientific Research in Education (Washington, D.C.: National Academies Press, 2002) discussed in Pamela Moss, "Understanding the Other/Understanding Ourselves: Toward a Constructive Dialogue about 'Principles' in Educational Research," in this symposium. The NRC report will be cited as SRE in the text for all subsequent references.
5. For an exception, see Gert J.J. Biesta and Nicholas C. Burbules, Pragmatism and Educational Research (Lanham, Maryland: Rowman and Littlefield, 2004).
6. Donald Campbell and Julian Stanley, Experimental and Quasi-experimental Designs for Research (Chicago: Rand McNally, 1963), 2 (emphasis added).
7. Ibid., 7.
8. Coalition for Evidence-Based Policy, Bringing Evidence-Driven Progress to Education: A Recommended Strategy for the U.S. Department of Education (Washington, D.C.: U.S. Department of Education, 2002).
9. It should be noted, however, that SRE does not defend educational research either - a decision that arguably encourages the view that the criticisms are well taken.
10. I unpack this point in some detail in Kenneth Howe, "A Critique of Experimentalism," Qualitative Inquiry 10, no. 1 (2004): 42-61. (Note that in that context I used "experimentalism" as I use "experimentism" in this essay.)
11. The notion that educational contexts should not be simplified to facilitate experimentation does not go without saying. For example Cook's article "Randomized Experiments in Educational Policy Research" can be read as suggesting just that.
12. I criticize the use of the analogy between educational and medical research in Howe, "A Critique of Experimentalism." SRE does not depend as heavily on it as other experimentist documents, especially the Coalition for Evidence-Based Policy's report Bringing Evidence-Driven Progress to Education.
13. For a discussion of Peterson's school voucher experiments, see, for example, the Economic Policy Institute, A Conversation on School Vouchers (Washington, D.C.: Government Printing Office, 2003).
14. Lee J. Cronbach, "Beyond the Two Disciplines of Scientific Psychology," American Psychologist 30, no. 2 (1975): 116-127.
15. For a sustained critique of experiments, see Lee J. Cronbach, Designing Evaluations of Social and Educational Programs (San Francisco: Jossey-Bass, 1982).
16. Indeed, this understanding of the importance of changing social contexts seems consistent with SRE's own analysis in chapter 4, but it is not consistent with the document's overall commitment to a cumulative education science.
17. By "interpretivism" I mean a broad methodological stance that incorporates the "insider's" or "intentional" perspective as fundamental to social research. For more on this point, see Kenneth Howe, "The Interpretive Turn and the New Debate in Education," Educational Researcher 27, no. 8 (1998): 13-20.
18. John Searle, Minds, Brains and Science (Cambridge: Harvard University Press, 1984), 64.
19. Georg Henrik vonWright, Explanation and Understanding (Ithaca: Cornell University Press, 1971), 85
20. My use of the notion of the "interpretive turn" derives from Paul Rabinow and William Sullivan, "The Interpretive Turn: Emergence of an Approach," in their edited volume, Interpretive Social Science (Los Angeles: University of California Press, 1979), 1-21.
21. Dorothy C. Holland and Margaret A. Eisenhart, Educated in Romance: Women, Achievement and College Culture (Chicago: University of Chicago Press, 1990).
22. I take educational research to be a species of social research, here and throughout.
23. In the same vein, Pamela Moss criticizes SRE for missing the point of hermeneutical alternatives and forcing them into the language of SRE. See Moss, "Understanding the Other/Understanding Ourselves," in this symposium.
24. John Dewey, "Search for the Great Community," in The Philosophy of John Dewey, ed. John J. McDermott (Chicago: University of Chicago Press, 1973), 620-642.
25. Educational philosophers, for example, can presumably do this sort of thing, but then their work is excluded from the purview of " scientific" research as construed by SRE.
26. Michael Root, Philosophy of Social Science (Cambridge, Massachusetts: Blackwell, 1993), 101.
27. In terms of Root's policy syllogism, scientific research in education is concerned with premise 1, "A causes B," and other forms of scholarship are concerned with premise 2, "B is desirable."
28. The idea of "value-minimalism" - arguably, the dominant contemporary view of the role assigned to values in evaluation and social research - is elaborated and criticized in Ernest R. House and Kenneth R. Howe, Values in Evaluation and Social Research (Thousand Oaks, California: Sage, 1999).
29. For an extended discussion of the evaluative dimension of the vocabulary of social research, see Kenneth Howe, Closing Methodological Divides: Toward Democratic Educational Research (Dordrecht, Netherlands: Kluwer, 2003).
30. Root, Philosophy of Social Science, 114.
31. Experimentism's methodological strictures lead it to pose "what works" questions in a way that narrows the field of prospective causal factors, which, in turn, leads to its political conservatism. The problem is not with the question of "what works" per se. "What works" questions can be posed more expansively, in a way that keeps the field of prospective causal factors open to socioeconomic structures. Consider, for example, how the "what works" question would have to be posed if the objective was to eliminate Jonathan Kozol's "savage inequalities." The currently popular reforms of test-based accountability and school choice, which do nothing to address structural conditions, would be rightly rejected as ludicrous answers.
32. I do not mean political in a "partisan" sense. It is very difficult to use the term "political" these days without it being construed as participating in an emotivist moral-political framework, whereby it has to be partisan, that is, without rational justification. I will say more on this point subsequently.
33. Larry Laudan, Beyond Positivism and Relativism (Boulder, Colorado: Westview Press, 1996), 141
34. Fernand Braudel, quoted in Stephen Toulmin, Return to Reason (Cambridge, Massachusetts: Harvard University Press, 2001), viii.
35. Toulmin, Return to Reason.
36. Ibid., 100.
37. See Schwandt, "A Diagnostic Reading of Scientifically Based Research for Education."
38. Hilary Putnam, The Collapse of the Fact/Value Dichotomy (Cambridge, Massachusetts: Harvard University Press, 2002), 110.
39. Ibid., 103.
40. Dewey, "Search for the Great Community," 637. Although Dewey's referent was natural science, his observation applies with at least equal force to social science.
41. I do not mean to suggest that there is no place for methodological expertise. For more on this issue, see House and Howe, Values in Evaluation and Social Research, and Howe, Closing Methodological Divides.
42. In "A Critique of Experimentalism," I sketch an overarching methodological approach that I call "mixed-methods interpretivism." It is an instance of what I call "experimentalism" in this article.
43. John Dewey, "The Problem of Freedom," in The Philosophy of John Dewey, ed. McDermott, 675
44. For such examples, see the section titled "Science - As You Like It" in Schwandt, "A Diagnostic Reading of Scientifically Based Research for Education."
45. The New York Times ran a full-page ad signed by various long-time school choice boosters and sponsored by the Center for Educational Reform, a staunchly partisan school choice advocacy group. The National Charter School Clearinghouse (NCSC), closely associated with the U.S. Department of Education, launched a letter campaign against the AFT study.
46. Indeed, I surmise that many experimentists see themselves as liberal, not conservative, and behave accordingly in the policies they support, who they vote for, and so on. In this connection, political liberals may engage in research rooted in conservative politics for strategicreasons - for damage control, to get alternative views on the table, and the like. For those who hope to influence policy, this seems justifiable (and also ethically treacherous, of course). But this kind of compromise should be seen (and felt) for what it is. Educational researchers cannot get off the hook by adopting the stance, encouraged by SRE, that they can keep their politics and their science separate. For a further discussion of this issue, see Howe, Closing Methodological Divides.
47. See, for example, Boruch, "The Virtues of Randomness," and Cook, "Randomized Experiments in Educational Policy Research."