Likelihood, model selection, and the Duhem-Quine problem

Journal of Philosophy

Volumn 101, Issue 5, 2004, Pages 221-241

  Sober, Elliott ^a,b  

^a STANFORD UNIVERSITY   (United States)

^b UNIVERSITY OF WISCONSIN MADISON   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 20744432982     PISSN: 0022362X     EISSN: 19398549     Source Type: Journal    
DOI: 10.5840/jphil2004101515     Document Type: Article

References (45)

1. The Aim and Structure of Physical Theory (Princeton: University Press, 1954).
2. "Two Dogmas of Empiricism," in From a Logical Point of View (Cambridge: Harvard, 1953), pp. 20-46
3. and Philosophy of Logic (Englewood Cliffs, NJ: Prentice-Hall, 1970).
4. Distributive holists may assert that the effects on H and A are qualitatively the same (that is, that both are confirmed or both are disconfirmed) or, more ambitiously, that the effects are quantitatively the same (that is, that the degree of confirmation of H is identical with the degree of confirmation of A). For more on this taxonomy of holisms, see my "Quine's Two Dogmas," Proceedings of the Aristotelian Society, LXXIV (2000): 237-80.
5. I do not concede that simplicity is always an extra-evidential consideration; the point here is that this is what holists happen to believe. For discussion, see my Reconstructing the Past: Parsimony, Evolution, and Inference (Cambridge: MIT, 1988)
6. and my "Instrumentalism, Parsimony, and the Akaike Framework," Philosophy of Science, LXIX (2002): S112-S123.
7. See, for example, J. Dorling, "Bayesian Personalism, the Methodology of Scientific Research Programs, and Duhem's Problem," Studies in the History and Philosophy of Science, X (1979): 177-87
8. Colin Howson and Peter Urbach, Scientific Reasoning: The Bayesian Approach, (La Salle, IL: Open Court, 1989)
9. John Earman, Bayes or Bust? (Cambridge: MIT, 1992)
10. and Michael Strevens, "A Bayesian Treatment of Auxiliary Hypotheses," British Journal for the Philosophy of Science, LII (2001): 515-37.
11. For a nonBayesian treatment, see Deborah Mayo's Error and the Growth of Experimental Knowledge (Chicago: University Press, 1996), which analyzes the Duhem-Quine problem within the context of frequentist statistics.
12. As a simple example of a Bayesian analysis, let us define the degree of confirmation that X receives from Y, c[X,Y], as the ratio Pr(X|Y)/Pr(X). So defined, c[X,Y] > 1 when Y positively confirms X and c[X,Y] < 1 when Y disconfirms X By Bayes's theorem, this ratio equals Pr(Y|X)/Pr(Y); thus, c[H, notO] > c[A, notO] if and only if Pr(notO|H) > Pr(notO|A), which expands to Pr(notO|H & A)Pr(A |H) + Pr(notO| H & notA)Pr(notA |H) > Pr(notO |H & A)Pr(H |A) + Pr(notO| notH & A)Pr(notH |A). Note the occurrence of Pr(A|H) and of Pr(H|A) in this expression. If we assume that H and A are probabilistically independent, the inequality reduces to Pr(notO |H & A)Pr(A) + Pr(notO| H & notA)Pr(notA) > Pr(notO |H & A)Pr(H) + Pr(notO| notH & A)Pr(notH). Notice the prior probabilities of A and of H. I do not mean to beg questions here about the proper definition of degree of confirmation, on which see Branden Fitelson, "The Plurality of Bayesian Measures of Confirmation and the Problem of Measure Sensitivity," Philosophy of Science, LXVI (1999): S362-S378. The point is just to identify the kinds of quantities that a Bayesian analysis must evaluate. Another Bayesian approach would be to compare, not the change in probability that O induces in H with the change it induces in A, but the absolute values of Pr(H|O) and Pr(A|O). Bayes's Theorem entails that Pr(H|O) > Pr(A|O) if and only if Pr(O|H)Pr(H) > Pr(O|A)Pr(A). Prior probabilities occur in this expression, and Pr(H|A) and Pr(A|H) are involved as well, since Pr(O|H) = Pr(O| H & A)Pr(A|H) + Pr(O | H & notA)Pr(notA | H) and Pr(O|A) = Pr(O | H & A)Pr(H|A) + Pr(O | notH & A)Pr(notH|A).
13. "Information Theory as an Extension of the Maximum Likelihood Principle," in B. Petrov and F. Csaki, eds., Second International Symposium on Information Theory (Budapest: Akademiai Kiado, 1973), pp. 267-81.
14. Here I adopt terminology from Hartry Field, who talks about world-to-head and head-to-world reliability. See his "'Narrow' Aspects of Intentionality and the Information-theoretic Approach to Content," in Enrique Villanueva, ed., Information, Semantics, and Epistemology (New York: Blackwell, 1990), pp. 102-16.
15. If our interest in using tuberculosis tests is to find out whether someone probably has tuberculosis, why are error characteristics defined in tenus of probabilities of the form Pr(± test outcome | ± tuberculosis), rather than in terms of probabilities of the form Pr(± tuberculosis | ± test outcome)? The reason is that the latter quantities depend on how rare or common tuberculosis is, but the former do not. It is a curious fact about our universe that probabilities of the form Pr(effect|cause) are often time-translationally invariant, whereas probabilities of the form Pr(cause| effect) rarely are. For discussion, see my "Temporally Oriented Laws," Synthese, XCIV (1993): 171-89.
16. See Ian Hacking, The Logic of Statistical Inference (New York: Cambridge, 1965);
17. Anthony Edwards, Likelihood (New York: Cambridge, 1972);
18. and Richard Royall, Statistical Evidence: A Likelihood Paradigm (Boca Raton, FL: Chapman and Hall, 1997).
19. See Philip Kitcher's The Advancement of Science: Science without legend, Objectivity without Illusions (New York: Oxford, 1993), pp. 228-33
20. and his "Real Realism: The Galilean Strategy," Philosophical Review, CX (2001): 151-98.
21. See Richard W. Miller, Fact and Method (Princeton: University Press, 1999)
22. and my "Bayesianism: Its Scope and Limits," in Richard Swinburne, ed., Bayes's Theorem (New York: Oxford, 2002), pp. 21-38.
23. See Isaac Levi and Sidney Morgenbesser, "Belief and Disposition," American Philosophical Quarterly, I (1964): 221-32
24. and Levi, Gambling with Truth (New York: Knopf, 1967).
25. W.M. Smart, "John Crouch Adams and the Discovery of Neptune," Occasional Notes of the Royal Astronomical Society, XI (1947): 33-88.
26. Adams and Le Verrier were able to accommodate the observed orbit of Uranus within the Newtonian framework by postulating an eighth planet, but did Uranus's orbit confirm the hypothesis that there is an eighth planet? It is tempting to answer this question in the negative and to insist that it was only the observation of Neptune that provided evidence. The broad epistemological question at issue here is whether using an observation to construct a hypothesis means that the observation fails to provide evidence for the hypothesis. See Christopher Hitchcock and Elliott Sober, "Prediction, Accommodation, and the Problem of Overfitting," British Journal for the Philosophy of Science, LV (2004): 1-34, for discussion.
27. See my "Indispensability and Mathematics," Philosophical Review, CII (1993): 35-57.
28. Whereas the Adams/Le Verrier approach to the orbit of Uranus involved "asymmetry by default," this was not the case with respect to later discussion of the anomalous perihelion of Mercury, in that modifications of the auxiliary assumptions and of Newtonian theory were both developed. See N. Rosevcare, Mercury's Perihelion from Le Verrier to Einstein (New York: Oxford, 1982)
29. and John Earman and Michel Janssen, "Einstein's Explanation of the Motion of Mercury's Perihelion," in Earman, Janssen, and John D. Norton, eds., The Attraction of Gravitation: New Studies in the History of General Relativity (Boston: Birkhäuser, 1993), pp. 129-72.
30. See Hans Reichenbach, Experience and Prediction (Chicago: University Press, 1938).
31. See M. Forster, "Bayes and Bust: Simplicity as a Problem for a Probabilist's Approach to Confirmation," British Journal for the Philosophy of Science, XLVI (1995): 399-429
32. and my "Bayesianism: Its Scope and Limits" (op. cit.).
33. See also Y. Sakamoto, M. Ishiguro, and G. Kitagawa, Akaike Information Criterion Statistics (NewYork: Springer, 1986);
34. and Kenneth P. Burnham and David R. Anderson, Model Selection and Inference: A Practical Information-Theoretic Approach (New York: Springer, 1998).
35. See Forster and Sober, "How to Tell When Simpler, More Unified, or Less Ad Hoc Theories Will Provide More Accurate Predictions," British Journal for the Philosophy of Science, XLV (1994): 1-36.
36. Does AIC play into the hands of holism by invoking simplicity? I would say not, in that the justification for AIC depends on empirical assumptions (though ones of great generality). In addition, I will be using AIC only as an example of a model selection criterion; cross-validation is another such criterion, and it involves no appeal to simplicity. As it happens, take-one-out cross-validation is asymptotically equivalent with AIC. See M. Stone's "Cross-validatory Choice and Assessment of Statistical Predictions (with Discussion)," Journal of the Royal Statistical Society, B XXXVI (1974): 111-47
37. and his "An Asymptotic Equivalence of Choice of Model by Cross-validation and Akaike's Criterion," Journal of the Royal Statistical Society, B XXXIX (1977): 44-47.
38. See my "The Contest between Likelihood and Parsimony," Systematic Biology (2004, forthcoming).
39. "Evolution of Protein Molecules," in H. Munro, ed., Mammalian Protein Metabolism (New York: Academic, 1969), pp. 21-132.
40. "A Simple Method for Estimating Evolutionary Rates of Base Substitutions through Comparative Studies of Nucleotide Sequences,"Journal of Molecular Evolution, XVI (1980): 111-20.
41. For a survey of the different models now on offer, see Roderic Page and Edward Holmes, Molecular Evolution: Phylogenetic Approach (Maiden, MA: Blackwell, 1998), pp. 148-62.
42. Philosophy of Logic, p. 7;
43. see also "Two Dogmas of Empiricism," p. 44.
44. The Philosophy of Space and Time (New York: Dover, 1958).
45. "Two Dogmas of Empiricism," p. 43.