Treatment of nuisance parameters in high energy physics, and possible justifications and improvements in the statistics literature

Statistical Problems in Particle Physics, Astrophysics and Cosmology - Proceedings of PHYSTAT 2005

Volumn , Issue , 2006, Pages 75-85

  Cousins, Robert D ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Nuisance parameters; Systematic uncertainties

Indexed keywords

COSMOLOGY; STATISTICAL PHYSICS;

HIGHER-DIMENSIONAL PROBLEMS; MEASUREMENTS OF; NUISANCE PARAMETER; SYSTEMATIC UNCERTAINTIES;

HIGH ENERGY PHYSICS;

EID: 84894206361     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1142/9781860948985_0017     Document Type: Conference Paper

References (82)

1. S.S. Wilks, "The large-sample distribution of the likelihood ratio for testing composite hypotheses", Annals of Math. Stat. 9 (1938) 60.
2. F. James, "MINUIT. Function Minimization and Error Analysis, " www.asdoc.web.cern.ch/wwwasdoc/minuit/minmain.html.
3. J.D. Kalbfleisch and J.D. Sprott, "Application of likelihood methods to models involving large numbers of parameters, " Jour. Roy. Stat. Soc. Series B 32, 175 (1970).
4. D. Basu "On the Elimination of Nuisance Parameters" JASA 72, 355 (1977).
5. F. James and M. Roos, "Errors on Ratios of Small Numbers of Events", Nuclear Physics B172 475 (1980).
6. F. James, "Interpretation of the shape of the likelihood function around its minimum, " Comput. Phys. Commun. 20 29 (1980).
7. O. Barndorff-Nielsen "On a Formula for the Distribution of the Maximum Likelihood Estimates" Biometrika 70, 343 (1983).
8. O. Barndorff-Nielsen "Inference on Full or Partial Parameters Based on the Standardized Signed Log Likelihood Ratio" Biometrika 73, 307 (1986).
9. D.R. Cox, N. Reid "Parameter Orthogonality and Approximate Conditional Inference", Jour. Roy. Stat. Soc. Series B 49, 1 (1987).
10. H. B. Prosper, "Small Signal Analysis In High-Energy Physics: A Bayesian Approach, " Phys. Rev. D 37, 1153 (1988);.
11. D. A. Williams, "Comment On 'Small Signal Analysis In High-Energy Physics: A Bayesian Approach', " Phys. Rev. D 38, 3582 (1988), and reply.
12. D.A.S. Fraser, N. Reid, "Adjustments to Profile Likelihood" Biometrika 76, 477 (1989).
13. G. Zech, "Upper Limits In Experiments With Background Or Measurement Errors, " Nucl. Instr. and Meth. A277 608 (1989).
14. P. McCullagh, R. Tibshirani, "A Simple Method for the Adjustment of Profile Likelihoods" Jour. Roy. Stat. Soc. Series B 52, 325 (1990).
15. B. Liseo "Elimination of Nuisance Parameters with Reference Priors" Biometrika 80, 295 (1993).
16. D.R. Cox, N. Reid, "A Note on the Difference Between Profile and Modified Profile Likelihood" Biometrika 79, 408 (1992).
17. R.D. Cousins and V.L. Highland, "Incorporating systematic uncertainties into an upper limit, " Nucl. Instrum. Meth. A 320, 331 (1992).
18. B. Liseo "Elimination of Nuisance Parameters with Reference Priors" Biometrika 80, 295 (1993);.
19. J.O. Berger, L.D. Brown, and R.L. Wolpert, "A unified conditional frequentist and Bayesian test for fixed and sequential simple hypothesis-testing", Ann. Stat. 22 1787 (1994).
20. V. Innocente and L. Lista, "Evaluation of the upper limit to rare processes in the presence of background, and comparison between the Bayesian and classical approaches, " Nucl. Instrum. Meth. A 340, 396 (1994).
21. N. Reid, "The Roles of Conditioning in Inference" Stat. Sci. 10, 138 (1995). A more recent, very concise summary of likelihood-based inference is in Ref. 38.
22. R. D. Cousins, "Why isn't every physicist a Bayesian?, " Am. J. Phys. 63, 398 (1995).
23. A.L. Read, "Modified Frequentist Analysis of Search Results (The CLS Method)", Workshop on Confidence Limits, CERN (2000). doc.cern.ch/ yellowrep/2000/2000-005/p81.pdf;.
24. A.L. Read, "Optimal statistical analysis of search results based on the likelihood ratio and its application to the search for the MSM Higgs boson at sqrt(s) =161 and 172 GeV", DELPHI collaboration note, 97-158 PHYS 737 (1997).
25. V. Highland, Nucl. Instr. and Meth. A398 429 (1997), followed by reply by G. Zech.
26. P. C. Bhat, H. B. Prosper and S. S. Snyder, "Bayesian analysis of multi-source data, " Phys. Lett. B 407, 73 (1997).
27. G. J. Feldman and R. D. Cousins, "A Unified approach to the classical statistical analysis of small signals, " Phys. Rev. D 57, 3873 (1998).
28. R. D. Cousins, "Improved central confidence intervals for the ratio of Poisson means, " Nucl. Instrum. Meth. A 417, 391 (1998).
29. J.O. Berger, B. Liseo, R.L. Wolpert "Integrated Likelihood Methods for Eliminating Nuisance Parameters" Stat. Sci. 14, 1 (1999).
30. B. P. Roe and M. B. Woodroofe, "Improved probability method for estimating signal in the presence of background, " Phys. Rev. D 60, 053009 (1999).
31. T. Junk, "Confidence level computation for combining searches with small statistics", Nucl. Instr. and Meth. A434 (1999) 435.
32. thed., (London:Arnold, 1999), and earlier editions by Kendall and Stuart.
33. T.A. Severini, "On the Relationship Between Bayesian and Non-Bayesian Elimination of Nuisance Parameters" Statistica Sinica 9, 713 (1999).
34. J. Conway, "Inclusion of systematic uncertainties in upper limits and hypothesis tests, " Workshop on Confidence Limits, CERN (2000).
35. R. Cousins, "Comments on methods for setting confidence limits, " Workshop on Confidence Limits, CERN (2000).
36. G. Zech, "Confronting classical and Bayesian confidence limits to examples, " Workshop on Confidence Limits, CERN (2000). arXiv:hep-ex/0004011.
37. www-zeus.romal.infn.it/agostini/prob+stat.html.
38. G. Feldman, "Multiple measurements and parameters in the unified approach, " Workshop on Confidence Limits, Fermilab (2000), conferences.fnal.gov/cl2k/copies/feldman2.pdf.
39. R.D. Cousins, "Additional comments on methods for setting confidence limits", Workshop on Confidence Limits, Fermilab (2000), conferences.fnal.gov/cl2k/copies/bcousins2.ps.
40. N. Reid "Likelihood" J. Am. Stat. Assoc. 95, 1335 (2000).
41. R. D. Cousins, "Comment on [Improved probability method for estimating signal in the presence of background], " Phys. Rev. D 62 (2000) 098301.
42. W. A. Rolke and A. M. Lopez, "Confidence intervals and upper bounds for small signals in the presence of background noise, " Nucl. Instrum. Meth. A 458, 745 (2001), arXiv:hep-ph/0005187.
43. B. P. Roe and M. B. Woodroofe, "Setting confidence belts, " Phys. Rev. D 63, 013009 (2001).
44. L.D. Brown, T.T. Cai, A. DasGupta "Interval Estimation for a Binomial Proportion" Stat. Sci. 16, 101 (2001).
45. R. Barlow, "A calculator for confidence intervals, " Comput. Phys. Commun. 149, 97 (2002), arXiv:hep-ex/0203002.
46. D. Nicolo, G. Signorelli, "An application of the strong confidence to the Chooz experiment with frequentist inclusion of systematics, " Conference on Advanced Statistical Techniques in Particle Physics, Durham, England (2002), www.ippp.dur.ac.uk/Workshops/02/statistics/.
47. L. Demortier, "Bayesian treatments of systematic uncertainties, " Conference on Advanced Statistical Techniques in Particle Physics, Durham, England (2002), www.ippp.dur.ac.uk/Workshops/02/statistics/.
48. A.L. Read, "Presentation of Search Results - The CLS Technique, " Conference on Advanced Statistical Techniques in Particle Physics, Durham, England (2002), www.ippp.dur.ac.uk/Workshops/02/statistics/.
49. G. Zech, "Frequentist and Bayesian confidence limits, " Eur. Phys. J. C 4, 12 (2002), arXiv:hep-ex/0106023.
50. K. S. Cranmer, "Frequentist hypothesis testing with background uncertainty, " PHYSTAT2003, SLAC (2003). arXiv:physics/0310108.
51. N. Reid, D.A.S. Eraser, "Likelihood inference in the presence of Nuisance parameters, " PHYSTAT2003, SLAC (2003). arXiv:physics/0312079.
52. L. Demortier, "Constructing ensembles of pseudo-experiments, " PHYSTAT2003, SLAC (2003) arXiv:physics/0312100.
53. J. Linnemann, "Measures of significance in HEP and astrophysics, " PHYSTAT2003, SLAC (2003) arXiv: physics/0312059.
54. P. Sinervo, "Definition and treatment of systematic uncertainties in high energy physics and astrophysics, " PHYSTAT2003, SLAC (2003).
55. G. Punzi, "Including systematic uncertainties in confidence limits, " CDF Statistics note (2003).
56. J. Conrad, O. Botner, A. Hallgren and C. Perez de los Heros, "Including systematic uncertainties in confidence interval construction for Poisson statistics, " Phys. Rev. D 67, 012002 (2003), arXiv:hep-ex/ 0202013.
57. J. Conrad, O. Botner, A. Hallgren and C. P. de los Heros, "Coverage of confidence intervals for Poisson statistics in presence of systematic uncertainties, " arXiv:hep-ex/0206034.
58. G. C. Hill, "Comment on 'Including systematic uncertainties in confidence interval construction for Poisson statistics', " Phys. Rev. D 67, 118101 (2003), arXiv:physics/0302057.
59. J.O. Berger, "Could Fisher, Jeffreys and Neyman Have Agreed on Testing?" Stat. Sci. 18, 1 (2003).
60. S.C. Dass and J.O. Berger, "Unified Conditional Frequentist and Bayesian Testing of Composite Hypotheses", Scand. J. Statist. 30 199 (2003).
61. J. Heinrich, C. Blocker, J. Conway, L. Demortier, L. Lyons, G. Punzi, and P. K. Sinervo, "Interval estimation in the presence of nuisance parameters. 1. Bayesian approach, " CDF/MEMO/STATISTICS/PUBLIC/7117 (2004), arXiv:physics/0409129.
62. L. Demortier, "A fully Bayesian computation of upper limits for Poisson processes", CDF/MEMO/STATISTICS/PUBLIC/5928 (2004).
63. D. A. S. Fraser, N. Reid and A. C. M. Wong, "Inference for bounded parameters, " Phys. Rev. D 69, 033002 (2004).
64. R. Barlow, "A note on AlnL = - 1/2 Errors, " arXiv:physics/0403046.
65. M.J. Bayarri, J.O. Berger "The Interplay of Bayesian and Frequentist Analysis" Stat. Sci. 19, 58 (2004).
66. D.A.S. Fraser, "Ancillaries and Conditional Inference" Stat. Sci. 19, 333 (2004);.
67. T.J. diCiccio and M.E. Thompson, "A Conversation with Donald A.S. Fraser, " Stat. Sci. 19, 370 (2004).
68. J. Conrad, "A program for confidence interval calculations for a Poisson process with background including systematic uncertainties: POLE 1.0, " Comput. Phys. Commun. 158, 117 (2004).
69. L. Lista, "Including Gaussian uncertainty on the background estimate for upper limit calculations using Poissonian sampling, " Nucl. Instrum. Meth. A 517, 360 (2004).
70. B. Liseo, "The Elimination of Nuisance Parameters", geostasto.eco.uniromal.it/utenti/liseo/pub.htm (2004).
71. W. A. Rolke, A. M. Lopez and J. Conrad, "Limits and Confidence Intervals in the Presence of Nuisance Parameters, " arXiv:physics/0403059 (v4, 7 Jul 2005).
72. C. Blocker, "Interval Estimation in the Presence of Nuisance Parameters. 2. Cousins and Highland Method, " CDF/MEMO/STATISTICS/PUBLIC/ 7539 (2005).
73. J. Conway, "Calculation of cross section upper limits combining channels incorporating correlated and uncorrelated systematic uncertainties", CDF/PUB/STATISTICS/PUBLIC/6428 (2005).
74. F. Tegenfeldt and J. Conrad, "On Bayesian treatment of systematic uncertainties in confidence interval calculations, " Nucl. Instrum. Meth. A 539, 407 (2005), arXiv:physics/0408039.
75. N. Reid, respondent to this paper, these proceedings.
76. K. Cranmer, "Statistical Challenges of the LHC", these proceedings.
77. L. Demortier, "Bayesian Reference Analysis", these proceedings.
78. G. Feldman, "Concluding Talk", these proceedings.
79. J. Heinrich, "The Bayesian approach to setting limits: what to avoid", these proceedings.
80. G. Punzi, "Ordering algorithms and Confidence Intervals in the presence of nuisance parameters", these proceedings.
81. DO collaboration, www-d0.fnal.gov.
82. CDF collaboration, www-cdf.fnal.gov.