The politics of measurable precision: The emergence of sampling techniques in Canada's dominion bureau of statistics

Canadian Historical Review

Volumn 79, Issue 4, 1998, Pages 691-725

  Beaud, Jean Pierre ^a   Prévost, Jean Guy ^a  

^a NONE

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0007346380     PISSN: 00083755     EISSN: 17101093     Source Type: Journal    
DOI: 10.3138/CHR.79.4.691     Document Type: Review

References (102)

1. See, for instance, J.W. Duncan and W.C. Shelton, Revolution in United States Government Statistics, 1926-1976 (Washington: Department of Commerce 1978), where probability sampling and its applications are described as the first component of the statistical revolution in the United States
2. Jerzy Neyman, 'On the Two Different Aspects of the Representative Method: The Method of Stratified Sampling and the Method of Purposive Selection,' Journal of the Royal Statistical Society 97 (1934): 558-625
3. German statistik, the reports of French intendants in the seventeenth and eighteenth centuries, and, in Canada, documents such as Robert Gourlay's Statistical Account of Upper Canada (London: Simpkin and Marshall 1822)
4. and Joseph Bouchette's Topographical Description of the Provinces of Upper and Lower Canada (London: Longman 1832; an earlier version restricted to Lower Canada had been published in 1815) exemplify the transition from one type of knowledge to the other
5. For a general overview, see Alain Desrosières, La politique des grands nombres (Paris: La Découverte 1993)
6. Theodore M. Porter, The Rise of Statistical Thinking (Princeton: Princeton University Press 1986)
7. and Trust in Numbers (Princeton: Princeton University Press 1995)
8. and Ian Hacking, The Taming of Chance (Cambridge: Cambridge University Press 1990)
9. Bruce Curtis, 'The Canada "Blue Books" and the Administrative Capacity of the Canadian State, 1822-67,' Canadian Historical Review 74, 4 (1993): 537
10. George Emery also adopts the Foucauldian power/knowledge paradigm in his Facts of Life: The Social Construction of Vital Statistics, Ontario, 1869-1952 (Montreal and Kingston: McGill-Queen's University Press 1993)
11. Coats's quotation comes from 'Dalhousie and Progress,' Dalhousie Review 17 (1937): 160
12. On the establishment of the DBS and the centralist crusade waged by Coats, see Jean-Pierre Beaud and Jean-Guy Prévost, 'La structuration de l'appareil statistique canadien, 1912-21,' Canadian Historical Review 74, 3 (1993): 395-413
13. Fernand Faure, 'The Development and Progress of Statistics in France,' in Joseph Koren, The Development and Progress of Statistics in Many Countries (1918; New York: Bart Franklin 1970), 219
14. Seng describes William Farr's 1861 analysis of fourteen subdistricts within the data collected during the census as an example of those 'so-called sample investigations.' You Poh Seng, 'Historical Survey of the Development of Sampling Theories and Practice,' Journal of the Royal Statistical Society, series A 114 (1951): 214-231
15. Neyman, 'On the Two Different Aspects of the Representative Method,' 568
16. Morris H. Hansen and William G. Madow, 'Some Important Events in the Historical Development of Sample Surveys,' in D.B. Owen, On the History of Statistics and Probability (New York: Dekker 1976), list seven major contributions they impute to Neyman's 1934 paper
17. See, notably, S.E. Fienberg and J.M. Tanur, 'A Historical Perspective on the Institutional Bases for Survey Research in the United States,' Survey Methodology 16 (1990): 31-46, and the subsequent comments by R.M. Groves, who criticizes them for not laying enough stress on the heroic role played by individuals B what he calls 'the human drama' (47-50)
18. On the rift between the proponents of quota samples and statisticians of the Census Bureau, especially Hansen, who were critical of its non-mathematical character, see Jean Converse, Survey Research in the United States: Roots and Emergence (Berkeley: University of California Press 1987), 230-1
19. T. Dalenius, Sampling in Sweden: Contributions to the Methods and Theories of Sample Survey Practice (Stockholm: Almqvist and Wicksell 1957), 34, acknowledges that, in Sweden, official statistics were admittedly influenced by the performance of the local Gallup Institute
20. Most of the literature downplays the role of polling and market research in the development of sampling in official statistics: either authors do not mention it at all (Seng, 'Historical Survey of the Development of Sampling Theories and Practice,'
21. and Frederick F. Stephan, 'History of the Uses of Modern Sampling Procedures,' Journal of the American Statistical Association 43 (1948): 12-39
22. Hansen and Madow, 'Some Important Events in the Historical Development of Sample Surveys')
23. or consider that probabilistic and quota techniques developed in parallel manner, without interaction (William Kruskal and Frederick Mosteller, 'Representative Sampling IV: The History of the Concept in Statistics, 1895-1939,'
24. International Statistical Review 48 (1980): 169-95
25. Philippe Tassi, 'De l'exhaustif au partiel: un peu d'histoire sur le développement des sondages,' Journal de la Société de statistique de Paris 129, 1-2 (1988): 116-32)
26. or, at best, concede that polls had an effect on acceptance by the lay public (Hansen, 'Some History and Reminiscences on Survey Sampling,' Statistical Science 2 [1987]: 184
27. Duncan and Shelton, Revolution in United States Government Statistics, 1926-1976, 41)
28. More imaginatively, Desrosières shows how nationwide electoral campaigns using radio, the development of extended consumer goods markets, and the rise of the welfare state all pertained to the same process of unification of the national territory (La politique des grands nombres, 271)
29. Duncan and Shelton offer the most extreme version of the internal account by reconstructing a genesis of sampling that goes from Gauss-Laplace to Pearson to Fisher to Neyman, and then to the American Bureau of the Census, as though the problem of representativity had never been encountered by official statisticians anywhere else in the world before the 1930s (Revolution in United States Government Statistics, 1926-1976, 32-5)
30. Kruskal and Mosteller, 'Representative Sampling IV.'
31. A quite different, and more satisfying, account can be found in Alain Desrosières, 'La partie pour le tout: comment généraliser? La préhistoire de la contrainte de représentativité,' in Jean Mairesse, dir., Estimations et sondages. Cinq contributions à l'histoire de la statistique (Paris: Economica 1988), 97-116
32. The behaviour of the participants in this debate must account for a part of this obscurity: Arthur Bowley, for instance, after having enthusiastically endorsed purposive selection and expended much effort trying to give it a sound mathematical basis, made a rapid and total conversion after hearing Neyman's paper, denying that he had placed much hope in the purposive method. Compare Bowley, 'Measurement of the Precision Attained in Sampling,' Bulletin de l'Institut international de statistique 22 (1926), supplement to 1re liv.: 1-62
33. Nathan Keyfitz and H.F. Greenway, 'Robert Coats and the Organization of Statistics,' Canadian Journal of Economics and Political Science 27 (1961): 320
34. Desrosières, La politique des grands nombres, 271
35. For instance, defending the accuracy of the 1931 Census of unemployment that was challenged by trade unions, M.C. Maclean, chief of census analysis, wrote: 'Census figures of unemployment ... are as satisfactory as other census or mass statistics, incidental inaccuracies disappearing in averages and trends.' National Archives of Canada (NA), Records of Statistics Canada, RG 31, vol.1418, file on unemployment, appendix VI, Unemployment in Canada B Some Fundamental Concepts, no date
36. R.H. Coats, 'Canadian Trade Statistics; Imports and Exports: What They Are; How to Use Them,' Journal of the Canadian Bankers Association 33 (1926): 309
37. A. Jensen, 'The Representative Method in Practice,' Bulletin de l'Institut international de statistique 22 (1926), 1re liv.: 427
38. Wright's letter is quoted by Kiaer in 'Sur les méthodes représentatives ou typologiques,' Bulletin de l'Institut international de statistique 13 (1903), 1re liv. I: 70
39. Kruskal and Mosteller, 'Representative Sampling IV,' 182
40. R. H. Coats, 'Enumeration and Sampling in the Field of the Census,' Journal of the American Statistical Association 26 (1931): 272
41. Coats qualified this by adding that the 60 per cent sample was 'large but probably non-representative,' implying that the 94 per cent was more representative. 'Enumeration and Sampling in the Field of the Census,' 40
42. S.A. Cudmore, in Inter-American Statistical Institute, Statistical Activities of the American Nations (Washington 1941), 150. Of course, this is exactly what should be considered as a biased sample
43. DBS, Wartime Activities of the Bureau of Statistics (Ottawa 1943), 7
44. Wartime Activities of the Bureau of Statistics, 3. Both the 1940 National Registration and the 1941 census were envisioned from the start as providing statistical universes from which samples could be drawn, in the sense that, given the wartime conditions and the need for rapid results, tabulating and analyzing from only a fraction of the data was imperative. That is how the inquiry on wages according to provinces mentioned by Cudmore seems to have been done. Few details survive about the exact sampling procedure followed in this case, but from the one used in the inquiry on housing (on which more detail will be given later), we can reasonably infer that it consisted of what we would today call systematic sampling B that is selecting every nth case of a series
45. See Maurice Halbwachs, La classe ouvrière et les niveaux de vie (Paris: Alcan 1912), especially the appendix
46. Herbert Brown Ames, The City below the Hill (1897; Toronto: University of Toronto Press 1972)
47. Phyllis Heaton, 'Standard of Living Studies and Their Significance, Including a Special Study in Montreal' (MA thesis, McGill University 1932), provides an assessment of these studies
48. Canada, House of Commons, Committee on Industrial and International Relations, Minutes of Proceedings and Evidence, 1926 Sessional Papers
49. quoted in Irving Abella and David Millar, The Canadian Worker in the Twentieth Century (Toronto: Oxford University Press 1970), 130
50. Statistics Canada, The Consumer Price Index. Revision based on 1974 Expenditures, Concepts and Procedures (Ottawa 1978), 8
51. On the Labour Gazette and its correspondents' network, see Paul Craven, 'An Impartial Umpire': Industrial Relations and the Canadian State 1900-1911 (Toronto: University of Toronto Press 1980)
52. Report of the Board of Inquiry into Cost of Living in Canada, vol.2: Rise in Prices and the Cost of Living, 1900-1914: A Statistical Examination of Economic Causes (Ottawa: King's Printer 1915), 72
53. Coats, 'Enumeration and Sampling in the Field of the Census,' 271-272
54. Maclean had prepared, at Coats's request, a dossier on the representative method, presenting data about fifty-three inquiries made in this manner. NA, Records of Statistics Canada, RG 31, box 17, file 1345 (Sampling), M.C. Maclean to R.H. Coats, 6 Oct. 1930
55. C. Gini, 'Une application de la méthode représentative aux matériaux du dernier recensement de la population italienne (1er décembre 1921),' Bulletin de l'Institut international de statistique 23 (1928), 2e liv.: 214
56. Referring to the most vocal of 'the old guard adherents of full coverage' that had opposed Kiaer at the late-nineteenth-century ISI meetings, Kruskal and Mosteller describe Gini's reaction as 'almost a return to the older von Mayr approach' ('Representative Sampling IV,' 174, 187)
57. Bulletin de l'Institut international de statistique 22 (1925), 1re liv.: 66
58. C.A. Verrijn Stuart, 'Note sur l'application de la méthode représentative,' Bulletin de l'Institut international de statistique ibid., 440-3. After Coats's paper was published, he received two warm letters of support from Zahn, who wrote that 'statistics based on that method are exposed to be taxed of propaganda and are apt to shake confidence in both that method and official statistics in general.' NA, RG 31, box 17, file 1345, F. Zahn to R.H. Coats, 28 Dec. 1931. It is of interest that in the 1934 book he devoted to the history of the ISI on the occasion of its fiftieth anniversary, Zahn comes back briefly to the episode of the 1925 session, describing it as something of a no-win game. See 50 années de l'Institut international de Statistique (Munich: ISI 1934)
59. See Hansen, 'Some History and Reminiscences on Survey Sampling,' 183
60. Hansen, 'Commentaires sur les articles de la section spéciale,' Techniques d'enquête 16 (1990): 89
61. Nathan Keyfitz would later describe Coats (together with Quetelet and Australia's F.H. Knibbs) as a 'gifted amateur' ('Government Statistics,' in William Kruskal and Judith M. Tanur, Encyclopedia of Statistics [New York: Free Press 1978]: 420). As for Gini, however, an examination of his bibliography reveals an intimate knowledge of probability calculus, on which he contributed a number of papers, especially from 1907 to 1910
62. On the story of Gallup's famous 1936 prediction, see Converse, Survey Research in the United States, 114-121
63. NA, RG 31, Minutes of the Committee on the National Income, 27 June 1940
64. The use of sampling to check results gathered by enumeration was implemented in the immediate postwar years. In 1946 and 1947, for instance, exhaustive inquiries about the earnings of doctors and dentists were conducted. In each case, the procedure followed was the same: a questionnaire was sent to all doctors and dentists (lists were provided by the Department of National Health and the Canadian Association of Dentists); then, to check if non-respondents (roughly half of the population) had the same characteristics as those who had answered, a 10 per cent random sample to be interviewed was taken from those who had not sent back the questionnaire. See DBS, Survey of Incomes in the Profession of Dentistry in Canada, 1941-1944 (Ottawa 1946)
65. and DBS, Survey of Incomes in the Medical Profession in Canada in 1939, 1944, 1945 and 1946 (Ottawa 1948)
66. The DBS was entrusted with the task of administering the registration cards which, complying with the 1940 National Registration Act, 8 million Canadians had filled in. The Population Register was designed to provide data on Canadians aged sixteen to sixty-nine in the context of the war effort. The DBS used it notably 'to supply government agencies with the names and addresses of 1.8 million people available for employment in war industries requiring specific skills.' Statistics Canada, 75 Years and Counting: A History of Statistics Canada (Ottawa 1993), 37
67. NA, RG 31, A National Population Register, Its Value and Organization, 1942, 4
68. Dominion Bureau of Statistics, Census of Canada 1941, vol.I (Ottawa 1947), xvii
69. Agriculture, Labour, Finance, Trade and Commerce each had a representative on the committee. Cudmore was president, and Marshall, who had taken the job of assistant dominion statistician in 1942, acted as secretary. Statistics Canada, 75 Years and Counting, 35
70. NA, RG 31, box 1416, file 1553, Statistical Preparations for Post-War Problems, nd
71. NA, RG 31, box 17, file 1345, A.J. Pelletier to S.A. Cudmore, 13 March 1942
72. NA, RG 31, memo to Herbert Marshall from Nathan Keyfitz, 27 Dec. 1943. Keyfitz was enthusiastic about the capacity of the new technique and suggested that the DBS could even undertake opinion polls. In a handwritten comment he put on his copy of the memo, Cudmore expressed strong reservations about this suggestion. According to him, it was not quite in line with the DBS official mandate
73. NA, RG 31, vol.1435, S.A. Cudmore to N. Keyfitz, 8 Jan. 1944
74. N. Keyfitz to S.A. Cudmore, 24 Feb. 1944
75. S.A. Cudmore to J.A. Mackinnon, 18 Feb. 1944
76. NA, RG 31, vol.1435, Memorandum Re Estimate for Field Sampling Organization, nd
77. Dominion Bureau of Statistics, Labour Force Survey, Labour Force Bulletin, no. 1, Jan. 1946 (SC 71-001), 2-4. The size of the sample was designed to provide adequate precision not only on the national level, but on the level of provinces as well (the present Labour Force Survey is based on a sample of no fewer than 58,800 households). More generally, the size of a sample is relative to the smallest subgroups to be analyzed
78. A reading of their respective obituaries reveals that both were presidents of the Canadian Political Science Association and of the American Statistical Association, and Fellows of the Royal Society of Canada. Coats was a statistical adviser to the League of Nations, while Marshall played an active role in the statistical activities of the United Nations. Although Marshall was thirteen years younger than Coats, he was somewhat of the same generation, having joined the DBS in 1921. Neither of them was thoroughly trained in mathematics. See Herbert Marshall, 'Robert Hamilton Coats, 1874-1960,' Canadian Journal of Economics and Political Science 26, 3 (1960): 482-3
79. and Walter Duffett, 'Herbert Marshall, 1887-1977,' Canadian Journal of Economics 11, 1 (1978): 91
80. Nathan Keyfitz, 'The Sampling Aproach to Economic Data,' Canadian Journal of Economics and Political Science 11 (1945): 467
81. Nathan Keyfitz, 'The Sampling Aproach to Economic Data,' Canadian Journal of Economics and Political Science Ibid., 469
82. Keyfitz quoted their 1943 paper, 'On the Theory of Sampling from Finite Populations,' published in the Annals of Mathematical Statistics
83. See Doug Owram, The Government Generation: Canadian Intellectuals and the State, 1900-1945 (Toronto: University of Toronto Press 1986)
84. The quotation, however, is from Kenneth Norrie and D. Owram, A History of the Canadian Economy (Toronto: Harcourt Brace Jovanovich 1991), 533-534
85. See D. Owram, 'Economic Thought in the 1930s: The Prelude to Keynesianism,' Canadian Historical Review 66 (1985): 344-77
86. See also, on Bryce, J.L. Granatstein, The Ottawa Men: The Civil Service Mandarins, 1935-1957 (Toronto: Oxford University Press 1982), 256-263
87. The ambiguity of the Liberal government's commitment with regard to full employment is discussed in Robert M. Campbell, L'objectif du plein emploi au Canada, 1945-1985. Historique, concepts et données comparatives (Ottawa: Conseil économique du Canada 1991)
88. For a more general examination of the degree to which Canada subscribed to Keynesian economics, see, also by Campbell: Grand Illusions: The Politics of the Keynesian Experience in Canada, 1945-1975 (Peterborough: Broadview Press 1987)
89. Keyfitz, 'The Sampling Aproach to Economic Data,' 476
90. On the history of Canadian economics, see Owram, 'Economic Thought in the 1930s'; Craufurd Goodwin, Canadian Economic Thought: The Political Economy of a Developing Nation, 1814-1914 (Durham: Duke University Press 1961)
91. K.W. Taylor, 'Economic Scholarship in Canada,' Canadian Journal of Economics and Political Science 26 (1960): 6-18
92. The quotation is from Owram, 'Economic Thought in the 1930s,' 351
93. But see in particular R.H. Coats's presidential address to the Canadian Political Association, 'Statistics Comes of Age,' Canadian Journal of Economics and Political Science 2 (1936): 269-87. Jacob Viner's dissertation on international trade, for which he had provided statistical material, was heralded by Coats as a fine example of statistics making theory fruitful (271)
94. R.H. Coats, 'Fifty years of Statistical Progress,' in Fifty Years Retrospect: Anniversary Volume, 1882-1932, Royal Society of Canada (Toronto 1932), 83
95. 'Our democracy feels that while a bevy of engineeers can plan almost anything imaginable in the way of physical construction, God alone knows enough to design a new social and economic order ... In a word the genius of democracy is held to lie in evolution, in making up its mind as it goes along ... But our actions affect our growth. Civilization itself is based on forethought, and the basis of forethought is knowledge. By all means let us follow our nose, but let it be an intelligent nose, able to pick up a true scent from a false. The method of trial and error, but not gratuitous error.' Coats, 'Statistics Comes of Age,' 282
96. Kim Farral, 'Theories, Concepts, and the Art of Measurement: Economic Theories' Relationship to the Labour Force Concepts,' Notes on Labour Statistics (Ottawa: Statistics Canada 1973), 20-28
97. See also Martin Comeau, 'Analyse politique de l'élaboration et de la mesure du taux de chômage au Canada' (MA thesis, Université du Québec à Montréal 1995)
98. R.H. Coats, 'Science and Society,' Canadian Journal of Economics and Political Science 5 (1939): 165. A 1938 memorandum prepared for the benefit of Coats clearly contrasted Keynes's theory of unemployment with the implicit classical theory that supported the DBS definition of unemployment (NA, RG 31, vol. 1418, file Unemployment, Memorandum to Dr. Coats on the Relationship of the Bureau's Conclusions on Unemployment to Current Economic Opinion, 11 Feb. 1938). The unnamed author of the memo was himself favourable to Keynes's view
99. James Struthers, No Fault of Their Own: Unemployment and the Canadian Welfare State, 1914-1941 (Toronto: University of Toronto Press 1983), 232. These doubts were still very much present in 1940, as witnessed by a DBS internal Memorandum on the Unemployment Insurance Act, 5 Sept. 1940 (NA, RG 31, vol. 1418, file Unemployment)
100. Farral, 'Theories, Concepts, and the Art of Measurement,' 21
101. The Belgian statistician Adolphe Quetelet (1796-1874), whose contribution to census methodology remains of value, can justly be described as the father of administrative statistics. See Jean-Pierre Beaud et Jean-Guy Prévost, 'Back to Quételet,' Recherches sociologiques 29, 2 (1998): 83-100. The title refers to Coats's misspelling of Quetelet's name by introducing an acute accent, a mistake still common
102. See Leslie Kish, 'Rolling Samples and Censuses,' Survey Methodology 16 (1990): 63-71