The Opportunities and Limitations of Using Mechanical Turk (MTURK) in Public Administration and Management Scholarship

International Public Management Journal

Volumn 20, Issue 3, 2017, Pages 489-511

  Stritch, Justin M ^a   Pedersen, Mogens Jin ^b   Taggart, Gabel ^a  

^a ARIZONA STATE UNIVERSITY   (United States)

^b AARHUS UNIVERSITY   (Denmark)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85015159484     PISSN: 10967494     EISSN: None     Source Type: Journal    
DOI: 10.1080/10967494.2016.1276493     Document Type: Article

References (119)

1. Allen, N. J., and J. P., Meyer. 1996. “Affective, Continuance, and Normative Commitment to the Organization:An Examination of Construct Validity.” Journal of Vocational Behavior 49(3):252–276.
2. Alm, J., K. M., Bloomquist, and M., McKee. 2011. “Comparing Student and Non-Student Reporting Behavior in Tax Compliance Experiments.” Pp. 93–98 in A. Plumley, ed., Recent Research on Tax Administration and Compliance. Washington, DC:IRS Research Bulletin.
3. Arceneaux, K., 2012. “Cognitive Biases and the Strength of Political Arguments.” American Journal of Political Science 56(2):271–285. doi:10.2307/23187099
4. Avellaneda, C. N., 2013. “Mayoral Decision-Making:Issue Salience, Decision Context, and Choice Constraint? An Experimental Study with 120 Latin American Mayors.” Journal of Public Administration Research and Theory 23(3):631–661. doi:10.1093/Jopart/Mus041
5. Baekgaard, M., 2015. “Performance Information and Citizen Service attitudes:Do Cost Information and Service Use Affect the Relationship?” International Public Management Journal 18(2):228–245. doi:10.1080/10967494.2015.1022676
6. Baekgaard, M., and S., Serritzlew. 2016. “Interpreting Performance Information:Motivated Reasoning or Unbiased Comprehension.” Public Administration Review 76(1):73–82.
7. Ball, S. B., and P.-A., Cech. 1996. “Subject Pool Choice and Treatment Effects in Economic Laboratory Research.” Research in Experimental Economics 6(3):239–292.
8. Bellé, N., 2013. “Experimental Evidence on the Relationship between Public Service Motivation and Job Performance.” Public Administration Review 73(1):143–153.
9. Bellé, N., 2014. “Leading to Make a Difference:A Field Experiment on the Performance Effects of Transformational Leadership, Perceived Social Impact, and Public Service Motivation.” Journal of Public Administration Research and Theory 24(1):109–136.
10. Berinsky, A. J., G. A., Huber, and G. S., Lenz. 2012. “Evaluating Online Labor Markets for Experimental Research:Amazon.com’s Mechanical Turk.” Political Analysis 20(3):351–368.
11. Berinsky, A. J., M. F., Margolis, and M. W., Sances. 2014. “Separating the Shirkers from the Workers? Making Sure Respondents Pay Attention on Self-Administered Surveys.” American Journal of Political Science 58(3):739–753. doi:10.1111/Ajps.12081
12. Blom-Hansen, J., R., Morton, and S., Serritzlew. 2015. “Experiments in Public Management Research.” International Public Management Journal 18(2):151–170. doi:10.1080/10967494.2015.1024904
13. Bouwman, R., and S., Grimmelikhuijsen. 2014. “Reviewing Experimental Public Administration Research:The Emergence of a Hybrid Tradition.” Paper presented at the IRSPM Conference, Ottawa, Canada, April 9–11, 2014.
14. Bovaird, T., 2007. “Beyond Engagement and Participation:User and Community Coproduction of Public Services.” Public Administration Review 67(5):846–860.
15. Bozeman, B., 1993. “A Theory of Government ‘Red Tape.’” Journal of Public Administration Research and Theory 3(3):273–304.
16. Brewer, G. A., 2011. “Parsing Public/Private Differences in Work Motivation and Performance:An Experimental Study.” Journal of Public Administration Research and Theory 21(Suppl 3):I347.
17. Brewer, G. A., and G. A., Brewer. 2011. “Parsing Public/Private Differences in Work Motivation and Performance:An Experimental Study.” Journal of Public Administration Research and Theory 21(Suppl 3):I347–I362. doi:10.1093/Jopart/Mur030
18. Brudney, J. L., and R. E., England. 1983. “Toward a Definition of the Coproduction Concept.” Public Administration Review 43(1):59–65.
19. Buhrmester, M., T., Kwang, and S. D., Gosling. 2011. “Amazon’s Mechanical Turk:A New Source of Inexpensive, Yet High-Quality, Data?” Perspectives on Psychological Science 6(1):3–5.
20. Button, K. S., J. P., Ioannidis, C., Mokrysz, B. A., Nosek, J., Flint, E. S., Robinson, and M. R., Munafò. 2013. “Power Failure:Why Small Sample Size Undermines the Reliability of Neuroscience.” Nature Reviews Neuroscience 14(5):365–376.
21. Callison-Burch, C., 2009. “Fast, Cheap, and Creative:Evaluating Translation Quality Using Amazon’s Mechanical Turk.” Paper presented at the Conference on Empirical Methods in Natural Language Processing, Singapore, August 6–7.
22. Casler, K., L., Bickel, and E., Hackett. 2013. “Separate but Equal? A Comparison of Participants and Data Gathered via Amazon’s MTurk, Social Media, and Face-to-Face Behavioral Testing.” Computers in Human Behavior 29(6):2156–2160.
23. Chandler, D., and A., Kapelner. 2013. “Breaking Monotony with Meaning:Motivation in Crowdsourcing Markets.” Journal of Economic Behavior and Organization 90:123–133.
24. Chandler, J., P., Mueller, and G., Paolacci. 2014. “Nonnaïveté among Amazon Mechanical Turk Workers:Consequences and Solutions for Behavioral Researchers.” Behavior Research Methods 46(1):112–130.
25. Chandler, J., and D., Shapiro. 2016. “Conducting Clinical Research Using Crowdsourced Convenience Samples.” The Annual Review of Clinical Psychology 12:53–81.
26. Christensen, R. K., S. W., Whiting, T., Im, E., Rho, J. M., Stritch, and J., Park. 2013. “Public Service Motivation, Task, and Non-Task Behavior:A Performance Appraisal Experiment with Korean MPA and MBA Students.” International Public Management Journal 16(1):28–52.
27. Clifford, S., R. M., Jewell, and P. D., Waggoner. 2015. “Are Samples Drawn from Mechanical Turk Valid for Research on Political Ideology?” Research and Politics 2(4):1–9.
28. Coursey, D. H., and S. K., Pandey. 2007. “Public Service Motivation Measurement.” Administration and Society 39(5):547–568. doi:10.1177/0095399707303634
29. Crump, M. J., J. V., Mcdonnell, and T. M., Gureckis. 2013. “Evaluating Amazon’s Mechanical Turk as a Tool for Experimental Behavioral Research.” Plos One 8(3):E57410.
30. Curtin, R., S., Presser, and E., Singer. 2005. “Changes in Telephone Survey Nonresponse over the Past Quarter Century.” Public Opinion Quarterly 69(1):87–98.
31. Davis, D. D., and C. A., Holt. 1993. Experimental Economics. Princeton, NJ:Princeton University Press.
32. De Leeuw, E. D., and W. D., Heer. 2002. “Trends in Household Survey Nonresponse:A Longitudinal and International Comparison.” Pp. 41–54 in R. M., Groves D. A., Dillman J. L., Eltinge and R. J. A., Little eds., Survey Nonresponse. Utrecht, The Netherlands:Utrecht University.
33. Dillman, D. A., J. D., Smyth, and L. M., Christian. 2014. Internet, Phone, Mail, and Mixed-Mode Surveys:The Tailored Design Method. Hoboken, NJ:John Wiley & Sons.
34. Dowling, C. M., and A., Wichowsky. 2015. “Attacks without Consequence? Candidates, Parties, Groups, and the Changing Face of Negative Advertising.” American Journal of Political Science 59(1):19–36. doi:10.1111/Ajps.12094
35. Edwards, P., I., Roberts, M., Clarke, C., Diguiseppi, R., Wentz, I., Kwan, R., Cooper, L. M., Feliex, and S., Pratap. 2009. “Methods to Increase Response to Postal and Electronic Questionnaires.” Cochrane Database System Review 3:1–12.
36. Eriksson, K., and B., Simpson. 2011. “Perceptions of Unfairness in Allocations between Multiple Recipients.” Cognitive Psychology 62(3):225–244.
37. Estellés-Arolas, E., and F., González-Ladrón-De-Guevara. 2012. “Towards an Integrated Crowdsourcing Definition.” Journal of Information Science 38(2):189–200.
38. Evanini, K., D., Higgins, and K., Zechner. 2010. “Using Amazon Mechanical Turk for Transcription of Non-Native Speech.” Paper presented at the annual conference of the North American Chapter of the Association for Computational Linguistics, Los Angeles, CA, June 1–6.
39. Favero, N., and J. B., Bullock. 2014. “How (Not) to Solve the Problem:An Evaluation of Scholarly Responses to Common Source Bias.” Journal of Public Administration Research and Theory 25(1):285–308.
40. Feeney, M. K., 2012. “Organizational Red Tape:A Measurement Experiment.” Journal of Public Administration Research and Theory 22(3):427–444.
41. Fishbach, A., M. D., Henderson, and M., Koo. 2011. “Pursuing Goals with Others:Group Identification and Motivation Resulting from Things Done versus Things Left Undone.” Journal of Experimental Psychology 140(3):520–534.
42. Fort, K., G., Adda, and K. B., Cohen. 2011. “Amazon Mechanical Turk:Gold Mine or Coal Mine?” Computational Linguistics 37(2):413–420.
43. Gallo, N., and D. E., Lewis. 2012. “The Consequences of Presidential Patronage for Federal Agency Performance.” Journal of Public Administration Research and Theory 22(2):219–243.
44. Gómez, A., M. L., Brooks, M. D., Buhrmester, A., Vázquez, J., Jetten, and W. B., Swann Jr. 2011. “On the Nature of Identity Fusion:Insights into the Construct and a New Measure.” Journal of Personality and Social Psychology 100(5):918–933.
45. Goodman, J. K., C. E., Cryder, and A., Cheema. 2013. “Data Collection in a Flat World:The Strengths and Weaknesses of Mechanical Turk Samples.” Journal of Behavioral Decision Making 26(3):213–224.
46. Gordon, S. C., 2009. “Assessing Partisan Bias in Federal Public Corruption Prosecutions.” American Political Science Review 103(4):534–554.
47. Grimmelikhuijsen, S., S., Jilke, A. L., Olsen, and L., Tummers. 2015. “Behavioral Public Administration:Combining Insights from Public Administration and Psychology.” Paper presented at the Academy of Management, Vancouver, BC, August 7–11.
48. Groves, R. M., F. J., Fowler Jr., M. P., Couper, J. M., Lepkowski, E., Singer, and R., Tourangeau. 2013. Survey Methodology. Hoboken, NJ:John Wiley & Sons.
49. Hansen, K. M., 2007. “The Effects of Incentives, Interview Length, and Interviewer Characteristics on Response Rates in a CATI-Study.” International Journal of Public Opinion Research 19(1):112–121.
50. Healy, A., and G. S., Lenz. 2014. “Substituting the End for the Whole:Why Voters Respond Primarily to the Election-Year Economy.” American Journal of Political Science 58(1):31–47. doi:10.1111/Ajps.12053
51. Henrich, J., S. J., Heine, and A., Norenzayan. 2010. “The Weirdest People in the World?” Behavioral and Brain Sciences 33(2–3):61–83.
52. Herd, P., T., Deleire, H., Harvey, and D. P., Moynihan. 2013. “Shifting Administrative Burden to the State:The Case of Medicaid Take‐Up.” Public Administration Review 73(S1):S69–S81.
53. Horton, J. J., D. G., Rand, and R. J., Zeckhauser. 2011. “The Online Laboratory:Conducting Experiments in a Real Labor Market.” Experimental Economics 14(3):399–425.
54. Huff, C., and D., Tingley. 2015. “‘Who Are These People?’ Evaluating the Demographic Characteristics and Political Preferences of MTurk Survey Respondents.” Research and Politics 2(3):1–12.
55. Jakobsen, M., and S. C., Andersen. 2013. “Coproduction and Equity in Public Service Delivery.” Public Administration Review 73(5):704–713. doi:10.1111/Puar.12094
56. Jakobsen, M., and R., Jensen. 2015. “Common Method Bias in Public Management Studies.” International Public Management Journal 18(1):3–30.
57. James, O., 2009. “Evaluating the Expectations Disconfirmation and Expectations Anchoring Approaches to Citizen Satisfaction with Local Public Services.” Journal of Public Administration Research and Theory 19(1):107–123.
58. James, O., 2011. “Performance Measures and Democracy:Information Effects on Citizens in Field and Laboratory Experiments.” Journal of Public Administration Research and Theory 21(3):399–418.
59. Jilke, S., G. G., Van Ryzin, and S., Van De Walle. 2016. “Responses to Decline in Marketized Public Services:An Experimental Evaluation of Choice Overload.” Journal of Public Administration Research and Theory 26:421–432.
60. Johnson, D. R., and L. A., Borden. 2012. “Participants at Your Fingertips Using Amazon’s Mechanical Turk to Increase Student–Faculty Collaborative Research.” Teaching of Psychology 39(4):245–251.
61. Kagel, J. H., and A. E., Roth. 1995. The Handbook of Experimental Economics. Princeton, NJ:Princeton University Press.
62. Kaufman, W., and M. K., Feeney. 2014. “Beyond the Rules:The Effect of Outcome Favourability on Red Tape Perceptions.” Public Administration 92(1):178–191.
63. Kaufmann, W., and L., Tummers. 2016. “The Negative Effect of Red Tape on Procedural Satisfaction.” Public Management Review. doi:10.1080/14719037.2016.1210907
64. Kaufmann, N., T., Schulze, and D., Veit. 2011. “More than Fun and Money:Worker Motivation in Crowdsourcing–A Study on Mechanical Turk.” Paper presented at the Americas Conference on Information Systems, Detroit, MI, August 4–7.
65. Kelman, S., 2007. “Public Administration and Organization Studies.” Pp. 225–67 in A., Brief and J. P., Walsh eds., Academy of Management Annals. New York, NY:Erlbaum.
66. Kelman, S., and J. N., Friedman. 2009. “Performance Improvement and Performance Dysfunction:An Empirical Examination of Distortionary Impacts of the Emergency Room Wait-Time Target in the English National Health Service.” Journal of Public Administration Research and Theory 19(4):917–946.
67. Kiefer, T., J., Hartley, N., Conway, and R. B., Briner. 2015. “Feeling the Squeeze:Public Employees’ Experiences of Cutback- and Innovation-Related Organizational Changes Following a National Announcement of Budget Reductions.” Journal of Public Administration Research and Theory 25(4):1279–1305. doi:10.1093/Jopart/Muu042
68. Kim, S., 2009. “Revising Perry’s Measurement Scale of Public Service Motivation.” The American Review of Public Administration 39(2):149.
69. Kim, S., 2011. “Testing A Revised Measure of Public Service Motivation:Reflective Versus Formative Specification.” Journal of Public Administration Research and Theory 21(3):521–546.
70. Kittur, A., E. H., Chi, and B., Suh. 2008. “Crowdsourcing User Studies with Mechanical Turk.” Paper presented at the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Florence, Italy, April 5–10.
71. Lam, S. S. K., 1998. “Test-Retest Reliability of the Organizational Commitment Questionnaire.” The Journal of Social Psychology 138(6):787–788. doi:10.1080/00224549809603264
72. Landers, R. N., and T. S., Behrend. 2015. “An Inconvenient Truth:Arbitrary Distinctions between Organizational, Mechanical Turk, and Other Convenience Samples.” Industrial and Organizational Psychology 8:142–164.
73. Levitt, S. D., and J. A., List. 2007. “What do Laboratory Experiments Measuring Social Preferences Reveal about the Real World?” The Journal of Economic Perspectives 21(2):153–174.
74. Litman, L., J., Robinson, and T., Abberback. 2016. “TurkPrime.com:A Versatile Crowdsourcing Data Acquisition Platform for the Behavioral Sciences.” Behavior Research Methods:1–10.
75. Little, G., L. B., Chilton, M., Goldman, and R. C., Miller. 2009. “Turkit:Tools for Iterative Tasks on Mechanical Turk.” Paper Presented at the Proceedings of the ACM SIGKDD Workshop on Human Computation, Paris, France, June 28.
76. Marge, M., S., Banerjee, and A. I., Rudnicky. 2010. “Using the Amazon Mechanical Turk for Transcription of Spoken Language.” Paper Presented at the IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), Dallas, TX, March 14–19.
77. Marvel, J. D., 2014. “The Boston Marathon Bombings:Who’s to Blame and Why it Matters for Public Administration.” Public Administration Review 74(6):713–25.
78. Marvel, J. D., 2015. “Public Opinion and Public Sector Performance:Are Individuals’ Beliefs about Performance Evidence-Based or the Product of Anti–Public Sector Bias?” International Public Management Journal 18(2):209–227.
79. Marvel, J. D., 2016. “Unconscious Bias in Citizens’ Evaluations of Public Sector Performance.” Journal of Public Administration Research and Theory 26(1):143–158.
80. Marvel, J. D., and A. M., Girth. 2016. “Citizen Attributions of Blame in Third-Party Governance.” Public Administration Review 76(1):96–108.
81. Mason, W., and S., Suri. 2012. “Conducting Behavioral Research on Amazon’s Mechanical Turk.” Behavior Research Methods 44(1):1–23.
82. Meier, K. J., and L. J., O’Toole, Jr. 2001. “Managerial Strategies and Behavior in Networks:A Model with Evidence from US Public Education.” Journal of Public Administration Research and Theory 11(3):271–294.
83. Meier, K. J., and L. J., O’Toole Jr. 2013. “I Think (I am Doing Well), Therefore I am:Assessing the Validity of Administrators’ Self-Assessments of Performance.” International Public Management Journal 16(1):1–27.
84. Moynihan, D., P., Herd, and H., Harvey. 2014. “Administrative Burden:Learning, Psychological, and Compliance Costs in Citizen-State Interactions.” Journal of Public Administration Research and Theory 25(1):43–69.
85. Moynihan, D. P., 2006. “What Do We Talk about When We Talk about Performance? Dialogue Theory and Performance Budgeting.” Journal of Public Administration Research and Theory 16(2):151–168.
86. Novotney, S., and C., Callison-Burch. 2010. “Cheap, Fast and Good Enough:Automatic Speech Recognition with Non-Expert Transcription.” Paper presented at the annual conference of the North American Chapter of the Association for Computational Linguistics, Los Angeles, CA, June 1–6.
87. Olsen, A. L., 2015a. "Citizen (Dis) Satisfaction:An Experimental Equivalence Framing Study.” Public Administration Review 75(3):469–478.
88. Olsen, A. L., 2015b. “’Simon Said,’ We Didn’t Jump.” Public Administration Review 75(2):325–326.
89. Osborne, S. P., 2008. The Third Sector in Europe:Prospects and Challenges. New York, NY:Routledge.
90. Paolacci, G., and J., Chandler. 2014. “Inside the Turk:Understanding Mechanical Turk as a Participant Pool.” Current Directions in Psychological Science 23(3):184–188.
91. Paolacci, G., J., Chandler, and P. G., Ipeirotis. 2010. “Running Experiments on Amazon Mechanical Turk.” Judgment and Decision Making 5(5):411–419.
92. Pedersen, M. J., 2015. “Activating the Forces of Public Service Motivation:Evidence from a Low-Intensity Randomized Survey Experiment.” Public Administration Review 75(5):734–46.
93. Pedersen, M. J., and J. M., Stritch. 2016. “Internal Management and Perceived Managerial Trustworthiness:Evidence from a Survey Experiment.” The American Review of Public Administration. doi:10.1177/0275074016657179.
94. Perry, J. L., 1996. “Measuring Public Service Motivation:An Assessment of Construct Reliability and Validity.” Journal of Public Administration Research and Theory 6(1):5–22.
95. Perry, J. L., 2012. “How Can We Improve Our Science to Generate More Usable Knowledge for Public Professionals?” Public Administration Review 72(4):479–482.
96. Perry, J. L., and L. R., Wise. 1990. “The Motivational Bases of Public Service.” Public Administration Review 50(3):367–373.
97. Plott, C. R., 1987. “Dimensions of Parallelism:Some Policy Applications of Experimental Methods.” Pp. 193–219 in A. E., Roth ed., Laboratory Experimentation in Economics:Six Points of View. New York, NY:Cambridge University Press.
98. Podsakoff, P. M., S. B., Mackenzie, J. Y., Lee, and N. P., Podsakoff. 2003. “Common Method Biases in Behavioral Research:A Critical Review of the Literature and Recommended Remedies.” Journal of Applied Psychology 88(5):879–903.
99. Podsakoff, P. M., S. B., Mackenzie, and N. P., Podsakoff. 2012. “Sources of Method Bias in Social Science Research and Recommendations on How to Control it.” Annual Review of Psychology 63:539–569.
100. Podsakoff, P. M., and D. W., Organ. 1986. “Self-Reports in Organizational Research:Problems and Prospects.” Journal of Management 12(4):531–544.
101. Reddel, T., and G., Woolcock. 2004. “From Consultation to Participatory Governance? A Critical Review of Citizen Engagement Strategies in Queensland.” Australian Journal of Public Administration 63(3):75–87.
102. Resh, W. G., J. D., Marvel, and B., Wen. 2016. “Do What You Mean, or Mean What You Say? When Self-Interest Crowds out Prosocial Motivation and Mission Match.” Paper presented at the 2016 Public Management Research Conference, Aarhus, Denmark, June 22–24.
103. Riccucci, N. M., G. G., Van Ryzin, and C. F., Lavena. 2014. “Representative Bureaucracy in Policing:Does it Increase Perceived Legitimacy?” Journal of Public Administration Research and Theory 24(3):537–551.
104. Riccucci, N. M., G. G., Van Ryzin, and H., Li. 2016. “Representative Bureaucracy and the Willingness to Coproduce:An Experimental Study.” Public Administration Review 76(1):121–130.
105. Robins, R. W., R. C., Fraley, B. W., Roberts, and K. H., Trzesniewski. 2001. “A Longitudinal Study of Personality Change in Young Adulthood.” Journal of Personality 69(4):617–640.
106. Ross, J., L., Irani, M., Silberman, A., Zaldivar, and B., Tomlinson. 2010. “Who are the Crowdworkers? Shifting Demographics in Mechanical Turk.” Paper presented at the Conference on Human Factors in Computing Systems, Atlanta, GA, April 10–15.
107. Schuerger, J. M., K. L., Zarrella, and A. S., Hotz. 1989. “Factors that Influence the Temporal Stability of Personality by Questionnaire.” Journal of Personality and Social Psychology 56(5):777–783.
108. Sears, D. O., 1986. “College Sophomores in the Laboratory:Influences of a Narrow Data Base on Social Psychology’s View of Human Nature.” Journal of Personality and Social Psychology 51(3):515–530.
109. Smith, C., D., Santucci, S., Xu, A., Cox, and K. A., Henderson. 2012. “’I Love My Job, But…:’ A Narrative Analysis of Women’s Perceptions of their Careers in Parks and Recreation.” Journal of Leisure Research 44(1):52–69.
110. Smith, J., 2016. “The Motivational Effects of Mission Matching:A Lab-Experimental Test of a Moderated Mediation Model.” Public Administration Review 76(4):626–637.
111. Smith, N. A., I. E., Sabat, L. R., Martinez, K., Weaver, and S., Xu. 2015. “A Convenient Solution:Using MTurk to Sample from Hard-to-Reach Populations.” Industrial and Organizational Psychology 8:220–228.
112. Smith, S., 2013. “4 Ways to Ensure Valid Responses for your Online Survey.” Qualtrics. https://www.qualtrics.com/blog/online-survey-valid-responses/
113. Taggart, G., J. M., Stritch, and B., Bozeman. 2015. National Administrative Studies Project (NASP)—Citizen:Waive 1 [data file and codebook]. Phoenix, AZ:Center for Organization Research and Design (CORD) at Arizona State University.
114. Thibodeau, P., M. M., Peebles, D. J., Grodner, and F. H., Durgin. 2015. “The Wished-for Always Wins Until the Winner was Inevitable All Along:Motivated Reasoning and Belief Bias Regulate Emotion During Elections.” Political Psychology 36(4):431–448.
115. Vandenabeele, W., 2008. “Development of a Public Service Motivation Measurement Scale:Corroborating and Extending Perry’s Measurement Instrument.” International Public Management Journal 11(1):143–167.
116. White, E., B. K., Armstrong, and R., Saracci. 2008. Principles of Exposure Measurement in Epidemiology:Collecting, Evaluating and Improving Measures of Disease Risk Factors. New York, NY:Oxford University Press.
117. Wright, B. E., R. K., Christensen, and S. K., Pandey. 2013. “Measuring Public Service Motivation:Exploring the Equivalence of Existing Global Measures.” International Public Management Journal 16(2):197–223.
118. Wright, B. E., L. J., Manigault, and T. R., Black. 2004. “Quantitative Research Measurement in Public Administration:An Assessment of Journal Publications.” Administration and Society 35(6):747–764.
119. Zhu, L., 2013. “Panel Data Analysis in Public Administration:Substantive and Statistical Considerations.” Journal of Public Administration Research and Theory 23(2):395–428. doi:10.1093/Jopart/Mus064