Money, lies, and replicability: On the need for imperically grounded experimental practices and interdisciplinary discourse

Behavioral and Brain Sciences

Volumn 24, Issue 3, 2001, Pages 433-451

  Hertwig, Ralph ^a   Ortmann, Andreas ^b  

^a MAX PLANCK INSTITUTE FOR HUMAN DEVELOPMENT   (Germany)

^b CHARLES UNIVERSITY   (Czech Republic)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; BEHAVIORAL SCIENCE; CREATIVITY; ECONOMIC ASPECT; EMPIRICISM; EXPERIMENT; HABIT; HUMAN; INFORMATION PROCESSING; MEDICAL DECISION MAKING; MEDICAL PRACTICE; MEDICAL PSYCHOLOGY; METHODOLOGY; MONEY; MOTIVATION; POLICY; REPRODUCIBILITY; STATISTICAL ANALYSIS;

EID: 0034779416     PISSN: 0140525X     EISSN: None     Source Type: Journal    
DOI: 10.1017/s0140525x01564146     Document Type: Article

References (369)

1. A visual introduction to dynamical systems theory for psychology
2. Ethical regulations and their impact on research practice
3. What the subjects have to say
4. Follow-up analysis of use of forewarning and deception in psychological experiments
5. Social decision heuristics in the use of shared resources
6. Ethical principles of psychologists and code of conduct
7. Warm-glow versus cold-prickle: The effects of positive and negative framing on cooperation in experiments
8. The sunk cost and Concorde effects: Are humans less rational than lower animals?
9. Effects of cognitive feedback on performance
10. Social learning theory
11. Control without deception: Individual behaviour in free-riding experiments revisited
12. Principles of automaticity
13. When do base rates affect predictions?
14. The forgotten variable in conformity research: Impact of task importance on social influence
15. Macro-economic policy
16. On the relationship between decisions in one-shot and repeated tasks: Experimental results and the possibility of general models
17. Consciousness, free choice, and automaticity
18. Some thoughts on ethics of research. After reading Milgram's "Behavioral study of obedience"
19. Principles of ethical conduct in the treatment of subjects: Reaction to the draft report of the Committee on Ethical Standards in Psychological Research
20. IRBs and social science research: The costs of deception
21. Research using intentional deception: Ethical issues revisited
22. Subjective probabilities inferred from estimates and bets
23. The impact of incentives upon risky choice experiments
24. The influence of compensation method and disclosure level on information search strategy and escalation of commitment
25. Trust, reciprocity, and social history
26. Preference reversal and arbitrage
27. Values and religious issues in psychotherapy and mental health
28. Time horizon and equilibrium selection in tacit coordination games: Experimental results
29. Understanding conjunction effects in probability judgment: The role of implicit mental models
30. The effects of routine strength on adaptation and information search in recurrent decision making
31. Explaining and predicting routinized decision making: A review of theories
32. I like it but I don't know why: A value-account approach to implicit attitude formation
33. Playing fair
34. Why experiment in economics?
35. Testing non-cooperative game theory: A preliminary study
36. Bayesian inference: Combining base rates with opinions of sources who vary in credibility
37. Source credibility in social judgment: Bias, expertise, and the judge's point of view
38. Social psychology and personality: Toward a convergence
39. The effects of costless pre-play communication. Experimental evidence from a game with Pareto-ranked equilibria
40. More enforcement with less law: On contract enforcement, trust and crowding
41. Experimental economics and deception
42. The ecological validity of jury simulations: Is the jury still out?
43. Null science: Psychology's statistical status quo draws fire
44. In one word: Not from experience
45. Dynamic decision making: Human control of complex systems
46. Man as a stabiliser of systems: From static snapshots of judgment processes to dynamic decision making
47. The misbehavior of organisms
48. Deception can be acceptable: Comment on Ortmann and Hertwig
49. The how and why of ecological memory
50. Going beyond the information given
51. Representative design and probabilistic theory in a functional psychology
52. Friend-or-foe intentionality priming in an extensive form trust game
53. Majority and minority influence, task representation and inductive reasoning
54. Do markets correct biases in probability judgment? Evidence from market experiments
55. Individual decision making
56. Rules for experimenting in psychology and economics, and why they differ
57. Experience-weighted attraction learning (EWA) in "normal-form" games: A unifying approach
58. The effects of financial incentives in experiments: A review and capital-labor-production framework
59. Cognition and framing in sequential bargaining for gains and losses
60. The curse of knowledge in economic settings: An experimental analysis
61. Reinforcement, reward, and intrinsic motivation: A meta-analysis
62. Raising the stakes in the ultimatum game: Experimental evidence from Indonesia
63. Convergent and discriminant validation by the multitrait-multimethod matrix
64. Instructions and reinforcement in the observing behavior of adults and children
65. Base-rate training without case cues reduces base-rate neglect
66. Methodological problems in AIDS behavioral research: Influences on measurement error and participation bias in studies of sexual behavior
67. Valuing the future: Temporal discounting of health and money
68. Preference reversals in monetary and life expectancy evaluations
69. Social preferences: Some simple tests and a new model
70. Reflections on language
71. The negative subject: Myth, reality, or a prior experimental experience effect?
72. Deception in psychological research: When is its use justified?
73. The subsidence of preference reversals in simplified and marketlike experimental settings: A note
74. Being there: Putting brain, body, and world together again
75. The statistical power of abnormal-social psychological research: A review
76. Hedge-clipping, tree-felling, and the management of ambiguity
77. Quasi-experimentation: Design and analysis issues for field settings
78. Demand characteristics and three conceptions of the frequently deceived subject
79. Cognitive adaptations for social exchange
80. Are humans good intuitive statisticians after all? Rethinking some conclusions from the literature on judgment under uncertainty
81. Cognition and behavior in normal-form games: An experimental study
82. The impact of accuracy and effort feedback and goals on adaptive decision behavior
83. Constructing the subject. Historical origins of psychological research
84. Field dependence and hindsight bias: Cognitive restructuring and the generation of reasons
85. Experimental economics
86. Rational choice in an uncertain world
87. The purpose of experiments: Ecological validity versus comparing hypotheses
88. Experimental demand, clear incentives, both, or neither?
89. Meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation
90. The undermining effect is a reality after all - Extrinsic rewards,task interest, and self-determination: Reply to Eisenberger, Pierce, and Cameron (1999) and Lepper, Henderlong, and Gingras (1999)
91. The support of autonomy and the control of behavior
92. Trust, reciprocity, and interpersonal history: Fool me once, shame on you, fool me twice, shame on me
93. Effects of feedback complexity on dynamic decision making
94. Ethics in social and behavioral research
95. Measuring beliefs in an experimental lost wallet game
96. A theory of sequential reciprocity
97. The aim and structure of physical theory
98. Physical theory and experiment
99. Conversational implicature, conscious representation and the conjuction fallacy
100. The use of base rate information as a function of experienced consistency and utility
101. Bidding in common value auctions: How the commercial construction industry corrects for the winners' curse
102. The reliability of probability preferences
103. Costs and payoffs are instructions
104. Dynamic decision theory and probabilistic information processing
105. Detrimental effects of reward: Reality or myth?
106. Effects of reward on intrinsic motivation - Negative, neutral, and positive: Comment on Deci, Koestner, and Ryan (1999)
107. Suspicion, affective response, and educational benefit as a result of deception in psychology research
108. The experimental contract: Subjects' expectations of and reactions to some behaviors of experimenters
109. Simultaneous over- and underconfidence: The role of error in judgment processes
110. Estimating sample-size requirements in research design: A study of intuitive statistical judgment
111. Human reasoning: The psychology of deduction
112. Reasoning, decision making and rationality
113. A theory of reciprocity
114. The structure-mapping engine: Algorithm and examples
115. Immediate reward followed by extinction vs. later reward without extinction
116. Judgment and decision making: Behavioral approaches
117. Wage rigidities in a competitive incomplete contract market. An experimental investigation
118. Cooperation and punishment in public goods experiments
119. Does fairness prevent market clearing? An experimental investigation
120. A theory of cognitive dissonance
121. Cognitive consequences of forced compliance
122. Prior deception and subsequent experimental performance: The "faithful" subject
123. The evolution with age of probabilistic, intuitively based misconceptions
124. Subjective sensitivity analysis
125. Participant partners: College students weigh the costs and benefits of deceptive research
126. Statistical methods and scientific inference
127. Not just for the money: An economic theory of personal motivation
128. The cost of price incentives: An empirical analysis of motivation crowding-out
129. The old lady visits your backyard: A tale of morals and markets
130. The theory of learning in games
131. Relation of cue to consequence in avoidance learning
132. Thinking and reasoning
133. Involvement in role-playing simulations: A demonstration with studies on obedience
134. Structural alignment in analogy and similarity
135. How to make cognitive illusions disappear: Beyond heuristics and biases
136. From tools to theories: A heuristic of discovery in cognitive psychology
137. On narrow norms and vague heuristics: A reply to Kahneman and Tversky (1996)
138. Presentation and content: The use of base rates as a continuous variable
139. How to improve Bayesian reasoning without instruction: Frequency formats
140. Overcoming difficulties in Bayesian reasoning: A reply to Lewis & Keren and Mellers & McGraw
141. Domain-specific reasoning: Social contracts, cheating, and perspective change
142. Cognition as intuitive statistics
143. Simple heuristics that make us smart
144. Ultimatum game with an ethnicity manipulation: Results from Bulgan Cum, Mongolia
145. Game theory evolving
146. Analysis of the word-frequency effect in recognition memory
147. Meta-analysis in social research
148. A fine is a price
149. Allocative efficiency of markets with zero-intelligence traders: Markets as a partial substitute for individual rationality
150. A model of noisy introspection
151. Ten little treasures of game theory and ten intuitive contradictions
152. Content and discontent: Indications and implications of domain specificity in preferential decision making
153. Relational similarity and the non-independence of features in similarity judgments
154. Examining the joint roles of responsibility and reinforcement history in recommitment
155. Can't stop on a dime: The roles of matching and momentum in persistence of commitment
156. An experientially derived base-rate error in humans
157. Learning to commit or avoid the base-rate error
158. What does and does not alleviate base-rate neglect under direct experience
159. Base rates versus sample accuracy: Competition for control in human matching to sample
160. Demons versus heuristics in artificial intelligence, behavioral ecology, and economics
161. Under what conditions does theory obstruct research progress?
162. Role-playing as an experimental strategy in social psychology
163. Recent psychological studies of behavior under uncertainty
164. Bayes rule as a description model: The representativeness heuristic
165. Testing Bayes rule and the representativeness heuristic: Some experimental evidence
166. Economic theory of choice and the preference reversal phenomenon
167. Dependence of empirical laws upon the source of experimental variation
168. The elusive thematic materials effect in the Wason's selection task
169. The problem of external validity (or 'parallelism') in experimental economics
170. A logit model of brand choice calibrated on scanner data
171. An experimental analysis of ultimatum bargaining
172. Auctioning ultimatum bargaining positions: How to act if rational decisions are unacceptable
173. When guessing is better than thinking: Multiple bases for frequency judgements
174. Social judgement theory
175. The predictive utility of generalized expected utility theories
176. Theory and misbehavior of first-price auctions
177. Theory and misbehavior of first-price auctions: Reply
178. Expected utility theory and the experimentalists
179. Experimental economics and contingent valuation
180. Experimental evidence on the existence of hypothetical bias in value elicitation methods
181. Choice as a function of reinforcement ratios in delayed matching to sample
182. Statistical methods for meta-analysis
183. Hindsight bias: An interaction of automatic and motivational factors?
184. Does culture matter in economic behavior? Ultimatum game bargaining among the Machiguenga of the Peruvian Amazon
185. The evolution of prestige: Freely conferred deference as a mechanism for enhancing the benefits of cultural transmission
186. The "conjunction fallacy" revisited: How intelligent inferences look like reasoning errors
187. Does deception destroy experimental control?
188. Economists' and psychologists' experimental practices: How they differ, why they differ, and how they could converge
189. Biases to the left, fallacies to the right: Stuck in the middle with null hypothesis significance testing. Commentary on Krueger on social-bias
190. More is not always better: The benefits of cognitive limits
191. Experiments in economics
192. Experimental economics and deception: A comment
193. The social context of reasoning: Conversational inference and rational judgement
194. The influence of the accuracy of individuating information on the use of base rate information in probability judgment
195. Preferences, property rights and anonymity in bargaining games
196. Social distance and other-regarding behavior in dictator games
197. The impact of exchange context on the activation of equity in ultimatum games
198. Hindsight bias: A price worth paying for fast and frugal memory
199. Communicating statistical information
200. Beyond discrete biases: Functional and dysfunctional aspects of judgmental heuristics
201. Learning from feedback: Exactness and incentives
202. Introduction: Perspectives from economics and psychology
203. Industrial organization: A survey of laboratory research
204. Risk aversion and incentive effects
205. Mental leaps: Analogy in creative thought
206. Psychologically rational choice: Selection between alternatives in a multiple-equilibrium game
207. Factors influencing the use of internal summary evaluation versus external information in choice
208. Distributed representations of structure: A theory of analogical access and mapping
209. Methods of meta-analysis: Correcting error and bias in research findings
210. Stärkere Framing Effekte durch mehr Nachdenken? Einflüβe der Bearbeitungszeit auf Lösungen des "Asian disease"- Problems
211. Hypothetical and real consequences in experimental auctions for insurance against low-probability risks
212. Consumer behavior: A quadrennium
213. Money vs. ganning: Effects of salient monetary payments in double oral auctions
214. The origin of consciousness in the breakdown of the bicameral mind
215. Are financial incentives related to performance? A meta-analytic review of empirical research
216. Consumer involvement and deception from implied advertising claims
217. Naive empiricism and dogmatism in confidence research: A critical examination of the hard-easy effect
218. Fairness and the assumptions of economics
219. Judgement under uncertainty: Heuristics and biases
220. On the psychology of prediction
221. Prospect theory: An analysis of choice under risk
222. On the reality of cognitive illusions: A reply to Gigerenzer's critique
223. Choices, values, and frames
224. Human use of human subjects: The problem of deception in social psychological experiments
225. Violation of utility theory in unique and repeated gambles
226. Ethical issues in behavioral research
227. In defense of deception
228. On the how and why (not) of learning from outcomes
229. The base rate fallacy reconsidered: Descriptive, normative, and methodological challenges
230. By all available means: Cameron and Pierce's defense of extrinsic motivators
231. Memory in naturalistic and laboratory contexts: Distinguishing the accuracy-oriented and quantity-oriented approaches to memory assessment
232. Memory metaphors and the real-life/laboratory controversy: Correspondence versus storehouse conceptions of memory
233. A course in microeconomic theory
234. Further tests of the separation theorem and the capital asset pricing model
235. The bet on bias: A foregone conclusion?
236. A re-assessment of role playing as a technique in social psychology
237. Subjects' expectations and the search for alternatives to deception in social psychology
238. Framing decisions Hypothetical and real
239. The structure of scientific revolutions
240. Further reflections on prospect theory. Working Paper
241. Uses, abuses and misuses of significance tests in the scientific community: Wont the Bayesian choice be mavoidable?
242. Public goods: A survey of experimental research
243. Undermining children's intrinsic interest with extrinsic reward: A test of the "overjustification" hypothesis
244. Understanding the effects of extrinsic rewards on intrinsic motivation - Uses and abuses of meta-analysis: Comment on Deci, Koestner, and Ryan (1999)
245. Intrinsic motivation and extrinsic rewards. A commentary on Cameron and Pierce's meta-analysis
246. Parts and wholes: Connectionist dynamics of economic behavior in context
247. Multiattribute decision making in context: A dynamic neural network methodology
248. Confidence in judgements based on incomplete information: An investigation using both hypothetical and real gambles
249. A dynamic theory of personality
250. Group decision and social change
251. Economics and psychology: Lessons for our own day from the early twentieth century
252. Reversals of preference between bids and choices in gambling decisions
253. Effect of feedback on base-rate utilization
254. Psychophysical scaling: Judgements of attributes or objects?
255. Small world
256. Experimental economics from the vantage-point of behavioural economics
257. Research on self-control: An integrating framework
258. Some lessons from past experiments and some challenges for the future
259. Procedural debiasing
260. The rhetoric of irrationality
261. Psychology and economics: Perspectives on risk, cooperation, and the marketplace
262. Determinants of human operant behavior
263. Suspicion in the psychological laboratory: Kelman's prophecy revisited
264. Base rates can affect individual predictions
265. Cognitive psychology: A neural network approach
266. Faculty discretionary time: Departments and the "academic ratchet"
267. Uninstructed human responding: Sensitivity to ratio and interval contingencies
268. Positive and negative decision frames: A verbal protocol analysis of the Asian disease problem of Tversky and Kahneman
269. The history of biological thought: Diversity, evolution, and inheritance
270. A functional imaging study of "theory of mind" in two-person reciprocal exchange
271. Reciprocity, trust, and payoff privacy in extensive form bargaining
272. Cooperation in single play, two-person extensive form games between anonymously matched players
273. Experimental economics and deception: A comment
274. Taking into account the strength of an alternative hypothesis
275. (Non)complementary updating of belief in two hypotheses
276. Do participants believe what experimenters tell them? Further tests of the relation between confidence in yes/no and forced-choice tasks
277. Relation between confidence in yes/no and forced-choice tasks
278. Sensitivity to changes in base-rate information
279. Problem structure and the use of base-rate information from experience
280. Respects for similarity
281. Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology
282. Dominance violations in judged prices of two- and three-outcome gambles
283. Do frequency representations eliminate conjunction effects? An exercise in adversarial collaboration
284. Behavioral study of obedience
285. Reinforcement-induced behavioral stereotypy: How not to teach people to discover rules
286. Do studies of statistical power have an effect on the power of studies?
287. Aspiration adaptation theory
288. Aspiration levels and risk taking: A theoretical model and empirical study on the behavior of government bond traders
289. Twenty years of deception research: A decline in subjects' trust?
290. Beyond greed and fear: Understanding behavioral finance and the psychology of investing
291. The analysis of proximities: Multidimensional scaling with an unknown distance function, I
292. Deception methods in psychology: Have they changed in 23 years?
293. A census of subject pool characteristics and policies
294. Nonparametric statistics for the behavioral sciences
295. Decision making behavior in a two-choice uncertain outcome situation
296. Bargaining and group decision making
297. Effects of deceiving and debriefing psychological subjects on performance in later experiments
298. Rational choice and the structure of environments
299. Models of man
300. Models of thought
301. Invariants of human behavior
302. Compatibility effects in judgement and choice
303. The theory of moral sentiments
304. Evolution and the theory of games
305. Need for cognition and choice framing effects
306. Amelioration of deception and harm in psychological research: The important role of debriefing
307. Experimental economics: Induced value theory
308. Microeconomic systems as an experimental science
309. Rational choice: The contrast between economics and psychology
310. Monetary rewards and decision cost in strategie interactions
311. Monetary rewards rewards and decision cost in experimental economics
312. Rewards, experience and decision costs in first price auctions
313. Behavioral confirmation in social interaction: From social perception to social reality
314. On players' models of other players: Theory and experimental evidence
315. Ineffective deception in conformity research: Some causes and consequences
316. Experimental economics: Hard science or wasteful tinkering?
317. Tradeoffs between quality and quantity of life: Development of the QQ Questionnaire for cancer patient attitude
318. Cognition and behavior analysis: A review of Rachlin's Judgment, decision and choice
319. Effects of attribute scales on process and performance in multiattribute choice
320. A model of financier incentive effects in decision making
321. Decision making and the search for fundamental regularities: What can be learned from a process perspective?
322. Probing suspicion among participants in deception research
323. The psychology of choice and the assumptions of economics
324. The theory of industrial organization
325. Précis of Simple heuristics that make us smart
326. How can we open up the adaptive toolbox?
327. The psychological foundations of culture
328. Features of similarity
329. Judgment under uncertainty: Heuristics and biases
330. The framing of decision and the rationality of choice
331. Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment
332. Rational choice and the framing of decisions
333. Advances in prospect theory: Cumulative representation of uncertainty
334. On the origin of conventions: Evidence from symmetric bargaining games
335. Implicit diagnosticity in an information-buying task: How do we use the information that we bring with us to a problem?
336. Diagnosticity, confidence, and the need for information
337. Deceiving experimental subjects
338. Cost and payoffs in perceptual research
339. Monetary rewards and decision cost in experimental economics
340. Conjoint-measurement framework for the study of probabilistic information processing
341. Using conjoint-measurement models to investigate a theory about probabilistic information processing
342. Research through deception
343. Processing probabilistic multidimensional information for decisions
344. The importance of cognitive illusions
345. From subjective probabilities to decision weights: The effect of asymmetric loss functions on the evaluation of uncertain outcomes and events
346. From Shakespeare to Spielberg: Some reflections on modes of decision making
347. Models and mosaics: Investigating cross-cultural differences in risk perception and risk preference
348. Reasons for rank-dependent utility evaluation
349. Subject effects in laboratory research: An examination of subjects role, demand characteristics, and valid inference
350. Evolutionary game theory
351. Lottery choice: Incentives, complexity and decision time
352. Thinking too much: Introspection can reduce the quality of preferences and decisions
353. Experiments in the laboratory and the real world
354. Subjective memorability and the mirror effect
355. The automaticity of everyday life - Advances in social cognition
356. Elimination and inclusion procedures in judgment
357. The evolution of conventions
358. Overcoming the early entrant advantage: The role of alignable and nonalignable differences
359. Transitive and intransitive preferences in choices between simple and compound lotteries: Experimental evidence and neural network modelling
360. A violation of the monotonicity axiom: Experimental evidence on the conjunction fallacy
361. The psychological and economical perspective on human decisions in social and interactive contexts