Optimal indolence: A normative microscopic approach to work and leisure

Journal of the Royal Society Interface

Volumn 11, Issue 91, 2014, Pages

  Niyogi, Ritwik K ^a   Breton, Yannick Andre ^b   Solomon, Rebecca B ^b   Conover, Kent ^b   Shizgal, Peter ^b   Dayan, Peter ^a  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b Concordia University ^*  (Canada)

Author keywords

Economics; Leisure; Microscopic; Normative; Reinforcement learning; Work

Indexed keywords

BIOPHYSICS; BIOTECHNOLOGY; ECONOMICS;

LEISURE; MICROFEATURES; MICROSCOPIC; NORMATIVE; WORK;

REINFORCEMENT LEARNING;

ARTICLE; DECISION MAKING; DECISION TREE; LEISURE; MATHEMATICAL MODEL; THEORY; TIME; WORK; ALGORITHM; ANIMAL; BEHAVIOR; BRAIN; HUMAN; LEARNING; MARKOV CHAIN; PHYSIOLOGY; PROBABILITY; REINFORCEMENT; REWARD; THEORETICAL MODEL; TIME FACTOR;

ECONOMICS; LEISURE; MICROSCOPIC; NORMATIVE; REINFORCEMENT LEARNING; WORK;

ALGORITHMS; ANIMALS; BEHAVIOR; BRAIN; DECISION MAKING; HUMANS; LEARNING; LEISURE ACTIVITIES; MARKOV CHAINS; MODELS, THEORETICAL; PROBABILITY; REINFORCEMENT (PSYCHOLOGY); REWARD; STOCHASTIC PROCESSES; TIME FACTORS;

EID: 84891891750     PISSN: 17425689     EISSN: 17425662     Source Type: Journal    
DOI: 10.1098/rsif.2013.0969     Document Type: Article

References (71)

1. Frank RH. 2005 Microeconomics and behavior. New York, NY: McGraw-Hill Higher Education.
2. Kagel JH, Battalio RC, Green L. 1995 Economic choice theory: an experimental analysis of animal behavior. Cambridge, UK: Cambridge University Press.
3. Battalio RC, Green L, Kagel JH. 1981 Income - leisure tradeoffs of animal workers. The Am. Econ. Rev. 71, 621-632.
4. Camerer C, Babcock L, Loewenstein G, Thaler R. 1997 Labor supply of New York city cabdrivers: one day at a time. Q. J. Econ. 112, 407-441. (doi:10.1162/003355397555244)
5. Green L, Kagel JH, Battalio RC. 1987 Consumption - leisure tradeoffs in pigeons: effects of changing marginal wage rates by varying amount of reinforcement. J. Exp. Anal. Behav. 47, 17-28. (doi:10.1901/jeab.1987.47-17)
6. Skinner BF. 1938 The behavior of organisms: an experimental analysis. New York, NY: Appleton-Century-Crofts.
7. Skinner BF. 1981 Selection by consequences. Science 213, 501-504. (doi:10.1126/science.7244649)
8. Herrnstein RJ. 1961 Relative and absolute strength of response as a function of frequency of reinforcement. J. Exp. Anal. Behav. 4, 267-272. (doi:10.1901/jeab.1961.4-267)
9. Herrnstein RJ. 1974 Formal properties of the matching law. J. Exp. Anal. Behav. 21, 159-164. (doi:10.1901/jeab.1974.21-159)
10. Baum WM, Rachlin HC. 1969 Choice as time allocation. J. Exp. Anal. Behav. 12, 861-874. (doi:10.1901/jeab.1969.12-861)
11. Baum WM. 1974 On two types of deviation from the matching law: bias and undermatching. J. Exp. Anal. Behav. 22, 231-242. (doi:10.1901/jeab.1974.22-231)
12. Baum WM. 1981 Optimization and the matching law as accounts of instrumental behavior. J. Exp. Anal. Behav. 36, 387-403. (doi:10.1901/jeab.1981. 36-387)
13. Green L, Rachlin H. 1991 Economic substitutability of electrical brain stimulation, food, and water. J. Exp. Anal. Behav. 55, 133-143. (doi:10.1901/jeab.1991.55-133)
14. McDowell JJ. 1986 On the falsifiability of matching theory. J. Exp. Anal. Behav. 45, 63-74. (doi:10.1901/jeab.1986.45-63)
15. Dallery J, McDowell JJ, Lancaster JS. 2000 Falsification of matching theory's account of single-alternative responding: Herrnstein's k varies with sucrose concentration. J. Exp. Anal. Behav. 73, 23-43. (doi:10.1901/jeab.2000. 73-23)
16. McDowell JJ. 2005 On the classic and modern theories of matching. J. Exp. Anal. Behav. 84, 111-127. (doi:10.1901/jeab.2005.59-04)
17. Haccou P, Meelis E. 1992 Statistical analysis of behavioural data: an approach based on time-structured models. New York, NY: Oxford University Press.
18. Conover KL, Shizgal P. 2005 Employing labor-supply theory to measure the reward value of electrical brain stimulation. Games Econ. Behav. 52, 283-304. (doi:10.1016/j.geb.2004.08.003)
19. Arvanitogiannis A, Shizgal P. 2008 The reinforcement mountain: allocation of behavior as a function of the rate and intensity of rewarding brain stimulation. Behav. Neurosci. 122, 1126-1138. (doi:10.1037/a0012679)
20. Breton YA, Marcus JC, Shizgal P. 2009 Rattus Psychologicus: construction of preferences by self-stimulating rats. Behav. Brain Res. 202, 77-91. (doi:10.1016/j.bbr.2009.03.019)
21. Hernandez G, Breton YA, Conover K, Shizgal P. 2010 At what stage of neural processing does cocaine act to boost pursuit of rewards? PLoS ONE 5, e15081. (doi:10.1371/journal.pone.0015081)
22. Trujillo-Pisanty I, Hernandez G, Moreau-Debord I, Cossette MP, Conover K, Cheer JF, Shizgal P. 2011 Cannabinoid receptor blockade reduces the opportunity cost at which rats maintain operant performance for rewarding brain stimulation. J. Neurosci. 31, 5426-5435. (doi:10.1523/JNEUROSCI.0079-11.2011)
23. Hernandez G, Trujillo-Pisanty I, Cossette MP, Conover K, Shizgal P. 2012 Role of dopamine tone in the pursuit of brain stimulation reward. J. Neurosci. 32, 11 032-11 041. (doi:10.1523/JNEUROSCI.1051-12.2012)
24. Niv Y, Daw ND, Joel D, Dayan P. 2007 Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology 191, 507-520. (doi:10.1007/s00213-006-0502-4)
25. Baum WM. 2002 From molecular to molar: a paradigm shift in behavior analysis. J. Exp. Anal. Behav. 78, 95-116. (doi:10.1901/jeab.2002.78-95)
26. Baum WM. 2001 Molar versus molecular as a paradigm clash. J. Exp. Anal. Behav. 75, 338-378 (discussion 367-378). (doi:10.1901/jeab.2001.75-338)
27. Baum WM. 2004 Molar and molecular views of choice. Behav. Process. 66, 349-359. (doi:10.1016/j.beproc.2004.03.013)
28. Baum WM. 1995 Introduction to molar behavior analysis. Mex. J. Behav. Anal. 21, 7-25.
29. Baum WM. 1976 Time-based and count-based measurement of preference. J. Exp. Anal. Behav. 26, 27-35. (doi:10.1901/jeab.1976.26-27)
30. Rachlin H. 1978 A molar theory of reinforcement schedules. J. Exp. Anal. Behav. 30, 345-360. (doi:10.1901/jeab.1978.30-345)
31. Hineline PN. 2001 Beyond the molar-molecular distinction: we need multiscaled analyses. J. Exp. Anal. Behav. 75, 342-347 (discussion 367-378).
32. Ferster C, Skinner BF. 1957 Schedules of reinforcement. New York, NY: Appleton-Century-Crofts.
33. Gilbert TF. 1958 Fundamental dimensional properties of the operant. Psychol. Rev. 65, 272-282. (doi:10.1037/h0044071)
34. Shull RL, Gaynor ST, Grimes JA. 2001 Response rate viewed as engagement bouts: effects of relative reinforcement and schedule type. J. Exp. Anal. Behav. 75, 247-274. (doi:10.1901/jeab.2001.75-247)
35. Williams J, Sagvolden G, Taylor E, Sagvolden T. 2009 Dynamic behavioural changes in the spontaneously hyperactive rat: 1. Control by place, timing, and reinforcement rate. Behav. Brain Res. 198, 273-282. (doi:10.1016/j.bbr.2008.08. 044)
36. Williams J, Sagvolden G, Taylor E, Sagvolden T. 2009 Dynamic behavioural changes in the spontaneously hyperactive rat: 2. Control by novelty. Behav. Brain Res. 198, 283-290. (doi:10.1016/j.bbr.2008.08.045)
37. Williams J, Sagvolden G, Taylor E, Sagvolden T. 2009 Dynamic behavioural changes in the spontaneously hyperactive rat: 3. Control by reinforcer rate changes and predictability. Behav. Brain Res. 198, 291-297. (doi:10.1016/j.bbr. 2008.08.046)
38. Olds J, Milner P. 1954 Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. J. Comp. Phys. Psychol. 47, 419-427. (doi:10.1037/h0058775)
39. Breton Y, Conover K, Shizgal P. 2009 Probability discounting of brain stimulation reward in the rat. In 39th Ann. Meeting of the Society for Neuroscience (Neuroscience 2009), 17-21 October, Chicago, IL, presentation 892.14/GG98. Washington, DC: Society for Neuroscience.
40. Breton YA. Molar and molecular models of performance for rewarding brain stimulation. PhD thesis, Concordia University, 2013.
41. Solomon RB, Conover K, Shizgal P. In preparation. Measurement of subjective opportunity costs in rats working for rewarding brain stimulation.
42. Gallistel CR, Leon M. 1991 Measuring the subjective magnitude of brain stimulation reward by titration with rate of reward. Behav. Neurosci. 105, 913-925. (doi:10.1037/0735-7044.105.6.913)
43. Simmons JM, Gallistel CR. 1994 Saturation of subjective reward magnitude as a function of current and pulse frequency. Behav. Neurosci. 108, 151-160. (doi:10.1037/0735-7044.108.1.151)
44. Hamilton AL, Stellar JR, Hart EB. 1985 Reward, performance, and the response strength method in self-stimulating rats: validation and neuroleptics. Phys. Behav. 35, 897-904. (doi:10.1016/0031-9384(85)90257-4)
45. Mark TA, Gallistel CR. 1993 Subjective reward magnitude of medial forebrain stimulation as a function of train duration and pulse frequency. Behav. Neurosci. 107, 389-401. (doi:10.1037/0735-7044.107.2.389)
46. Leon M, Gallistel CR. 1992 The function relating the subjective magnitude of brain stimulation reward to stimulation strength varies with site of stimulation. Behav. Brain Res. 52, 183-193. (doi:10.1016/S0166-4328(05)80229-3)
47. Sonnenschein B, Conover K, Shizgal P. 2003 Growth of brain stimulation reward as a function of duration and stimulation strength. Behav. Neurosci. 117, 978-994. (doi:10.1037/0735-7044.117.5.978)
48. Sutton RS, Barto AG. 1998 Reinforcement learning: an introduction, vol. 28. Cambridge, UK: Cambridge University Press.
49. Puterman ML. 2005 Markov decision processes: discrete stochastic dynamic programming. Wiley Series in Probability and Statistics. New York, NY: Wiley-Blackwell.
50. Guitart-Masip M, Fuentemilla L, Bach DR, Huys QJM, Dayan P, Dolan RJ, Duzel E. 2011 Action dominates valence in anticipatory representations in the human striatum and dopaminergic midbrain. J. Neurosci. 31, 7867-7875. (doi:10.1523/JNEUROSCI.6376-10.2011)
51. Shidara M, Aigner TG, Richmond BJ. 1998 Neuronal signals in the monkey ventral striatum related to progress through a predictable series of trials. J. Neurosci. 18, 2613-2625.
52. Breland K, Breland M. 1961 The misbehavior of organisms. Am. Psychol. 16, 681-684. (doi:10.1037/h0040090)
53. Dayan P, Niv Y, Seymour B, Daw ND. 2006 The misbehavior of value and the discipline of the will. Neural Netw. 19, 1153-1160. (doi:10.1016/j.neunet.2006. 03.002)
54. Takikawa Y, Kawagoe R, Itoh H, Nakahara H, Hikosaka O. 2002 Modulation of saccadic eye movements by predicted reward outcome. Exp. Brain Res. 142, 284-291. (doi:10.1007/s00221-001-0928-1)
55. Gibbon J. 1977 Scalar expectancy theory and Weber's law in animal timing. Psychol. Rev. 84, 279-325. (doi:10.1037/0033-295X.84.3.279)
56. Miyazaki K, Miyazaki KW, Doya K. 2011 Activation of dorsal raphe serotonin neurons underlies waiting for delayed rewards. J. Neurosci. 31, 469-479. (doi:10.1523/JNEUROSCI.3714-10.2011)
57. Miyazaki KW, Miyazaki K, Doya K. 2012 Activation of dorsal raphe serotonin neurons is necessary for waiting for delayed rewards. J. Neurosci. 32, 10451-10457. (doi:10.1523/JNEUROSCI.0915-12.2012)
58. Fletcher PJ. 1995 Effects of combined or separate 5,7-dihydroxytryptamine lesions of the dorsal and median raphe nuclei on responding maintained by a DRL 20s schedule of food reinforcement. Brain Res. 675, 45-54. (doi:10.1016/0006- 8993(95)00037-Q)
59. Jolly DC, Richards JB, Seiden LS. 1999 Serotonergic mediation of DRL 72s behavior: receptor subtype involvement in a behavioral screen for antidepressant drugs. Biol. Psychiatry 45, 1151-1162. (doi:10.1016/S0006-3223(98)00014-6)
60. Ho MY, Al-Zahrani SS, Al-Ruwaitea AS, Bradshaw CM, Szabadi E. 1998 5-Hydroxytryptamine and impulse control: prospects for a behavioural analysis. J. Psychopharmacol. 12, 68-78. (doi:10.1177/026988119801200109)
61. Bizot JC, Thibot MH, Le Bihan C, Soubri P, Simon P. 1988 Effects of imipramine-like drugs and serotonin uptake blockers on delay of reward in rats. Possible implication in the behavioral mechanism of action of antidepressants. J. Pharmacol. Exp. Ther. 246, 1144-1151.
62. Bizot J, Le Bihan C, Puech AJ, Hamon M, Thibot M. 1999 Serotonin and tolerance to delay of reward in rats. Psychopharmacology 146, 400-412. (doi:10.1007/PL00005485)
63. Diener E, Wirtz D, Oishi S. 2001 End effects of rated life quality: the James Dean effect. Psychol. Sci. 12, 124-128. (doi:10.1111/1467-9280.00321)
64. Kacelnik A, Marsh B. 2002 Cost can increase preference in starlings. Anim. Behav. 63, 245-250. (doi:10.1006/anbe.2001.1900)
65. Dupas P, Robinson J. 2013 Daily needs, income targets and labor supply: evidence from Kenya. See http://www.nber.org/papers/w19264.
66. Sutton R, Precup D, Singh S. 1999 Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning. Artif. Intell. 112, 181-211. (doi:10.1016/S0004-3702(99) 00052-1)
67. Cools R, Nakamura K, Daw ND. 2011 Serotonin and dopamine: unifying affective, activational, and decision functions. Neuropsychopharmacology 36, 98-113. (doi:10.1038/npp.2010.121)
68. Dayan P. 2012 Instrumental vigour in punishment and reward. Eur. J. Neurosci. 35, 1152-1168. (doi:10.1111/j.1460-9568.2012.08026.x)
69. Berridge KC. 2007 The debate over dopamine's role in reward: the case for incentive salience. Psychopharmacology 191, 391-431. (doi:10.1007/s00213-006- 0578-x)
70. Lyon M, Robbins T. 1975 The action of central nervous system stimulant drugs: a general theory concerning amphetamine effects. Curr. Dev. Psychopharmcol. 2, 79-163.
71. Salamone JD, Correa M. 2002 Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine. Behav. Brain Res. 137, 3-25. (doi:10.1016/S0166-4328(02)00282-6)