Neural basis of utility estimation

Current Opinion in Neurobiology

Volumn 7, Issue 2, 1997, Pages 198-208

  Shizgal, Peter ^a  

^a and Applied Physiology   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ACTION POTENTIAL; BEHAVIOR; BRAIN DEPTH STIMULATION; MEDIAL FOREBRAIN BUNDLE; NERVE FIBER; NONHUMAN; PRIORITY JOURNAL; REINFORCEMENT; REVIEW;

EID: 0031005829     PISSN: 09594388     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0959-4388(97)80008-6     Document Type: Article

References (76)

1. Barth J. Revised edition The End of the Road. 1967;Doubleday, Garden City, NY.
2. Bentham J. Bums JH, Hart HLA. An Introduction to the Principles of Morals and Legislation. 1996;Clarendon Press, Oxford. [Originally published in 1789.].
3. Cabanac M. Pleasure: the common currency. J Theor Biol. 155:1992;173-200.
4. Conover KL, Shizgal P. Competition and summation between rewarding effect of sucrose and lateral hypothalamic stimulation in the rat. Behav Neurosci. 108:1994;537-548.
5. Conover KL, Woodside B, Shizgal P. Effects of sodium depletion on competition and summation between rewarding effects of salt and lateral hypothalamic stimulation in the rat. Behav Neurosci. 108:1994;549-558.
6. Conover KL, Shizgal P. Differential effects of postingestive feedback on the reward value of sucrose and lateral hypothalamic stimulation in the rat. Behav Nuerosci. 108:1994;559-572.
7. of special interest Woodside B, Renaudin A, Shizgal P. Administration of ovarian steroid hormones does not change the reward effectiveness of lateral hypothalamic stimulation in ovariectomized rats. Psychobiology. 24:1996;202-210 Hormonal manipulations that alter sucrose preference and body weight in female rats fail to produce meaningful changes in the threshold for self-stimulation of the medial forebrain bundle. This result is consistent with the findings of Conover and co-workers [5,6] showing that self-stimulation of the medial forebrain bundle is relatively unaffected by physiological manipulations that profoudly alter the rewarding properties of tastants.
8. of special interest Bless EP, McGinnis KA, Mitchell AL, Hartwell M, Mitchell JB. The effects of gonadal steroids on brain stimulation reward in female rats. Behav Brain Res. 82:1997;235-244 Curves relating the rate of lever pressing to the frequency of lateral hypothalamic stimulation were collected from female rats. The curves obtained during estrus were shifted slightly leftward from those collected during diestrus. Similar shifts were observed when curves obtained from ovariectomized rats given estradiol and progesterone were compared to curves obtained from ovariectomized controls. These results were interpreted as evidence that ovarian hormones potentiate the rewarding effect of the lateral hypothalamic stimulation. However, the size of the shifts in the representative curves shown is very small, falling well within the range of variation that served as the criterion for within-session stability. Moreover, larger shifts than this have been produced by performance manipulations, such as adding weight to the lever.
9. Fouriezos G, Bielajew C, Pagotto W. Task difficulty increases thresholds of rewarding brain stimulation. Behav Brain Rev. 37:1990;1-7.
10. of special interest Abrahamsen GC, Berman Y, Carr KD. Curve-shift analysis of self-stimulation in food-restricted rats: relationships between daily meal, plasma corticosterone and reward sensitization. Brain Res. 695:1995;186-194 A demonstration that severe food restriction lowers the threshold for self-stimulation of the lateral hypothalamus. In their present form, the models proposed by Conover et al. [5,6] do not account for this finding. One avenue for investigation in future work is to determine whether differences in electrode placement in this study and those of Conover et al. [5,6] might account for the differential sensitivity to physiological modulation. Another issue that merits attention is whether the effect reported by the authors of this study reflects a specific enhancement of gustatory reward or a more generalized potentiation of appetitive behavior.
11. Rossi J, Panksepp J. Analysis of the relationships between self-stimulation sniffing and brain-stimulation sniffing. Physiol Behav. 51:1992;805-813.
12. Carr KD, Wolinsky TD. Chronic food restriction and weight loss produce opioid facilitation of perifornical hypothalamic self-stimulation. Brin Res. 607:1993;141-148.
13. Yeomans JS. Principles of Brain Stimulation. 1990;Oxford University Press, New York.
14. of outstanding interest Shizgal P, Conover K. On the neural computation of utility. Curr Dir Psychol Sci. 5:1996;37-43 This paper extends the proposal of Conover, Woodside, and Shizgal [5] concernng a common currency for the evaluation of tastants and rewarding electrical brain stimulation. It is argued that rewarding electrical stimulation is able to produce a meaningful neural signal because of the unidimensional nature of coding in the stimulated system. The weighting of the gustatory stimulus by an animal's physiological state appears to occur before the combination of the electrical and gustatory reward signals.
15. Gallistel CR, Shizgal P, Yeomans JS. A portrait of the substrate for self-stimulation. Psychol Rev. 88:1981;228-273.
16. Simmons JM, Gallistel CR. Saturation of subjective reward magnitude as a function of current and pulse frequency. Behav Neurosci. 108:1994;151-160.
17. Heyman GM, Monaghan MM. Reinforcer magnitude (sucrose concentration) and the matching law theory of response strength. J Exp Anal Behav. 61:1994;505-516.
18. Hamilton AL, Stellar JR, Hart EB. Reward, performance, and the response strength method in self-stimulating rats: validation and neuroleptics. Physiol Behav. 35:1985;897-904.
19. Herrnstein RJ. Relative and absolute strength of response as a function of frequency of reinforcement. J Exp Anal Behav. 4:1961;267-272.
20. Herrnstein RJ. On the law of effect. J Exp Anal Behav. 13:1970;243-266.
21. Herrnstein RJ. Formal properties of the matching law. J Exp Anal Behav. 21:1974;159-164.
22. Baum WM, Rachlin H. Choice as time allocation. J Exp Anal Behav. 12:1969;861-874.
23. Miller HL. Matching-based hedonic scaling in the pigeon. J Exp Anal Behav. 26:1976;335-347.
24. Gallistel CR. Foraging for brain stimulation: toward a neurobiology of computation. Cognition. 50:1994;151-170.
25. Gallistel CR, Leon M. Measuring the subjective magnitude of brain stimulation reward by titration with rate of reward. Behav Neurosci. 105:1991;913-925.
26. Mark TA, Gallistel CR. Subjective reward magnitude of medial forebrain stimulation as a function of train duration and pulse frequency. Behav Neurosci. 107:1993;389-401.
27. Davison M, McCarthy D. The Matching Law. 1988;Lawrence Erlbaum Associates, Hillsdale, NJ.
28. Shizgal P, Matthews G. Electrical stimulation of the rat diencephalon: differential effects of interrupted stimulation on on- and off-responding. Brain Res. 129:1977;319-333.
29. Kahneman D, Fredrickson BL, Schreiber CA, Redelmeier DA. When more pain is preferred to less: adding a better end. Psychol Sci. 4:1993;401-405.
30. Mark TA, Gallistel CR. Kinetics of matching. J Exp Pyschol. 20:1994;1-17.
31. Mazur JE. Choice between single and multiple delayed reinforcers. J Exp Anal Behav. 46:1986;67-78.
32. Mazur JE, Stellar JR, Waraczynski M. Self-control choice with electrical stimulation of the brain. Behav Processes. 15:1987;143-153.
33. of special interest Meck WH. Neuropharmacology of timing and time perception. Cogn Brain Res. 3:1996;227-242 A review of the psychological, pharmacological, and anatomical properties of the interval timer that probably measures inter-reinforcement intervals and response - reinforcement delays.
34. Church RM. Properties of the internal clock. Gibbon J, Allan L. Timing and Time Perception. 1984;566-582 New York Academy of Sciences, New York.
35. Gibbon J. Scalar expectancy theory and Weber's law in animal timing. Psychol Rev. 84:1977;279-325.
36. Green L, Rachlin H. Economic substitutability of electrical brain stimulation, food, and water. J Exp Anal Behav. 55:1991;133-143.
37. Rachlin H, Kagel JH, Battalio RC. Substitutability in time allocation. Psychol Rev. 87:1980;355-374.
38. Salzman CD, Murasugi CM, Britten KH, Newsome WT. Microstimulation in visual area MT: effects of direction discrimination performance. J Neurosci. 12:1992;2331-2355.
39. Newsome WT, Salzman CD. The neuronal basis of motion perception. Ciba Found Symp. 174:1993;217-246.
40. Gibbon J, Church RM, Fairhust S, Kacelnik A. Scalar expectancy theory and choice between delayed rewards. Psychol Rev. 95:1988;102-114.
41. Rolls ET, Burton MJ, Mora F. Neurophysiological analysis of brain-stimulation reward in the monkey. Brain Res. 194:1980;339-357.
42. Ono T, Nakamura K, Nishijo H, Fukuda M. Hypothalamic neuron involvement in integration of reward, aversion, and cue signals. J Neurophysiol. 56:1986;63-79.
43. of special interest Boye SM, Rompré PP. Mesencephalic substrate of reward: axonal connections. J Neurosci. 16:1996;3511-3520 Using a behavioral adaptation of the collision method, the authors show that reward-related fibers directly link self-stimulation sites in the caudal mesencephalon and the ventral tegmental area. At least some of the these fibers appear to extend as far as the lateral hypothalamus.
44. Murray B, Shizgal P. Evidence implicating both slow- and fast-conducting fibers in the rewarding effect of medial forebrain bundle stimulation. Behav Brain Res. 63:1994;47-60.
45. of special interest Murray B, Shizgal P. Behavioral measures of conduction velocity and refractory period for reward-relevant axons in the anterior LH and VTA. Physiol Behav. 59:1996;643-652 An extension of earlier work tracing the trajectory of reward-related fibers in the medial forebrain bundle by means of a behavioral adaptation of the collision method. The results suggest that reward-related fibers directly link the rostral portion of the lateral hypothalamus to the the ventral tegmental area.
46. of special interest Wise RA. Addictive drugs and brain stimulation reward. Annu Rev Neurosci. 19:1996;319-340 A review of the relationship between the rewarding effects of drugs of abuse and electrical brain stimulation.
47. Rompré PP, Miliaressis E. Pontine and mesencephalic substrates of self-stimulation. Brain Res. 359:1985;246-259.
48. Yeomans JS. Quantitative measurement of neural post-stimulation excitability with behavioral methods. Physiol Behav. 15:1975;593-602.
49. Yeomans JS. Absolute refractory periods of self-stimulation neurons. Physiol Behav. 22:1979;911-919.
50. Bielajew C, Shizgal P. Behaviorally derived measures of conduction velocity in the substrate for rewarding medial forebrain bundle stimulation. Brain Res. 237:1982;107-119.
51. Shizgal P, Bielajew C, Corbertt D, Skelton R, Yeomans J. Behavioral methods for inferring anatomical linkage between rewarding brain stimulation sites. J Comp Physiol Psychol. 94:1980;227-237.
52. Bielajew C, Shizgal P. Evidence implicating descending fibers in self-stimulation of the medial forebrain bundle. J Neurosci. 6:1986;919-929.
53. Murray B, Shizgal P. Anterolateral lesions of the medial forebrain bundle increase the frequency threshold for self-stimulation of the lateral hypothalamus and ventral tegmental area in the rat. Psychobiology. 19:1991;135-146.
54. of special interest Murray B, Shizgal P. Attenuation of medial forebrain bundle reward by anterior lateral hypothalamic lesions. Behav Brain Res. 75:1996;33-47 A further demonstration that lesios rostral to medial forebrain bundle self-stimulation sites can increase the threshold for the rewarding effect.
55. of outstanding interest Arvanitogiannis A, Waraczynski M, Shizgal P. Effects of excitotoxic lesions of the basal forebrain on MFB self-stimulation. Physiol Behav. 59:1996;795-806 A key finding in support of the 'descending path' hypothesis: excitotoxic lesions in the basal forebrain can produce large increases in the threshold for self-stimulation of the lateral hypothalamus. To account for the large across-subject variance in the magnitude of the lesion effects, the authors propose a model that incorporates multiple subpopulations of reward-related cells and a non-linear reward-growth function.
56. of special interest Arvanitogiannis A, Flores C, Pfaus JG, Shizgal P. Increased ipsilateral expression of Fos following lateral hypothalamic self-stimulation. Brain Res. 720:1996;148-154 Using expression of the immediate-early gene product Fos as an index of neural activation, this paper demonstrates that rewarding electrical stimulation of the lateral hypothalamus produces neural activation in a variety of forebrain regions, predominantly, but not exclusively, in the stimulated hemisphere. Among the areas where the stimulation triggered expression of the Fos protein is the region where lesions increase the threshold for self-stimulation of the medial forebrain bundle.
57. Flores C, Arvanitogiannis A, Shizgal P. Fos-like immunoreactivity in forebrain regions following self-stimulation of the lateral hypothalamus and the ventral tegmental area. Behav Brain Res. 1997;. in press.
58. of special interest Murray B, Shizgal P. Physiological measures of conduction velocity and refractory period for putative reward-relevant MFB axons arising in the rostral MFB. Physiol Behav. 59:1996;427-437 In this study, single-unit recording is employed to measure excitability and conduction properties of fibers arising in the basal forebrain region where lesions reduce the reward effectiveness of stimulation delivered at more caudal sites along the medial forebrain bundle. The axonal refractory periods and conduction velocities of the antidromically activated neurons overlap the values that have been estimated for reward-related fibers by behavioral means. Such cells could compose part of the 'descending path' hypothesized to link the lateral hypothalamic and ventral tegmental levels of the medial forebrain bundle via reward-related fibers.
59. of outstanding interest Gallistel CR, Leon M, Lim BT, Sim JC, Waraczynski M. Destruction of medial forebrain bundle caudal to the site of stimulation reduces rewarding efficacy but destruction rostrally does not. Behav Neurosci. 110:1996;1-25 The authors report that lesions of the ventral tegmental area produce relatively large and consistent increases in the threshold for lateral hypothalamic self-stimulation, whereas lateral hypothalamic lesions produce smaller and much less consistent increases in the threshold for self-stimulation of the ventral tegmental area and/or caudal medial forebrain bundle. These results are interpreted as evidence against the descending path hypothesis, and an imaginative alternative is proposed. However, the new model cannot account for the dozens of cases in which rostrally placed lesions have increased the self-stimulation threshold at more caudal sites along the medial forebrain bundle. Moreover, the authors appear to disregard damage to dopaminergic neurons as a factor contributing to the effects of their ventral tegmental area lesions.
60. Waraczynski M, Conover K, Shizgal P. Rewarding effectiveness of caudal MFB stimulation is unaltered following DMH lesions. Physiol Behav. 52:1992;211-218.
61. Wise RA, Rompré PP. Brain dopamine and reward. Annu Rev Psychol. 40:1989;191-225.
62. of outstanding interest Malette J, Miliaressis E. Interhemispheric links in brain stimulation reward. Behav Brain Res. 68:1995;117-137 Using stimulation trains consisting of pulse pairs delivered to the medial forebrain bundle in each hemisphere, bilateral summation of rewarding effects was studied. Contrary to previous reports, summation at many pairs of sites dependent on the pulse-pair interval. With electrodes at different levels of the neuraxis, the pulse-pair interval at which summationn increased often depended on the order in which the two sites were stimulated. These interesting effects are interpreted in the light of a model of branchpoint failure in neurons with multiple axonal branches. However, key assumptions underlying the branchpoint model appear implausible, and a simpler model, based on synaptic convergence, can account for the results. In the simplest anatomical arrangements compatible with both models, the behaviorally relevant direction of conduction is ascending at the level of the ventral tegmental area.
63. of special interest Buckenham K, Yeomans JS. An uncrossed tectopontine pathway mediates ipsiversive circling. Behav Brain Res. 54:1993;11-22.
64. Yeomans JS. Electrically evoked behaviors: axons and synapses mapped with collision tests. Behav Brain Res. 67:1995;121-132 A review showing how pulse-pair stimulation can be used to trace both axonal and synaptic connections in the neural circuitry subserving behavioral effects of electrical brain stimulation.
65. of special interest Anderson RM, Fatigati MD, Rompré P-P. Estimates of the axonal refractory period of midbrain dopamine neurons: their relevance to brain stimulation reward. Brain Res. 718:1996;83-88 A carefully executed study in which the refractory periods of midbrain dopaminergic axons were estimated. Unlike conventional estimates, the ones obtained by the method employed in this study are relatively unaffected by the properties of the soma and initial segment. Thus, these values can be compared meaningfully to those derived from behavioral experiments on self-stimulation of the medial forebrain bundle, which also are likely to reflect axonal properties.
66. Yeomans JS, Maidment NT, Bunney BS. Excitability properties of medial forebrain bundle axons of A9 and A10 dopamine cells. Brain Res. 450:1988;86-93.
67. Yeomans JS, Mathur A, Tampakeras M. Rewarding brain stimulation: role of tegmental cholinergic neurons that activate dopamine neurons. Behav Neurosci. 197:1993;1077-1087.
68. Miliaressis E, Emond C, Merali Z. Re-evaluation of the role of dopamine in intracranial self-stimulation using in vivo microdialysis. Behav Brain Res. 46:1991;43-48.
69. Wise RA. Action of drugs of abuse on brain reward systems. Pharmacol Biochem Behav. 13:1980;213-223.
70. Blackburn JR, Pfaus JG, Philips AG. Dopamine functions in appetitive and defensive behaviours. Prog Neurobiol. 39:1992;247-279.
71. of special interest Montague PR, Dayan P, Sejnowski TJ. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci. 16:1996;1936-1947 An interesting model in which dopamine neurons signal the difference between experienced and expected rewards. If this model is correct, then the primary reward signal has already been computed before it is relayed to midbrain dopamine neurons.
72. Schultz W, Apicella P, Ljungberg T. Responses of monkey dopamine nuerons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci. 13:1993;900-913.
73. Gallistel CR. The role of the dopaminergic projections in MFB self-stimulation. Behav Brain Res. 20:1986;313-321.
74. of special interest Yeomans J, Baptista M. Both nicotinic and muscarinic receptors in ventral tegmental area contribute to brain-stimulation reward. Pharmacol Biochem Behav. 1997; Evidence that cholinergic input to the ventral tegmental area can modulate the rewarding effect of medial forebrain bundle stimulation.
75. Kofman O, McGlynn SM, Olmstead MC, Yeomans JS. Differential effects of atropine, procaine and dopamine in the rat ventral tegmentum on lateral hypothalamic rewarding brain stimulation. Behav Brain Res. 38:1990;55-68.
76. of special interest Lepore M, Franklin KBJ. N-methyl-D-aspartate lesions of the pedunculopontine nucleus block acquisition and impair maintenance of responding reinforced with brain stimulation. Neuroscience. 71:1996;147-155 Performance of a task in which lever pressing was rewarded by a frequency-modulated train of lateral hypothalamic stimulation was monitored following excitotoxic lesions of the pedunculopontine region; the stimulation was patterned to mimic the waxing and waning of drug action during absorption and elimination. That bilateral lesions blocked acquisition of the task in naive subjects and attenuated performance for BSR in rats trained prior to the lesion is consistent with Yeomans' hypothesis (see [67]) that cholinergic neurons in this region constitute a critical link in the neural circuitry responsible for the rewarding effect. Nonetheless, it is not clear why unilateral lesions were ineffective whereas unilateral administration of drugs that alter the firing of cholinergic somata in the pedunculopontine region has been shown to alter BSR [67]. The use of response rate as the behavioral measure of reward in this study complicates interpretation of the findings. Response rate can be insensitive to changes in reward and may be altered by manipulations that change performance capacity rather than the intensity of reinforcement.