Characterizing the dynamics of mental representations: The temporal generalization method

Trends in Cognitive Sciences

Volumn 18, Issue 4, 2014, Pages 203-210

  King, J R ^a,b   Dehaene, S ^a,b,c,d  

^a INSERM   (France)

^b DIF   (France)

^c UNIVERSITÉ PARIS SACLAY   (France)

^d Université Paris 11 ^*  (France)

Author keywords

Decoding; EEG; MEG; Multivariate pattern analyses; Parallel processing; Serial processing; Temporal generalization

Indexed keywords

BRAIN MAPPING; CURRICULA; DECODING; ELECTROENCEPHALOGRAPHY; ELECTROPHYSIOLOGY; MAGNETOENCEPHALOGRAPHY; MULTIVARIANT ANALYSIS; NEUROPHYSIOLOGY;

MEG; MULTIVARIATE PATTERN ANALYSIS; PARALLEL PROCESSING; SERIAL PROCESSING; TEMPORAL GENERALIZATION;

BRAIN;

ATTENTION; BRAIN REGION; CLASSIFIER; COGNITION; DECISION MAKING; DREAM; ELECTROENCEPHALOGRAM; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; IMAGERY; INFERIOR TEMPORAL CORTEX; MENTAL REPRESENTATION; NONHUMAN; PHONEME; PREFRONTAL CORTEX; PSYCHOLOGICAL REFRACTORY PERIOD; PSYCHOPHYSIOLOGY; REM SLEEP; RESPONSE TIME; REVIEW; SIGNAL DETECTION; STIMULUS RESPONSE; SUPERIOR TEMPORAL GYRUS; THETA RHYTHM; VISUAL STIMULATION; WORKING MEMORY;

DECODING; EEG; MEG; MULTIVARIATE PATTERN ANALYSES; PARALLEL PROCESSING; SERIAL PROCESSING; TEMPORAL GENERALIZATION;

BRAIN; BRAIN MAPPING; ELECTROENCEPHALOGRAPHY; HUMANS; MAGNETOENCEPHALOGRAPHY; MENTAL PROCESSES; NONLINEAR DYNAMICS;

EID: 84897072624     PISSN: 13646613     EISSN: 1879307X     Source Type: Journal    
DOI: 10.1016/j.tics.2014.01.002     Document Type: Review

References (83)

1. Donders F.C. On the speed of mental processes. Acta Psychol. (Amst.) 1969, 30:412-431.
2. Sternberg S. The discovery of processing stages: extensions of Donders' method. Acta Psychol. (Amst.) 1969, 30:276-315.
3. Pashler H. Dual-task interference in simple tasks: data and theory. Psychol. Bull. 1994, 116:220-244.
4. Townsend J.T. Serial vs parallel processing: sometimes they look like Tweedledum and Tweedledee but they can (and should) be distinguished. Psychol. Sci. 1990, 1:46-54.
5. McClelland J.L. On the time-relations of mental processes: an examination of systems of processes in cascade. Psychol. Rev. 1979, 86:287-330.
6. Dehaene S. The organization of brain activations in number comparison: event-related potentials and the additive-factors methods. J. Cogn. Neurosci. 1996, 8:47-68.
7. Nishitani N., Hari R. Viewing lip forms: cortical dynamics. Neuron 2002, 36:1211-1220.
8. Kanwisher N., et al. The fusiform face area: a module in human extrastriate cortex specialized for face perception. J. Neurosci. 1997, 17:4302.
9. Quian Quiroga R., Panzeri S. Extracting information from neuronal populations: information theory and decoding approaches. Nat. Rev. Neurosci. 2009, 10:173-185.
10. Haynes J-D.D., Rees G. Decoding mental states from brain activity in humans. Nat. Rev. Neurosci. 2006, 7:523-534.
11. Kriegeskorte N., Kievit R.A. Representational geometry: integrating cognition, computation, and the brain. Trends Cogn. Sci. 2013, 17:401-412.
12. Kamitani Y., Tong F. Decoding the visual and subjective contents of the human brain. Nat. Neurosci. 2005, 8:679-685.
13. Harrison S.A., Tong F. Decoding reveals the contents of visual working memory in early visual areas. Nature 2009, 458:632-635.
14. Kok P., et al. Less is more: expectation sharpens representations in the primary visual cortex. Neuron 2012, 75:265-270.
15. Freeman J., et al. Orientation decoding depends on maps, not columns. J. Neurosci. 2011, 31:4792-4804.
16. Brouwer G.J., Heeger D.J. Decoding and reconstructing color from responses in human visual cortex. J. Neurosci. 2009, 29:13992-14003.
17. Gallant J., et al. Bayesian reconstruction of perceptual experiences from human brain activity. Foundations of Augmented Cognition. Neuroergonomics and Operational Neuroscience 2009, 393. Springer.
18. Kay K.N., Gallant J.L. I can see what you see. Nat. Neurosci. 2009, 12:245.
19. Miyawaki Y., et al. Visual image reconstruction from human brain activity using a combination of multiscale local image decoders. Neuron 2008, 60:915-929.
20. Thirion B., et al. Inverse retinotopy: inferring the visual content of images from brain activation patterns. Neuroimage 2006, 33:1104-1116.
21. Gramfort A., et al. Decoding visual percepts induced by word reading with fMRI. 2012 Second International Workshop on Pattern Recognition in NeuroImaging 2012, 13-16. IEEE Computer Society.
22. Van Gerven M.A.J., et al. Dynamic decoding of ongoing perception. Neuroimage 2011, 57:950-957.
23. Nishimoto S., et al. Reconstructing visual experiences from brain activity evoked by natural movies. Curr. Biol. 2011, 21:1641-1646.
24. Lee S-H., et al. Disentangling visual imagery and perception of real-world objects. Neuroimage 2012, 59:4064-4073.
25. Albers A.M., et al. Shared representations for working memory and mental imagery in early visual cortex. Curr. Biol. 2013, 23:1427-1431.
26. Christophel T.B., et al. Decoding the contents of visual short-term memory from human visual and parietal cortex. J. Neurosci. 2012, 32:12983-12989.
27. Horikawa T., et al. Neural decoding of visual imagery during sleep. Science 2013, 340:639-642.
28. Formisano E., et al. "Who" is saying "what"? Brain-based decoding of human voice and speech. Science 2008, 322:970-973.
29. Giordano B.L., et al. Abstract encoding of auditory objects in cortical activity patterns. Cereb. Cortex 2013, 23:2025-2037.
30. Rissman J., et al. Detecting individual memories through the neural decoding of memory states and past experience. Proc. Natl. Acad. Sci. U.S.A. 2010, 107:9849-9854.
31. Knops A., et al. Recruitment of an area involved in eye movements during mental arithmetic. Science 2009, 324:1583-1585.
32. Cole M.W., et al. Multi-task connectivity reveals flexible hubs for adaptive task control. Nat. Neurosci. 2013, 16:1348-1355.
33. Zhang J., et al. Choosing the rules: distinct and overlapping frontoparietal representations of task rules for perceptual decisions. J. Neurosci. 2013, 33:11852-11862.
34. Charles L., et al. Decoding the dynamics of action, intention, and error-detection for conscious and subliminal stimuli. J. Neurosci. 2014, 34:1158-1170.
35. Carlson T.A. High temporal resolution decoding of object position and category. J. Vis. 2011, 11:1-17.
36. Isik L., et al. The dynamics of invariant object recognition in the human visual system. J. Neurophysiol. 2013, 10.1152/jn.00394.2013.
37. Garcia J.O., et al. Near-real-time feature-selective modulations in human cortex. Curr. Biol. 2013, 23:515-522.
38. Ramkumar P., et al. Feature-specific information processing precedes concerted activation in human visual cortex. J. Neurosci. 2013, 33:7691-7699.
39. Duncan K.K., et al. Identifying spatially overlapping local cortical networks with MEG. Hum. Brain Mapp. 2010, 31:1003-1016.
40. Philiastides M.G., et al. Neural representation of task difficulty and decision making during perceptual categorization: a timing diagram. J. Neurosci. 2006, 26:8965-8975.
41. Ratcliff R., et al. Quality of evidence for perceptual decision making is indexed by trial-to-trial variability of the EEG. Proc. Natl. Acad. Sci. U.S.A. 2009, 106:6539-6544.
42. Philiastides M.G., Sajda P. Temporal characterization of the neural correlates of perceptual decision making in the human brain. Cereb. Cortex 2006, 16:509-518.
43. Carlson T., et al. Representational dynamics of object vision: the first 1000ms. J. Vis. 2013, 13:1-19.
44. Deng S., et al. EEG classification of imagined syllable rhythm using Hilbert spectrum methods. J. Neural Eng. 2010, 7:046006.
45. Waldert S., et al. Hand movement direction decoded from MEG and EEG. J. Neurosci. 2008, 28:1000-1008.
46. King J-R., et al. Single-trial decoding of auditory novelty responses facilitates the detection of residual consciousness. Neuroimage 2013, 83C:726-738.
47. Chan A.M., et al. Decoding word and category-specific spatiotemporal representations from MEG and EEG. Neuroimage 2011, 54:3028-3039.
48. Sudre G., et al. Tracking neural coding of perceptual and semantic features of concrete nouns. Neuroimage 2012, 62:451-463.
49. Guimaraes M.P., et al. Single-trial classification of MEG recordings. IEEE Trans. Biomed. Eng. 2007, 54:436-443.
50. Schaefer R.S., et al. Name that tune: decoding music from the listening brain. Neuroimage 2011, 56:843-849.
51. Schyns P.G., et al. Cracking the code of oscillatory activity. PLoS Biol. 2011, 9:e1001064.
52. Fuentemilla L., et al. Theta-coupled periodic replay in working memory. Curr. Biol. 2010, 20:606-612.
53. Hämäläinen M., et al. Magnetoencephalography - theory, instrumentation, and applications to noninvasive studies of the working human brain. Rev. Mod. Phys. 1993, 65:413-497.
54. Niessing J., Friedrich R.W. Olfactory pattern classification by discrete neuronal network states. Nature 2010, 465:47-52.
55. Chang E.F., et al. Categorical speech representation in human superior temporal gyrus. Nat. Neurosci. 2010, 13:1428-1432.
56. Bouchard K.E., et al. Functional organization of human sensorimotor cortex for speech articulation. Nature 2013, 495:327-332.
57. King J-R., et al. Two distinct dynamic modes subtend the detection of unexpected sounds. PLoS ONE 2014, 10.1371/journal.pone.0085791.
58. Zhang Y., et al. Object decoding with attention in inferior temporal cortex. Proc. Natl. Acad. Sci. U.S.A. 2011, 108:8850-8855.
59. Meyers E.M., et al. Dynamic population coding of category information in inferior temporal and prefrontal cortex. J. Neurophysiol. 2008, 100:1407-1419.
60. Stokes M.G., et al. Dynamic coding for cognitive control in prefrontal cortex. Neuron 2013, 78:364-375.
61. Nikolić D., et al. Distributed fading memory for stimulus properties in the primary visual cortex. PLoS Biol. 2009, 7:e1000260.
62. Wilson M.A., McNaughton B.L. Reactivation of hippocampal ensemble memories during sleep. Science 1994, 265:676-679.
63. Louie K., Wilson M.A. Temporally structured replay of awake hippocampal ensemble activity during rapid eye movement sleep. Neuron 2001, 29:145-156.
64. Poe G.R., et al. Experience-dependent phase-reversal of hippocampal neuron firing during REM sleep. Brain Res. 2000, 855:176-180.
65. Lee A.K., Wilson M.A. Memory of sequential experience in the hippocampus during slow wave sleep. Neuron 2002, 36:1183-1194.
66. Davidson T.J., et al. Hippocampal replay of extended experience. Neuron 2009, 63:497-507.
67. Pfeiffer B.E., Foster D.J. Hippocampal place-cell sequences depict future paths to remembered goals. Nature 2013, 497:74-79.
68. Foster D.J., Wilson M.A. Reverse replay of behavioural sequences in hippocampal place cells during the awake state. Nature 2006, 440:680-683.
69. Dave A.S., Margoliash D. Song replay during sleep and computational rules for sensorimotor vocal learning. Science 2000, 290:812-816.
70. Shepard R.N. Toward a universal law of generalization for psychological science. Science 1987, 237:1317-1323.
71. Haynes J-D., et al. Reading hidden intentions in the human brain. Curr. Biol. 2007, 17:323-328.
72. Kriegeskorte N., et al. Information-based functional brain mapping. Proc. Natl. Acad. Sci. U.S.A. 2006, 103:3863-3868.
73. Schurger A., et al. Reducing multi-sensor data to a single time course that reveals experimental effects. BMC Neurosci. 2013, 14:122.
74. DiCarlo J.J., et al. How does the brain solve visual object recognition?. Neuron 2012, 73:415-434.
75. Li N., DiCarlo J.J. Unsupervised natural visual experience rapidly reshapes size-invariant object representation in inferior temporal cortex. Neuron 2010, 67:1062-1075.
76. Chang C., Lin C. LIBSVM: a library for support vector machines. Computer 2001, 2:1-30.
77. Knill D.C., Pouget A. The Bayesian brain: the role of uncertainty in neural coding and computation. Trends Neurosci. 2004, 27:712-719.
78. Heinzle J., et al. Topographically specific functional connectivity between visual field maps in the human brain. Neuroimage 2011, 56:1426-1436.
79. Freiwald W.A., et al. A face feature space in the macaque temporal lobe. Nat. Neurosci. 2009, 12:1187-1196.
80. Brincat S.L., Connor C.E. Underlying principles of visual shape selectivity in posterior inferotemporal cortex. Nat. Neurosci. 2004, 7:880-886.
81. Mineault P.J., et al. Hierarchical processing of complex motion along the primate dorsal visual pathway. Proc. Natl. Acad. Sci. U.S.A. 2012, 109:E972-E980.
82. Rust N.C., Dicarlo J.J. Selectivity and tolerance ("invariance") both increase as visual information propagates from cortical area V4 to IT. J. Neurosci. 2010, 30:12978-12995.
83. Williams M.A., et al. Only some spatial patterns of fMRI response are read out in task performance. Nat. Neurosci. 2007, 10:685-686.