Learning from the past: Approaches for reproducibility in computational neuroscience

20 Years of Computational Neuroscience

Volumn , Issue , 2013, Pages 73-102

  Crook, Sharon M ^a   Davison, Andrew P ^b   Plesser, Hans E ^c  

^a Arizona State University   (United States)

^b Unite de Neurosciences Integratives et Computationnelles   (France)

^c NORWEGIAN UNIVERSITY OF LIFE SCIENCES   (Norway)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85010065988     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1007/978-1-4614-1424-7_4     Document Type: Chapter

References (96)

1. Bower JM, Beeman D (1997) The book of GENESIS: exploring realistic neural models with the GEneral NEural SImulation System. Springer, New York
2. Bhalla U, Bilitch DH, Bower J (1992) Rallpacks: a set of benchmarks for neural simulators. Trends Neurosci 15:453-458
3. Bray T, Paoli J, Sperberg-McQueen C (1998) Extensible markup language (XML) 1.0. http:// www.w3.org/TR/REC-xml
4. Breslow NE (2010) Commentary. Biostatistics 11(3):379-380. doi:10.1093/biostatistics/kxq025
5. Brette R (2006) Exact simulation of integrate-and-fire models with synaptic conductances. Neural Comput 18:2004-2027
6. Brette R (2007) Exact simulation of integrate-and-fire models with exponential currents. Neural Comput 19:2604-2609
7. Brette R, Rudolph M, Carnevale T, Hines M, Beeman D, Bower JM, Diesmann M, Morrison A, Goodman PH, Harris FC Jr, Zirpe M, Natschläger T, Pecevski D, Ermentrout B, Djurfeldt M, Lansner A, Rochel O, Vieville T, Muller E, Davison AP, Boustani SE, Destexhe A (2007) Simulation of networks of spiking neurons: a review of tools and strategies. J Comput Neurosci 23:349-398
8. Brüderle D, Muller E, Davison A, Muller E, Schemmel J, Meier K (2009) Establishing a novel modeling tool: a Python-based interface for a neuromorphic hardware system. Front Neuroinform 3:17. doi:10.3389/neuro.11.017.2009
9. Brunel N (2000) Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons. J Comput Neurosci 8:183-208
10. Cannon R, Howell F, Goddard N, De Schutter E (2002) Non-curated distributed databases for experimental data and models in neuroscience. Network 13:415-428
11. Cannon RC, Gewaltig MO, Gleeson P, Bhalla US, Cornelis H, Hines ML, Howell FW, Muller E, Stiles JR, Wils S, Schutter ED (2007) Interoperability of neuroscience modeling software: current status and future directions. Neuroinformatics 5:127-138
12. Carnevale N, Hines M (2006) The NEURON book. Cambridge University Press, Cambridge, UK
13. Claerbout JF, Karrenbach M (1992) Electronic documents give reproducible research a new meaning. In: SEG expanded abstracts, Society of Exploration Geophysicists, vol 11, pp 601-604
14. Cox DR, Donnelly C (2010) Commentary. Biostatistics 11(3):381-382. doi:10.1093/biostatistics/ kxq026
15. Craver CF (2007) Explaining the brain: mechanisms and the Mosaic Unity of Neuroscience. Oxford University Press, New York
16. Crook S, Howell F (2007) XML for data representation and model specification. In: Crasto C (ed) Methods in molecular biology book series: neuroinformatics. Humana Press, Totowa, NJ, pp 53-66
17. Crook S, Gleeson P, Howell F, Svitak J, Silver R (2007) MorphML: Level 1 of the NeuroML standards for neuronal morphology data and model specification. Neuroinformatics 5:96-104
18. D'Haene M (2010) Efficient simulation strategies for spiking neural networks. PhD thesis, Universiteit Gent
19. Davison A, Morse T, Migliore M, Marenco L, Shepherd G, Hines M (2002) ModelDB: a resource for neuronal and network modeling. In: Kötter R (ed) Neuroscience databases: a practical guide. Kluwer Academic, Norwell, MA, pp 99-122
20. Davison A, Brüderle D, Eppler J, Kremkow J, Muller E, Pecevski D, Perrinet L, Yger P (2009) PyNN: a common interface for neuronal network simulators. Front Neuroinform 2:11. doi:10.3389/neuro.11.011.2008
21. de Leeuw J (2001) Reproducible research: the bottom line. Tech. Rep., Department of Statistics, UCLA, UC Los Angeles. http://escholarship.org/uc/item/9050x4r4
22. DeAngelis CD, Fontanarosa PB (2010) The importance of independent academic statistical analysis. Biostatistics 11(3):383-384. doi:10.1093/biostatistics/kxq027
23. Djurfeldt M, Lansner A (2007) Workshop report: 1st INCF Workshop on Large-scale Modeling of the Nervous System. Available from Nature Precedings http://dx.doi.org/10.1038/npre. 2007.262.1
24. Donoho DL (2010) An invitation to reproducible computational research. Biostatistics 11(3):385-388. doi:10.1093/biostatistics/kxq028
25. Donoho DL, Maleki A, Rahman IU, Shahram M, Stodden V (2009) 15 years of reproducible research in computational harmonic analysis. Comput Sci Eng 11:8-18. doi:10.1109/ MCSE.2009.15
26. Drummond C (2009) Replicability is not reproducibility: nor is it good science. In: Proceedings of the evaluation methods for machine learning workshop at the 26th ICML, Montreal, CA
27. Eppler JM, Helias M, Muller E, Diesmann M, Gewaltig MO (2008) PyNEST: a convenient interface to the NEST simulator. Front Neuroinform 2:12. doi: 10.3389/neuro.11.012.2008
28. Ferrenberg AM, Landau DP, Wong YJ (1992) Monte Carlo simulations: hidden errors from "good" random number generators. Phys Rev Lett69(23):3382-3384. doi:10.1103/PhysRevLett.69.3382
29. Finney A, Hucka M, Bornstein B, Keating S, Shapiro B, Matthews J, Kovitz B, Schilstra M, Funahashi A, Doyle J, Kitano H (2006) Software infrastructure for effective communication and reuse of computational models. In: Szallasi Z, Stelling J, Periwal V (eds) Systems modeling in cell biology: from concepts to nuts and bolts. MIT Press, Boston, pp 369-378
30. Funk W, Dammann V, Donnevert G (2006) Quality assurance in analytical chemistry, 2nd edn. Wiley-VCH, Weinheim
31. Galluppi F, Rast A, Davies S, Furber S (2010) A general-purpose model translation system for a universal neural chip. In: Wong K, Mendis B, Bouzerdoum A (eds) Neural information processing. Theory and algorithms, Lecture notes in computer science, vol 6443. Springer, Berlin, pp 58-65
32. Gardner D, Knuth KH, Abato M, Erde SM, White T, DeBellis R, Gardner EP (2001) Common data model for neuroscience data and data model exchange. J Am Med Inform Assoc 8:17-33
33. Gardner D, Akil H, Ascoli G, Bowden D, Bug W, Donohue D, Goldberg D, Grafstein B, Grethe J, Gupta A, Halavi M, Kennedy D, Marenco L, Martone M, Miller P, Müller H, Robert A, Shepherd G, Sternberg P, Van Essen D, Williams R (2008) The neuroscience information framework: a data and knowledge environment for neuroscience. Neuroinformatics 6(3):149-160
34. Gewaltig MO, Diesmann M (2007) NEST (NEural Simulation Tool). Scholarpedia 2(4):1430
35. Gewaltig MO, Koerner EE (2008) Self-sustained dynamics of sparsely connected networks without external noise. In: 2008 Neuroscience meeting planner, Society for Neuroscience, Washington, DC, program No. 220.1
36. Gleeson P, Steuber V, Silver RA (2007) neuroConstruct: a tool for modeling networks in 3D space. Neuron 54:219-235
37. Gleeson P, Crook S, Cannon RC, Hines ML, Billings GO, Farinella M, Morse TM, Davison AP, Ray S, Bhalla US, Barnes SR, Dimitrova YD, Silver RA (2010) NeuroML: a language for describing data driven models of neurons and networks with a high degree of biological detail. PLoS Comput Biol 6(6):e1000815. doi:10.1371/journal.pcbi.1000815
38. Goddard N, Hucka M, Howell F, Cornells H, Shankar K, Beeman D (2001) NeuroML: model description methods for collaborative modelling in neuroscience. Philos Trans R Soc Lond B Biol Sci 356:1209-1228
39. Goodman D (2010a) Code generation: a strategy for neural network simulators. Neuroinformatics 8:183-196. doi:10.1007/s12021-010-9082-x
40. Goodman SN (2010b) Commentary. Biostatistics 11(3):389-390. doi:10.1093/biostatistics/ kxq030
41. Goodman D, Brette R (2008) Brian: a simulator for spiking neural networks in Python. Front Neuroinform 2:5. doi:10.3389/neuro.11.005.2008
42. Gorchetchnikov A, The INCF Multiscale Modeling Taskforce (2010) NineML-a description language for spiking neuron network modeling: the user layer. BMC Neurosci 11(suppl 1):P71
43. Grebogi C, Hammel SM, Yorke JA, Sauer T (1990) Shadowing of physical trajectories in chaotic dynamics: containment and refinement. Phys Rev Lett 65(13):1527-1530. doi:10.1103/ PhysRevLett.65.1527
44. Groves T (2010) The wider concept of data sharing: view from the BMJ. Biostatistics 11(3):391-392. doi:10.1093/biostatistics/kxq031
45. Gruntz D, Waldvogel J (2004) Orbits in the planar three-body problem. In: Gander W, Hrebicek J (eds) Solving problems in scientific computing using Maple and MATLAB, 4th edn. Springer, Berlin, pp 51-72
46. Hansel D, Mato G, Meunier C, Neltner L (1998) On numerical simulations of integrate-and-fire neural networks. Neural Comput 10:467-483
47. Hanuschkin A, Kunkel S, Helias M, Morrison A, Diesmann M (2010) A general and efficient method for incorporating exact spike times in globally time-driven simulations. Front Neuroinform 4:113. doi:10.3389/fninf.2010.00113
48. Hedley W, Nelson M, Nielsen P, Bullivant D, Hunter P (2000) XML languages for describing biological models. In: Proceedings of the Physiological Society of New Zealand, vol 19
49. Heijmen T (2011) Soft errors from space to ground: historical overview, empirical evidence, and future trends (chap 1). In: Nicolaidis M (ed) Soft errors in modern electronic systems, Frontiers in electronic testing, vol 41. Springer, New York, pp 1-25
50. Hines M (1989) A program for simulation of nerve equations with branching geometries. Int J Biomed Comput 24:55-68
51. Hines ML, Carnevale NT (1997) The NEURON simulation environment. Neural Comput 9(6):1179-1209
52. Hines ML, Carnevale NT (2000) Expanding NEURON's repertoire of mechanisms with NMODL. Neural Comput 12:995-1007
53. Hines ML, Morse T, Migliore M, Carnevale NT, Shepherd GM (2004) ModelDB: a database to support computational neuroscience. J Comput Neurosci 17(1):7-11. doi:10.1023/B:JCNS.0000023869.22017.2e
54. Hucka M, Finney A, Sauro H, Bolouri H, Doyle J, Kitano H, Arkin A (2003) The systems biology markup language (SBML): a medium for representation and exchange of biochemical network models. Bioinformatics 19:524-531
55. Humphreys P (2004) Extending ourselves: computational science, empiricism, and scientific method. Oxford University Press, Oxford
56. Hund F (1996) Geschichte der physikalischen Begriffe. Spektrum Akademischer Verlag, Heidelberg
57. Ioannidis JPA (2005) Why most published research findings are false. PLoS Med 2:e124. doi:10.1371/journal.pmed.0020124
58. Jones E, Oliphant T, Peterson P et al (2001) SciPy: open source scientific tools for Python. http:// www.scipy.org/
59. Keiding N (2010a) Reproducible research and the substantive context. Biostatistics 11(3):376-378. doi:10.1093/biostatistics/kxq033
60. Keiding N (2010b) Reproducible research and the substantive context: response to comments. Biostatistics 11(3):395-396. doi:10.1093/biostatistics/kxq034
61. Kilkenny C, Browne WJ, Cuthill IC, Emerson M, Altman DG (2010) Improving bioscience research reporting: the ARRIVE guidelines for reporting animal research. PLoS Biol 8(6):e1000412. doi:10.1371/journal.pbio.1000412
62. Köhn D, Le Novère N (2008) SED-ML-an XML format for the implementation of the MIASE guidelines. In: Heiner M, Uhrmacher A (eds) Computational methods in systems biology, Lecture notes in computer science, vol 5307. Springer, Berlin, pp 176-190
63. Küppers G, Lenhard J (2005) Validation of simulation: patterns in the social and natural sciences. J Artif Soc Soc Simul 8(4):3
64. Lapicque L (1907) Recherches quantitatives sur l'excitation électrique des nerfs traitée comme une polarization. J Physiol Pathol Gen 9:620-635
65. Le Novere N, Bornstein B, Broicher A, Courtot M, Donizelli M, Dharuri H, Li L, Sauro H, Schilstra M, Shapiro B, Snoep J, Hucka M (2006) BioModels database: a free, centralized database of curated, published, quantitative kinetic models of biochemical and cellular systems. Nucleic Acids Res 34(Database issue):D689-D691
66. Lehrer J (2010) The truth wears off. The New Yorker, 13 Dec 2010:52
67. Lloyd C, Halstead M, Nielsen P (2004) CellML: its future, present and past. Prog Biophys Mol Biol 85:433-450
68. Lloyd CM, Lawson JR, Hunter PJ, Nielsen PF (2008) The CellML model repository. Bioinformatics 24(18):2122-2123. doi:10.1093/bioinformatics/btn390
69. Ludäscher B, Altintas I, Berkley C, Higgins D, Jaeger-Frank E, Jones M, Lee E, Tao J, Zhao Y (2006) Scientific workflow management and the Kepler system. Concurrency Comput Pract Exp 18(10):1039-1065
70. Marenco L, Wang R, Shepherd G, Miller P (2010) The NIF DISCO framework: facilitating automated integration of neuroscience content on the web. Neuroinformatics 8(2):101-112
71. Maunsell J (2010) Announcement regarding supplemental material. J Neurosci 30:10599-10600
72. Mesirov JP (2010) Accessible reproducible research. Science 327(5964):415-416. doi:10.1126/ science.1179653
73. Migliore M, Morse TM, Davison AP, Marenco L, Shepherd GM, Hines ML (2003) ModelDB: making models publicly accessible to support computational neuroscience. Neuroinformatics 1(1):135-139
74. Miner R (2005) The importance of MathML to mathematical communication. Notices AMS 52:532-538
75. Morrison A, Straube S, Plesser HE, Diesmann M (2007) Exact subthreshold integration with continuous spike times in discrete time neural network simulations. Neural Comput 19:47-79
76. Morse TM (2007) Model sharing in computational neuroscience. Scholarpedia 2(4):30-36
77. Nordlie E, Plesser HE (2010) Visualizing neuronal network connectivity with connectivity pattern tables. Front Neuroinform 3:39. doi:10.3389/neuro.11.039.2009
78. Nordlie E, Gewaltig MO, Plesser HE (2009) Towards reproducible descriptions of neuronal network models. PLoS Comput Biol 5(8):e1000456. doi:10.1371/journal.pcbi.1000456
79. Oinn T, Greenwood M, Addis M, Alpdemir M, Ferris J, Glover K, Goble C, Goderis A, Hull D, Marvin D, Li P, Lord P, Pocock MR, Senger M, Stevens R, Wipat A, Wroe C (2006) Taverna: lessons in creating a workflow environment for the life sciences. Concurr Comput Pract Exp 18(10):1067-1100. doi:10.1002/cpe.993
80. Pecevski D, Natschläger T, Schuch K (2009) PCSIM: a parallel simulation environment for neural circuits fully integrated with Python. Front Neuroinform 3:11. doi:10.3389/neuro.11.011.2009
81. Peng RD (2009) Reproducible research and biostatistics. Biostatistics 10(3):405-408. doi: 10.1093/ biostatistics/kxp014
82. Peng RD (2010) Discussion of Keiding. Biostatistics 11(3):393-394. doi:10.1093/biostatistics/ kxq032
83. Peterson B, Healy M, Nadkarni P, Miller P, Shepherd G (1996) ModelDB: an environment for running and storing computational models and their results applied to neuroscience. J Am Med Inform Assoc 3:389-398
84. Petzold L (1983) Automatic selection of methods for solving stiff and nonstiff systems of ordinary differential equations. SIAM J Sci Comput 4:136-148. doi:10.1137/0904010
85. Plesser HE, Austvoll K (2009) Specification and generation of structured neuronal network models with the NEST topology module. BMC Neurosci 10(suppl 1):P56. doi: 10.1186/1471-2202-10-S1-P56
86. Quirk JJ (2005) Computational science: "same old silence, same old mistakes; something more is needed". In: Plewa T, Linde T, Weirs VG (eds) Adaptive mesh refinement-theory and applications. Springer, Berlin, pp 3-28
87. Raikov I, The INCF Multiscale Modeling Taskforce (2010) NineML-a description language for spiking neuron network modeling: the abstraction layer. BMC Neurosci 11(suppl 1):P66
88. Rotter S, Diesmann M (1999) Exact digital simulation of time-invariant linear systems with applications to neuronal modeling. Biol Cybern 81:381-402
89. Schrödinger E (1915) Zur Theorie der Fall-und Steigversuche an Teilchen mit Brownscher Bewegung. Phys Zeit 16:289-295
90. Shelley MJ, Tao L (2001) Efficient and accurate time-stepping schemes for integrate-and-fire neuronal networks. J Comput Neurosci 11:111-119
91. Silva CT, Freire J, Callahan S (2007) Provenance for visualizations: reproducibility and beyond. Comput Sci Eng 9(5):82-90
92. Stodden V (2009a) Enabling reproducible research: open licensing for scientific innovation. Int J Commun Law Policy 13:2-25, http://www.ijclp.net/files/ijclp_web-doc_1-13-2009.pdf
93. Stodden V (2009b) The legal framework for reproducible research in the sciences: licensing and copyright. IEEE Comput Sci Eng 11(1):35-40
94. Usui S (2003) Visiome: neuroinformatics research in vision project. Neural Netw 16:1293-1300
95. Vandewalle P, Kovacevic J, Vetterli M (2009) Reproducible research in signal processing. IEEE Signal Proc Mag 26(3):37-47. doi:10.1109/MSP.2009.932122
96. Wilson MA, Bhalla US, Uhley JD, Bower JM (1989) GENESIS: a system for simulating neural networks. In: Touretzky D (ed) Advances in neural information processing systems. Morgan Kaufmann, San Mateo, CA, pp 485-492