Solving the RNA design problem with reinforcement learning

PLoS Computational Biology

Volumn 14, Issue 6, 2018, Pages

  Eastman, Peter ^a   Shi, Jade ^b   Ramsundar, Bharath ^a   Pande, Vijay S ^a  

^a Stanford University   (United States)

^b STANFORD UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

REINFORCEMENT LEARNING; STRUCTURAL DESIGN;

DESIGN PROBLEMS; FUTURE IMPROVEMENTS; IN-SILICO; LEARN+; REINFORCEMENT LEARNINGS; SECONDARY STRUCTURES; SINGLE MODELS; STRUCTURE DESIGN; TARGET STRUCTURE; TRAINING SETS;

RNA;

ARTICLE; HUMAN; PROTEIN SECONDARY STRUCTURE; REINFORCEMENT; ALGORITHM; CHEMISTRY; COMPUTER AIDED DESIGN; COMPUTER SIMULATION; CONFORMATION; DEVICES; LEARNING; MACHINE LEARNING; PHYSIOLOGY; PROBLEM SOLVING; RNA FOLDING;

RNA;

ALGORITHMS; COMPUTER SIMULATION; COMPUTER-AIDED DESIGN; LEARNING; MACHINE LEARNING; NUCLEIC ACID CONFORMATION; PROBLEM SOLVING; RNA; RNA FOLDING;

EID: 85049380253     PISSN: 1553734X     EISSN: 15537358     Source Type: Journal    
DOI: 10.1371/journal.pcbi.1006176     Document Type: Article

References (28)

1. Goldberg MS, Xing D, Ren Y, Orsulic S, Bhatia SN, Sharp PA, Nanoparticle-mediated delivery of siRNA targeting Parp1 extends survival of mice bearing tumors derived from Brca1-deficient ovarian cancer cells. Proc Natl Acad Sci. 2011;108: 745–750. doi: 10.1073/pnas.1016538108 pmid: 21187397
2. Win MN, Smolke CD, Higher-Order Cellular Information Processing with Synthetic RNA Devices. Science. 2008;322: 456–460. doi: 10.1126/science.1160311 pmid: 18927397
3. Delebecque CJ, Lindner AB, Silver PA, Aldaye FA, Organization of Intracellular Reactions with Rationally Designed RNA Assemblies. Science. 2011;333: 470–474. doi: 10.1126/science.1206938 pmid: 21700839
4. Hao C, Li X, Tian C, Jiang W, Wang G, Mao C, Construction of RNA nanocages by re-engineering the packaging RNA of Phi29 bacteriophage. Nat Commun. 2014;5. doi: 10.1038/ncomms4890 pmid: 24835104
5. Dixon N, Duncan JN, Geerlings T, Dunstan MS, McCarthy JEG, Leys D, et al. Reengineering orthogonally selective riboswitches. Proc Natl Acad Sci. 2010;107: 2830–2835. doi: 10.1073/pnas.0911209107 pmid: 20133756
6. Lorenz R, Bernhart SH, Höner zu Siederdissen C, Tafer H, Flamm C, Stadler PF, et al. ViennaRNA Package 2.0.Algorithms Mol Biol. 2011;6: 26. doi: 10.1186/1748-7188-6-26 pmid: 22115189
7. Domin G, Findeiß S, Wachsmuth M, Will S, Stadler PF, Mörl M, Applicability of a computational design approach for synthetic riboswitches. Nucleic Acids Res. 2017;45: 4108–4119. doi: 10.1093/nar/gkw1267 pmid: 27994029
8. Dotu I, Garcia-Martin JA, Slinger BL, Mechery V, Meyer MM, Clote P, Complete RNA inverse folding: computational design of functional hammerhead ribozymes. Nucleic Acids Res. 2014;42: 11752–11762. doi: 10.1093/nar/gku740 pmid: 25209235
9. Penchovsky R, Breaker RR, Computational design and experimental validation of oligonucleotide-sensing allosteric ribozymes. Nat Biotechnol. 2005;23: 1424–1433. doi: 10.1038/nbt1155 pmid: 16244657
10. Wachsmuth M, Findeiß S, Weissheimer N, Stadler PF, Mörl M, De novo design of a synthetic riboswitch that regulates transcription termination. Nucleic Acids Res. 2013;41: 2541–2551. doi: 10.1093/nar/gks1330 pmid: 23275562
11. Hofacker IL, Fontana W, Stadler PF, Bonhoeffer LS, Tacker M, Schuster P, Fast folding and comparison of RNA secondary structures. Monatshefte Für Chem Chem Mon. 1994;125: 167–188. doi: 10.1007/BF00818163
12. Andronescu M, Fejes AP, Hutter F, Hoos HH, Condon A, A New Algorithm for RNA Secondary Structure Design. J Mol Biol. 2004;336: 607–624. doi: 10.1016/j.jmb.2003.12.041 pmid: 15095976
13. Busch A, Backofen R, INFO-RNA—a fast approach to inverse RNA folding. Bioinformatics. 2006;22: 1823–1831. doi: 10.1093/bioinformatics/btl194 pmid: 16709587
14. Matthies MC, Bienert S, Torda AE, Dynamics in Sequence Space for RNA Secondary Structure Design. J Chem Theory Comput. 2012;8: 3663–3670. doi: 10.1021/ct300267j pmid: 26593011
15. Zadeh JN, Steenberg CD, Bois JS, Wolfe BR, Pierce MB, Khan AR, et al. NUPACK: Analysis and design of nucleic acid systems. J Comput Chem. 2011;32: 170–173. doi: 10.1002/jcc.21596 pmid: 20645303
16. Taneda A, MODENA: a multi-objective RNA inverse folding. Adv Appl Bioinforma Chem AABC. 2010;4: 1–12.
17. Lee J, Kladwang W, Lee M, Cantu D, Azizyan M, Kim H, et al. RNA design rules from a massive open laboratory. Proc Natl Acad Sci. 2014;111: 2122–2127. doi: 10.1073/pnas.1313039111 pmid: 24469816
18. Anderson-Lee J, Fisker E, Kosaraju V, Wu M, Kong J, Lee J, et al. Principles for Predicting RNA Secondary Structure Design Difficulty. J Mol Biol. 2016;428: 748–757. doi: 10.1016/j.jmb.2015.11.013 pmid: 26902426
19. Krizhevsky A, Sutskever I, Hinton GE, Pereira F, Burges CJC, Bottou L, Weinberger KQ, ImageNet Classification with Deep Convolutional Neural Networks. In: Advances in Neural Information Processing Systems 25. Curran Associates, Inc.; 2012. pp. 1097–1105. Available: http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf
20. Graves A, Fernández S, Gomez F, Schmidhuber J, Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. ACM Press; 2006. pp. 369–376. doi: 10.1145/1143844.1143891
21. Wu Y, Schuster M, Chen Z, Le QV, Norouzi M, Macherey W, et al. Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. ArXiv160908144 Cs. 2016; Available: http://arxiv.org/abs/1609.08144
22. Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D, et al. Playing Atari with Deep Reinforcement Learning. ArXiv13125602 Cs. 2013; Available: http://arxiv.org/abs/1312.5602
23. Gu S, Holly E, Lillicrap T, Levine S, Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates. ArXiv161000633 Cs. 2016; Available: http://arxiv.org/abs/1610.00633
24. Silver D, Schrittwieser J, Simonyan K, Antonoglou I, Huang A, Guez A, et al. Mastering the game of Go without human knowledge. Nature. 2017;550: 354–359. doi: 10.1038/nature24270 pmid: 29052630
25. He K, Zhang X, Ren S, Sun J, Deep Residual Learning for Image Recognition. ArXiv151203385 Cs. 2015; Available: http://arxiv.org/abs/1512.03385
26. Mnih V, Badia AP, Mirza M, Graves A, Lillicrap TP, Harley T, et al. Asynchronous Methods for Deep Reinforcement Learning. ArXiv160201783 Cs. 2016; Available: http://arxiv.org/abs/1602.01783
27. DeepChem: Democratizing Deep-Learning for Drug Discovery, Quantum Chemistry, Materials Science and Biology [Internet]. deepchem; 2017. Available: https://github.com/deepchem/deepchem
28. Abadi Martín, Agarwal Ashish, Barham Paul, Brevdo Eugene, Chen Zhifeng, Citro Craig, et al. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems [Internet]. 2015. Available: https://www.tensorflow.org/