A generative vision model that trains with high data efficiency and breaks text-based CAPTCHAs

Science

Volumn 358, Issue 6368, 2017, Pages

  George, Dileep ^a   Lehrach, Wolfgang ^a   Kansky, Ken ^a   Lázaro Gredilla, Miguel ^a   Laan, Christopher ^a   Marthi, Bhaskara ^a   Lou, Xinghua ^a   Meng, Zhaoshi ^a   Liu, Yi ^a   Wang, Huayan ^a   Lavin, Alex ^a   Phoenix, D Scott ^a  

^a VICARIOUS AI   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; DATA; HEURISTICS; LEARNING; MACHINE LEARNING; MODEL; PROBABILITY; SEGMENTATION;

ARTICLE; ARTIFICIAL NEURAL NETWORK; BENCHMARKING; COMPUTER HEURISTICS; DATA BASE; HUMAN; MACHINE LEARNING; PRIORITY JOURNAL; PRODUCTIVITY; RECOGNITION; RECURSIVE CORTICAL NETWORK; TEXT MESSAGING; COMPUTER; STATISTICAL MODEL; VISION;

COMPUTERS; HUMANS; MACHINE LEARNING; MODELS, STATISTICAL; VISUAL PERCEPTION;

EID: 85032354846     PISSN: 00368075     EISSN: 10959203     Source Type: Journal    
DOI: 10.1126/science.aag2612     Document Type: Article

References (62)

1. B. M. Lake, R. Salakhutdinov, J. B. Tenenbaum, Human-level concept learning through probabilistic program induction. Science 350, 1332-1338 (2015). doi: 10.1126/science.aab3050; pmid: 26659050
2. K. Chellapilla, P. Simard, “Using machine learning to break visual human interaction proofs (HIPs),” in Advances in Neural Information Processing Systems 17 (2004), pp. 265-272.
3. E. Bursztein, M. Martin, J. C. Mitchell, “Text-based CAPTCHA strengths and weaknesses,” in Proceedings of the 18th ACM Conference on Computer and Communications Security (ACM, 2011), pp. 125-138.
4. G. Mori, J. Malik, “Recognizing objects in adversarial clutter: Breaking a visual CAPTCHA,” in 2003 IEEE Conference on Computer Vision and Pattern Recognition (IEEE Computer Society, 2003), pp. I-134–I-141.
5. V. Mansinghka, T. D. Kulkarni, Y. N. Perov, J. Tenenbaum, “Approximate Bayesian image interpretation using generative probabilistic graphics programs,” in Advances in Neural Information Processing Systems 26 (2013), pp. 1520–1528.
6. I. Goodfellow, Y. Bulatov, J. Ibarz, S. Arnoud, V. Shet, “Multi-digit number recognition from street view imagery using deep convolutional neural networks,” paper presented at the International Conference on Learning Representations (ICLR) 2014, Banff, Canada, 14 to 16 April 2014.
7. E. Bursztein, J. Aigrain, A. Moscicki, J. C. Mitchell, “The end is nigh: Generic Solving of text-based CAPTCHAs,” paper presented at the 8th USENIX Workshop on Offensive Technologies (WOOT’14), San Diego, CA, 19 August 2014.
8. D. R. Hofstadter, Metamagical Themas: Questing for the Essence of Mind and Pattern (Basic Books, 1985).
9. T. S. Lee, D. Mumford, Hierarchical Bayesian inference in the visual cortex. JOSA A 20, 1434–1448 (2003). doi: 10.1364/JOSAA.20.001434; pmid: 12868647
10. T. L. Griffiths, N. Chater, C. Kemp, A. Perfors, J. B. Tenenbaum, Probabilistic models of cognition: Exploring representations and inductive biases. Trends Cogn. Sci. 14, 357–364 (2010). doi: 10.1016/j.tics.2010.05.004; pmid: 20576465
11. T. S. Lee, “The visual system’s internal model of the world,” in Proceedings of the IEEE, vol. 103 (IEEE, 2015), pp. 1359–1378
12. D. Kersten, A. Yuille, Bayesian models of object perception. Curr. Opin. Neurobiol. 13, 150–158 (2003). doi: 10.1016/S0959-4388(03)00042-4; pmid: 12744967
13. C. D. Gilbert, W. Li, Top-down influences on visual processing. Nat. Rev. Neurosci. 14, 350–363 (2013). doi: 10.1038/nrn3476; pmid: 23595013
14. V. A. F. Lamme, P. R. Roelfsema, The distinct modes of vision offered by feedforward and recurrent processing. Trends Neurosci. 23, 571–579 (2000). doi: 10.1016/S0166-2236(00)01657-X; pmid: 11074267
15. P. R. Roelfsema, V. A. F. Lamme, H. Spekreijse, Object-based attention in the primary visual cortex of the macaque monkey. Nature 395, 376–381 (1998). doi: 10.1038/26475; pmid: 9759726
16. E. H. Cohen, F. Tong, Neural mechanisms of object-based attention. Cereb. Cortex 25, 1080–1092 (2015). doi: 10.1093/cercor/bht303; pmid: 24217991
17. D. J. Field, A. Hayes, R. F. Hess, Contour integration by the human visual system: Evidence for a local “association field”. Vision Res. 33, 173–193 (1993). doi: 10.1016/0042-6989(93)90156-Q; pmid: 8447091
18. C. D. Gilbert, T. N. Wiesel, Columnar specificity of intrinsic horizontal and corticocortical connections in cat visual cortex. J. Neurosci. 9, 2432–2442 (1989). pmid: 2746337
19. E. Craft, H. Schütze, E. Niebur, R. von der Heydt, A neural model of figure-ground organization. J. Neurophysiol. 97, 4310–4326 (2007). doi: 10.1152/jn.00203.2007; pmid: 17442769
20. V. A. F. Lamme, V. Rodriguez-Rodriguez, H. Spekreijse, Separate processing dynamics for texture elements, boundaries and surfaces in primary visual cortex of the macaque monkey. Cereb. Cortex 9, 406–413 (1999). doi: 10.1093/cercor/9.4.406; pmid: 10426419
21. E. A. DeYoe, D. C. Van Essen, Concurrent processing streams in monkey visual cortex. Trends Neurosci. 11, 219–226 (1988). doi: 10.1016/0166-2236(88)90130-0; pmid: 2471327
22. X. Huang, M. A. Paradiso, V1 response timing and surface filling-in. J. Neurophysiol. 100, 539–547 (2008). doi: 10.1152/jn.00997.2007; pmid: 18509081
23. H. Zhou, H. S. Friedman, R. von der Heydt, Coding of border ownership in monkey visual cortex. J. Neurosci. 20, 6594–6611 (2000). pmid: 10964965
24. S.-C. Zhu, D. Mumford, A Stochastic Grammar of Images (Now Publishers, 2007).
25. L. L. Zhu, Y. Chen, A. Yuille, Recursive compositional models for vision: Description and review of recent work. J. Math. Imaging Vis. 41, 122–146 (2011). doi: 10.1007/s10851-011-0282-2
26. A. L. Yuille, “Towards a theory of compositional learning and encoding of objects,” in 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops) (IEEE, 2011), pp. 1448–1455.
27. R. Salakhutdinov, J. B. Tenenbaum, A. Torralba, “Learning to learn with compound HD models,” in Advances in Neural Information Processing Systems 24 (2012), pp. 1–9.
28. Y. Chen, L. Zhu, C. Lin, A. Yuille, H. Zhang, “Rapid inference on a novel AND/OR graph for object detection, segmentation and parsing,” in Advances in Neural Information Processing Systems 20 (2007), pp. 289–296.
29. L. Zhu, A. Yuille, “Recursive compositional models: Representation, learning, and inference,” in 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2009), p. 5.
30. Z. Tu, X. Chen, A. L. Yuille, S.-C. Zhu, Image parsing: Unifying segmentation, detection, and recognition. Int. J. Comput. Vis. 63, 113–140 (2005). doi: 10.1007/s11263-005-6642-x
31. S. Fidler, A. Leonardis, “Towards scalable representations of object categories: Learning a hierarchy of parts,” in 2007 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2007), pp. 1–8.
32. Y. Jin, S. Geman, “Context and hierarchy in a probabilistic image model,” in 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2 (IEEE, 2006), pp. 2145–2152.
33. Supplementary materials.
34. Z. Si, S.-C. Zhu, Learning AND-OR templates for object recognition and detection. IEEE Trans. Pattern Anal. Mach. Intell. 35, 2189–2205 (2013).
35. S. Geman, Invariance and selectivity in the ventral visual pathway. J. Physiol. Paris 100, 212–224 (2006). doi: 10.1016/j.jphysparis.2007.01.001; pmid: 17336506
36. T. Serre, L. Wolf, S. Bileschi, M. Riesenhuber, T. Poggio, Robust object recognition with cortex-like mechanisms. IEEE Trans. Pattern Anal. Mach. Intell. 29, 411–426 (2007). doi: 10.1109/TPAMI.2007.56; pmid: 17224612
37. C. Guo, S.-C. Zhu, Y. N. Wu, Primal sketch: Integrating structure and texture. Comput. Vis. Image Underst. 106, 5–19 (2007). doi: 10.1016/j.cviu.2005.09.004
38. J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference (Morgan Kaufmann, 1988).
39. T. Wu, S.-C. Zhu, A numerical study of the bottom-up and top-down inference processes in and-or graphs. Int. J. Comput. Vis. 93, 226–252 (2011). doi: 10.1007/s11263-010-0346-6
40. J. Xu, T. L. Wickramarathne, N. V. Chawla, Representing higher-order dependencies in networks. Sci. Adv. 2, e1600028 (2016). doi: 10.1126/sciadv.1600028; pmid: 27386539
41. D. Sontag, A. Globerson, T. Jaakkola, “Introduction to dual decomposition for inference,” in Optimization for Machine Learning (MIT Press, 2010), pp. 1–37.
42. E. Bienenstock, S. Geman, D. Potter, “Compositionality, MDL priors, and object recognition,” in Advances in Neural Information Processing Systems 10 (1997), pp. 838–844.
43. J. Tsotsos, A. Rothenstein, Computational models of visual attention. Scholarpedia 6, 6201 (2011). doi: 10.4249/scholarpedia.6201
44. B. A. Olshausen, C. H. Anderson, D. C. Van Essen, A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information. J. Neurosci. 13, 4700–4719 (1993). pmid: 8229193
45. Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998). doi: 10.1109/5.726791
46. K. Simonyan, A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” paper presented at the International Conference on Learning Representations (ICLR) 2015, San Diego, CA, 7 to 9 May 2015.
47. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, F.-F. Li, “Imagenet: A large-scale hierarchical image database,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2009), pp. 248–255.
48. A. Wong, A. L. Yuille, “One shot learning via compositions of meaningful patches,” in 2015 IEEE International Conference on Computer Vision (IEEE, 2015), pp. 1197–1205.
49. D. P. Kingma, M. Welling, “Stochastic gradient VB and the variational auto-encoder,” paper presented at the International Conference on Learning Representations (ICLR) 2014, Banff, Canada, 14 to 16 April 2014.
50. K. Gregor, I. Danihelka, A. Graves, D. J. Rezende, D. Wierstra, “DRAW: A recurrent neural network for image generation,” in Proceedings of the 32nd International Conference on Machine Learning (Proceedings of Machine Learning Research, 2015), pp. 1462–1471.
51. C. K. Williams, M. K. Titsias, Greedy learning of multiple objects in images using robust statistics and factorial learning. Neural Comput. 16, 1039–1062 (2004). doi: 10.1162/089976604773135096; pmid: 15070509
52. T. Gao, B. Packer, D. Koller, “A segmentation-aware object detection model with occlusion handling,” in 2011 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 1361–1368.
53. R. Fransens, C. Strecha, L. Van Gool, “A mean field EM-algorithm for coherent occlusion handling in MAP-estimation problems,” in 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1 (IEEE, 2006), pp. 300–307.
54. D. Karatzas, F. Shafait, S. Uchida, M. Iwamura, L. Gomez i Bigorda, S. R. Mestre, J. Mas, D. F. Mota, J. A. Almazan, L.-P. de las Heras, “ICDAR 2013 Robust Reading Competition,” in 2013 12th International Conference on Document Analysis and Recognition (IEEE, 2013), pp. 1484–1493.
55. M. Jaderberg, K. Simonyan, A. Vedaldi, A. Zisserman, “Deep structured output learning for unconstrained text recognition,” paper presented at the International Conference on Learning Representations (ICLR) 2015, San Diego, CA, 7 to 9 May 2015.
56. J. Wang, A. Yuille, “Semantic part segmentation using compositional model combining shape and appearance,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2015), pp. 1788–1797.
57. D. Tabernik, A. Leonardis, M. Boben, D. Skočaj, M. Kristan, Adding discriminative power to a generative hierarchical compositional model using histograms of compositions. Comput. Vis. Image Underst. 138, 102–113 (2015). doi: 10.1016/j.cviu.2015.04.006
58. S. M. Ali Eslami, N. Heess, T. Weber, Y. Tassa, K. Kavukcuoglu, “Attend, infer, repeat: Fast scene understanding with generative models,” paper presented at the 30th Conference on Neural Information Processing Systems (NIPS 2016), Barcelona, Spain, 5 to 10 December 2016.
59. M. L’azaro-Gredilla, Y. Liu, D. S. Phoenix, D. George, “Hierarchical compositional feature learning,” arXiv:1611.02252 [cs.LG] (7 November 2016).
60. S. Ullman, Visual routines. Cognition 18, 97–159 (1984). doi: 10.1016/0010-0277(84)90023-4; pmid: 6543165
61. S. Litvak, S. Ullman, Cortical circuitry implementing graphical models. Neural Comput. 21, 3010–3056 (2009). doi: 10.1162/neco.2009.05-08-783; pmid: 19686065
62. D. George, J. Hawkins, Towards a mathematical theory of cortical micro-circuits. PLOS Comput. Biol. 5, e1000532 (2009). doi: 10.1371/journal.pcbi.1000532; pmid: 19816557