Parallel architecture for hierarchical optical flow estimation based on FPGA

IEEE Transactions on Very Large Scale Integration (VLSI) Systems

Volumn 20, Issue 6, 2012, Pages 1058-1067

  Barranco, Francisco ^a   Tomasi, Matteo ^a   Diaz, Javier ^a   Vanegas, Mauricio ^a,b   Ros, Eduardo ^a  

^a UNIVERSITY OF GRANADA   (Spain)

^b UNIVERSITY OF GENOA   (Italy)

Author keywords

Field programmable gate arrays (FPGAs); Machine vision; Real time systems; Reconfigurable architectures

Indexed keywords

COMPARATIVE STUDIES; FRAMES PER SECONDS; HARDWARE RESOURCES; MULTISCALES; OPTICAL FLOW ESTIMATION; SYSTEM ARCHITECTURES;

COMPUTER VISION; FIELD PROGRAMMABLE GATE ARRAYS (FPGA); INTERACTIVE COMPUTER SYSTEMS; PARALLEL ARCHITECTURES; REAL TIME SYSTEMS; RECONFIGURABLE ARCHITECTURES;

MOTION ESTIMATION;

EID: 84861185160     PISSN: 10638210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TVLSI.2011.2145423     Document Type: Article

References (48)

1. S. P. Sabatini, F. Solari, and G. M. Bisio, "Spatiotemporal neuromorphic operators for the detection of motion-in-depth," in Proc. 2nd ICSC Symp. Neural Comput., 2000, pp. 874-880.
2. P. C. Merrell and D. Lee, "Structure from motion using optical flow probability distributions," in Intelligent Computing: Theory and Applications III, K. L. Priddy, Ed. Orlando, FL: SPIE, 2005, vol. 5803, pp. 39-48. (Pubitemid 41537960)
3. K. Pauwels, N. Kruger, M. Lappe, F. Worgotter, and M. M. V. Hulle, "A cortical architecture on parallel hardware for motion processing in real-time," J. Vision, vol. 10, no. 18, pp. 1-21, 2010.
4. K. Pauwels and M. M. V. Hulle, "Optimal instantaneous rigid motion estimation insensitive to local minima," Comput. Vision Image Understanding, vol. 104, no. 1, pp. 77-86, 2006. (Pubitemid 44332978)
5. Y. Matsushita, E. Ofek, W. Ge, X. Tang, and H.-Y. Shum, "Full-frame video stabilization with motion inpainting," IEEE Trans. Patt. Anal. Mach. Intell., vol. 28, no. 7, pp. 1150-1163, Jul. 2006.
6. T. Brox, B. Rosenhahn, D. Cremers, and H. Seidel, "High accuracy optical flow serves 3-D pose tracking: Exploiting contour and flow based constraints," in Proc. Euro. Conf. Comput. Vision (ECCV), 2006, pp. 98-111. (Pubitemid 44029720)
7. A. Kokaram, "On missing data treatment for degraded video and film archives: A survey and a new bayesian approach," IEEE Trans. Image Process., vol. 13, no. 3, pp. 397-415, 2004.
8. J. M. L. K. Nakayama, "Optical velocity patterns, velocity-sensitive neurons and space perception," Perception, vol. 3, pp. 63-80, 1974.
9. J. Aloimonos, I.Weiss, and A. Bandyopadhyay, "Active vision," Int. J. Comput. Vision, vol. 1, pp. 333-356, 1988.
10. F. Barranco, J. Diaz, E. Ros, and B. del Pino, "Visual system based on artificial retina for motion detection," IEEE Trans. Syst., Man, Cybern., vol. 39, no. 3, pp. 752-762, Jun. 2009.
11. A. Giachetti, M. Campani, and V. Torre, "The use of optical flowfor the autonomous navigation," in Proceedings of the 3rd European Conference on Computer Vision, ser. (ECCV). New York: Springer-Verlag, 1994, vol. 1, pp. 146-151.
12. A. Wali and A. M. Alimi, "Event detection from video surveillance data based on optical flow histogram and high-level feature extraction," in Proc. 20th Int. Workshop Database Expert Syst. Appl., 2009, pp. 221-225.
13. J. Diaz, E. Ros, R. Agis, and J. Bernier, "Superpipelined high-performance optical-flow computation architecture," Comput. Vision Image Understand., vol. 112, no. 3, pp. 262-273, 2008.
14. H. Frenz, M. Lappe, M. Kolesnik, and T. Bhrmann, "Estimation of travel distance from visual motion in virtual environments," ACM Trans. Appl. Perception, vol. 4, pp. 419-436, 2007.
15. B. D. Lucas and T. Kanade, "An iterative image registration technique with an application to stereo vision," in Proc. 7th Int. Joint Conf. Artif. Intell.-Vol. 2 (IJCAI), 1981, pp. 674-679. (Pubitemid 14478744)
16. S. S. Beauchemin and J. L. Barron, "The computation of optical flow," ACM Comput. Surveys, vol. 27, no. 3, pp. 433-466, 1995.
17. J. L. Barron, D. J. Fleet, and S. S. Beauchemin, "Performance of optical flow techniques," Int. J. Comput. Vision, vol. 12, pp. 43-77, 1994.
18. J. Bergen, P. Anandan, K. Hanna, and R. Hingorani, "Hierarchical model-based motion estimation," in Computer Vision ECCV'92, ser. Lecture Notes in Computer Science, G. Sandini, Ed. New York: Springer-Verlag, 1992, vol. 588, pp. 237-252.
19. J. Y. Bouguet, "Pyramidal implementation of the Lucas-Kanade feature tracker: Description of the algorithm," OpenCV Document, Intel Microprocessor Research Labs, 2000.
20. S. Baker and I. Matthews, "Equivalence and efficiency of image alignment algorithms," in Proc. IEEE Comput. Soc. Conf. Comput. Vision Pattern Recog. (CVPR), 2001, vol. 1, pp. 1090-1097.
21. F. Dellaert and R. T. Collins, "Fast image-based tracking by selective pixel integration," in Proc. ICCVWorkshop Frame Rate Process., 1999, pp. 1-22.
22. G. Hager and P. Belhumeur, "Efficient region tracking with parametric models of geometry and illumination," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 10, pp. 1025-1039, Oct. 1998. (Pubitemid 128741363)
23. Y. Murachi, Y. Fukuyama, R.Yamamoto, J. Miyakoshi, H. Kawaguchi, H. Ishihara, M. Miyama, Y. Matsuda, and M.Yoshimoto, "A vga 30-fps realtime optical-flow processor core for moving picture recognition," IEICE Trans. Electron., vol. 91, pp. 457-464, 2008.
24. K. Pauwels and M. M. V. Hulle, "Realtime phase-based optical flow on the GPU," in Proc. IEEE Comput. Soc. Conf. Comput. Vision Pattern Recog. Workshops (CVPRW), 2008, pp. 1-8.
25. M. Anguita, J. Diaz, E. Ros, and F. J. Fernandez-Baldomero, "Optimization strategies for high-performance computing of optical-flow in general-purpose processors," IEEE Trans. Circuits Syst. for Video Technol., vol. 19, no. 10, pp. 1475-1488, Oct. 2009.
26. T.Kohlberger, C. Schnorr, A. Bruhn, and J.Weickert, "Domain decomposition for variational optical-flow computation," IEEE Trans. Image Process., vol. 14, no. 8, pp. 1125-1137, Aug. 2005. (Pubitemid 41236143)
27. J. Diaz, E. Ros, F. Pelayo, E. M. Ortigosa, and S. Mota, "FPGA-based real-time optical-flow system," IEEE Trans. Circuits Syst. for Video Technol., vol. 16, no. 2, pp. 274-279, Feb. 2006. (Pubitemid 43269787)
28. B. K. P. Horn and B. G. Schunck, "Determining optical flow," Artif. Intell., vol. 17, pp. 185-203, 1981.
29. J. W. Brandt, "Improved accuracy in gradient-based optical flow estimation," Int. J. Comput. Vision, vol. 25, pp. 5-22, 1997. (Pubitemid 127506558)
30. Sevensols, Granada, Spain, "Seven solutions," 2011. [Online]. Available: http://www.sevensols.com/
31. E. Ortigosa, A. Canas, E. Ros, P. Ortigosa, S. Mota, and J. Diaz, "Hardware description of multi-layer perceptrons with different abstraction levels," Microprocess. Microsyst., vol. 30, no. 7, pp. 435-444, 2006. (Pubitemid 44189496)
32. J. Diaz, "Multimodal bio-inspired vision system. High performance motion and stereo processing architecture," Ph.D. dissertation, Dept. Comput. Arch. Technol., Univ. Granada, Granada, Spain, 2006.
33. E. P. Simoncelli, "Design of multi-dimensional derivative filters," in Proc. IEEE Int. Conf. Image Process., 1994, pp. 790-794.
34. Xilinx, San Jose, CA, "FPGA and CPLD solutions from Xilinx, Inc.," 2011. [Online]. Available: http://www.xilinx.com/
35. M. Otte and H.-H. Nagel, "Optical flowestimation: Advances and comparisons," in Proceedings of the Third European Conference on Computer Vision, ser. ECCV'94. New York: Springer-Verlag, 1994, pp. 51-60.
36. M. Vanegas, M. Tomasi, J. Diaz, and E. Ros, "Multi-port abstraction layer for FPGA intensive memory exploitation applications," J. Syst. Arch., vol. 56, no. 9, pp. 442-451, 2010.
37. P. J. Burt, Edward, and E. H. Adelson, "The laplacian pyramid as a compact image code," IEEE Trans. Commun., vol. 31, no. 4, pp. 532-540, Apr. 1983.
38. D. A. Marzat J. and Y. Dumortier, "Real-time dense and accurate parallel optical flow using cuda," in Proc. 17th Int. Conf. Central Euro. Comput. Graphics, Visualization, Comput. Vision (WSCG), 2009, pp. 105-111.
39. M. Heindlmaier, L. Yu, and K. Diepold, "The impact of nonlinear filtering and confidence information on optical flow estimation in a lucas & kanade framework," in Proc. 16th IEEE Int. Conf. Image Process., 2009, pp. 1573-1576.
40. M. Tomasi, M.Vanegas, F. Barranco, J. Diaz, and E. Ros, "High-performance optical-flow architecture based on a multiscale, multi-orientation phase-based model," IEEE Trans. Circuits Syst. for Video Technol., vol. 20, no. 12, pp. 1797-1807, Dec. 2010.
41. G. Botella, A. Garcia, M. Rodriguez-Alvarez, E. Ros, U. Meyer-Baese, and M. C. Molina, "Robust bioinspired architecture for optical-flow computation," IEEE Trans. Very Large Scale Integr. (VLSI) Syst., vol. 18, no. 4, pp. 616-629, Apr. 2010.
42. V. Mahalingam, K. Bhattacharya, N. Ranganathan, H. Chakravarthula, R. Murphy, and K. Pratt, "A VLSI architecture and algorithm for lucaskanade-based optical flowcomputation," IEEE Trans. Very Large Scale Integr. (VLSI) Syst., vol. 18, no. 1, pp. 29-38, Jan. 2010.
43. P. Gwosdek, A. Bruhn, and J. Weickert, "Variational optic flow on the sony playstation 3," J. Real-Time Image Process., vol. 5, pp. 163-177, 2010.
44. J. Chase, B. Nelson, J. Bodily, Z. Wei, and D.-J. Lee, "Real-time optical flow calculations on FPGA and GPU architectures: A comparison study," in Proc. 16th Int. Symp. Field-Program. Custom Comput. Mach. (FCCM), 2008, pp. 173-182.
45. M. Martineau, Z. Wei, D.-J. Lee, and M. Martineau, "A fast and accurate tensor-based optical flow algorithm implemented in fpga," in Proc. IEEE Workshop Appl. Comput. Vision (WACV), 2007, pp. 18-18.
46. A. Bruhn, J.Weickert, T. Kohlberger, and C. Schnrr, "A multigrid platform for real-time motion computation with discontinuity-preserving variational methods," Int. J. Comput. Vision, vol. 70, pp. 257-277, 2006. (Pubitemid 44131407)
47. H. Niitsuma and T. Maruyama, "High speed computation of the optical flow," in Proc. Image Anal. Process. (ICIAP), 2005, vol. 3617, pp. 287-295. (Pubitemid 43901988)
48. "Open rt-Vision Project," 2011. [Online]. Available: http://code. google.com/p/open-rtvision/