Real-time architecture for a robust multi-scale stereo engine on FPGA

IEEE Transactions on Very Large Scale Integration (VLSI) Systems

Volumn 20, Issue 12, 2012, Pages 2208-2219

  Tomasi, Matteo ^a   Vanegas, Mauricio ^a   Barranco, Francisco ^a   Díaz, Javier ^a   Ros, Eduardo ^a  

^a UNIVERSITY OF GRANADA   (Spain)

Author keywords

Embedded and real time systems; multi scale; stereo image processing

Indexed keywords

AFFINE DEFORMATION; EPIPOLAR CONSTRAINTS; FRAME RATE; ILLUMINATION CHANGES; IMAGE PAIRS; IMAGE STRUCTURES; LARGE DISPARITY; MULTIPLE USER; MULTISCALES; ON CHIPS; PHASE INFORMATION; REAL-TIME ARCHITECTURE; REAL-TIME IMPLEMENTATIONS; STATE OF THE ART; STATE-OF-THE-ART APPROACH; STEREO ALGORITHMS; STEREO IMAGE PROCESSING; SUBPIXEL ACCURACY; SUPERSCALAR; WORKING FREQUENCY;

IMAGE PROCESSING; REAL TIME CONTROL;

ALGORITHMS;

EID: 84864925782     PISSN: 10638210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TVLSI.2011.2172007     Document Type: Article

References (49)

1. A. Cozzi, B. Crespi, F. Valentinotti, and F. Wörgötter, "Performance of phase-based algorithms for disparity estimation," Mach. Vision Appl., vol. 9, no. 5-6, pp. 334-340, 1997. (Pubitemid 127561579)
2. D. J. Fleet and A. D. Jepson, "Stability of phase information," PA M I, vol. 15, no. 12, pp. 1253-1268, 1993.
3. "Drivsco Project," 2007. [Online]. Available: http://www.pspc.dibe. unige.it/drivsco/
4. F. Valentinotti and S. Taraglio, "Phase difference stereo disparity computation on a simd parallel machine," in Proc. Int. Conf. Exhib. HighPerform. Comput. Network. (HPCN Europe), 1997, pp. 127-136. (Pubitemid 127066557)
5. K. Pauwels and M. M. V. Hulle, "Realtime phase-based optical flow on the gpu," in Proc. Comput. Vision Pattern Recog. Workshop, 2008, pp. 1-8.
6. J. Anderson and F. Najm, "Low-power programmable FPGA routing circuitry," IEEE Trans. Very Large Scale Integr. (VLSI) Syst., vol. 17, no. 8, pp. 1048-1060, Aug. 2009.
7. C. You, J.-R. Guo, R. Kraft, M. Chu, B. Goda, and J. McDonald, "A 12-gb/s demux implemented with sige high-speed FPGA circuits," IEEE Trans. Very Large Scale Integr. (VLSI) Syst., vol. 15, no. 9, pp. 1051-1054, Sep. 2007. (Pubitemid 47295157)
8. H. Parandeh-Afshar, A. Verma, P. Brisk, and P. Ienne, "Improving FPGA performance for carry-save arithmetic," IEEE Trans. Very Large Scale Integr. (VLSI) Syst., vol. 18, no. 4, pp. 578-590, Apr. 2010.
9. P. Kumar and K. Sridharan, "VLSI-efficient scheme and FPGA realization for robotic mapping in a dynamic environment," IEEE Trans. Very Large Scale Integr. (VLSI) Syst., vol. 15, no. 1, pp. 118-123, Jan. 2007. (Pubitemid 46376146)
10. D. K. Masrani and W. J. MacLean, "A real-time large disparity range stereo-system using FPGAs," in Proc. IV IEEE Int. Conf. Comput. Vision Syst. (ICVS), 2006, p. 13. (Pubitemid 44499935)
11. S. Park and H. Jeong, "Real-time stereo vision FPGA chip with low error rate," in Proc. Int. Conf. Multimedia Ubiquitous Eng., 2007, pp. 751-756. (Pubitemid 350301571)
12. J. Díaz, E. Ros, R. Carrillo, and A. Prieto, "Real-time system for highimage resolution disparity estimation," IEEE Trans. Image Process., vol. 16, no. 1, pp. 280-285, Jan. 2007. (Pubitemid 46437490)
13. P. Perona, "Deformable kernels for early vision," PAMI, vol. 17, pp. 488-499, 1991.
14. V. Torre and T. Poggio, "On edge detection," PAMI, vol. 8, pp. 147-163, 1984.
15. J. J. Clark, "Authenticating edges produced by zero-crossing algorithms," PA M I, vol. 11, no. 1, pp. 43-57, 1989.
16. S. P. Sabatini, G. Gastaldi, F. Solari, K. Pauwels, M. M. Van Hulle, J. Diaz, E. Ros, N. Pugeault, and N. Krüger, "A compact harmonic code for early vision based on anisotropic frequency channels," in Proc. Comput. Vis. Image Understanding, Jun. 2010, vol. 114, pp. 681-699 [Online]. Available: http://dx.doi.org/10.1016/j.cviu.2010.03.008
17. D. J. Fleet, A. D. Jepson, and M. R. Jenkin, "Phase-based disparity measurement," CVGIP: Image Understanding, vol. 53, no. 2, pp. 198-210, Mar. 1991.
18. J. Monaco, A. C. Bovik, and L. K. Cormack, "Nonlinearities in stereoscopic phase-differencing," IEEE Trans. Image Process., vol. 17, no. 9, pp. 1672-1684, Sep. 2008.
19. F. Solari, S. Sabatini, and G. Bisio, "Fast technique for phase-based disparity estimation with no explicit calculation of phase," Electron. Lett., vol. 37, no. 23, pp. 1382-1383, 2001. (Pubitemid 33076149)
20. P. J. Burt and E. H. Adelson, "The Laplacian pyramid as a compact image code," IEEE Trans. Commun., vol. 31, pp. 532-540, 1983.
21. E. H. Adelson, C. H. Anderson, J. R. Bergen, P. J. Burt, and J. M. Ogden, "Pyramid methods in image processing," RCA Engineer, vol. 29, no. 6, pp. 33-41, 1984.
22. J. R. Bergen, P. Anandan, K. J. Hanna, and R. Hingorani, "Hierarchical model-based motion estimation," in Proc. 2nd Euro. Conf. Comput. Vision, 1992, pp. 237-252.
23. K. Pauwels and M. M. V. Hulle, "Optic flow from unstable sequences containing unconstrained scenes through local velocity constancy maximization," in Proc. British Mach. Vision Conf., 2006, pp. 4-7.
24. Xilinx, Inc., San Jose, CA, "Xilinx webpage," 2010 [Online]. Available: http://www.xilinx.com
25. Mentor Graphics, "Manual Reference," 2010. [Online]. Available: http://www.mentor.com/products/fpga/handel-c/upload/handelc-ref-erence.pdf
26. E. Ortigosa, A. Cañas, E. Ros, P. Ortigosa, S. Mota, and J. Díaz, "Hardware description of multi-layer perceptrons with different abstraction levels," Microprocess. Microsyst., vol. 30, no. 7, pp. 435-444, 2006. (Pubitemid 44189496)
27. M. Tomasi, M. Vanegas, F. Barranco, J. Diaz, and E. Ros, "Highperformance optical-flow architecture based on a multi-scale, multi-orientation phase-based model," IEEE Trans. Circuits Syst. for Video Technol., vol. 20, no. 12, pp. 1797-1807, Dec. 2010.
28. M. Vanegas, M. Tomasi, J. Díaz, and E. Ros, "Multi-port abstraction layer for FPGA intensive memory exploitation applications," J. Syst. Arch., vol. 56, no. 9, pp. 442-451, 2010.
29. M. Beauchamp, S. Hauck, K. Underwood, and K. Hemmert, "Architectural modifications to enhance the floating-point performance of FPGAs," IEEE Trans. Very Large Scale Integr. (VLSI) Syst., vol. 16, no. 2, pp. 177-187, Feb. 2008.
30. J. Díaz, "Multimodal bio-inspired vision system. High performance motion and stereo processing architecture," Ph.D. dissertation, Comput. Arch. Technol. Dept., Univ. Granada, Granada, Spain, 2006.
31. J. Valls, M. Kuhlmann, and K. K. Parhi, "Evaluation of cordic algorithms for FPGA design," J. VLSI Signal Process. Syst., vol. 32, no. 3, pp. 207-222, 2002. (Pubitemid 35312030)
32. Seven solutions 2010. [Online]. Available: http://www.sevensols.com
33. Middlebury Vision 2010. [Online]. Available: http://vision.middle-bury. edu/stereo/
34. D. Scharstein and R. Szeliski, "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms," Int. J. Comput. Vision, vol. 47, no. 1-3, pp. 7-42, 2002. (Pubitemid 34428419)
35. C. Georgoulas and I. Andreadis, "A real-time fuzzy hardware structure for disparity map computation," J. Real-Time Image Process., pp. 1-17, 2010.
36. T. Vaudrey, C. Rabe, R. Klette, and J. Milburn, "Differences between stereo and motion behaviour on synthetic and real-world stereo sequences," in Proc. Image Vision Comput., 2008, pp. 1-6.
37. S. Jin, J. Cho, X. D. Pham, K. M. Lee, S.-K. Park, M. Kim, and J. W. Jeon, "FPGA design and implementation of a real-time stereo vision system," IEEE Trans. Circuits Syst. for Video Technol., vol. 20, no. 1, pp. 15-26, Jan. 2010.
38. K. Ambrosch and W. Kubinger, "Accurate hardware-based stereo vision," Comput. Vis. Image Understanding, vol. 114, pp. 1303-1316, Nov. 2010.
39. N.-C. Chang, T.-H. Tsai, B.-H. Hsu, Y.-C. Chen, and T.-S. Chang, "Algorithm and architecture of disparity estimation with mini-census adaptive support weight," IEEE Trans. Circuits Systems for Video Technol., vol. 20, no. 6, pp. 792-805, Jun. 2010.
40. M. Gong and Y.-H. Yang, "Real-time stereo matching using orthogonal reliability-based dynamic programming," IEEE Trans. Image Process., vol. 16, no. 3, pp. 879-884, Mar. 2007. (Pubitemid 46431980)
41. M. Gong and Y. H. Yang, "Fast unambiguous stereo matching using reliability-based dynamic programming," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 6, pp. 998-1003, Jun. 2005. (Pubitemid 40889850)
42. L. Wang, M. Liao, M. Gong, R. Yang, and D. Nister, "High-quality real-time stereo using adaptive cost aggregation and dynamic programming," in Proc. 3rd Int. Symp. 3D Data Process., Visualization, Transmission (3DPVT), 2006, pp. 798-805.
43. M. Z. Brown, D. Burschka, G. D. Hager, and S. Member, "Advances in computational stereo," PA M I, vol. 25, pp. 993-1008, 2003.
44. S.-K. Han, S. Woo, M.-H. Jeong, and B.-J. You, "Improved-quality real-time stereo vision processor," in Proc. 22nd Int. Conf. VLSI Design, 2009, pp. 287-292.
45. A. Darabiha, J. MacLean, and J. Rose, "Reconfigurable hardware implementation of a phase-correlation stereo algorithm," Mach. Vision Appl., vol. 17, no. 2, pp. 116-132, 2006.
46. S. Li, "Binocular spherical stereo," IEEE Trans. Intell. Transport. Syst., vol. 9, no. 4, pp. 589-600, Dec. 2008.
47. O. Marques, "Stereo depth with a unified architecture gpu," in Proc. Comput. Vision Pattern Recog. Workshop, 2008, vol. 0, pp. 1-6.
48. C. Murphy, D. Lindquist, A. M. Rynning, T. Cecil, S. Leavitt, and M. L. Chang, "Low-cost stereo vision on an fpga," in Proc. 15th Annu. IEEE Symp. Field-Program. Custom Comput. Mach., 2007, pp. 333-334.
49. J. I. Woodfill, G. Gordon, D. Jurasek, T. Brown, R. Buck, and T. Inc, "The tyzx deepsea g2 vision system, a taskable, embedded stereo camera," in Proc. IEEE Comput. Soc. Workshop Embed. Comput. Vision, Conf. Comput. Vision Pattern Recog., 2006, pp. 126-126.