A real-time remote rendering system for interactive mobile graphics

ACM Transactions on Multimedia Computing, Communications and Applications

Volumn 8, Issue 3 SUPPL. 3, 2012, Pages

  Shi, Shu ^a   Nahrstedt, Klara ^b   Campbell, Roy ^b  

^a Ricoh Innovations Inc   (United States)

^b UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

Author keywords

3D image warping; Interaction latency; Mobile devices; Remote rendering

Indexed keywords

3-D IMAGE; 3D GRAPHICS; ALGORITHM PERFORMANCE; COMPUTING BEHAVIOR; DEPTH IMAGE; GRAPHICS RENDERING; IMAGE BASED RENDERING; INTERACTION LATENCY; LOW-LATENCY; MOBILE CLIENT; MOBILE DISPLAYS; MOBILE GRAPHICS; MOBILE PLATFORM; PREDICTION ALGORITHMS; REMOTE RENDERING; REMOTE RENDERING SYSTEMS; RENDERING QUALITY; VIEWPOINT SELECTION;

ALGORITHMS; MOBILE DEVICES; THREE DIMENSIONAL;

THREE DIMENSIONAL COMPUTER GRAPHICS;

EID: 84870232628     PISSN: 15516857     EISSN: 15516865     Source Type: Journal    
DOI: 10.1145/2348816.2348825     Document Type: Article

References (34)

1. AZUMA, R. T. 1997. A survey of augmented reality. Presence: Teleoper. Virtual Environ. 6, 4, 355-385.
2. BAO, P. AND GOURLAY, D. 2004. Remote walkthrough over mobile networks using 3-D image warping and streaming. Vis. Image Signal Processing, IEE Proceedings 151, 4, 329-336.
3. BARATTO, R. A., KIM, L. N., AND NIEH, J. 2005. Thinc: A virtual display architecture for thin-client computing. In Proceedings of the 20th ACM Symposium on Operating Systems Principles (SOSP'05). ACM, New York, 277-290.
4. BEIGBEDER, T., COUGHLAN, R., LUSHER, C., PLUNKETT, J., AGU, E., AND CLAYPOOL, M. 2004. The effects of loss and latency on user performance in unreal tournament 2003. In Proceedings of NetGames'04. 144-151.
5. BOUKERCHE, A. AND PAZZI, R. W. N. 2006. Remote rendering and streaming of progressive panoramas for mobile devices. In Proceedings of the 14th Annual ACM International Conference on Multimedia (Multimedia'06). ACM, New York, 691-694.
6. CHANG, C.-F. AND GER, S.-H. 2002. Enhancing 3D graphics on mobile devices by image-based rendering. In Proceedings of PCM'02. 1105-1111.
7. CHEN, S. E. 1995. Quicktime Vr: An image-based approach to virtual environment navigation. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'95). ACM, New York, 29-38.
8. DUGUET, F. AND DRETTAKIS, G. 2004. Flexible point-based rendering on mobile devices. IEEE Trans. Comput. Graph. Appl. 24, 4, 57-63.
9. EISERT, P. AND FECHTELER, P. 2008. Low delay streaming of computer graphics. In Proceedings of the 15th IEEE International Conference on Image Processing, (ICIP 2008). 2704-2707.
10. ENGEL, K., WESTERMANN, R., AND ERTL, T. 1999. Isosurface extraction techniques for web-based volume visualization. IEEE Visualiz.
11. Epic Games. 2007. Unreal tournament 3. www.unrealtournament.com/.
12. KARLTON, P. WOMACK, P., AND LEECH, J. 2005. Opengl graphics with the x window system (version 1.4). http://www.opengl.org/documentation/specs/.
13. KUM, S.-U. AND MAYER-PATEL, K. 2005. Real-time multidepth stream compression. ACM Trans. Multimedia Comput. Commun. Appl. 1, 128-150.
14. KURILLO, G., VASUDEVAN, R., LOBATON, E., AND BAJCSY, R. 2008. A framework for collaborative real-time 3D teleimmersion in a geographically distributed environment.. In Proceedings of the 10th IEEE International Symposium on Multimedia (ISM'08). 111-118.
15. LAMBERTI, F. AND SANNA, A. 2007. A streaming-based solution for remote visualization of 3D graphics on mobile devices. IEEE Trans. Vis. Comput. Graph. 13, 2, 247-260.
16. LEVOY, M. 1995. Polygon-assisted jpeg and mpeg compression of synthetic images. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'95). ACM, New York, 21-28.
17. LIEN, J.-M., KURILLO, G., AND BAJCSY, R. 2007. Seleton-based data compression for multi-camera tele-immersion system. In Proceedings of the 3rd International Conference on Advances in Visual Computing (ISVC'07). Vol. 1, Springer-Verlag, Berlin, Heidelberg, 714-723.
18. MARK, W. R. 1999. Post-rendering 3D image warping: Visibility, reconstruction, and performance for depth-image warping. Ph. D. dissertation. Department of Computer Science. University of North Carolina at Chapel Hill.
19. MARQUEZ, J., DOMENECH, J., GIL, J., AND PONT, A. 2008. Exploring the benefits of caching and prefetching in the mobile web. In Proceedings of WCITD'08.
20. MCMILLAN, L. 1997. An image-based approach to three-dimensional computer graphics. Ph. D. Dissertation. Department of Computer Science. University of North Carolina at Chapel Hill.
21. MCMILLAN, L. AND BISHOP, G. 1995. Plenoptic modeling: an image-based rendering system. In Proceedings of the 22nd Annual Conference on Computer Graphics and Internetive Techniques (SIGGRAPH'95). 39-46.
22. NOIMARK, Y. AND COHEN-OR, D. 2003. Streaming scenes to mpeg-4 video-enabled devices. IEEE Comput. Graph. Appl. 23, 58-64.
23. OHAZAMA, C. 1999. Opengl vizserver. White paper, Silicon Graphics, Inc.
24. PENTA, S. K. AND NARAYANAN, P. 2005. Compression of multiple depth maps for ibr. Vis. Comput. 21, 611-618. 10.1007/s00371-005-0337-8.
25. REDERT, A., DE BEECK, M. O., FEHN, C., IJSSELSTEIJN, W., POLLEFEYS, M., GOOL, L. J. V., OFEK, E., SEXTON, I., AND SURMAN, P. 2002. Attest: Advanced three-dimensional television system technologies. In Proceedings of 1st International Symposium on 3D Data Processing Visualization Transmission (3DPVT). IEEE Computer Society, 313-319.
26. RIVA, O. AND KANGASHARJU, J. 2008. Challenges and lessons in developing middleware on smart phones. IEEE Computer 41, 10, 77-85.
27. ScanView. 2003. A system for remote visualization of scanned 3D models. http://graphics.stanford.edu/software/scanview/.
28. SCHMALSTIEG, D. 1997. The remote rendering pipeline - managing geometry and bandwidth in distributed virtual environments. Ph. D. dissertation, Institute of Computer Graphics and Algorithms, Vienna University of Technology, Favoritenstrasse 9-11/186, A-1040 Vienna, Austria.
29. SHADE, J., GORTLER, S., HE, L., AND SZELISKI, R. 1998. Layered depth images. In Proceedings of the 28th Annual Conference on Computer Graphics and Internetive Techniques (SIGGRAPH'98). 231-242.
30. SHI, S., HSU, C.-H., NAHRSTEDT, K., AND CAMPBELL, R. 2011a. Using graphics rendering contexts to enhance the real-time video coding for mobile cloud gaming. In Proceedings of the 19th ACM International Conference on Multimedia (MM'11). ACM, New York, NY, USA, 103-112.
31. SHI, S., NAHRSTEDT, K., AND CAMPBELL, R. H. 2011b. Distortion over latency: Novel metric for measuring interactive performance in remote rendering systems.. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME'11).
32. SINGHAL, S. AND ZYDA, M. 1999. Networked Virtual Environments: Design and Implementation. ACM Press/Addison-Wesley Publishing Co., New York, NY.
33. SMIT, F. A., VAN LIERE, R., BECK, S., AND FRÖHLICH, B. 2009. An image-warping architecture for Vr: Low latency versus image quality. In Proceedings of the IEEE Symposium on Virtual Reality (VR). IEEE, 27-34.
34. ZHU, M., MONDET, S., MORIN, G., OOI, W. T., AND CHENG, W. 2011. Towards peer-assisted rendering in networked virtual environments. In Proceedings of the 19th ACM International Conference on Multimedia (MM'11). ACM, New York, 183-192.