Parallel L-BFGS-B algorithm on GPU

Computers and Graphics (Pergamon)

Volumn 40, Issue 1, 2014, Pages 1-9

  Fei, Yun ^a   Rong, Guodong ^b   Wang, Bin ^a   Wang, Wenping ^c  

^a TSINGHUA UNIVERSITY   (China)

^b SAMSUNG RESEARCH AMERICA   (United States)

^c UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

CVT; GPU; L BFGS B; Nonlinear optimization

Indexed keywords

COMPUTER GRAPHICS; COMPUTER GRAPHICS EQUIPMENT; CONJUGATE GRADIENT METHOD; NONLINEAR PROGRAMMING; PROGRAM PROCESSORS; VARIABLE SPEED TRANSMISSIONS;

CENTROIDAL VORONOI TESSELLATION; DESIGN DECISIONS; LIMITED MEMORY BROYDEN-FLETCHER-GOLDFARB-SHANNO; NON-LINEAR OPTIMIZATION; OPTIMIZATION METHOD; PARALLEL IMPLEMENTATIONS; RESEARCH TOPICS; SCIENTIFIC COMPUTATION;

GRAPHICS PROCESSING UNIT;

EID: 84893541927     PISSN: 00978493     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.cag.2014.01.002     Document Type: Article

References (60)

1. ALGLIB Project. Unconstrained optimization: L-BFGS and CG. 2013. {http://www.alglib.net/optimization/lbfgsandcg.php#header3 {.
2. D.C. Liu, and J. Nocedal On the limited memory BFGS method for large scale optimization Math Program 45 1 1989 503 528 (Pubitemid 20660315)
3. R.H. Byrd, P. Lu, J. Nocedal, and C. Zhu A limited memory algorithm for bound constrained optimization SIAM J Sci Comput 16 5 1995 1190 1208
4. C. Zhu, R.H. Byrd, P. Lu, and J. Nocedal Algorithm 778 L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization ACM Trans Math Softw 23 4 1997 550 560 (Pubitemid 127447768)
5. Hestenes MR, Stiefel E. Methods of conjugate gradients for solving linear systems; 1952.
6. D.W. Marquardt An algorithm for least-squares estimation of nonlinear parameters SIAM J Soc Ind Appl Math 11 2 1963 431 441
7. Y. Liu, W. Wang, B. Lévy, F. Sun, D. Yan, and L. Lu et al. On centroidal Voronoi tessellation - energy smoothness and fast computation ACM Trans Graph 28 4 2009 101
8. Yang C, Duraiswami R, DeMenthon D, Davis L. Mean-shift analysis using quasi-Newton methods. In: Proceedings of ICIP 03, vol. 2. IEEE; 2003. p. II-447.
9. Chen Y.W, Xu R, Tang SY, Morikawa S, Kurumi Y. Non-rigid MR-CT image registration for MR-guided liver cancer surgery. In: Proceedings of ICME 07. IEEE; 2007, p. 1756-60.
10. Hyneman W, Itokazu H, Williams L, Zhao X. Human face project. In: ACM SIGGRAPH 05 courses. ACM; 2005, p. 5.
11. L. Wang, K. Zhou, Y. Yu, and B. Guo Vector solid textures ACM Trans Graph 29 4 2010 86
12. Du Q, Wang X. Centroidal Voronoi tessellation based algorithms for vector fields visualization and segmentation. In: Proceedings of Vis 04. IEEE; 2004. p. 43-50. (Pubitemid 40497906)
13. Q. Du, V. Faber, and M. Gunzburger Centroidal Voronoi tessellations applications and algorithms SIAM Rev 41 4 1999 637 676 (Pubitemid 30536738)
14. Q. Du, M. Gunzburger, L. Ju, and X. Wang Centroidal Voronoi tessellation algorithms for image compression, segmentation, and multichannel restoration J Math Imaging Vis 24 2 2006 177 194
15. J. Wang, L. Ju, and X. Wang An edge-weighted centroidal Voronoi tessellation model for image segmentation IEEE Trans Image Process 18 8 2009 1844 1858
16. Alliez P, De Verdire E, Devillers O, Isenburg M. Isotropic surface remeshing. In: Proceedings of SMI 03, 2003. p. 49-58.
17. Q. Du, and D. Wang Anisotropic centroidal Voronoi tessellations and their applications SIAM J Sci Comput 26 3 2005 737 761 (Pubitemid 40738105)
18. Lp centroidal Voronoi tessellation and its applications ACM Trans Graph 29 4 2010 119
19. S. Hiller, H. Hellwig, and O. Deussen Beyond stippling methods for distributing objects on the plane Comput Graph Forum 22 3 2003 515 522
20. Secord A. Weighted Voronoi stippling. In: Proceedings of NPAR 02. ACM; 2002. p. 37-43.
21. Battiato S, Di Blasi G, Farinella GM, Gallo G. Digital mosaic frameworks - an overview. In: Comput graph forum, vol. 26. Wiley Online Library; 2007, p. 794-812. (Pubitemid 350274204)
22. Deussen O, Isenberg T. Halftoning and stippling. In: Image and video-based artistic stylisation. Springer; 2013, p. 45-61.
23. Cevahir A, Nukada A, Matsuoka S. Fast conjugate gradients with multiple GPUs. In: Proceedings of ICCS 09, 2009. p. 893-903.
24. J. Bolz, I. Farmer, E. Grinspun, and P. Schröoder Sparse matrix solvers on the GPU conjugate gradients and multigrid ACM Trans Graph 22 3 2003 917 924
25. Li B, Young AA, Cowan BR. GPU accelerated non-rigid registration for the evaluation of cardiac function. In: Proceedings of MICCAI 08, 2008. p. 880-7.
26. J.L. Morales, and J. Nocedal Remark on "Algorithm 778 L-BFGS-B: Fortran subroutines for large-scale bound constrained optimization" ACM Trans Math Softw 38 1 2011 1 4
27. G. Rong, Y. Liu, W. Wang, X. Yin, X.D. Gu, and X. Guo GPU-assisted computation of centroidal Voronoi tessellation IEEE Trans Vis Comput Graph 17 3 2011 345 356
28. Averick BM, Carter RG, Moré JJ, Xue GL. The MINPACK-2 test problem collection. Technical Report MCS-P153-0692. Argonne National Laboratory; 1992.
29. C. Broyden, J. Dennis Jr., and J. Moré On the local and superlinear convergence of quasi-Newton methods IMA J Appl Math 12 3 1973 223 245
30. Jiang L, Byrd RH, Eskow E, Schnabel RB. A preconditioned L-BFGS algorithm with application to molecular energy minimization. Technical Report CU-CS-982-04. Department of Computer Science, University of Colorado; 2004.
31. Gao G, Reynolds A. An improved implementation of the LBFGS algorithm for automatic history matching. In: Proceedings of ATCE 04, 2004. p. 1-18.
32. Schraudolph N, Yu J, Günter S. A stochastic quasi-Newton method for online convex optimization. In: Proceedings of AISTATS 07, 2007. p. 433-40.
33. Liu Y. HLBFGS. 2010 {http://research.microsoft.com/en-us/UM/people/ yangliu/software/HLBFGS/}.
34. J. Morales A numerical study of limited memory BFGS methods Appl Math Lett 15 4 2002 481 487
35. Hillesland KE, Molinov S, Grzeszczuk R. Nonlinear optimization framework for image-based modeling on programmable graphics hardware. In: ACM SIGGRAPH 05 courses, 2005.
36. Goodnight N, Woolley C, Lewin G, Luebke D, Humphreys G. A multigrid solver for boundary value problems using programmable graphics hardware. In: Proceedings of HPG 03, 2003. p. 102-11.
37. J. Krüger, and R. Westermann Linear algebra operators for GPU implementation of numerical algorithms ACM Trans Graph 22 3 2003 908 916
38. Feng Z, Li P. Multigrid on GPU: tackling power grid analysis on parallel SIMT platforms. In: Proceedings of ICCAD 08, 2008. p. 647-54.
39. L. Buatois, G. Caumon, and B. Levy Concurrent number cruncher a GPU implementation of a general sparse linear solver Int J Parallel Emergent Distrib Syst 24 3 2009 205 223
40. Ament M, Knittel G, Weiskopf D, Strasser W. A parallel preconditioned conjugate gradient solver for the Poisson problem on a multi-GPU platform. In: Proceedings of PDP 10, 2010. p. 583-92.
41. M. Dehnavi, M. Fernandez, and D. Giannacopoulos Enhancing the performance of conjugate gradient solvers on graphic processing units IEEE Trans Magn 47 5 2011 1162 1165
42. A. Cevahir, A. Nukada, and S. Matsuoka High performance conjugate gradient solver on multi-GPU clusters using hypergraph partitioning Comput Sci Res Dev 25 1 2010 83 91
43. M. Verschoor, and A. Jalba Analysis and performance estimation of the conjugate gradient method on multiple GPUs Parallel Comput 38 10-11 2012 52 575
44. Yatawatta S, Kazemi S, Zaroubi S. GPU accelerated nonlinear optimization in radio interferometric calibration. In: Proceedings of IPC 12, 2012. p. 1-6.
45. Sellitto M. Accelerating an imaging spectroscopy algorithm for submerged marine environments using heterogeneous computing. [Masters thesis]. Department of Electrical and Computer Engineering, Northeastern University; 2012.
46. Wetzl J, Taubmann O, Haase S, Köhler T, Kraus M, Hornegger J. GPU accelerated time-of-flight super-resolution for image-guided surgery. In: Tolxdorff T, Deserno TM, editors. Bildverarbeitung für die Medizin, 2013. p. 21-6.
47. Harris M. Optimizing parallel reduction in CUDA. NVIDIA Corporation; 2007 {http://developer.download.nvidia.com/assets/cuda/files/reduction.pdf}.
48. CUDA C programming guide. NVIDIA Corporation; 2007 {http://docs.nvidia. com/cuda/cuda-c-programming-guide/index.html}.
49. Gunnels J, Lin C, Morrow G, van de Geijn R. A flexible class of parallel matrix multiplication algorithms. In: Proceedings of IPPS/SPDP 98, 1998. p. 110-6.
50. Humphrey J, Price D, Spagnoli K, Paolini A, Kelmelis E. CULA: hybrid GPU accelerated linear algebra routines. In: Proceedings of SPIE 10, 2010, p. 770502.
51. CUBLAS Library. NVIDIA Corporation; 2008 {http://docs.nvidia.com/cuda/ cublas/index.html}.
52. Volkov V, Demmel JW. Benchmarking GPUs to tune dense linear algebra. In: Proceedings of SC 08, 2008. p. 31:1-11.
53. Sengupta S. Efficient primitives and algorithms for many-core architectures [Ph.D. thesis]. Davis: University of California; 2010.
54. Thrust. NVIDIA Corporation; 2009 {http://docs.nvidia.com/cuda/thrust/ index.html}.
55. Henry S. Parallelizing Choleskys decomposition algorithm. Technical Report. INRIA Bordeaux; 2009.
56. Rong G, Tan TS. Jump flooding in GPU with applications to Voronoi diagram and distance transform. In: Proceedings of I3D 06, 2006. p. 109-116.
57. Rong G, Tan TS. Variants of jump flooding algorithm for computing discrete Voronoi diagrams. In: Proceedings of ISVD 07, 2007. p. 176-81.
58. Yuan Z, Rong G, Guo X, Wang W. Generalized Voronoi diagram computation on GPU. In: Proceedings of ISVD 11, 2011. p. 75-82.
59. Frey P, Borouchaki H. Surface mesh evaluation. In: Proceedings of IMR 97, 1997. p. 363-74.
60. Dolan E, Moré J, Munson T. Benchmarking optimization software with COPS 3.0. Technical Report ANL/MCS-TM-273. Argonne National Laboratory; 2004.