Optimal accumulation of Jacobian matrices by elimination methods on the dual computational graph

Mathematical Programming

Volumn 99, Issue 3, 2004, Pages 399-421

  Naumann, Uwe ^a  

^a ARGONNE NATIONAL LABORATORY   (United States)

Author keywords

Automatic differentiation; Computational graphs; Elimination techniques; Jacobian matrices

Indexed keywords

AUTOMATIC DIFFERENTIATION; COMPUTATIONAL GRAPHS; ELIMINATION TECHNIQUES; JACOBIAN MATRICES;

ALGORITHMS; COMPUTATION THEORY; COMPUTER SIMULATION; DYNAMIC PROGRAMMING; FUNCTIONS; GRAPH THEORY; MATHEMATICAL MODELS; SET THEORY; SIMULATED ANNEALING; VECTORS;

MATRIX ALGEBRA;

EID: 8344274741     PISSN: 00255610     EISSN: 14364646     Source Type: Journal    
DOI: 10.1007/s10107-003-0456-9     Document Type: Article

References (36)

1. Aho, A., Sethi, R., Ullman, J.: Compilers, Principles, Techniques and Tools. Addison-Wesley, Reading MA, 1986
2. Albrecht, A., Gottschling, P., Naumann, U.: Markowitz-type heuristics for computing Jacobian matrices efficiently. In: Proceedings of International Conference on Computational Science, Springer, LNCS. 2658, 575-584 (2003)
3. Ausiello, G., Crescenzi, P., Gambosi, G., Kann, V., Marchetti-Spaccamela, A., Protasi, M.: Complexity and Approximation. Springer, Berlin, 1999
4. Berz, M., Bischof, C., Corliss, G., Griewank, A.: Computational Differentiation: Techniques, Applications, and Tools. Proceedings Series, Philadelphia, 1996, SIAM
5. Bischof, C., Haghighat, M.: Hierarchical approaches to Automatic Differentiation. In: [4] pp. 82-94, 1996
6. Bischof, C., Khademi, P., Boucharicha, A., Carle, A.: Efficient Computation of Gradients and Jacobians by Dynamic Exploitation of Sparsity in Automatic Differentiation. Optim. Meth. Softw. 7, 1-39 (1997)
7. Corliss, G., Faure, C., Griewank, A., Hascoet, L., Naumann, U., editors: Automatic Differentiation of Algorithms. From Simulation to Optimization, Springer, New York, 2002
8. Corliss, G., Griewank, A. editors: Automatic Differentiation: Theory Implementation and Application. Proceedings Series, Philadelphia, 1991, SIAM
9. Curtis, A., Powell, M., Reid, J.: On the Estimation of Sparse Jacobian Matrices. J. Inst. Math. Appl. 13, 117-119 (1974)
10. Van der Wijngaart, R., Saphir, W.: On the Efficacy of source code optimizations for cache-based processors. Technical report, NAS, June 2000
11. Garey, M., Johnson, D.: Computers and Intractability - A Guide to the Theory of NP-completeness. W. H. Freeman and Company, San Francisco, 1979
12. Gilbert, J.: A note on the NP-completeness of vertex elimination on directed graphs. J. Alg. Disc. Meth. 1(3), 292-294 (SIAM), September, 1980
13. Griewank, A.: Evaluating Derivatives. Principles and Techniques of Algorithmic Differentiation. Number 19 in Frontiers in Applied Mathematics. (SIAM) Philadelphia, 2000
14. Griewank, A., Naumann, U.: Accumulating Jacobians as chained sparse matrix products. Math. Program. 3(95), 555-571 2003 (Springer)
15. Griewank, A., Reese, S.: On the calculation of Jacobian matrices by the Markovitz rule. In: [8], 1991, pp. 126-135
16. Herley, K.: A Note on the NP-completeness of optimum Jacobian accumulation by vertex elimination. Presentation at: Theory Institute on Combinatorial Challenges in Computational Differentiation. 1993
17. Horowitz, E., Sahni, S.: Fundamentals of Computer Algorithms. Computer Science Press, Rockville, 1978
18. Iri, M.: History of Automatic Differentiation and rounding error estimation. In: [8], 1991, pp. 3-17
19. Jessani, R., Putrino, M.: Comparison of single- and dual-pass multiply-add fused floating-point units. IEEE Trans. Comp. 47(9), 927-936 (1998)
20. NAG Numerical Libraries, http://www.nag.co.uk/numeric/numerical_libraries.asp, NAG Ltd., Oxford, UK
21. Marshall, J., Hill, C., Perelman, L., Adcroft, A.: Hydrostatic, quasi-hydrostatic and nonhydrostatic ocean modeling. J. Geophys. Res. 102 C3(5), 733-5, 752 (1997)
22. Metropolis, N., Rosenbluth, A., Rosenbluth, M., Teller, A., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21, 1087-92 (1953)
23. Miller, W., Wrathall, C.: Software for Roundoff Analysis of Matrix Algorithms. Academic Press, New York, 1980
24. Naumann, U.: Efficient Calculation of Jacobian Matrices by Optimized Application of the Chain Rule to Computational Graphs. PhD thesis, Technical University Dresden, Feb. 1999
25. Naumann, U.: An Enhanced Markowitz rule for accumulating Jacobians efficiently. In: K. Mikula, (ed.) ALGORITHMY'2000 Conference on Scientific Computing, Slovak University of Technology, Bratislava, Slovakia, September 2000, pp. 320-329
26. Naumann, U.: Cheaper Jacobians by Simulated Annealing. SIAM J. Opt. 13(3), 660-674, March 2002
27. Naumann, U.: Elimination techniques for cheap Jacobians. In: [7] 2002, pp. 247-253
28. Naumann, U.: Optimal Pivoting in Tangent-Linear and Adjoint Systems of Nonlinear Equations. Preprint ANL-MCS/P944-0402, Argonne National Laboratory, 2002
29. Naumann, U., Albrecht, A.: Combinatorial optimization methods for fast derivative code. Case for Support EPSRC Grant GR/R38101/01, University of Hertfordshire, Hatfield, UK, 2001
30. Naumann, U., Gottschling, P.: Prospects for Simulated Annealing in Automatic Differentiation. In: K. Steinhøfel, (ed.), SAGA 2002 Stochastic Algorithms, Foundations and Applications, volume 2264 of LNCS. Springer, Berlin, 2001
31. Newsam, G., Ramsdell, J.: Estimation of sparse jacobian Matrices. SIAM J. Alg. Dis. Meth. 4, 404-417 (1983)
32. Rose, D., Tarjan, R.: Algorithmic aspects of vertex elimination on directed graphs. J. Appl. Math. 34(1), 176-197 (SIAM) January 1978
33. Basic Linear Algebra Subprograms, http://www.netlib.org/blas/
34. Tadjouddine, M., Forth, S., Pryce, J.: AD Tools and Prospects for Optimal AD in CFD Flux Calculations. In: [7], 2002, pp. 255-261
35. Tadjouddine, M., Forth, S., Pryce, J., Reid, J.: performance issues for vertex elimination methods in computing Jacobians using Automatic Differentiation. In: Proceedings of the ICCS 2000 Conference, Volume 2330 of Springer LNCS, 2002, pp. 1077-1086
36. Wengert, R.: A Simple automatic derivative evaluation program. Comm. ACM 7, 463-464 (1964)