Jacobian code generated by source transformation and vertex elimination can be as efficient as hand-coding

ACM Transactions on Mathematical Software

Volumn 30, Issue 3, 2004, Pages 266-299

  Forth, Shaun A ^a   Tadjouddine, Mohamed ^a   Pryce, John D ^a   Reid, John K ^b,c  

^a CRANFIELD UNIVERSITY   (United Kingdom)

^b JKR Associates   (United Kingdom)

^c RUTHERFORD APPLETON LABORATORY   (United Kingdom)

Author keywords

Jacobian; Source transformation; Vertex elimination

Indexed keywords

HAND-CODING; JACOBIAN CODES; SOURCE TRANSFORMATION; VERTEX ELIMINATION;

ALGORITHMS; DATA ACQUISITION; DATA COMPRESSION; DIFFERENTIATION (CALCULUS); LINEAR ALGEBRA; PROGRAM COMPILERS;

CODES (SYMBOLS);

EID: 8344224448     PISSN: 00983500     EISSN: None     Source Type: Journal    
DOI: 10.1145/1024074.1024076     Document Type: Article

References (44)

1. AHO, A. V, SETHI, R., AND ULLMAN, J. D. 1995. Compilers: Principles, Techniques, and Tools. Addison-Wesley, Reading, Mass.
2. AVERICK, B. M., CARTER, R. G., MORÉ, J. J., AND XUE, G.-L. 1992. The MINPACK-2 test problem collection. Preprint MCS-P153-0692, ANL/MCS-TM-150, Rev. 1, Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, Ill. See ftp://info.mcs.anl.gov/pub/MINPACK-2/tprobs/P153.ps.Z.
3. BENDTSEN, C. AND STAUNING, O. 1996. FADBAD, A flexible C++ package for automatic differentiation. Tech. Rep. IMM-REP-1996-17, Technical University of Denmark, IMM, Departement of Mathematical Modeling, Lyngby.
4. BERZ, M., BISCHOF, C., CORLISS, G., AND GRIEWANK, A., EDS. 1996. Computational Differentiation: Techniques, Applications, and Tools. SIAM, Philadelphia, Pa.
5. BISCHOF, C., BÜCKER, H., LANG, B., RASCH, A., AND VEHRESCHILD, A. 2002. Combining source transformation and operator overloading techniques to compute derivatives for MATLAB programs. In. Proceedings of the 2nd IEEE International Workshop on Source Code Analysis and Manipulation (SCAM 2002). IEEE Computer Society Press, Los Alamitos, Calif, pp. 65-72.
6. BISCHOF, C. H. 1991. Issues in parallel automatic differentiation. In Automatic Differentiation of Algorithms: Theory, Implementation, and Application, A. Griewank and G. F. Corliss, Eds. SIAM, Philadelphia, Pa., 100-113.
7. BISCHOF, C. H., CARLE, A., HOVLAND, P. D., KHADEML, P., AND MAUER, A. 1998. ADIFOR 2.0 user's guide (Revision D). Tech. Rep., Mathematics and Computer Science Division Technical Memorandum no. 192 and Center for Research on Parallel Computation Technical Report CRPC-95516-S. See www.mcs. anl. gov/adifor.
8. BISCHOF, C. H., CARLE, A., KHADEMI, P., AND MAUER, A. 1996a. ADIFOR 2.0: Automatic differentiation of Fortran 77 programs. IEEE Computat. Sci. Eng. 3, 3, 18-32.
9. BISCHOF, C. H. AND HAGHIGHAT, M. R. 1996. Hierarchical approaches to automatic differentiation. In Computational Differentiation: Techniques, Applications, and Tools, M. Berz, C. Bischof, G. Corliss, and A. Griewank, Eds. SIAM, Philadelphia, Pa., 83-94.
10. BISCHOF, C. H., KHADEMI, P.M., BOUARICHA, A., AND CARLE, A. 1996b. Efficient computations of gradients and Jacobians by dynamic exploitation of sparsity in automatic differentiation. Optimiza. Meth. Softw. 7,1-39.
11. BISCHOF, C. H., ROH, L., AND MAUER, A. 1997. ADIC - An extensible automatic differentiation tool for ANSI-C. Softw.- Pract. Exp. 27, 12, 1427-1456. See www-fp.mcs.anl.gov/division/ software.
12. COLEMAN, T F., GARBOW, B. S., AND MORÉ, J. J. 1984. Software for estimating sparse Jacobian matrices. ACM Trans. Math. Softw. 10, 3, 329-345.
13. COLEMAN, T. F. AND VERMA, A. 1998. ADMAT: An automatic differentiation toolbox for MATLAB. Tech. Rep., Computer Science Department, Cornell University.
14. CORLISS, G., FAURE, C., GRIEWANK, A., HASCOËT, L., AND NAUMANN, U., Eds. 2001. Automatic Differentiation: From Simulation to Optimization. Computer and Information Science. Springer, New York.
15. DUFF, I., ERISMAN, A. M., AND REID, J. 1989. Direct methods for sparse matrices. Monographs on numerical analysis. Oxford University Press.
16. FAURE, C. AND PAPEGAY, Y. 1998. Odyssée User's Guide. Version 1.7. Rapport technique RT-0224, INRIA, Sophia-Antipolis, France. Sept. See www. inria.fr/RRRT/RT-0224.it.html, and www.inria. fr/safir/SAM/Odyssee/odyssee.html.
17. FORTH, S. A. 2001. User guide for MAD-A Matlab automatic differentiation toolbox. Applied Mathematics and Operational Research Report AMOR 2001/5, Cranfield University (RMCS Shrivenham), Swindon, SN6 8LA, UK June.
18. FORTH, S. A. AND TADJOUDDINE, M. 2003. CFD Newton solvers with EliAD, an elimination automatic differentiation tool. In Computational Fluid Dynamics 2002: Proceedings of the 2nd International Conference on Computational Fluid Dynamics, ICCFD (Sydney, Australia). S. Armfield, P. Morgan, and K. Srinivas, Eds., Springer-Verlag, New York, 134-139.
19. GIERING, R. AND KAMINSKI, T. 1998. Recipies for adjoint code construction. ACM Trans. Math. Softw. 24, 4, 437-474. (Also appeared as Max-Planck Institut für Meteorologie Hamburg, Technical Report No. 212, 1996.)
20. GIERING, R. AND KAMINSKI, T. 1998. Recipies for adjoint code construction. ACM Trans. Math. Softw. 24, 4, 437-474. (Also appeared as Max-Planck Institut für Meteorologie Hamburg, Technical Report No. 212, 1996.)
21. GOEDECKER, S. AND HOISIE, A. 2001. Performance Optimisation of Numerically Intensive Codes. Software, Environments, Tools. SIAM, Philadelphia, Pa., ISBN 0-89871-482-3.
22. GRIEWANK, A. 2000. Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation. Number 19 in Frontiers in Appl. Math. SIAM, Philadelphia, Pa.
23. GRIEWANK, A. AND CORLISS, G. F., Eds. 1991. Automatic Differentiation of Algorithms: Theory, Implementation, and Application. SIAM, Philadelphia, Pa.
24. GRIEWANK, A., JUEDES, D., AND UTKE, J. 1996. ADOL-C, a package for the automatic differentiation of algorithms written in C/C++. ACM Trans. Math. Softw. 22, 2, 131-167.
25. GBIEWANK, A. AND REESE, S. 1991. On the calculation of Jacobian matrices by the Markowitz rule. In Automatic Differentiation of Algorithms: Theory, Implementation, and Application, A. Griewank and G. F. Corliss, Eds. SIAM, Philadelphia, Pa., 126-135.
26. HASCOET, L., NAUMANN, U., AND PASCUAL, V. 2003. TBR analysis in reverse mode automatic differentiation. In Future Generation Computer Systems. Elsevier, Amsterdam, The Netherlands.
27. HOVLAND, P. D. AND MCINNES, L. C. 2001. Parallel, simulation of compressible flow using automatic differentiation and PBTSc. Parall. Comput. 27, 4 (Mar.), 503-519.
28. KNUTH, D. E. 1997. The Art of Computer Programming, Volume 1: Fundamental Algorithms. Adison-Wesley, Reading, Mass.
29. MARKOWITZ, H. 1957. The elimination form of the inverse and its application. Manage. Sci. 3, 257-269
30. NAUMANN, U. 1999. Efficient calculation of Jacobian matrices by optimized application of the chain rule to computational graphs. Ph.D. thesis, Technical University of Dresden.
31. NAUMANN, U. 2001. Elimination techniques for cheap Jacobians. In Automatic Differentiation: From Simulation to Optimization, G. Corliss, C. Faure, A. Griewank, L. Hascoët, and U. Naumann, Eds. Springer-Verlag, New. York, Chapter 29, 241-246.
32. NAUMANN, U. 2003. Statement-level optimality of tangent-linear and adjoint models. Argonne National Laboratory, preprint, ANL/MCS-P-1066, June.
33. NAUMANN, U. 2004. Optimal accumulation of Jacobian matrices by elimination methods on the dual computational graph. Math. Prog. 99, 3 (Apr.), 399-421.
34. PARR, T., LILLY, J., WELLS, P., KLAREN, R., ILLOUZ, M., MITCHELL, J., STANCHFIELD, S., COKER, J., ZUKOWSKI, M., AND FLACK, C. 2000, ANTLR Reference Manual. Tech. Rep., MageLang Institute's jGuru.com. January. See www. antlr. org/doc/index. html.
35. PRYCE, J. D. AND REID, J. K. 1998. ADO1, A Fortran 90 code for automatic differentiation. Tech. Rep. RAL-TR-1998-057, Rutherford Appleton Laboratory, Chilton, Didcot, Oxfordshire, OX11 OQX, England, See htp://matisa.cc.rl.ac.uk/ pub/reports/prRAL98057.ps.gz.
36. REID, J. 2003. On the stability of automatic differentiation. In preparation.
37. ROE, P. L. 1981. Approximate Riemann solvers, parameter vectors, and difference schemes. J.Computat. Phys. 43, 357-372.
38. TADJOUDDINE, M. 1999. La différentiation automatique. Ph.D. dissertation, Université de Nice, Sophia Antipolis, France.
39. TADJOUDDINE, M., EYSETTE, F., AND FAURE, C. 1998. Sparse Jacobian computation in automatic differentiation by static program analysis. In Proceedings of the 5th International Static Analysis Symposium. Number 1503 in Lecture Notes in Computer Science. Springer-Verlag, New York, 311-326.
40. TADJOUDDINE, M., FORTH, S. A., AND PRYCE, J. D. 2001. AD tools and prospects for optimal AD in CFD flux Jacobian calculations. In Automatic Differentiation: From Simulation to Optimization, G. Corliss, C. Faure, A. Griewank, L. Hascoët, and U. Naumann, Eds. Springer-Verlag, New York, Chapter 30, 247-252.
41. TADJOUDDINE, M., FORTH, S. A., AND PRYCE, J. D. 2003. Hierarchical automatic differentiation by vertex elimination and source transformation. In Proceedings of Computational Science and Its Applications-ICCSA 2003. Lecture Notes in Computer Science, vol. 2. Springer-Verlag, New York, 115-124.
42. TADJOUDDINE, M., FORTH, S. A., PRYCE, J. D., AND REID, J. K. 2002. Performance issues for vertex elimination methods in computing Jacobians using automatic differentiation. In Proceedings of the 2nd International Conference on Computational Science, P. M. Sloot, Ed. Lecture Notes in Computer Science, vol. 2. Springer-Verlag, New York, 1077-1086.
43. VERMA, A. 1998. ADMAT: Automatic differentiation in MATLAB using object oriented methods. In SIAM Interdisciplinary Workshop on Object Oriented Methods for Interoperability. SIAM, National Science Foundation, Yorktown Heights, N.Y., 174-183.
44. WILKINSON, J. 1965. The Algebraic Eigenvalue Problem. Oxford University Press.