What color is your Jacobian? Graph coloring for computing derivatives

SIAM Review

Volumn 47, Issue 4, 2005, Pages 629-705

  Gebremedhin, Assefaw Hadish ^a   Manne, Fredrik ^b   Pothen, Alex ^a  

^a ECS Building   (United States)

^b UNIVERSITY OF BERGEN   (Norway)

Author keywords

Approximation algorithms; Automatic differentiation; Distance coloring; Finite differences; Hessians; Jacobians; Matrix partitioning problems; Sparsity; Symmetry

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; DIFFERENTIATION (CALCULUS); FINITE DIFFERENCE METHOD; GRAPH THEORY; MATRIX ALGEBRA;

AUTOMATIC DIFFERENTIATION; GRAPH COLORING; HESSIAN MATRIX; JACOBIAN MATRIX; MATRIX PARTITIONING PROBLEMS; SPARSITY;

COLORING;

EID: 28244465516     PISSN: 00361445     EISSN: None     Source Type: Journal    
DOI: 10.1137/S0036144504444711     Document Type: Review

References (124)

1. A. Aggarwal, A. Bar-Noy, D. Coppersmith, R. Ramaswami, B. Schieber, and M. Sudan, Efficient routing and scheduling algorithms for optical networks, in Proceedings of the 5th Annual ACM-SIAM Symposium on Discrete Algorithms, SIAM, Philadelphia, 1994, pp. 412-423.
2. G. Agnarsson, R. Greenlaw, and M. M. Halldórsson, On powers of chordal graphs and their colorings, Congr. Numer., 100 (2000), pp. 41-65.
3. G. Agnarsson and M. M. Halldórsson, Coloring powers of planar graphs, SIAM J. Discrete Math., 16 (2003), pp. 651-662.
4. M. O. Albertson and D. M. Berman, The acyclic chromatic number, Congr. Numer., 17 (1976), pp. 51-69.
5. 4's, Electron. J. Combin., 11 (2004), article R26.
6. N. Alon, C. McDiarmid, and B. Reed, Acyclic coloring of graphs, Random Structures Algorithms, 2 (1991), pp. 277-288.
7. K. Appel and W. Haken, Every planar map is four colorable, Part 1: Discharging, Illinois J. Math., 21 (1977), pp. 429-490.
8. K. Appel and W. Haken, The four color proof suffices, Math. Intell., 8 (1986), pp. 10-20.
9. K. Appel and W. Haken, Every Planar Map Is Four Colorable, AMS, Providence, RI, 1989.
10. K. Appel, W. Haken, and J. Koch, Every planar map is four colorable, Part II: Reducibility, Illinois J. Math., 21 (1977), pp. 491-567.
11. O. Axelsson and U. Nävert, On a graphical package for nonlinear partial differential equation problems, in Proceedings of the IFIP Congress 77, B. Gilchrist, ed., Information Processing, North-Holland, Amsterdam, 1977, pp. 103-108.
12. M. Bellare, O. Goldreich, and M. Sudan, Free bits, PCPs and nonapproximability - towards tight results, SIAM J. Comput., 27 (1998), pp. 804-915.
13. C. Berge, Hypergraphs, North-Holland, Amsterdam, 1989.
14. B. Berger and J. Rompel, A better performance guarantee for approximate graph coloring, Algorithmica, 5 (1990), pp. 459-466.
15. J. Bialogrodzki, Path coloring and routing in graphs, in Graph Colorings, M. Kubale, ed., Contemp. Math. 352, AMS, Providence, RI, 2004, pp. 139-152.
16. C. Bischof, A. Carle, P. Khademi, and A. Mauer, ADIFOR 2.0: Automatic differentiation of Fortran 77 programs, IEEE Comput. Sci. Engrg., 3 (1996), pp. 18-32.
17. C. Bischof, L. Roh, and A. Mauer, ADIC: An extensible automatic differentiation tool for ANSI-C, Software - Practice and Experience, 27 (1997), pp. 1427-1456.
18. E. Boman, D. Bozdaǧ, U. Catalyurek, A. H. Gebremedhin, and F. Manne, A scalable parallel graph coloring algorithm for distributed memory computers, in Proceedings of Euro-Par 2005 Parallel Processing, J. Cunha and P. Medeiros, eds., Lecture Notes in Comput. Sci. 3648, Springer-Verlag, Berlin, 2005, pp. 241-251.
19. O. V. Borodin, On acyclic colorings of planar graphs, Discrete Math., 25 (1979), pp. 211-236.
20. D. Bozdaǧ, U. Catalyurek, A. H. Gebremedhin, F. Manne, E. Boman, and F. Özgüner, A parallel distance-2 graph coloring algorithm for distributed memory computers, in Proceedings of HPCC-05, the 2005 International Conference on High Performance Computing and Communications, L.T. Yang et al., eds., Lecture Notes in Comput. Sci. 3726, Springer-Verlag, Berlin, 2005, pp. 796-806.
21. B. J. Breitkreutz, C. Stark, and M. Tyers, Osprey: A network visualization system, Genome Bio., 4 (2003), article R22.
22. D. Brélaz, New methods to color the vertices of a graph, Comm. ACM, 22 (1979), pp. 251-256.
23. M. I. Burstein, Every 4-valent graph has an acyclic 5-coloring, Soobsc. Akad. Nauk Gruzin. SSR, 93 (1979), pp. 21-24 (in Russian, with Georgian and English summaries).
24. C. J. Chaitin, M. Auslander, A. K. Chandra, J. Cocke, M. E. Hopkins, and P. Mark-Stein, Register allocation via coloring, Comput. Lang., 6 (1981), pp. 47-57.
25. N. Chiba, T. Nishizeki, and N. Saito, A linear 5-coloring algorithm for planar graphs, J. Algorithms, 2 (1981), pp. 317-327.
26. T. F. Coleman, Large Sparse Numerical Optimization, Lecture Notes in Comput. Sci. 165, Springer-Verlag, New York, 1984.
27. T. F. Coleman and J.-Y. Cai, The cyclic coloring problem and estimation of sparse Hessian matrices, SIAM J. Algebraic Discrete Methods, 7 (1986), pp. 221-235.
28. T. F. Coleman, B. Garbow, and J. J. Moré, Software for estimating sparse Jacobian matrices, ACM Trans. Math. Software, 10 (1984), pp. 329-347.
29. T. F. Coleman, B. Garbow, and J. J. Moré, Software for estimating sparse Hessian matrices, ACM Trans. Math. Software, 11 (1985), pp. 363-377.
30. T. F. Coleman and J. J. Moré, Estimation of sparse Jacobian matrices and graph coloring problems, SIAM J. Numer. Anal., 20 (1983), pp. 187-209.
31. T. F. Coleman and J. J. Moré, Estimation of sparse Hessian matrices and graph coloring problems, Math. Program., 28 (1984), pp. 243-270.
32. T. F. Coleman and A. Verma, The efficient computation of sparse Jacobian matrices using automatic differentiation, SIAM J. Sci. Comput., 19 (1998), pp. 1210-1233.
33. G. Corliss, C. Faure, A. Griewank, L. Hascoët, and U. Naumann, Automatic Differentiation of Algorithms: From Simulation to Optimization, Springer-Verlag, New York, 2002.
34. J. Culberson, Graph coloring page, http://www.es.ualberta.ca/~joe/ Coloring/index.html.
35. A. R. Curtis, M. J. D. Powell, and J. K. Reid, On the estimation of sparse Jacobian matrices, J. Inst. Math. Appl., 13 (1974), pp. 117-119.
36. T. Davis, University of Florida Sparse Matrix Collection, NA Digest, 97 (1997); http://www.cise.ufl.edu/research/sparse/matrices.
37. R. Diestel, Graph Theory, 2nd ed., Springer-Verlag, New York, 2000.
38. P. Duchet, Hypergraphs, in Handbook of Combinatorics, Volume I, R. L. Graham, M. Grötschel, and L. Lovász, eds., MIT Press, Cambridge, MA, 1996, pp. 381-432.
39. E. Elmroth, F. Gustavson, I. Jonsson, and B. Kågström, Recursive blocked algorithms and hybrid data structures for dense matrix library software, SIAM Rev., 46 (2004), pp. 3-45.
40. P. Erdös and A. Hajnal, On chromatic number of graphs and set systems, Acta. Math. Acad. Sci. Hungar., 17 (1966), pp. 61-99.
41. G. Fertin, A. Raspaud, and B. Reed, On star coloring of graphs, in Graph-Theoretic Concepts in Computer Science, 27th International Workshop, Lecture Notes in Comput. Sci. 2204, Springer-Verlag, Berlin, 2001, pp. 140-153.
42. H. J. Finck and H. Sachs, Über eine von H.S. Wilf angegebene Schranke für die chromatische Zahl endlicher Graphen, Math. Nachr., 39 (1969), pp. 373-386.
43. R. Fritsch and G. Fritsch, The Four Color Theorem: History, Topological Foundations, and Idea of Proof, Springer-Verlag, New York, 1998.
44. A. Gamst, Some lower bounds for a class of frequency assignment problems, IEEE Trans. Vehicular Tech., 35 (1986), pp. 8-14.
45. A. H. Gebremedhin, Practical Parallel Algorithms for Graph Coloring Problems in Numerical Optimization, Ph.D. thesis, University of Bergen, Norway, 2003.
46. A. H. Gebremedhin and F. Manne, Scalable parallel graph coloring algorithms, Concurrency: Practice and Experience, 12 (2000), pp. 1131-1146.
47. A. H. Gebremedhin, F. Manne, and A. Pothen, Parallel distance-k coloring algorithms for numerical optimization, in Euro-Par 2002 Parallel Processing, B. Monien and R. Feld-mann, eds., Lecture Notes in Comput. Sci. 2400, Springer-Verlag, Berlin, 2002, pp. 912-921.
48. A. H. Gebremedhin, A. Tarafdar, F. Manne, and A. Pothen, New Acyclic and Star Coloring Algorithms with Application to Computing Hessians, Tech. Report, Old Dominion University, Norfolk, VA, 2005.
49. U. Geitner, J. Utke and A. Griewank, Automatic computation of sparse Jacobians by applying the method of Newsam and Ramsdell, in Computational Differentiation: Techniques, Applications, and Tools, M. Berz, C. Bischof, G. Corliss, and A. Griewank, eds., SIAM, Philadelphia, 1996, pp. 161-172.
50. R. Giering and T. Taminski, Recipes for Adjoint Code Construction, Tech. Report 212, Max-Planck Institut für Meteorologie, Hamburg, 1996.
51. R. K. Gjertsen Jr., M. T. Jones, and P. E. Plassmann, Parallel heuristics for improved, balanced graph colorings, J. Parallel Distrib. Comput., 37 (1996), pp. 171-186.
52. D. Goldfarb and P. L. Toint, Optimal estimation of Jacobian and Hessian matrices that arise in finite difference calculations, Math. Comp., 43 (1984), pp. 69-88.
53. A. Griewank, Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, SIAM, Philadelphia, 2000.
54. A. Griewank and G. F. Corliss, eds, Automatic Differentiation of Algorithms: Theory, Implementation, and Application, SIAM, Philadelphia, 1991.
55. A. Griewank, D. Juedes, and J. Utke, ADOL-C: A package for the automatic differentiation of algorithms written in C/C++, ACM Trans. Math. Software, 22 (1996), pp. 131-167.
56. B. Grünbaum, Acyclic colorings of planar graphs, Israel J. Math., 14 (1973), pp. 390-408.
57. G. Hajós, Über eine Konstruktion nicht n-färbbarer Graphen, Wiss. Z. Martin-Luther-Univ. Halle-Wittenberg Math.-Nat. Reihe, 10 (1961), pp. 116-117.
58. J. Hall, J. Hartline, A. R. Karlin, J. Saia, and J. Wilkes, On algorithms for efficient data migration, in Proceedings of the 12th Annual ACM-SIAM Symposium on Discrete Algorithms, SIAM, Philadelphia, 2001, pp. 620-629.
59. M. M. Halldórsson, A still better performance guarantee for approximate graph coloring, Inform. Process. Lett., 45 (1993), pp. 19-23.
60. J. L. Hennessy and D. A. Patterson, Computer Architecture: A Quantitative Approach, 3rd ed., Morgan-Kaufmann, San Francisco, 2002.
61. S. Hossain, On the Computation of Sparse Jacobian Matrices and Newton Steps, Ph.D. thesis, Tech. Report 146, Department of Informatics, University of Bergen, Norway, 1998.
62. S. Hossain and T. Steihaug, Computing a sparse Jacobian matrix by rows and columns, Optim. Methods Softw., 10 (1998), pp. 33-48.
63. S. Hossain and T. Steihaug, Reducing the number of AD passes for computing a sparse Jacobian matrix, in Automatic Differentiation of Algorithms: From Simulation to Optimization, G. Corliss, C. Faure, A. Griewank, L. Hascoët, and U. Naumann, eds., Springer-Verlag, New York, 2002, pp. 263-270.
64. S. Hossain and T. Steihaug, Sparsity issues in computation of Jacobian matrices, in Proceedings of the 2002 International Symposium on Symbolic and Algebraic Computation, T. Mora, ed., ACM, New York, 2002, pp. 123-130.
65. S. Hossain and T. Steihaug, Optimal Direct Determination of Sparse Jacobian Matrices, Tech. Report 254, Department of Informatics, University of Bergen, Norway, 2003.
66. P. D. Hovland, Personal communication, 2004.
67. P. D. Hovland, Combinatorial problems in automatic differentiation, presented at the SIAM Workshop on Combinatorial Scientific Computing, 2004.
68. P. D. Hovland, Automatic Differentiation of Parallel Programs, Ph.D. thesis, University of Illinois at Urbana-Champaign, Urbana, IL, 1997.
69. P. D. Hovland and L. C. McInnes, Parallel simulation of compressible flow using automatic differentiation and PETSc, Parallel Comput., 27 (2001), pp. 503-519.
70. D. Hysom and A. Pothen, A scalable parallel algorithm for incomplete factor preconditioning, SIAM J. Sci. Comput., 22 (2001), pp. 2194-2215.
71. T. Jensen and B. Toft, Graph Coloring Problems, Wiley-Interscience, New York, 1995.
72. D. S. Johnson, Worst case behaviour of graph coloring algorithms, Congr. Numer., 10 (1974), pp. 513-527.
73. M. T. Jones and P. E. Plassmann, A parallel graph coloring heuristic, SIAM J. Sci. Comput., 14 (1993), pp. 654-669.
74. M. T. Jones and P. E. Plassmann, Scalable iterative solution of sparse linear systems, Parallel Comput., 20 (1994), pp. 753-773.
75. D. Karger, R. Motwani, and M. Sudan, Approximate graph coloring by semidefinite programming, J. ACM, 45 (1998), pp. 246-265.
76. D. K. Kaushik, Personal communication, 2004.
77. D. K. Kaushik, D. E. Keyes, and B. F. Smith, On the interaction of architecture and algorithm in the domain-based parallelization of an unstructured grid incompressible flow code, in Proceedings of the 10th International Conference on Domain Decomposition Methods, AMS, Providence, RI, 1998, pp. 311-319.
78. D. E. Keyes, P. D. Hovland, L. C. McInnes, and W. Samyono, Using automatic differentiation for second-order matrix-free methods in PDE-constrained optimization, in Automatic Differentiation of Algorithms: From Simulation to Optimization, G. Corliss, C. Faure, A. Griewank, L. Hascoët, and U. Naumann, eds., Springer-Verlag, New York, 2001, pp. 35-50.
79. D. A. Knoll and D. E. Keyes, Jacobian-free Newton-Krylov methods: A survey of approaches and applications, J. Comput. Phys., 193 (2004), pp. 357-397.
80. A. V. Kosowski and K. Manuszewski, Classical coloring of graphs, in Graph Colorings, M. Kubale, ed., Contemp. Math. 352, AMS, Providence, RI, 2004, pp. 1-19.
81. A. V. Kostochka, Acyclic 6-coloring of planar graphs, Metody Diskret. Analiz., 28 (1976), pp. 40-56 (in Russian).
82. A. V. Kostochka, Upper Bounds of Chromatic Functions of Graphs, Ph.D. thesis, Novosibirsk, 1978 (in Russian).
83. M. Krivelevich and B. Sudakov, The chromatic number of random hypergraphs, Random Structures Algorithms, 12 (1998), pp. 381-483.
84. S. O. Krumke, M. V. Marathe, and S. S. Ravi, Models and approximation algorithms for channel assignment in radio networks, Wireless Networks, 7 (2001), pp. 567-574.
85. M. Kubale, Graph colouring, in Encyclopedia of Microcomputers, Vol. 8, A. Kent and J. Williams, eds., Marcel Dekker, New York, 1991, pp. 47-69.
86. M. Kubale, Graph Colorings, AMS, Providence, RI, 2004.
87. S. L. Lee and P. D. Hovland, Sensitivity analysis using parallel ODE solvers and automatic differentiation in C: SensPVODE and ADIC, in Automatic Differentiation of Algorithms: Prom Simulation to Optimization, G. Corliss, C. Faure, A. Griewank, L. Hascoët, and U. Naumann, eds., Springer-Verlag, New York, 2001, Chap. 26, pp. 223-229.
88. D. R. Lick and A. T. White, k-degenerate graphs, Canad. J. Math., 22 (1970), pp. 1082-1096.
89. Y. L. Lin and S. S. Skiena, Algorithms for square roots of graphs, SIAM J. Discrete Math., 8 (1995), pp. 99-118.
90. L. Lovász and M. D. Plummer, Matching Theory, North-Holland, Amsterdam, 1986.
91. M. Luby, A simple parallel algorithm for the maximal independent set problem, SIAM J. Comput., 15 (1986), pp. 1036-1053.
92. B. Manvel, Extremely greedy coloring algorithms, in Graphs and Applications, Proceedings of the 1st Colorado Symposium on Graph Theory, F. Harary and J. Maybee, eds., John Wiley, New York, 1985, pp. 257-270.
93. D. W. Matula, A min-max theorem for graphs with application to graph coloring, SIAM Rev., 10 (1968), pp. 481-482.
94. D. W. Matula, G. Marble, and J. Isaacson, Graph coloring algorithms, in Graph Theory and Computing, R. Read, ed., Academic Press, New York, 1972, pp. 109-122.
95. D. W. Matula, Y. Shiloach, and R. E. Tarjan, Two Linear-Time Algorithms for Five-Coloring a Planar Graph, Tech. Report STAN-CS-80-830, Computer Science Department, Stanford University, Stanford, CA, 1980.
96. D. W. Matula, Y. Shiloach, and R. E. Tarjan, Analysis of two linear-time algorithms for five-coloring a planar graph, Congr. Numer., 33 (1981), p. 407 (abstract).
97. S. T. McCormick, Optimal approximation of sparse Hessians and its equivalence to a graph coloring problem, Math. Program., 26 (1983), pp. 153-171.
98. J. Mitchem, Every planar graph has an acyclic 8-coloring, Duke Math. J., 41 (1974), pp. 177-181.
99. R. Motwani, Average-case analysis of algorithms for matchings and related problems, J. ACM, 41 (1994), pp. 1329-1356.
100. U. Naumann, Cheaper Jacobians by simulated annealing, SIAM J. Optim., 13 (2002), pp. 660-674.
101. U. Naumann, Optimal accumulation of Jacobian matrices by elimination methods on the dual computational graph, Math. Program., 99 (2004), pp. 399-421.
102. J. Neśetŕil and P. Ossan de Mendez, Colorings and homomorphisms of minor closed classes, in Discrete and Computational Geometry: The Goodman-Pollack Festschrift, B. Aronov, S. Basu, J. Pach, and M. Sharir, eds., Springer-Verlag, Berlin, 2003, pp. 651-664.
103. G. N. Newsam and J. D. Ramsdell, Estimation of sparse Jacobian matrices, SIAM J. Algebraic Discrete Methods, 4 (1983), pp. 404-418.
104. J. J. Nocedal and S. J. Wright, Numerical Optimization, Springer-Verlag, New York, 1999.
105. W. Nooy, A. Mrvar, and V. Batageli, Exploring Social Network Analysis with Pajek, Cambridge University Press, Cambridge, UK, 2004.
106. B. Norris, S. Balay, S. Benson, L. Freitag, P. D. Hovland, L. C. McInnes, and B. Smith, Parallel components for PDEs and optimization: Some issues and experiences, Parallel Comput., 28 (2002), pp. 1811-1831.
107. M. J. D. Powell and P. L. Toint, On the estimation of sparse Hessian matrices, SIAM J. Numer. Anal., 16 (1979), pp. 1060-1074.
108. P. Raghavan and E. Upfal, Efficient routing in all-optical networks, in Proceedings of the 26th Symposium on the Theory of Computing (STOC'94), ACM, New York, 1994, pp. 134-143.
109. E. Ramadan, A. Tarafdar, and A. Pothen, A hypergraph model for the yeast protein complex network, in Proceedings of the IPDPS Workshop on High Performance Computational Biology, CD-ROM, IEEE, 2004.
110. N. Rostaing, S. Dalmas and A. Galligo, Automatic differentiation in Odyssee, Tellus, 45a (1993), pp. 558-568.
111. Y. Saad, ILUM: A multi-elimination ILU preconditioner for general sparse matrices, SIAM J. Sci. Comput., 17 (1996), pp. 830-847.
112. J. Schmidt-Pruzan, E. Shamir, and E. Upfal, Random hypergraph coloring algorithms and the weak chromatic number, J. Graph Theory, 8 (1985), pp. 347-362.
113. G. Szekeres and H. S. Wilf, An inequality for the chromatic number of a graph, J. Combin. Theory, 4 (1968), pp. 1-3.
114. C. Thomassen, Every planar graph is 5-choosable, J. Combin. Theory Ser. B, 62 (1994), pp. 180-181.
115. B. Toft, Coloring, stable sets and perfect graphs, in Handbook of Combinatorics, Vol. I, R. L. Graham, M. Grötschel, and L. Lovász, eds., MIT Press, Cambridge, UK, 1996, pp. 233-288.
116. M. Trick, Network resources for coloring a graph, http://mat.gsia.cmu. edu/COLOR/color.html.
117. V. V. Vazirani, Approximation Algorithms, Springer-Verlag, Berlin, 2001.
118. A. Walther, ADOL-C: A Package for Automatic Differentiation of Algorithms Written in C/C++, http://www.math.tu-dresden.de/wir/project/ adolc/.
119. G. Wegner, Graphs with Given Diameter and a Coloring Problem, Tech. Report, University of Dortmund, Germany, 1977.
120. D. J. A. Welsh and M. B. Powell, An upper bound for the chromatic number of a graph and its application to timetabling problems, Comput. J., 10 (1967), pp. 85-86.
121. A. Wigderson, Improving the performance guarantee for approximate graph coloring, J. ACM, 30 (1983), pp. 729-735.
122. R. A. Wilson, Graphs, Colourings, and the Four-Colour Theorem, Oxford University Press, Oxford, 2002.
123. L. A. Wolsey, Integer Programming, Wiley-Interscience, New York, USA, 1998.
124. M. Yannakakis, Computing the minimum fill-in is NP-complete, SIAM J. Algebraic Discrete Methods, 2 (1981), pp. 77-79.