The extended Ritz method for functional optimization: Overview and applications to single-person and team optimal decision problems

Optimization Methods and Software

Volumn 24, Issue 1, 2009, Pages 15-43

  Baglietto, M ^a   Sanguineti, M ^a   Zoppoli, R ^a  

^a UNIVERSITY OF GENOA   (Italy)

Author keywords

Curse of dimensionality; Dynamic programming; Dynamic routing; Stochastic T stage decision problems; Suboptimal solutions; Water reservoirs management

Indexed keywords

APPLICATIONS; APPROXIMATION ALGORITHMS; APPROXIMATION THEORY; COMMUNICATION; COMPUTER NETWORKS; DYNAMIC PROGRAMMING; FUNCTIONS; NONLINEAR PROGRAMMING; OPTIMIZATION; POLYNOMIAL APPROXIMATION; PROBABILITY DENSITY FUNCTION; STOCHASTIC CONTROL SYSTEMS; SYSTEMS ENGINEERING; TELECOMMUNICATION NETWORKS;

CURSE OF DIMENSIONALITY; DYNAMIC ROUTING; STOCHASTIC T-STAGE DECISION PROBLEMS; SUBOPTIMAL SOLUTIONS; WATER RESERVOIRS MANAGEMENT;

RESERVOIRS (WATER);

EID: 57649217589     PISSN: 10556788     EISSN: 10294937     Source Type: Journal    
DOI: 10.1080/10556780802328900     Document Type: Article

References (66)

1. A. Alessandri, M. Baglietto, T. Parisini, and R. Zoppoli, A neural state estimator with bounded errors for nonlinear systems, IEEE Trans. Automat. Control 44 (1999), pp. 2028-2042.
2. A. Alessandri, M. Cuneo, S. Pagnan, and M. Sanguined, A recursive algorithm/or nonlinear least-squares problems, Comput. Optim. Appl. 38 (2007), pp. 195-216.
3. A. Alessandri, T. Parisini, M. Sanguined, and R. Zoppoli, Neural strategies for nonlinear optimal filtering, Proceedings of IEEE International Conference on Systems Engineering, Kobe, Japan, 1992, pp. 44-49.
4. A. Alessandri and M. Sanguined, Optimization of approximating networks for optimal fault diagnosis, Optim. Methods Softw. 20 (2005), pp. 241-266.
5. A. Alessandri, M. Sanguined, and M. Maggiore, Optimization-based learning with bounded error for feedforward neural networks, IEEE Trans. Neural Net. 13 (2002), pp. 261-273.
6. G. Apostolopoulos, D. Williams, S. Kamat, R. Guerin, A. Orda, and T. Przygienda, QoS routing mechanisms and OSPF extensions, RFC 2676, August 1999.
7. H. Attouch, Variational Convergence for Functions and Operators, Pitman Publishing, London, England, 1984.
8. M. Baglietto, G. Battistelli, R. Bolla, R. Bruschi, F. Davoli, and R. Zoppoli, A neural network solution to QoS-IP team-optimal dynamic routing, Proceedings of Joint 44th IEEE Conference on Decision and Control and European Control Conference, Seville, Spain, 2005, pp. 7452-7459.
9. M. Baglietto, C. Cervellera, M. Sanguined, and R. Zoppoli, Water reservoirs management under uncertainty by approximating networks and learning from data, in Topics on System Analysis and Integrated Water Resource Management, Chapter 6, A. Castelletti and R. Soncini Sessa, eds. Elsevier Science Ltd., 2006, pp. 117-139.
10. M. Baglietto, T. Parisini, and R. Zoppoli, Numerical solutions to the Witsenhausen counterexample by approximating networks, IEEE Trans. Automat. Control 46 (2001), pp. 1471-1477.
11. -. Distributed-information neural control. The case of dynamic routing in traffic networks, IEEE Trans. Neural Net. 12 (2001), pp. 485-502.
12. A.R. Barron, Universal approximation bounds for superpositions of a sigmoidal'function, IEEE Trans. Infom. Theory 39 (1993), pp. 930-945.
13. R.W. Beard and T.W. Mclain, Successive Galerkin approximation algorithms for nonlinear optimal and robust control, Int. J. Control 71 (1998), pp. 717-743.
14. R. Bellman, Dynamic Programming, Princeton University Press, Princeton, NJ, 1957.
15. L. Breiman, Hinging hyperplanes for regression, classification, and function approximation, IEEE Trans. Inform. Theory 39 (1993), pp. 993-1013.
16. V.C.P. Chen, D. Ruppert, and C.A. Shoemaker, Applying experimental design and regression splines to highdimensional continuous-state stochastic dynamic programming, Oper. Res. 47 (1999), pp. 38-53.
17. R.A. DeVore, R. Howard, and C. Micchelli, Optimal nonlinear approximation, Manuscipta Math. 63 (1989), pp. 469-478.
18. U. Felgenhauer, On Ritz type discretizations for optimal control problems, Proceeding of 18th IFIP-ICZ Conference, Research Notes in Mathematics, Chapman-Hall, Vol. 386, 1999, pp. 91-99.
19. E. Foufoula-Georgiou and P.K. Kitanidis, Gradient dynamic programming for stochastic optimal control of multidimensional water resources systems, Water Resour. Res. 24 (1988), pp. 1345-1359.
20. I.M. Gelfand and S.V. Fomin, Calculus of Variations, Prentice Hall, Englewood Cliffs, NJ, 1963.
21. F. Girosi, M. Jones, and T. Poggio, Regularization theory and neural networks architectures, Neural Comput. 7 (1995). pp. 219-269.
22. S. Giulini and M. Sanguined, Approximation schemes for functional optimization problems, J. Optim. Theory Appl. to appear.
23. G. Gnecco and M. Sanguineti, Accuracy of suboptimal solutions to kernel principal component analysis, Comput. Optim. Appl. 2008, DOI:10.1007/s10589-007-9108-y.
24. -. The weight-decay technique in learning from data: An optimization point of view, Comp. Manag. Sci. 2008, DOI:10.1007/s10287-008-0072-5.
25. -. Approximation error bounds via Rademacher complexity, Appl. Math. Sci. 2 (2008), pp. 153-176.
26. -. Estimates of the approximation error using Rademacher complexity: learning vector-valued functions, J. Inequal. Appl. to appear.
27. L. Grippo, Convergent on-line algorithms for supervised learning in neural networks, IEEE Trans. Neural Net. 11 (2002), pp. 1284-1299.
28. W. W. Hager, The Ritz-Trefftz method for state and control constrained optimal control problems, SIAM J. Numer. Anal. 12 (1975), pp. 854-867.
29. A. Iftar and E.J. Davison, A decentralized discrete-time controller for dynamic routing, Int. J. Control 69 (1998), pp. 599-632.
30. S. A. Johnson, J.R. Stedinger, C. Shoemaker, Y. Li, and J.A. Tejada-Guibert, Numerical solution of continuous-state dynamic programs using linear and spline interpolation, Oper. Res. 41 (1993), pp. 484-500.
31. L.K. Jones, A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training, Ann. Stat. 20 (1992), pp. 608-613.
32. P.C. Kainen, V. Ku̇rková, and V.M. Sanguineti, Minimization of error junctionals over variable-basis functions, SIAM J. Optim. 14 (2003), pp. 732-742.
33. -. Rates of approximate minimization of error functlonals over Boolean variable-basis functions, J. Mathe. Model. Algorithms 4 (2005), pp. 355-368.
34. -. Complexity of Gaussian radial-basis networks approximating smooth functions, J. Complexity, to appear.
35. P.C. Kainen, V. Ku̇rková, and A. Vogt, Approximation by neural networks is not continuous, Neurocomputing 29 (1999), pp. 47-56.
36. -. Geometry and topology of continuous best and near best approximations, J. Approx. Theory 105 (2000), pp. 252-262.
37. V. Ku̇rková, Dimension-independent rates of approximation by neural networks. In Computer-Intensive Methods in Control and Signal Processing: Curse of Dimensionality, K. Warwick and M. Kárný, eds., Birkhäuser, Boston, 1997, pp. 261-270.
38. V. Ku̇rková, P.C. Kainen, and V. Kreinovich, Estimates of the number of hidden units and variation with respect to half-spaces, Neural Net. 10 (1997), pp. 1061-1068.
39. V. Ku̇rková and M. Sanguineti, Bounds on rates of variable-basis and neural-network approximation, IEEE Trans. Inform. Theory 47 (2001), pp. 2659-2665.
40. -. Comparison of worst-case errors in linear and neural network approximation, IEEE Trans. Inform. Theory 48 (2002), pp. 264-275.
41. -. Learning with generalization capability by kernel methods of bounded complexity, J. Complexity 21 (2005), pp. 350-367.
42. -. Error estimates for approximate optimization by the extended Ritz method. SIAM J. Optim. 15 (2005), pp. 461-487.
43. -. Approximate minimization of the regularized expected error over kernel models, Math. Oper. Res. to appear.
44. V. Ku̇rková, P. Savický, and K. Hlává cková, Representations and rates of approximation of real-valued Boolean functions by neural networks, Neural Netw. 11 (1998), pp. 651-659.
45. H.J. Kushner and G.G. Yin, Stochastic Approximation Algorithms and Applications, Springer Verlag, New York, 1997.
46. V. Maiorov and A. Pinkus, Lower bounds for approximation by MLP neural networks, Neurocomputing 25 (1999), pp. 81-91.
47. J. Marschak, Elements for a theory of teams, Manag. Sci. 1 (1955), pp. 127-137.
48. J. Marschak and R. Radner, Economic Theory of Teams, Yale University Press, 1972.
49. P. Niyogi and F. Girosi, On the relationship between generalization error, hypothesis complexity, and sample complexity for radial basis functions, Neural Comput. 8 (1996), pp. 819-842.
50. C.H. Papadimitriou and J.N. Tsitsiklis, Intractable problems in control theory, SIAM J. on Control Optim. 24 (1986), pp. 639-654.
51. T. Parisini and R. Zoppoli, Neural networks for feedback feedforward nonlinear control systems, IEEE Trans. Neural Net. 5 (1994), pp. 436-449.
52. -. Neural approximations for infinite-horizon optimal control of nonlinear stochastic systems, IEEE Trans. Neural Netw. 9 (1998), pp. 1388-1408.
53. C.R. Philbrick, Jr. and P.K. Kitanidis, Improved dynamic programming methods for optimal control of lumpedparameter stochastic systems, Oper. Res. 49 (1999), pp. 398-412.
54. A. Pinkus, Approximation by ridge functions, in Surface Fitting and Multiresolution Methods, A. Le Méhauté, C. Rabut, and L.L. Schumaker, eds., Vanderbilt University Press, Nashville, TN, 1997, pp. 1-14.
55. -. Approximation theory of the MLP model in neural networks, Acta Numer. 8 (1999), pp. 143-195.
56. R. Radner, Team decision problems, Ann. Mathe. Stat. 33 (1962), pp. 857-881.
57. W. Ritz, Über Eine Neue Methode zur Lösung Gewisser Variationsprobleme der Mathematischen Physik, J. für die reine und angewandte Mathematik 135 (1909), pp. 1-61.
58. M. Sanguineti, Universal approximation by ridge computational models: a survey, Open Appl. Math. J. 2 (2008), pp. 31-58.
59. A. Segall, The modeling of adaptive routing in data-communication networks. IEEE Trans. Commun. 25 (1977), pp. 85-95.
60. A. Singh, G. Mittal, and S.K. Nandi, QoS and traffic engineering: MPLS, diffserv and constraint based routing, May 2000. Available at http://keskus.hut.fi/tutkimus/ironet/ns2/qosr.pdf.
61. H.R. Sirisena and F.S. Chou, Convergence of the control parametrization Ritz method for nonlinear optimal control problems, J. Optim. Theory Appl. 29 (1979), pp. 369-382.
62. H.S. Witsenhausen. A counterexample in stochastic optimum control, SIAM J. Control. 6 (1968), pp. 131-147.
63. S. Yakowitz, Dynamic programming applications in water resources, Water Resour. Res. 18 (1982), pp. 673-696.
64. R. Zoppoli and T. Parisini, Learning techniques and neural networks for the solution of Nstage nonlinear nonquadratic optimal control problems, in Systems, Models and Feedback: Theory and Applications, A. Isidori and T.J. Tarn, Eds., Birkhäuser, Boston, 1992, pp. 193-210.
65. R. Zoppoli, M. Sanguineti, and T. Parisini, Approximating networks and the extended Ritz method for the solution of functional optimization problems, J. Optim. Theory Appl. 112 2002, pp. 403-439.
66. The Network Simulator 2. Available at http://www.isi.edu/nsnam/ns/ .