Calibrating artificial neural networks by global optimization

Expert Systems with Applications

Volumn 39, Issue 1, 2012, Pages 25-32

  Pintér, János D ^a,b  

^a Pintér Consulting Services Inc   (Canada)

^b OZYEGIN UNIVERSITY   (Turkey)

Author keywords

ANN implementation in Mathematica; Artificial neural networks; Calibration of ANNs by global optimization; Lipschitz Global Optimizer (LGO) solver suite; MathOptimizer Professional (LGO linked to Mathematica); Numerical examples

Indexed keywords

ACTUAL SYSTEM; ARTIFICIAL NEURAL NETWORK; BLACK BOXES; CALIBRATION PROBLEMS; FUNCTIONAL RELATIONSHIP; GLOBAL-LOCAL; INPUT AND OUTPUTS; LIPSCHITZ; MATHEMATICA; MULTI-MODAL OPTIMIZATION; NUMERICAL APPROXIMATIONS; NUMERICAL EXAMPLE; OPTIMIZATION SOFTWARE; SYSTEM OUTPUT; TECHNICAL COMPUTING SYSTEMS; TEST FUNCTIONS; TRAINING SETS;

CALIBRATION; GLOBAL OPTIMIZATION; OPTIMIZATION;

NEURAL NETWORKS;

EID: 81855207560     PISSN: 09574174     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.eswa.2011.06.050     Document Type: Article

References (61)

1. A. Abraham Meta learning evolutionary artificial neural networks Neurocomputing 56 2004 1 38
2. M.M. Ali, C. Khompatraporn, and Z.B. Zabinsky A numerical evaluation of several stochastic algorithms on selected continuous global optimization test problems Journal of Global Optimization 31 2005 635 672
3. P. Auer, M. Herbster, and M. Warmuth Exponentially many local minima for single neurons D. Touretzky, Advances in neural information processing systems Vol. 8 1996 MIT Press Cambridge, MA 316 322
4. Y. Bar-Yam Dynamics of complex systems 1999 Westview Press Boulder, CO
5. M. Bianchini, and M. Gori Optimal learning in artificial neural networks: A review of theoretical results Neurocomputing 13 1996 313 346
6. aglayan, M. O.; & Pintér, J. D. (2010). Development and calibration of a currency trading strategy by global optimization. Research Report, Özyegin University, Istanbul. Submitted for publication.
7. I. Castillo, F.J. Kampas, and J.D. Pintér Solving circle packing problems by global optimization: Numerical results and industrial applications European Journal of Operational Research 191 2008 786 802
8. A. Cichocki, and R. Unbehauen Neural networks for optimization and signal processing 1993 John Wiley & Sons Chichester, UK.
9. H. Cruse Neural networks as cybernetic systems 2nd revised ed. 2006 Brains, Minds and Media Bielefeld, Germany
10. G. Cybenko Approximation by superpositions of a sigmoidal function Mathematics of Control, Signals, and Systems 2 1989 303 314
11. P.H. Franses, and D. van Dijk Nonlinear time series models in empirical finance 2000 Cambridge University Press Cambridge, UK
12. K. Funahashi On the approximate realization of continuous mappings by neural networks Neural Networks 2 1989 183 192
13. R.M. Golden Mathematical methods for neural network analysis and design 1996 MIT Press Cambridge, MA
14. P. Goossens, J. McPhee, J.D. Pintér, and C. Schmitke Driving innovation: How mathematical modeling and optimization increase efficiency and productivity in vehicle design 2007 Technical Memorandum Maplesoft, Waterloo, ON
15. L. Grippo A class of unconstrained minimization methods for neural network training Optimization Methods and Software 4 2 1994 135 150
16. L. Hamm, B.W. Brorsen, and M.T. Hagan Comparison of stochastic global optimization methods to estimate neural network weights Neural Processing Letters 26 2007 145 158
17. J. Hertz, A. Krogh, and R.G. Palmer Introduction to the theory of neural computation 1991 Addison-Wesley Reading, MA
18. K. Hornik, M. Stinchcombe, and H. White Multilayer feedforward networks are universal approximators Neural Networks 2 1989 359 366
19. R. Horst, and P.M. Pardalos Handbook of global optimization Vol. 1 1995 Kluwer Academic Publishers Dordrecht
20. G. Isenor, J.D. Pintér, and M. Cada A global optimization approach to laser design Optimization and Engineering 4 2003 177 196
21. I. Jordanov, and R. Brown Neural network learning using low-discrepancy sequence N. Foo, AI'99, Lecture notes in artificial intelligence Vol. 1747 1999 Springer-Verlag Berlin, Heidelberg 255 267
22. Y. Kajitani, K.W. Hipel, and A.I. McLeod Forecasting nonlinear time series with feed-forward neural networks: a case study of Canadian lynx data Journal of Forecasting 24 2005 105 117
23. Kampas, F. J.; & Pintér, J. D. (2010). Advanced nonlinear optimization in Mathematica. Research Report, Özyegin University, Istanbul. Submitted for publication.
24. F.J. Kampas, and J.D. Pintér Configuration analysis and design by using optimization tools in mathematica The Mathematica Journal 10 1 2006 128 154
25. B. Karthikeyan, S. Gopal, and S. Venkatesh Partial discharge pattern classification using composite versions of probabilistic neural network inference engine Expert Systems with Applications 34 2008 1938 1947
26. Ch. Khompatraporn, J.D. Pintér, and Z.B. Zabinsky Comparative assessment of algorithms and software for global optimization Journal of Global Optimization 31 2005 613 633
27. A.N. Kolmogorov On the representation of continuous functions of many variables by superposition of continuous functions of one variable and addition Doklady Akademii Nauk SSR 114 1957 953 956
28. B. Lang Monotonic multi-layer perceptron networks as universal approximators W. Duch, Artificial neural networks: Formal models and their applications - ICANN 2005, Lecture notes in computer science Vol. 3697 2005 Springer-Verlag Berlin, Heidelberg 31 37
29. T.L. Mason, C. Emelle, J. van Berkel, A.M. Bagirov, F. Kampas, and J.D. Pintér Integrated production system optimization using the Lipschitz global optimizer and the discrete gradient method Journal of Industrial and Management Optimization 3 2 2007 257 277
30. T. Masters Signal and image processing with neural networks 1994 John Wiley & Sons New York
31. K. Mehrotra, C.K. Mohan, and S. Ranka Elements of artificial neural nets 1997 MIT Press Cambridge, MA
32. P.M. Pardalos, and H.E. Romeijn Handbook of global optimization Vol. 2 2002 Kluwer Academic Publishers Dordrecht
33. G. Pantoleontos, P. Basinas, G. Skodras, P. Grammelis, J.D. Pintér, S. Topis, and G.P. Sakellaropoulos A global optimization study on the devolatilisation kinetics of coal, biomass and waste fuels Fuel Processing Technology 90 2009 762 769
34. J. Park, and I.W. Sandberg Universal approximation using radial-basis-function networks Neural Computation 3 2 1991 246 257
35. J. Park, and I.W. Sandberg Approximation and radial-basis-function networks Neural Computation 5 2 1993 305 316
36. J.D. Pintér Global optimization in action 1996 Kluwer Academic Publishers Dordrecht
37. J.D. Pintér LGO - A program system for continuous and Lipschitz optimization I.M. Bomze, T. Csendes, R. Horst, P.M. Pardalos, Developments in global optimization 1997 Kluwer Academic Publishers Dordrecht 183 197
38. J.D. Pintér Global optimization: Software, test problems, and applications P.M. Pardalos, H.E. Romeijn, Handbook of global optimization Vol. 2 2002 Kluwer Academic Publishers Dordrecht 515 569
39. J.D. Pintér Globally optimized calibration of nonlinear models: Techniques, software, and applications Optimization Methods and Software 18 3 2003 335 355
40. J.D. Pintér Nonlinear optimization in modeling environments: Software implementations for compilers, spreadsheets, modeling languages, and integrated computing systems V. Jeyakumar, A.M. Rubinov, Continuous optimization: Current trends and modern applications 2005 Springer Science + Business Media New York 147 173
41. J.D. Pintér Nonlinear optimization with GAMS/LGO Journal of Global Optimization 38 2007 79 101
42. J.D. Pintér Software development for global optimization P.M. Pardalos, T.F. Coleman, Global optimization: Methods and applications Fields institute communications Vol. 55 2009 Springer Berlin 183 204
43. Pintér, J. D. (2010a). LGO - A model development and solver system for nonlinear (global and local) optimization. User's guide. (Current version) Distributed by Pintér Consulting Services, Inc.; Canada. .
44. Pintér, J. D. (2010b). Spreadsheet-based modeling and nonlinear optimization with Excel-LGO. User's guide. With technical contributions by B.C. al and F.J. Kampas (Current version) Distributed by Pintér Consulting Services, Inc.; Canada; .
45. Pintér, J.D. (2010c). MATLAB-LGO User's Guide. With technical contributions by B.C. al and F.J. Kampas. (Current version) Distributed by Pintér Consulting Services, Inc.; Canada. .
46. J.D. Pintér, and F.J. Kampas Nonlinear optimization in Mathematica with Mathoptimizer Professional Mathematica in Education and Research 10 2 2005 1 18
47. J.D. Pintér, D. Linder, and P. Chin Global Optimization Toolbox for Maple: An introduction with illustrative applications Optimization Methods and Software 21 4 2006 565 582
48. Prechelt, L. (1994). PROBEN1 - A set of benchmarks and benchmarking rules for neural network training algorithms. Technical Report 21/94, Fakultät fur Informatik, Universität Karlsruhe, Germany.
49. B.D. Ripley Pattern recognition and neural networks 1996 Cambridge University Press Cambridge, MA
50. K. Röpke DoE - Design of experiments 2005 IAV GmbH, Verlag Moderne Industrie. © sv Corporate Media, GmbH Munich, Germany
51. R.S. Sexton, B. Alidaee, R.E. Dorsey, and J.D. Johnson Global optimization for artificial neural networks: A tabu search application European Journal of Operational Research 106 1998 570 584
52. M. Smith Neural networks for statistical modeling 1993 Van Nostrand Reinhold New York
53. W.-H. Steeb The nonlinear workbook 3rd ed. 2005 World Scientific Singapore
54. I. Sutskever, and G.E. Hinton Deep, narrow sigmoid belief networks are universal approximators Neural Computation 20 2008 2629 2636
55. J. Tervo, P. Kolmonen, T. Lyyra-Laitinen, J.D. Pintér, and T. Lahtinen An optimization-based approach to the multiple static delivery technique in radiation therapy Annals of Operations Research 119 2003 205 227
56. Ch.D. Tilakaratne, M.A. Mammadov, and S.A. Morris Predicting trading signals of stock market indices using neural networks AI 2008: Advances in artificial intelligence W. Wobcke, M. Zhang, Lecture notes in computer science Vol. 5360 2008 Springer-Verlag Berlin, Heidelberg 522 531
57. Toh, K.-A. (1999). Fast deterministic global optimization for FNN training. In Proceedings of IEEE international conference on systems, man, and cybernetics, Tokyo, Japan (pp. V413-V418).
58. M.N. Vrahatis, G.S. Androulakis, J.N. Lambrinos, and G.D. Magoulas A class of gradient unconstrained minimization algorithms with adaptive stepsize Journal of Computational and Applied Mathematics 114 2000 367 386
59. T.L.H. Watkin Optimal learning with a neural network Europhysics Letters 21 8 1993 871 876
60. Wolfram Research (2011). Mathematica (Current version 8.0.1). Distributed by WRI, Champaign, IL.
61. Ye, H.; & Lin, Z. (2003). Global optimization of neural network weights using subenergy tunneling function and ripple search. In Proceedings of the IEEE ISCAS, Bangkok, Thailand, May 2003 (pp. V725-V728.