Generalized constraint neural network regression model subject to linear priors

IEEE Transactions on Neural Networks

Volumn 22, Issue 12 PART 2, 2011, Pages 2447-2459

  Qu, Ya Jun ^a   Hu, Bao Gang ^a  

^a INSTITUTE OF AUTOMATION   (China)

Author keywords

Linear constraints; linear priors; nonlinear regression; radial basis function networks; transparency

Indexed keywords

COMPUTATIONAL COSTS; GENERALIZED CONSTRAINT; HIGHER-DEGREE; LAGRANGE; LEAST-SQUARES APPROACH; LINEAR CONSTRAINTS; LINEAR PRIORS; MODELING APPROACH; MONOTONICITY; MULTIPLIER TECHNIQUES; NON-LINEAR REGRESSION; NUMERICAL INVESTIGATIONS; RADIAL BASIS FUNCTIONS; REAL-WORLD DATASETS; REGRESSION MODEL; REGRESSION PROBLEM; STRUCTURAL MODES;

LAGRANGE MULTIPLIERS; RADIAL BASIS FUNCTION NETWORKS; REGRESSION ANALYSIS; TRANSPARENCY;

NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; REGRESSION ANALYSIS; STATISTICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; LINEAR MODELS; REGRESSION ANALYSIS;

EID: 83655201363     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2011.2167348     Document Type: Article

References (50)

1. R. O. Duda, P. E. Hart, and D. Stork, Pattern Classification, 2nd ed. New York: Wiley, 2001.
2. S. Haykin, Neural Networks: A Comprehensive Foundation, 2nd ed. Englewood Cliffs, NJ: Prentice-Hall, 1999.
3. B. G. Hu, H. B. Qu, Y. Wang, and S. H. Yang, "A generalized- constraint neural network model: Associating partially known relationships for nonlinear regressions," Inf. Sci., vol. 179, no. 12, pp. 1929-1943, May 2009.
4. W. Pedrycz and R. A. Aliev, "Logic-oriented neural networks for fuzzy neurocomputing," Neurocomputing, vol. 73, nos. 1-3, pp. 10-23, Dec. 2009.
5. E. Krupka and N. Tishby, "Incorporating prior knowledge on features into learning," in Proc. Int. Conf. Articial Intell. Stat., 2007, pp. 1-8.
6. V. Vapnik and A. Vashist, "A new learning paradigm: Learning using privileged information," Neural Netw., vol. 22, nos. 5-6, pp. 544-557, Jul. 2009.
7. F. Lauer and G. Bloch, "Incorporating prior knowledge in support vector machines for classification: A review," Neurocomputing, vol. 71, nos. 7-9, pp. 1578-1594, Mar. 2008. (Pubitemid 351373181)
8. B. G. Hu, Y. Wang, S. H. Yang, and H. B. Qu, "How to add transparency to artificial neural networks," Pattern Recognit. Artif. Intell., vol. 20, no. 1, pp. 72-84, 2007. (Pubitemid 46577304)
9. T. M. Mitchell, Machine Learning. New York: McGraw-Hill, 1997.
10. T. Yu, "Incorporating prior domain knowledge into inductive machine learning: Its implementation in contemporary capital markets," Ph.D. thesis, Faculty Inf. Technol., Univ. Technology, Sydney, Australia, 2007.
11. F. Lauer and G. Bloch, "Incorporating prior knowledge in support vector regression," Mach. Learn., vol. 70, no. 1, pp. 89-118, 2008. (Pubitemid 50004032)
12. L. M. Fu, "Rule generation from neural networks," IEEE Trans. Syst., Man, Cybern., vol. 24, no. 8, pp. 1114-1124, Aug. 1994.
13. A. Tickle, R. Andrews, M. Golea, and J. Diederich, "The truth will come to light: Directions and challenges in extracting the knowledge embedded within trained artificial neural networks," IEEE Trans. Neural Netw., vol. 9, no. 6, pp. 1057-1068, Nov. 1998. (Pubitemid 128742391)
14. R. Setiono, W. K. Leow, and J. M. Zurada, "Extraction of rules from artificial neural networks for nonlinear regression," IEEE Trans. Neural Netw., vol. 13, no. 3, pp. 564-577, May 2002. (Pubitemid 34669648)
15. D. Martens, B. Baesens, and T. V. Gestel, "Decompositional rule extraction from support vector machines by active learning," IEEE Trans. Knowl. Data Eng., vol. 21, no. 2, pp. 178-191, Feb. 2009.
16. T. Q. Huynh and J. A. Reggia, "Guiding hidden layer representations for improved rule extraction from neural networks," IEEE Trans. Neural Netw., vol. 22, no. 2, pp. 264-275, Feb. 2011.
17. T. Mitchell, "The need for biases in learning generalizations," Dept. Comput. Sci., Rutgers Univ., Cream Ridge, NJ, Tech. Rep. CBM-TR-117, 1980.
18. T. Poggio and F. Girosi, "Networks for approximation and learning," Proc. IEEE, vol. 78, no. 9, pp. 1481-1497, Sep. 1990.
19. V. Vapnik, Statistical Learning Theory. New York: Wiley, 1998.
20. Y. Abu-Mostafa, "A method for learning from hint," in Proc. Adv. Neural Inf. Process. Syst., vol. 5. 1992, pp. 73-80.
21. G. Towell and J. Shavlik, "Knowledge-based artificial neural networks," Artif. Intell., vol. 70, nos. 1-2, pp. 119-165, Oct. 1994.
22. J.-S. R. Jang, "ANFIS: Adaptive-network-based fuzzy inference system," IEEE Trans. Syst., Man, Cybern., vol. 23, no. 3, pp. 665-685, May-Jun. 1993.
23. D. J. C. MacKay, "A practical Bayesian framework for backpropagation networks," Neural Comput., vol. 4, no. 3, pp. 448-472, 1992.
24. D. Psichogios and L. H. Ungar, "A hybrid neural network-first principles approach to process modeling," AIChE J., vol. 38, no. 10, pp. 1499-1511, Oct. 1992.
25. M. L. Thompson and M. A. Kramer, "Modeling chemical processes using prior knowledge and neural networks," AIChE J., vol. 40, no. 8, pp. 1328-1340, Aug. 1994.
26. B. Scholkopf, P. Simard, A. Smola, and V. Vapnik, "Prior knowledge in support vector kernels," in Proc. Adv. Neural Inf. Process. Syst., vol. 10. 1998, pp. 640-646.
27. P. Niyogi, F. Girosi, and T. Poggio, "Incorporating prior information in machine learning by creating virtual examples," Proc. IEEE, vol. 86, no. 11, pp. 2196-2209, Nov. 1998.
28. M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul, "An introduction to variational methods for graphical models," in Learning in Graphical Models, M. I. Jordan, Ed. Cambridge, MA: MIT Press, 1999.
29. Y. J. Qu and B. G. Hu, "RBF networks for nonlinear models subject to linear constraints," in Proc. IEEE Int. Conf. Granular Comput., Nanchang, China, Aug. 2009, pp. 482-487.
30. Y. J. Qu, B. Dai, and B. G. Hu, "Neural-network based regression model with prior from ranking information," in Proc. Int. Joint Conf. Neural Netw., Barcelona, Spain, Jul. 2010, pp. 3005-3012.
31. B. Scholkopf and A. Smola, Learning with Kernel, Support Vector Machines, Regularization, Optimization, and Beyond. Cambridge, MA: MIT Press, 2002.
32. S. Chen, C. J. Harris, and X. Hong. (2009). Grey-Box Radial Basis Function Modelling: The Art of Incorporating Prior Knowledge Online]. Available: http://eprints.ecs.soton.ac.uk/17774/1/gbrbfSSP09.pdf
33. X. J. Zhu and B. G. Andrew, "Kernel regression with order preferences," in Proc. Assoc. Advance. Artif. Intell., 2007, pp. 681-686.
34. J. Bi, K. Bennett, M. Embrechts, C. Breneman, and M. Song, "Dimensionality reduction via sparse support vector machines," J. Mach. Learn. Res., vol. 3, pp. 1229-1243, Mar. 2003.
35. V. Gomez-Verdejo, M. Martinez-Ramon, J. Arenas-Garcia, M. Lazaro-Gredilla, and H. Molina-Bulla, "Support vector machines with constraints for sparsity in the primal parameters," IEEE Trans. Neural Netw., vol. 22, no. 8, pp. 1269-1283, Aug. 2011.
36. A. Minin, M. Velikova, B. Lang, and H. Daniels, "Comparison of universal approximators incorporating partial monotonicity by structure," Neural Netw., vol. 23, no. 4, pp. 471-475, May 2010.
37. M. Velikova, H. Daniels, and A. Feelders, "Mixtures of monotone networks for prediction," Int. J. Comput. Intell., vol. 3, no. 3, pp. 204-214, 2006.
38. C. Dugas, Y. Bengio, F. Bélisle, C. Nadeau, and R. Garcia, "Incorporating functional knowledge in neural networks," J. Mach. Learn. Res., vol. 10, pp. 1239-1262, Jun. 2009.
39. H. Daniels and M. Velikova, "Monotone and partially monotone neural networks," IEEE Trans. Neural Netw., vol. 21, no. 6, pp. 906-917, Jun. 2010.
40. O. L. Mangasarian, J. W. Shavlik, and E. W. Wild, "Knowledge-based kernel approximation," J. Mach. Learn. Res., vol. 5, pp. 1127-1141, Sep. 2004.
41. O. L. Mangasarian and E. W. Wild, "Nonlinear knowledge in kernel approximation," IEEE Trans. Neural Netw., vol. 18, no. 1, pp. 300-306, Jan. 2007. (Pubitemid 46062936)
42. Z. Sun, Z. K. Zhang, H. G. Wang, and M. Jiang, "Cutting plane method for continuously constrained kernel-based regression," IEEE Trans. Neural Netw., vol. 21, no. 2, pp. 238-247, Feb. 2010.
43. J. Weston, O. Chapelle, A. Elisseeff, B. Scholkopf, and V. Vapnik, "Kernel dependency estimation," in Proc. Adv. Neural Inf. Syst., vol. 15. 2003, pp. 873-880.
44. S. Ferrari and M. Jensenius, "A constrained optimization approach to preserving prior knowledge during incremental training," IEEE Trans. Neural Netw., vol. 19, no. 6, pp. 996-1009, Jun. 2008.
45. M. X. Zhu and D. L. Zhang, "Study on the algorithms of selecting the radial basis function center," J. Anhui Univ., vol. 24, no. 1, pp. 72-78, 2000.
46. R. F. Stengel, Optimal Control and Estimation. New York: Dover, 1994, pp. 36-41.
47. D. P. Bertsekas, Constrained Optimization and Lagrange Multiplier Methods. Belmont, MA: Athena Scientific, 1996.
48. R. Fletcher, Practical Methods of Optimization, 2nd ed. Chichester, U.K.: Wiley, 1987.
49. K. S. Narendra and K. Parthasarathy, "Identification and control of dynamical systems using neural networks," IEEE Trans. Neural Netw., vol. 1, no. 1, pp. 4-27, Mar. 1990. (Pubitemid 20689507)
50. M. Bortman and M. Aladjem, "A growing and pruning method for radial basis function networks," IEEE Trans. Neural Netw., vol. 20, no. 6, pp. 1039-1045, Jun. 2009.