The constraint based decomposition (CBD) training architecture

Neural Networks

Volumn 14, Issue 4-5, 2001, Pages 527-550

  Drǎghici, Sorin ^a  

^a WAYNE STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CONSTRAINT THEORY; CONVERGENCE OF NUMERICAL METHODS; PROBLEM SOLVING;

CONSTRAINT BASED DECOMPOSITION (CBD);

NEURAL NETWORKS;

ACCURACY; ALGORITHM; ARTIFICIAL NEURAL NETWORK; COMPUTER ANALYSIS; COST; INFORMATION PROCESSING; PRIORITY JOURNAL; PROBLEM SOLVING; REVIEW;

EID: 2142796735     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(01)00040-5     Document Type: Article

References (84)

1. An A., Cercone N. ELEM2: a learning system for more accurate classifications. Mercer R.E., Neufeld E. Lecture notes in artificial intelligence. vol. 1418:1998;426-441 Springer-Verlag, Berlin.
2. Baffes, P. T., & Zelle, J. M. (1992). Growing layers of perceptrons; Introducing the extentron algorithm. In International Joint Conference on Neural Networks (vol. 2, pp. 392-397). Baltimore.
3. Barkema G.T., Andree H.M.A., Taal A. The patch algorithm: fast design of binary feedforward neural networks. Network. 5:1993;393-407.
4. Bennett K.P., Bredensteiner E.J. Gorini C., Hart E., Meyer W., Phillips T. Geometry at work. 1998;Mathematical Association of America, Washington DC.
5. Biehl M., Opper M. Tilinglike learning in the parity machine. Physical Review A. 44:(10):1991;6888-6894.
6. Bishop C.M. Novelty detection and neural network validation. IEE Proceedings on Vision, Image and Signal Processing. 141:(4):1994;217-222.
7. Blake, C., Keogh, E., & Merz. C. (1998). UCI repository of machine learning databases.
8. Borsuk K. Multidimensional analytic geometry. 1969;Polish Scientific Publishers, Warsaw.
9. Brady, M., Raghavan, R., & Slawny, J. (1988). Gradient descent fails to separate. In Proceedings of the IEEE International Conference on Neural Networks (vol. 1, pp. 649-656).
10. Brady M.L., Raghavan R., Slawny J. Back propagation fails to separate where perceptrons succeed. IEEE Transactions on Circuits and Systems. 36:(5):1989;665-674.
11. Breiman L., Friedman J.H., Olsen R.A., Stone C.J. Classification and regression trees. 1984;Wadsworth International, Monterey, CA.
12. Broomhead D.S., Lowe D. Multivariable functional interpolation and adaptive networks. Complex Systems. 2:1988;321-323.
13. Buhot A., Gordon M.B. Storage capacity of a constructive learning algorithm. Journal of Physics A - Mathematical and General. 33:(9):2000;1713-1727.
14. Campbell C., Vicente C.P. The target switch algorithm: a constructive learning procedure for feed-forward neural networks [Letter]. Neural Computation. 7:(6):1995;1245-1264.
15. Clark P., Niblett T. The CN2 induction algorithm. Machine Learning. 3:1989;261-283.
16. Courrieu P. Three algorithms for estimating the domain of validity of feedforward neural networks. Neural Networks. 7:(1):1994;169-174.
17. Draghici, S. (1994). The constraint based decomposition training architecture. In Proceedings of World Congress on Neural Networks (vol. 3, pp. 545-555). San Diego, CA.
18. Draghici, S. (1995). Using constraints to improve generalization and training of feedforward neural networks: constraint based decomposition and complex backpropagation. PhD Thesis, University of St. Andrews.
19. Draghici, S. (1996). Enhancement of the constraint based decomposition training architecture. In Proceedings of IEEE International Conference on Neural Networks IJCNN '96 (vol. 1, pp. 317-322). New York: IEEE Press.
20. Draghici, S. (1998). On the complexity-capability relationship of adaptive systems based on VLSI friendly neural networks. In Lecture notes in artificial intelligence, Proceedings of XII Canadian Conference on Artificial Intelligence AI'98 (vol. 1418, pp. 285-297). Springer-Verlag.
21. Draghici, S., Beiu, V., & Sethi, I. (1997). A VLSI optimal constructive algorithm for classification problems. In Smart engineering systems: neural networks, fuzzy logic, data mining and evolutionary programming (pp. 141-151). New York: ASME Press.
22. Duong T.A., Daud T. Cascade error projection: a learning algorithm for hardware implementation. Mira J., Sánchez-Andrées J.V. Foundations and tools for neural modeling, Lecture Notes in Computer Science. 1999;450-457 Springer-Verlag, Berlin.
23. Eklund, P. W. (2000). Comparative study of public-domain supervised machine-learning accuracy on the UCI database. In Proceedings of the data mining and knowledge discovery: theory, tools and technology (pp. 39-50). Orlando, FL: SPIE.
24. Fahlman S.E. The recurrent cascade-correlation architecture. Lippmann R.P., Moody J.E., Touretzky D.S. Advances in neural information processing systems 3. 1991;190-196 Morgan Kaufmann, Denver, CO.
25. Fahlman, S. E., & Lebiere, C. (1990). The cascade-correlation learning architecture (Tech. Rep.). Carnegie Mellon University.
26. Fiesler, E. (1994). Comparative bibliography of ontogenetic neural networks. In Proceedings of the International Conference on Neural Networks (ICANN '94) (pp. 793-796). Sorrento, Italy.
27. Fiesler E., Beale R. Handbook of neural computation. 1996;Oxford University Press and The Inst. of Physics. Publ, New York.
28. Frattale-Mascioli F.M., Martinelli G. The oil-spot algorithm. IEEE Transactions on Neural Networks. 6:1995;794-797.
29. Frean M. The Upstart algorithm: a method for constructing and training feedforward neural networks. Neural Computation. 2:1990;198-209.
30. Gallant, S. I. (1986). Optimal linear discriminants. In Eighth International Conference on Pattern Recognition (pp. 849-852). IEEE.
31. Hammadi N., Ohmaneunda T., Kaneo K., Ito H. Dynamic constructive fault tolerant algorithm for feedforward neural networks. IEICE Transactions on Information and Systems. E81D:(1):1998;115-123.
32. Heath, S. K. D., & Salzberg, S. (1993). Induction of oblique decision tree. In IJCAI-93, Washington, DC.
33. Hosseini S., Jutten C. Weight freezing in constructive neural networks: a novel approach. Lecture notes in computer science: engineering applications of bio-inspired artificial neural networks. 1999;1607:10-1607:20 Springer-Verlag, Berlin.
34. Hwang J.-N., You S.-S., Lay S.-R., Jou I.-C. The Cascade-Correlation learning: a projection pursuit learning perspective. IEEE Transactions on Neural Networks. 7:(2):1996;278-289.
35. Höhfeld M., Fahlman S.E. Learning with limited numerical precision using the Cascade-Correlation algorithm. IEEE Transactions on Neural Networks. 3:(4):1992;602-611.
36. Knuth D.E. Big omicron and big omega and big theta. SIGACT News. 8:(2):1976;18-24.
37. Kohonen T. Learning vector quantization. Neural Networks. 1:(Suppl. 1):1988;303.
38. Kohonen T. Learning vector quantization. The handbook of brain theory and neural networks. 1995;537-540 MIT Press, Cambridge, MA.
39. Kwok T.-Y., Yeung D.-Y. Constructive algorithms for structuring learning in feedforward neural networks for regression problems. IEEE Transactions on Neural Networks. 8:(3):1997;630-645.
40. Kwok T.Y., Yeung D.Y. Objective functions for training new hidden units in constructive neural networks. IEEE Transactions on Neural Networks. 8:(5):1997;1131-1148.
41. Lang K.J., Witbrock M.J. Learninig to tell two spirals apart. Proceedings of the 1988 Connectionist Models Summer School. 1989;52-61 Morgan Kaufmann, San Mateo.
42. Lehtokangas M. Fast initialization for cascade-correlation learning [Brief paper]. IEEE-NN. 10:(2):1999;410.
43. Liang H., Dai G.L. Improvement of cascade correlation learning algorithm with an evolutionary initialization. Information Sciences. 112:(1-4):1998;1-6.
44. Marchand M., Golea M., Rújan P. A convergence theorem for sequential learning in two-layer perceptrons. Europhysics Letters. 11:(6):1990;487-492.
45. Mezard M., Nadal J.P. Learning in feedforward layered networks: the tiling algorithm. J. Phys. A: Math. Gen. 22:1989;2191-2203.
46. Mitchell T.M. Version spaces: a candidate elimination approach to rule learning. IJCAI. 1:1977;1139-1151.
47. Mitchell, T. M. (1978). Version spaces: an approach to concept learning. Unpublished doctoral dissertation, Stanford University.
48. Moody J., Darken C.J. Fast learning in networks of locally-tuned processing units. Neural Computation. 1:1989;281-294.
49. Murthy, K. V. S. (1995). On growing better decision trees from data. Unpublished doctoral dissertation, Johns Hopkins University.
50. Murthy, S. K., Kasif, S., Salzberg, S., & Beigel, R. (1993). OCI: randomized induction of oblique decision trees. In Proceedings of the Eleventh National Conference on Artificial Intelligence (pp. 322-327). Washington, DC.
51. Musavi M., Ahmed W., Chan K.H., Faris K., Hummels D. On the training of radial basis function classifiers. Neural Networks. 5:1992;595-603.
52. Muselli M. On sequential construction of binary neural networks. IEEE Transactions on Neural Networks. 6:1995;678-690.
53. Parekh, R. G., Yang, J., & Honavar, V. (1995). Constructive neural network learning algorithms for multi-category pattern classification (Postscript). Department of Computer Science, Iowa State University, Ames, Iowa: Tech. Rep. ISU-CS-TR 95-15.
54. Parekh R., Yang J., Honavar V. Constructive neural-network learning algorithms for pattern classification. IEEE Transactions on Neural Networks. 11:(2):2000;436.
55. Poggio T., Girosi F. Networks for approximation and learning. Proceedings of IEEE. 78:(9):1990;1481-1497.
56. Prechelt L. Investigation of the cascor family of learning algorithms. Neural Networks. 10:(5):1997;885-896.
57. Precup, D., & Utgoff, P. E. (1998). Classification using Φ-machines and constructive function approximation. In Proc. 15th International Conference on Machine Learning (pp. 439-444). San Francisco, CA.
58. Preparata F.P., Shamos M.I. Computational geometry - an introduction. 1985;Springer-Verlag, New York.
59. Quinlan J.R. C4.5: programs for machine learning. 1993;Morgan Kaufmann, San Mateo.
60. Rauber, T. W., Barata, M. M., & Steiger-Garcao, A. (1993). A tool-box for analysis and visualization of sensor data in supervision. In Proceedings of International Conference of Fault Diagnosis, Toulouse, France.
61. Romaniuk S.T., Hall L.O. Divide and conquer neural networks. Neural Networks. 6:1993;1105-1116.
62. Rújan P., Marchand M. Learning by minimizing resources in neural networks. Complex Systems. 3:1989;229-241.
63. Sethi I.K. Entropy nets: from decision trees to neural networks. Proceedings of the IEEE. 78:1990;1605-1613.
64. Sethi, I., & Otten, M. (1990). Comparison between entropy nets and decision tree classifiers. In International Joint Conference on Neural Networks (vol. 3, pp. 41-46). San Diego, CA.
65. Siestma J., Dow R.J.F. Creating artificial neural networks that generalise. Neural Networks. 4:1991;67-79.
66. Sirat J.A., Nadal J.P. The Upstart algorithm: a method for constructing and training feedforward neural networks. Networks: Computation in Neural Systems. 1:(4):1990;423-438.
67. Śmieja F.J. Neural network constructive algorithms: trading generalization for learning efficiency? Circuits Systems Signal Processing. 12:(2):1993;331-374.
68. Springer P.L., Gulati S. Parallelizing the cascade-correlation algorithm using time warp. Neural Networks. 8:(4):1995;571-577.
69. Tan A.-H. Cascade ARTMAP: integrating neural computation and symbolic knowledge processing. IEEE Transactions on Neural Networks. 8:(2):1997;237-250.
70. Tetko I.V., Villa A.E.P. An efficient partition of training data set improves speed and accuracy of cascade-correlation algorithm. Neural Processing Letters. 6:(1/2):1997;51-59.
71. Tetko I.V., Villa A.E.P. An enhancement of generalization ability in cascade correlation algorithm by avoidance of overfitting/overtraining problem. Neural Processing Letters. 6:(1/2):1997;43-50.
72. Thimm G., Fiesler E. Two neural network construction methods. Neural Processing Letters. 6:(1/2):1997;25-31.
73. Treadgold N.K., Gedeon T.D. Extending and bench-marking the CasPer algorithm. Sattar A. Proceedings of the 10th Australian Joint Conference on Artificial Intelligence: Advanced Topics in Artificial Intelligence (AI-97). vol. 1342:1997;398-406 Springer, Berlin.
74. Treadgold N.K., Gedeon T.D. Constructing higher order neurons of increasing complexity in cascade networks. Lecture Notes in Computer Science. 1416:1998;557-563.
75. Treadgold N.K., Gedeon T.D. Exploring constructive cascade networks. IEEE-NN. 10:(6):1999;1335.
76. Treadgold N.K., Gedeon T.D. A constructive cascade network with adaptive regularization. to appear as Lecture Notes in Computer Science Mira J., Sanchez-Andres J. Proceedings of IWANN'99. 1999b;40-49 Springer-Verlag, Alicante, Spain.
77. Utgoff P.E. Incremental induction of decision trees. Machine Learning. 4:1989;161-186.
78. Utgoff, P. E., & Brodley, C. E. (1991). Linear machine decision trees (Technical report no. UM-CS-1991-010), University of Massachusetts, Amherst, Computer Science.
79. Utgoff, P. E., & Precup, D. (1997). Constructuve function approximation (Technical report no. 97-04). Amherst, MA: Department of Computer Science, University of Massachusetts.
80. Utgoff P.E., Precup D. Constructive function approximation. Liu H., Motoda H. Feature extraction, construction and selection: a data mining perspective. vol. 453:1998;Kluwer Academic, Norwell, MA.
81. Utgoff P.E., Berkman N.C., Clouse J.A. Decision tree induction based on efficient tree restructuring. Machine Learning. 29:1997;5-44.
82. Yang, J., Parekh, R., & Honavar, V. (1998). DistAI: an interpattern distance-based constructive learning algorithm. In World Congress on Computational Intelligence (pp. 2208-2213). Anchorage, Alaska.
83. Yang J., Parekh R., Honavar V., Dobbs D. Data-driven theory refinement using KBDistAI. Lecture Notes in Computer Science. 1642:1999;331-342.
84. Young S., Downs T. CARVE - a constructive algorithm for real-valued examples. IEEE Transactions on Neural Networks. 9:(6):1998;1180-1190.