Neural network studies. 2. Variable selection

Journal of Chemical Information and Computer Sciences

Volumn 36, Issue 4, 1996, Pages 794-803

  Tetko, Igor V ^a,b   Villa, Alessandro E P ^b   Livingstone, David J ^c,d  

^a INSTITUTE OF BIOORGANIC CHEMISTRY AND PETROCHEMISTRY   (Ukraine)

^b UNIVERSITY OF LAUSANNE   (Switzerland)

^c ChemQuest   (United Kingdom)

^d UNIVERSITY OF PORTSMOUTH   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

DRUG;

ARTICLE; ARTIFICIAL NEURAL NETWORK; CHEMISTRY; FACTUAL DATABASE; NONLINEAR SYSTEM; STATISTICAL MODEL; STRUCTURE ACTIVITY RELATION;

DATABASES, FACTUAL; LINEAR MODELS; NEURAL NETWORKS (COMPUTER); NONLINEAR DYNAMICS; PHARMACEUTICAL PREPARATIONS; STRUCTURE-ACTIVITY RELATIONSHIP;

EID: 0030188379     PISSN: 00952338     EISSN: None     Source Type: Journal    
DOI: 10.1021/ci950204c     Document Type: Article

References (48)

1. Hyde, R. M.; Livingstone, D. J. Perspectives in QSAR: Computational Chemistry and Pattern Recognition J. Comp.-Aided Mol. Design 1988, 2, 145-155.
2. Aivazyan, S. A.; Buchstaber, V. M.; Yenyukov, I. S.; Meshalkin, L. D. Applied Statistics. Classification and Reduction of Dimensionality. Finansy i statistika: Moscow, 1989.
3. Vapnik, V. Estimation of Dependencies Based on Empirical Data. Springer-Verlag: New York, 1982.
4. Kolmogorov, A. N. On the Representations of Continuous Functions of Many Variables by Superposition of Continuous Functions of One Variable and Addition. Dokl. Adad. Nauk USSR 1957, 114, 953-956.
5. Hecht-Nielsen, R. Kolmogorov's Mapping Neural Network Existence Theorem. Proceedings of the International Conference on Neural Networks; IEEE Press: New York, 1987; pp 11-14.
6. Manallack, D. T.; Livingstone, D. J. Artificial Neural Networks: Application and Chance Effects for QSAR Data Analysis. Med. Chem. Res. 1992, 2, 181-190.
7. Livingstone, D. J. and Manallack, D. T. Statistics Using Neural Networks: Chance Effects. J. Med. Chem. 1993, 36. 1295-1297.
8. Tetko, I. V.; Livingstone, D. J.; Luik, A. I. Neural Network Studies. 1. Comparison of Overfitting and Overtraining J. Chem. Inf. Comput. Sci. 1995, 35, 826-833.
9. Rogers, D.; Hopfinger, A. J. Application of Genetic Function Approximation lo Quantitative Structure-Activity Relationships and Quantitalive Structure-Property Relationships. J. Chem. Inf. Comput. Sci. 1994, 34, 854-856.
10. Ambati, B. K.; Ambati, J.; Mokhtar, M. M. Heuristic Combinatorial Optimisation by Simulated Darwinian Evolution: A Polynomial Time Algorithm for the Travelling Salesman Problem. Biological Cybernetics 1991, 65, 31-35.
11. Fogel, D. B. Empirical Estimation of the Computation Required to Discover Approximate Solutions to the Travelling Salesman Problem Using Evolutionary Programming. Proc. of 2nd Ann. Conf, on Evolutionary Programming, Fogel, D. B., Atmar, W. Eds.; Evol. Prog. Soc.: La Jolla, CA, 1993; pp 56-61.
12. Kubinyi, H. Variable Selection in QSAR Studies. I. An Evolutionary Algorithm. Quant. Struct.-Act. Relat., 1994, 13, 285-294.
13. Tetko, I. V.; Tanchuk, V. Yu.; Luik, A. I. Application of an Evolutionary Algorithm to the Structure-Activity Relationship. Proc. of 3rd Ann. Conf. on Evol. Program. Sebald, A. V., Fogel, L. J., Eds.; World Scientific: River Edge, NJ, 1994; pp 109-119.
14. Wikel, J. H.; Dow, E. R. The Use of Neural Networks for Variable Selection in QSAR. Bioorg. Med. Chem. Lett. 1993, 3, 645-651.
15. One epoch corresponds to the presentation of all input patterns to the ANN followed by updating of the net weights (batch training).
16. Manallack, D. T.; Livingstone, D. J. Neural Networks - A Tool for Drug Design. In: Advanced Computer-Assisted techniques in Drug Design; Van de Waterbeemd, H., Ed.; VCH Weinheim, 1994; pp 293-319.
17. Hansen, L. K.; Salamon, P. Neural Networks Ensembles. IEEE Trans. Pattern Anal. Machine Intell. 1990, 12, 993-1001.
18. Perrone, M. P. General Averaging Results for Convex Optimisation. In Proceedings of the 1993 Connectionist Models Summer School; Erlbaum Associates: Hillsdale, NJ, 1994; pp 364-371.
19. Tetko, I. V.; Luik, A. I.; Poda, G. I. Applications of Neural Networks in Structure - Activity Relationships of a Small Number of Molecules. J. Med. Chem. 1993, 36, 811-814.
20. Tetko, I. V. Application of Neural Networks in Structure - Activity Relationship Studies. PhD Dissertation. Institute of Bioorganic & Petroleum Chemistry: Kiev; 1994.
21. i, is a target value, and summation is over all patterns in the analyzed data set.
22. Thodberg, H. H. A Review of Bayesian Neural Networks with an Application to Near Infrared Spectroscopy. IEEE Trans. Neural Networks, in press.
23. Borggaard, C.; Thodberg, H. H. Optimal Minimal Neural Interpretation of Spectra. Anal. Chem. 1992, 64, 545-551.
24. LeChun, Y.; Denker, J. S.; Solla, S. A. Optimal Brain Damage. In: Advances in Neural Processing Systems 2 (NIPS*2): Touretzky, D. S., Ed.; Morgan-Kaufmann: 1990, pp 598-605.
25. Hassibi, B.; Stork, D. Second Order Derivatives for Network Pruning: Optimal Brain Surgeon. In Advances in Neural Processing Systems 5 (NIPS*5); Hanson, S. Cowan, J., Giles, C., Eds.; Morgan-Kaufmann Publishers: San Mateo, CA. 1993; pp 164-171.
26. Hansen, L. K.; Rasmussen, C. E. Pruning from Adaptive Regularization. Neural. Comp. 1994, 6, 1223-1233.
27. Chauvin, Y. A. Back-Propagation Algorithm with Optimal Use of Hidden Units. In Advances In Neural Processing Systems 1 (NIPS*1); Touretzky, D. S., Ed.; Morgan-Kaufmann: 1989; pp 519-526.
28. Weigen, S. A.; Rumelhart, D. E.; Huberman, B. A. Generalization by Weight-Elimination with Application to forecasting. In Advances in Neural Processing Systems 3 (NIPS*3); Lippann, R., Moody, J., Touretzek, D. S., Eds.; Morgan-Kaufmann: 1991; pp 875-882.
29. Reed, R. Pruning Algorithms - a Survey. IEEE Trans. Neural Networks. 1993, 4, 740-747.
30. Sietsma, J.; Dow, R. Creating Artifical Neural Networks that Generalize. Neural Networks 1991, 4, 67-79.
31. Kruschke, J. K.; Movellan, J. R. Benefits of Gain: Speeded Learning and Minimal Hidden Layers in Back-Propagation Networks. IEEE Trans. Syst. Man Cybern. 1991, 21, 273-280.
32. Tetko, I. V.; Tanchuk, V. Yu.; Luik, A. I. Simple Heuristic Methods for Input Parameters' Estimation in Neural Networks. Proceedings of the 1994 IEEE (WCCI) International Conference on Neural Networks; IEEE Press: 1994; Vol. 1, pp 376-381.
33. Tetko, I. V.; Tanchuk, V. Yu.; Chentsova, N. P.; Antonenko, S. V.; Poda, G. I.; Kukhar, V. P.; Luik, A. I. HIV-1 Reverse Transcriptase Inhibitor Design Using Artificial Neural Networks. J. Med. Chem. 1994, 37, 2520-2526.
34. Nagai, T.; Yamamoto, Y.; Katayama, H.; Adachi, M.; Aihara, K. A Nove; Method to Analyse Response Patterns of Taste Neurons by Artifical Neural Networks. NeuroReport 1992, 3, 745-748.
35. Mozer, M.; Smolensky, P. Skeletonization: a Technique for Trimming the Fat from a Network via Relevance Assessment. In Advances in Neural Processing Systems 1 (NIPS*1); Touretzky, D. S., Ed.; Morgan-kaufmann Publishers: San Mateo, CA, 1989; pp 107-115.
36. Karnin, E. D. A simple Procedure for Pruning Back-Propagation Trained Neutral Networks. IEEE Trans. Neural Networks 1990, 1, 239-242.
37. For example: Jonhos, N. N.; Leone, F. G. Statistic and Experimental Design in Engineering and the Physical Sciences; John Wiley & Sons: New York, 1977.
38. 2 was recommended: Cramer III. R. D.; DePriest, S. A.; Patterson, D. E.; Hecht, P. The Developing Practice of Comparative Field Analysis, In 3D QSAR in Drug Design: Theory Methods and Applications; Kubinyi, H., Ed.; ESCOM: The Netherlands, 1993; pp 443-486.
39. 2 was recommended: Cramer III. R. D.; DePriest, S. A.; Patterson, D. E.; Hecht, P. The Developing Practice of Comparative Field Analysis, In 3D QSAR in Drug Design: Theory Methods and Applications; Kubinyi, H., Ed.; ESCOM: The Netherlands, 1993; pp 443-486.
40. The training procedure works with only half the number of cases of the learning data set (25 in this example) for network training, while the other cases are used only to monitor network training. See Figure 3 and more details in ref 8.
41. Nakao, H.; Arakawa, M.; Nakamura, T.; Fukushima, M. Antileukemic Agents. II. New 2,5-bis(1-aziridinyl)-p-benzoquinone Derivatives. Chem. Pharm. Bull. 1972, 20, 1968-1979.
42. Liu, Q.; Hirono, S.; Moriguchi, I. Comparison of Functional-Link Net and the Generalised Delta Rule Net in Quantitative Structure-Activity Relationship Studies. Chem. Pharm. Bull. 1992, 40, 2962-2969.
43. Li, R. L.; Hansch, C.; Kaufman, B. T. A Comparison of the Inhibitory Action of 5-(substituted-benzyl)-2,4-diaminopyrimidines on Dihydrofolate Reductase from Chicken Liver with That from Bovine Liver. J. Med. Chem. 1982, 25, 435-440.
44. Hirst, J. D.; King, R. D.; Sternberg, M. J. E. Quantative Structure - Activity Relationships by Neural Networks and Inductive Logic Programming. 1. The Inhibition of Dihydrofolate Reductase by Pyrimidines. J. Comput.-Aided Mol Design. 1994, 8, 405-420.
45. Goodford, P. J. A. Computational Procedure for Determining Energetically Favorale Binding Sites on Biologically Important Macromolecules. J. Med. Chem. 1985, 28, 849-857.
46. Bodor, N.; Harget, A.; Huang, M.-J. Neural Network Studies. 1. Estimation of the Aqueous Solubility of Organic Compounds J. Am. Chem. Soc. 1991, 113, 9480-9483.
47. Brill, F. Z.; Brown, D. E.; Martin, W. V. Fast Genetic Selection of Features for Neural Network Classfier. IEEE Trans Neural Networks 1992, 3, 324-328.
48. Manallack, D. T.; Livingstone, D. J. Limitations of Functional-Link Nets as Applied to QSAR Data Analysis. Quant. Struct.-Act. Relat. 1994, 13, 18-21.