Kernel Adaptive Filtering: A Comprehensive Introduction

Kernel Adaptive Filtering: A Comprehensive Introduction

Volumn , Issue , 2010, Pages 1-209

  Liu, Weifeng ^a   Príncipe, José C ^a   Haykin, Simon ^a  

^a NONE

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84888754031     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/9780470608593     Document Type: Book

References (204)

1. T. Adali , X. Liu , and M. K. Sonmez Conditional distribution learning with neural networks and its application to channel equalization IEEE Transactions on Signal Processing , 45 : 1051 - 1064 , Apr. 1997.
2. M. Aizerman , E. Braverman , and L. Rozonoer Theoretical foundations of the potential function method in pattern recognition learning Automation and Remote Control , 25 : 821 - 837 , 1964.
3. H. Akaike A new look at the statistical model identifi cation IEEE Transactions on Automatic Control , 19 : 716 - 723 , 1974.
4. D. L. Alspach and H. W. Sorenson Nonlinear Baysian estimation using Gaussian sum approximations IEEE Transactions on Automatic Control , 17 ( 4 ): 439 - 448 , 1972.
5. S. An , W. Liu , and S. Venkatesh Fast cross-validation algorithms for least squares support vector machine and kernel ridge regression Pattern Recognition , 40 : 2154 - 2162 , 2007.
6. I. Arasaratnam and S. Haykin Cubature Kalman fi lters IEEE Transactions on Automatic Control , 54 ( 6 ): 1254 - 1269 , 2009.
7. A. Argyriou , C. A. Micchelli , and M. Pontil Learning convex combinations of continuously parameterized basic kernels In Proceedings of the 18th Conference on Learning Theory , pages 338 - 352 , 2005.
8. N. Aronszajn Theory of reproducing kernels Transactions of the American Mathematical Society , 68 : 337 - 404 , 1950.
9. J. F. Barrett The use of functionals in the analysis of non-linear physical systems. International Journal of Electronics , 15 : 567 - 615 , 1963.
10. S. Benedetto and E. Biglieri Nonlinear equalization of digital satellite channels IEEE Journal on Selected Areas in Communications , 1 : 57 - 62 , Jan. 1983.
11. A. Beuter , L. Glass , M. Mackey , and M. S. Titcombe Nonlinear Dynamics in Physiology and Medicine New York : Springer , 2003.
12. S. Billings and S. Fakhouri Identifi cation of systems containing linear dynamics and static nonlinear elements Automatica , 18 : 15 - 26 , 1982.
13. S. A. Billings and C. F. Fung Recurrent radial basis function networks for adaptive noise cancellation Neural Networks , 8 : 273 - 290 , June 1995.
14. A. Bordes , S. Ertekin , J. Weston , and L. Bottou Fast kernel classifi ers with online and active learning Journal of Machine Learning Research , 6 : 1579 - 1619 , 2005.
15. B. E. Boser , I. M. Guyon , and V. N. Vapnik A training algorithm for optimal margin classifi ers In Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory , pages 144 - 152 , 1992.
16. L. Bottou Large-scale machine learning and stochastic algorithms , 2008 Tutorial on NIPS 2008.
17. P. Brazdil Learning in multi-agent environments In Proceedings of the European Working Session on Learning , pages 598 - 605 , 1991.
18. D. Broomhead and D. Lowe Multivariable functional interpolation and adaptive networks Complex Systems , 2 : 321 - 355 , 1988.
19. C. J. C. Burges A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery , 2 ( 2 ): 121 - 167 , 1998.
20. C. Campbell , N. Cristianini , and A. Smola Query learning with large margin classifi ers. In Proceedings of the 17th International Conference on Machine Learning , pages 111 - 118 , 2000.
21. G. A. Carpenter and S. Grossberg A massively parallel architecture for a self-organizing neural pattern recognition machine Computer Vision, Graphics and Image Processing , 37 : 54 - 115 , 1987.
22. G. Cavallanti , N. Cesa-Bianchi , and C. Gentile Tracking the best hyperplane with a simple budget perceptron Machine Learning , 69 : 143 - 167 , 2007.
23. G. C. Cawley and N. L. C. Talbot Effi cient leave-one-out cross-validation of kernel Fisher discriminant classifi ers Patten Recognition , 36 : 2585 - 2592 , 2003.
24. I. Cha and S. Kassam Channel equalization using adaptive complex radial basis function networks IEEE Journal on Selected Areas in Communications , 13 : 122 - 131 , Jan. 1995.
25. O. Chapelle , V. Vapnik , O. Bousquet , and S. Mukherjee Choosing multiple parameters for support vector machines Machine Learning , 46 : 131 - 159 , 2002.
26. S. Chen , B. Mulgrew , and P. M. Grant A clustering technique for digital communications channel equalization using radial basis function networks IEEE Transactions on Neural Networks , 4 : 570 - 590 , July 1993a.
27. S. Chen , B. Mulgrew , and S. McLaughlin Adaptive Bayesian equalizer with decision feedback IEEE Transactions on Signal Processing , 41 : 2918 - 2927 , Sept. 1993b.
28. Y.-H. Cheng and C.-S. Lin A learning algorithm for radial basis function networks: with the capability of adding and pruning neurons In Proceedings of IEEE International Conference on Neural Networks 1994 , pages 797 - 801 , 1994.
29. V. Cherkassky and F. Mulier Learning from Data: Concepts, Theory, and Methods. New York : Wiley-Interscience , 1998.
30. J. A. Cherry Distortion Analysis of Weakly Nonlinear Filters Using Volterra Series. Master's Thesis, Carleton University, 1994.
31. M. Coker and D. Simkins A nonlinear adaptive noise canceller In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing 1980 , volume 5, pages 470 - 473 , 1980.
32. T. M. Cover and P. E. Hart Nearest neighbor pattern classifi cation IEEE Transactions on Information Theory , 13 : 21 - 27 , 1967.
33. T. M. Cover and J. A. Thomas Elements of Information Theory New York : Wiley-Interscience , 1991.
34. N. Cristianini and J. Shawe-Taylor An Introduction to Support Vector Machines and Other Kernel-based Learning Methods Cambridge, MA : Cambridge University Press , 2000.
35. R. S. Crowder Predicting the Mackey-Glass time series with cascade-correlation learning In Proceedings of 1990 Connectionist Models Summer School , 1990.
36. L. Csato and M. Opper Sparse online Gaussian processes Neural Computation , 14 : 641 - 668 , 2002.
37. O. Dekel , S. Shalev-Shwartz , and Y. Singer The Forgetron: A kernel-based perceptron on a fi xed budget In Advances in Neural Information Processing Systems 18 , pages 1342 - 1372 , Cambridge, MA : MIT Press , 2006.
38. A. Dempster , N. Laird , and D. Rubin Maximum likelihood from incomplete data via the EM algorithm Journal of the Royal Statistical Society, Series B , 39 ( 1 ): 1 - 38 , 1977.
39. C. Dima and M. Hebert Active learning for outdoor obstacle detection In Proceedings of Science and Systems I , Aug. 2005.
40. A. Doucet , C. Andrieu , and S. Godsill On sequential Monte Carlo sampling methods for Bayesian fi ltering Statistics and Computing , 10 ( 3 ): 197 - 208 , 2000a.
41. A. Doucet , J. F. G. de Freitas , and N. J. Gordon Sequential Monte Carlo Methods in Practice , 3rd edition New York : Springer-Verlag , 2000b.
42. R. O. Duda , P. E. Hart , and D. G. Stork Pattern Classifi cation New York : Wiley, Interscience , 2000.
43. M. A. El-Gamal The role of priors in active Bayesian learning in the sequential statistical decision framework In J. W. T. Grandy and L. H. Schick , editors, Maximum Entropy and Bayesian Methods , pages 33 - 38 Reading, MA : Kluwer , 1991.
44. Y. Engel , S. Mannor , and R. Meir The kernel recursive least-squares algorithm IEEE Transactions on Signal Processing , 52 ( 8 ): 2275 - 2285 , 2004.
45. H. W. Engl , M. Hanke , and A. Neubauer Regularization of Inverse Problems. New York : Springer , 2000.
46. M. J. Er , Z. Li , H. Cai , and Q. Chen Adaptive noise cancellation using enhanced dynamic fuzzy neural networks IEEE Transactions on Fuzzy Systems , 13 : 331 - 342 , June 2005.
47. D. Erdogmus , D. Rende , J. C. Principe , and T. F. Wong Nonlinear channel equalization using multilayer perceptrons with information-theoretic criterion In Proceedings of IEEE Workshop on Neural Networks for Signal Processing 2001 , pages 443 - 451 , 2001.
48. T. Evgeniou , M. Pontil , and T. Poggio Regularization networks and support vector machines In Advances in Computational Mathematics , pages 1 - 50 Cambridge, MA : MIT Press , 2000.
49. S. E. Fahlman and C. Lebiere The cascade-correlation learning architecture In Advances in Neural Information Processing Systems 2 , pages 524 - 532 New York : Morgan Kaufmann , 1990.
50. J. D. Farmer Chaotic attractors of an infi nite-dimensional dynamical system Physica D , 4 ( 3 ), 1982.
51. J. D. Farmer and J. J. Sidorowich Predicting chaotic time series Physical Review Letter , 59 ( 8 ): 845 - 848 , 1987.
52. V. V. Fedorov Theory of Optimal Experiment New York : Academic Press , 1972.
53. S. Fine and K. Scheinberg Effi cient SVM training using low-rank kernel representations Journal of Machine Learning Research , 2 : 242 - 264 , 2001.
54. M. O. Franz and B. Schölkopf A unifying view of Wiener and Volterra theory and polynomial kernel regression Neural Computation , 18 : 3097 - 3118 , 2006.
55. T.-T. Frieb and R. F. Harrison A kernel-based ADALINE In Proceedings European Symposium on Artifi cial Neural Networks 1999 , pages 245 - 250 , Apr. 1999.
56. K. Fukumizu Active learning in multilayer perceptrons In D. S. Touretzky , M. C. Mozer , and M. E. Hasselmo , editors, Advances in Neural Information Processing Systems , volume 8 , pages 295 - 301 Cambridge, MA : MIT Press , 1996.
57. D. Gabor Holographic model of temporal recall Nature , 217 : 584 - 585 , 1968.
58. D. Gabor , W. P. L. Wilby , and R. Woodcock A universal non-linear fi lter, predictor, and simulator which optimizes itself by a learning process In Proceedings of the Institution of Electrical Engineers , volume 108, pages 422 - 435 , 1960.
59. S. L. Gay and S. Tavathia The fast affi ne projection algorithm In Proceedings of the International Conference on Accoustics, Speech and Signal Processing 1995 , volume 5, pages 3023 - 3026 , 1995.
60. A. Gersho Adaptive equalization of highly dispersive channels for data transmission. Bell System Technical Journal , 48 : 55 - 70 , 1969.
61. C. Giraud-Carrier A note on the utility of incremental learning AI Communications , 13 ( 4 ): 215 - 233 , 2000.
62. F. Girosi , M. Jones , and T. Poggio Regularization theory and neural networks architectures Neural Computation , 7 : 219 - 269 , 1995.
63. T. Glasmachers Second-order SMO improves SVM online and active learning Neural Computation , 20 : 374 - 382 , 2008.
64. L. Glass and M. Mackey From Clocks to Chaos: The Rhythms of Life Princeton, NJ : Princeton University Press , 1988.
65. G. Golub and C. Loan Matrix Computations Washington, DC : John Hopkins University Press , 1996.
66. G. Goodwin and K. Sin Adaptive Filtering: Prediction and Control Upper Saddle River, NJ : Prentice Hall , 1984.
67. N. J. Gordon , D. J. Salmond , and A. F. M. Smith Novel approach to nonlinear/non-Gaussian Bayesian state estimation IEE Proceedings F: Radar and Signal Processing , 140 ( 5 ): 107 - 113 , 1993.
68. S. Grossberg Competitive learning: From interactive activation to adaptive resonance. Cognitive Science , 11 : 23 - 63 , 1987.
69. P. D. Grünwald The Minimum Description Length Principle Cambridge, MA : MIT Press , 2007.
70. K. Hagiwara and K. Kuno Regularization learning and early stopping in linear networks In Proceedings of International Joint Conference on Neural Networks 2000 , volume 4, pages 511 - 516 , 2000.
71. W. Härdle Applied Nonparametric Regression Cambridge, UK : Cambridge University Press , 1992.
72. B. Hassibi and G. Stork Second order derivatives for network pruning: Optimal brain surgeon In Neural Information Processing Systems , pages 164 - 171 , 1992.
73. S. Haykin Neural Networks: A Comprehensive Foundation , 1st edition Upper Saddle River, NJ : Prentice Hall , 1994.
74. S. Haykin Adaptive Filter Theory , 3rd edition Upper Saddle River, NJ : Prentice Hall , 1996.
75. S. Haykin Neural Networks: A Comprehensive Foundation , 2nd edition Upper Saddle River, NJ : Prentice Hall , 1998.
76. S. Haykin Adaptive Filter Theory , 4th edition Upper Saddle River, NJ : Prentice Hall , 2002.
77. S. Haykin Neural Networks and Learning Machines , 3rd edition Upper Saddle River , NJ : Prentice Hall , 2009.
78. S. Haykin , A. H. Sayed , J. R. Zeidler , P. Yee , and P. C. Wei Adaptive tracking of linear time-variant systems by extended RLS algorithms IEEE Transactions on Signal Processing , 45 : 1118 - 1128 , 1997.
79. R. Herbrich Learning Kernel Classifi ers: Theory and Algorithms Cambridge, MA : MIT Press , 2002.
80. E. Herrmann Local bandwidth choice in kernel regression estimation Journal of Computational and Graphical Statistics , 6 ( 1 ): 35 - 54 , 1997.
81. G. E. Hinton and T. J. Sejnowski Learning and relearning in Boltzmann machines. In Parallel Distributed Processing , volume 1 , pages 282 - 317 Cambridge, MA : MIT Press , 1986.
82. A. E. Hoerl and R. W. Kennard Ridge regression: Biased estimation for nonorthogonal problems Technometrics , 12 : 55 - 67 , 1970.
83. G. Huang , P. Saratchandran , and N. Sundararajan A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation IEEE Transactions on Neural Networks , 16 : 57 - 67 , 2005.
84. K. Ito and K. Xiong Gaussian fi lters for nonlinear fi ltering problems IEEE Transactions on Automatic Control , 45 ( 5 ): 910 - 927 , 2000.
85. L. Itti and P. Baldi Bayesian surprise attracts human attention In Advances in Neural Information Processing Systems, Vol. 19 , pages 1 - 8 Cambridge, MA : MIT Press , 2006.
86. A. G. Journel and C. J. Huijbregts Mining Geostatistics New York : Academic Press , 1978.
87. S. Julier and J. Uhlmann A new extension of the Kalman fi lter to nonlinear systems. In International Symposium of the Aerospace/Defense Sensing, Simul. and Controls , 1997.
88. R. E. Kalman A new approach to linear fi ltering and prediction problems Transactions of the ASME - Journal of Basic Engineering , 82 : 35 - 45 , 1960.
89. R. E. Kalman and R. S. Bucy New results in linear fi ltering and prediction theory Transactions of the ASME - Journal of Basic Engineering , 83 : 95 - 107 , 1961.
90. N. B. Karayiannis and G. W. Mi Growing radial basis neural networks: Merging supervised and unsupervised learning with network growth techniques IEEE Transactions on Neural Networks , 8 : 1492 - 1506 , 1997.
91. G. Kechriotis , E. Zarvas , and E. S. Manolakos Using recurrent neural networks for adaptive communication channel equalization IEEE Transactions on Neural Networks , 5 : 267 - 278 , March 1994.
92. J. L. Kelly and R. F. Logan Self-adaptive echo canceller. U.S. Patent 3,500,000 , 1970.
93. J. Kivinen , A. Smola , and R. C. Williamson Online learning with kernels IEEE Transactions on Signal Processing , 52 ( 8 ): 2165 - 2176 , Aug. 2004.
94. A. N. Kolmogorov Interpolation und extrapolation von station ä ren zuf ä ligen folgen. Izv. Akad. Nauk SSSR , 5 : 3 - 14 , 1941.
95. K. J. Lang and G. E. Hinton The development of the time-delay neural network architecture for speech recognition Tech. report CMU-CS-88-152 , Carnegie-Mellon University , Pittsburgh, PA , 1988.
96. S. L. Lauritzen Time series analysis in 1880: A discussion of contributions made by T. N. Thiele International Statistical Review , 49 : 319 - 333 , 1981.
97. Y. LeCun , J. S. Denker , and S. A. Solla Optimal brain damage In Advances in Neural Information Processing Systems 2 , pages 598 - 605 , New York : Morgan Kaufmann , 1990.
98. T. C. Lee , A. M. Peterson , and J. C. Tsai Amulti-layer feed-forward neural network with dynamically adjustable structures In Proceedings of IEEE International Conference on Systems, Man and Cybernetics , pages 367 - 369 , 1990.
99. T. Li and J. Jiang Adaptive Volterra fi lters for active control of nonlinear noise processes IEEE Transactions on Signal Processing , 49 : 1667 - 1676 , Aug. 2001.
100. W. A. Light Some aspects of radial basis functions approximation In S. P. Singh , editor, Approximation Theory, Spline Functions and Applications , pages 163 - 190. Reading, MA : Kluwer , 1992.
101. M. Lindenbaum Selective sampling for nearest neighbor classifi ers In Proceedings of the Sixteenth National Conference on Artifi cial Intelligence and the Eleventh Innovative Applications of Artifi cial Intelligence Conference Innovative Applications of Artifi cial Intelligence , pages 366 - 371 , 1999.
102. D. V. Lindley On the measure of information provided by an experiment The Annals of Mathematical Statistics , 27 ( 4 ): 986 - 1005 , 1956.
103. W. Liu and J. C. Príncipe Extended recursive least squares algorithm in RKHS In 1st IAPR Workshop on Cognitive Information Processing , 2008a.
104. W. Liu and J. C. Príncipe The kernel affi ne projection algorithms EURASIP Journal on Advances in Signal Processing , 2008 , 2008b. URL http://www.hindawi.com/ GetArticle.aspx?doi=10.1155/2008/784292.
105. W. Liu , P. Pokharel , and J. C. Príncipe The kernel least mean square algorithm IEEE Transactions on Signal Processing , 56 ( 2 ): 543 - 554 , 2008.
106. W. Liu , I. Park , Y. Wang , and J. C. Príncipe Extended kernel recursive least squares algorithm IEEE Transactions on Signal Processing , 57 ( 10 ) , October 2009.
107. E. N. Lorenz Deterministic nonperiodic fl ow Journal of the Atmospheric Sciences , 20 : 130 - 141 , 1963.
108. R. W. Lucky Automatic equalization for digital communication Bell System Technical Journal , 44 : 547 - 588 , 1965.
109. D. MacKay Information-based objective functions for active data selection Neural Computation , 4 ( 4 ): 590 - 604 , 1992a.
110. D. MacKay Bayesian interpolation Neural Computation , 4 : 415 - 447 , 1992b.
111. M. C. Mackey and L. Glass Pathological physiological conditions resulting from instabilities in physiological control system Science , 197 ( 4300 ): 287 - 289 , 1977.
112. V. Z. Marmarelis Identifi cation of nonlinear biological systems using Laguerre expansions of kernels Annals of Biomedical Engineering , 21 : 573 - 589 , 1993.
113. T. Martinetz and K. Schulten A " Neural-Gas " network learns topologies Artifi cial Neural Networks , I : 397 - 402 , 1991.
114. T. M. Martinetz , S. G. Berkovich , and K. J. Schulten " Neural-Gas " network for vector quantization and its application to time-series prediction IEEE Transactions on Neural Networks , 4 ( 4 ): 558 - 569 , 1993.
115. G. Matheron The intrinsic random functions and their applications Advances in Applied Probability , 5 : 439 - 468 , 1973.
116. J. M. McCool , B. Widrow , J. R. Zeidler , R. H. Hearn , and D. M. Chabries Adaptive line enhancer. U.S. Patent 4,238,746 , 1980.
117. C. Micchelli and M. Pontil Learning the kernel function via regularization Journal of Machine Learning Research , 6 : 1099 - 1125 , 2005.
118. C. A. Micchelli Interpolation of scattered data: distance matrices and conditionally positive defi nite functions Constructive Approximation , 2 : 11 - 22 , 1986.
119. S. Mika , G. Ratsch , J. Weston , B. Schölkopf , and K. R. M ü ller Fisher discriminant analysis with kernels In Proceedings of IEEE Neural Networks for Signal Processing Workshop 1999 , pages 8 - 10 , 1999.
120. M. Minsky and S. Papert Perceptrons: An Introduction to Computational Geometry. Cambridge, MA : MIT Press , 1969.
121. J. E. Moody and T. S. Rögnvaldsson Smoothing regularizers for projective basis function networks In Advances in Neural Information Processing Systems 9 , pages 585 - 591 Cambridge, MA : MIT Press , 1997.
122. S. Mukherjee , E. Osuna , and F. Girosi Nonlinear prediction of chaotic time series using support vector machines In J. Príncipe , L. Giles , N. Morgan , and E. Wilson , editors, IEEE Workshop on Neural Networks for Signal Processing VII , pages 511 - 519 , 1997.
123. K. R. M ü ller , A. J. Smola , G. R ä tsch , B. Schölkopf , J. Kohlmorgen , and V. Vapnik. Predicting time series with support vector machines In Proceedings of ICANN 1997 , pages 999 - 1004 , 1997.
124. A. O'Hagan Curve fi tting and optimal design for prediction Journal of the Royal Statistical Society B , 40 : 1C42 , 1978.
125. K. Ozeki and T. Umeda An adaptive fi ltering algorithm using an orthogonal projection to an affi ne subspace and its properties Electronics and Communications in Japan , 67-A ( 5 ), 1984.
126. G. Palm Evidence, information, and surprise Biological Cybernetics , 42 ( 1 ): 57 - 68 , 1981.
127. E. Parzen Statistical methods on time series by Hilbert space methods Technical Report 23 , Stanford University , CA , 1959.
128. R. L. Plackett Some theorems in least squares Biometrika , 37 : 149 - 157 , 1950.
129. J. Platt A resource-allocating network for function interpolation Neural Computation , 3 ( 2 ): 213 - 225 , 1991.
130. T. Poggio and F. Girosi Networks for approximation and learning Proceedings of IEEE , 78 ( 9 ): 1481 - 1497 , 1990.
131. T. Poggio and S. Smale The mathematics of learning: Dealing with data Transactions of the American Mathematical Society , 50 : 537 - 544 , Nov. 2003.
132. P. Pokharel , W. Liu , and J. C. Príncipe Kernel LMS In Proceedings of the International Conference on Accoustics, Speech and Signal Processing 2007 , volume 3, pages 1421 - 1424 , 2007.
133. P. Pokharel , W. Liu , and J. C. Príncipe Kernel least mean square algorithm with constrained growth Signal Processing , 89 : 257 - 265 , 2009.
134. M. J. D. Powell Radial basis functions for multivariable interpolation: A review In Clarendon Press Institute Of Mathematics And Its Applications Conference Series , pages 143 - 167 , 1985.
135. J. C. Príncipe , B. de Vries , J. M. Kuo , and P. G. de Oliveira Modeling applications with the focused gamma net In Advances in Neural Information Processing Systems , volume 4 , pages 143 - 150 , 1992.
136. J. Proakis Digital Communications New York : McGraw-Hill , 2000.
137. J. Proakis and J. Miller An adaptive receiver for digital signaling through channels with intersymbol interference IEEE Transactions on Information Theory , 15 : 484 - 497 , 1969.
138. J. Quinonero-Candela and C. E. Rasmussen A unifying view of sparse approximate Gaussian process regression Journal of Machine Learning Research , 6 : 1939 - 1959 , 2005.
139. J. Racine An effi cient cross-validation algorithm for window width selection for nonparametric kernel regression Communications in Statistics: Simulation and Computation , 22 : 1107 - 1114 , 1993.
140. L. Ralaivola and F. D'Alche Buc Time series fi ltering, smoothing and learning using the kernel Kalman fi lter In Proceedings 2005 IEEE International Joint Conference on Neural Networks , pages 1449 - 1454 , 2005.
141. N. Ranasinghe and W.-M. Shen Surprise-based learning for developmental robotics. In 2008 ECSIS Symposium on Learning and Adaptive Behaviors for Robotic Systems , pages 65 - 70 , 2008.
142. C. E. Rasmussen and C. Williams Gaussian Processes for Machine Learning. Cambridge, MA : MIT Press , 2006.
143. S. Raudys and T. Cibas Regularization by early stopping in single layer perceptron training In Proceedings of the 1996 International Conference on Artifi cial Neural Networks , volume 1112, pages 77 - 82 , 1996.
144. C. Richard , J. C. M. Bermudez , and P. Honeine Online prediction of time series data with kernels IEEE Transactions on Signal Processing , 57 ( 3 ): 1058 - 1066 , 2009.
145. R. M. Rifkin Everything Old Is New Again: A Fresh Look at Historical Approaches in Machine Learning PhD Dissertation, MIT, Cambridge, MA, 2002.
146. J. Rissanen Modeling by shortest data description Automatica , 14 : 465 - 471 , 1978.
147. G. Rombouts and M. Moonen A fast exact frequency domain implementation of the exponentially windowed affi ne projection algorithm In Proceedings. IEEE 2000 Adaptive Systems for Signal Processing, Communications, and Control Symposium , pages 342 - 346 , 2000.
148. S. Roweis and Z. Ghahramani A unifying review of linear Gaussian models Neural Computation , 11 ( 2 ), 1999.
149. J. Sacks , W. J. Welch , T. J. Mitchell , and H. P. Wynn Design and analysis of computer experiments Statistical Science , 4 : 409 - 435 , 1989.
150. N. P. Sands and J. M. Cioffi Nonlinear channel models for digital magnetic recording. IEEE Transactions on Magnetics , 29 : 3996 - 3998 , Nov. 1993.
151. S. G. Sankaran and A. A. L. Beex Convergence behavior of affi ne projection algorithms IEEE Transaction on Signal Processing , 48 ( 4 ): 1086 - 1096 , 2000.
152. A. Sayed Fundamentals of Adaptive Filtering New York : Wiley , 2003.
153. A. Sayed and T. Kailath A state-space approach to adaptive RLS fi ltering IEEE Signal Processing Magazine , 11 : 18 - 60 , 1994.
154. T. S. Schei A fi nite difference method for linearizing in nonlinear estimation algorithms Automatica , 33 ( 11 ): 2051 - 2058 , 1997.
155. M. Schetzen The Volterra and Wiener Theories of Nonlinear Systems Malabar, FL : Krieger Publishing , 2006.
156. B. Schölkopf and A. Smola Learning with Kernels: Support Vector Machines, Regularization, Optimization and Beyond Cambridge, MA : MIT Press , 2002.
157. B. Schölkopf , A. Smola , and K. M ü ller Nonlinear component analysis as a kernel eigenvalue problem Neural Computation , 10 : 1299 - 1319 , 1998.
158. B. Schölkopf , R. Herbrich , A. Smola , and R. Williamson A generalized representer theorem In Proceedings of the Annual Conference on Computational Learning Theory , pages 416 - 426 , 2001.
159. G. Schwarz Estimating the dimension of a model Annals of Statistics , 6 : 461 - 464 , 1978.
160. D. J. Sebald and J. A. Bucklew Support vector machine techniques for nonlinear equalization IEEE Journal on Selected Areas in Communications , 48 : 3217 - 3226 , Nov. 2000.
161. M. Seeger and C. Williams Fast forward selection to speed up sparse Gaussian process regression In Workshop on AI and Statistics 9 , 2003.
162. C. E. Shannon A mathematical theory of communication In Bell System Technical Journal , 379 - 423 , July 1948.
163. J. Shawe-Taylor and N. Cristianini Kernel Methods for Pattern Analysis Cambridge, UK : Cambridge University Press , 2004.
164. J. R. Shewchuk An introduction to the conjugate gradient method without the agonizing pain Technical report , Carnegie Mellon University, Pittsburgh, PA, 1994. http://www.cs.cmu.edu/~?jrs/jrspapers.html.
165. M. Simandl , J. Kralovec , and T. Söderstörm Advanced point-mass method for nonlinear state estimation Automatica , 42 ( 7 ): 1133 - 1145 , 2006.
166. H. Simon Why should machines learn? In R. S. Michalski , J. G. Carbonell , and T. M. Mitchell , editors, Machine Learning: An Artifi cial Intelligence Approach Wellsboro, PA : Tioga Publishing Company , 1983.
167. K. Slavakis and S. Theodoridis Sliding window generalized kernel affi ne projection algorithm using projection mappings EURASIP Journal on Advances in Signal Processing , 2008 http://www.hindawi.com/GetArticle.aspx?doi=10.1155/2008/ 735351.
168. A. J. Smola and P. L. Bartlett Sparse greedy Gaussian process regression In Advances in Neural Information Processing Systems 13 , pages 619 - 625 , 2001.
169. M. M. Sondhi An adaptive echo canceller Bell System Technical Journal , 46 : 497 - 511 , 1967.
170. M. M. Sondhi Closed loop adaptive echo canceller using generalized fi lter networks. U.S. Patent 3,499,999 , 1970.
171. H. W. Sorenson Kalman Filtering: Theory and Application New York : IEEE Press , 1985.
172. J. Stapleton and S. Bass Adaptive noise cancellation for a class of nonlinear, dynamic reference channels IEEE Transactions on Circuits and Systems , 32 : 143 - 150 , Feb. 1985.
173. I. Steinwart On the infl uence of the kernel on the consistency of support vector machines Journal of Machine Learning Research , 2 : 67 - 93 , 2001.
174. S. M. Stigler Gergonne's 1815 paper on the design and analysis of polynomial regression experiments Historia Mathematical 1 , 4 : 431 - 439 , 1974.
175. R. S. Sutton and A. G. Barto Reinforcement Learning: An Introduction Cambridge, MA : MIT Press , 1998.
176. J. A. K. Suykens , L. Lukas , and J. Vandewalle Sparse approximation using least squares support vector machines In IEEE International Symposium on Circuits and Systems 2000 , pages 757 - 760 , 2000.
177. F. Takens Detecting strange attractors in turbulence Dynamical Systems and Turbulence , pages 366 - 381 , 1981.
178. M. Tan Adaptation-based cooperative learning multiagent systems In P roceedings of the Tenth International Conference on Machine Learning , pages 330 - 337 , 1993.
179. M. Tanaka , S. Makino , and J. Kojima A block exact fast affi ne projection algorithm. IEEE Transactions on Speech and Audio Processing , 7 : 79 - 86 , 1999.
180. P. Tans Trends in atmospheric carbon dioxide - Mauna Loa. NOAA/ESRL, 2008. www.esrl.noaa.gov/gmd/ccgg/trends.
181. S. Theodoridis , C. M. S., and C. F. N. Cowan Nonlinear channel equalization using clustering techniques In Proceedings of IEEE International Conference on Communications 1992 , volume 3, pages 1277 - 1279 , 1992.
182. P. D. Thompson Optimum smoothing of two-dimensional fi elds Tellus , 8 : 384 - 393 , 1956.
183. A. Tikhonov and V. Arsenin Solution of Ill-posed Problems Washington, DC : Winston & Sons , 1977.
184. M. E. Tipping Sparse Bayesian learning and the relevance vector machine Journal of Machine Learning Research , 1 : 211 - 244 , 2001.
185. S. Tong and D. Koller Support vector machine active learning with applications to text classifi cation In P. Langley , editor, Proceedings of 17th International Conference on Machine Learning , pages 999 - 1006 , 2000.
186. W. Tucker A Rigorous ODE Solver and Smale's 14th Problem Foundations of Computational Mathematics , 2 : 53 - 117 , 2002.
187. S. Van Vaerenbergh , J. Via , and I. Santamaría A sliding-window kernel RLS algorithm and its application to nonlinear channel identifi cation In Proceedings of the International Conference on Accoustics, Speech and Signal Processing 2006 , volume 5, pages 789 - 792 , May 2006.
188. V. Vapnik The Nature of Statistical Learning Theory New York : Springer Verlag , 1995.
189. V. Vapnik Statistical Learning Theory New York : Wiley , 1998.
190. V. Volterra Sopra le funzioni che dipendono de altre funzioni. Rend. R. Academia dei Lincei 2 Sem. , 59 : 97 - 105 , 1887.
191. G. Wahba Spline Models for Observational Data Philadelphia, PA : SIAM , 1990.
192. E. A. Wan Temporal backpropogation for FIR neural networks In IEEE International Joint Conference on Neural Networks , volume I, pages 575 - 580 , 1990.
193. A. S. Weigend , D. E. Rumelhart , and B. A. Huberman Generalization by weight-elimination with application to forecasting In Advances in Neural Information Processing Systems 3 , pages 875 - 882 San Francisco, CA : Morgan Kaufmann , 1990.
194. P. Whittle Prediction and Regulation by Linear Least-Square Methods Oxford, UK : English Universities Press , 1963.
195. B. Widrow and M. E. Hoff Adaptive switching circuits IRE WESCON Convention Record , 4 : 96 - 104 , 1960.
196. B. Widrow and S. D. Stearns Adaptive Signal Processing Upper Saddle River, NJ : Prentice-Hall , 1985.
197. B. Widrow , J. Glover , J. R. , J. McCool , J. Kaunitz , C. Williams , R. Hearn , J. Zeidler , J. Eugene Dong , and R. Goodlin Adaptive noise cancelling: Principles and applications Proceedings of the IEEE , 63 : 1692 - 1716 , 1975.
198. N. Wiener Extrapolation, Interpolation and Smoothing of Stationary Time Series. Cambridge, MA : MIT Press , 1949.
199. N. Wiener Nonlinear Problems in Random Theory New York : Wiley , 1958.
200. C. Williams and M. Seeger Using the Nyström method to speed up kernel machines. In T. K. Leen , T. G. Dietterich , and V. Tresp , editors, Advances in Neural Information Processing Systems , pages 682 - 688 Cambridge, UK : MIT Press , 2001.
201. C. K. I. Williams and C. E. Rasmussen Gaussian processes for regression In D. S. Touretzky , M. C. Mozer , and M. E. Hasselmo , editors, Advances in Neural Information Processin g Systems 8 , pages 514 - 520 Cambridge, UK : MIT Press , 1996.
202. G. Williams Linear Algebra with Applications Sudbury, MA : Jones and Bartlett Publishers , 2007.
203. R. L. Winkler Introduction to Bayesian Inference and Decision , 2nd edition Gainesville , FL: Probabilistic Publishing , 2003.
204. Y. Yao , L. Rosasco , and A. Caponnetto On early stopping in gradient descent learning Constructive Approximation , 26 ( 2 ): 289 - 315 , 2007.