Improving signal prediction performance of neural networks through multiresolution learning approach

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 36, Issue 2, 2006, Pages 341-352

  Liang, Yao ^a   Liang, Xu ^b  

^a State University   (United States)

^b Berkeley   (United States)

Author keywords

Adjustabe activation functions; Generalization performance; Multiresolution learning; Neural network complexity; Neural networks; Nonlinear signal prediction; Wavelet analysis

Indexed keywords

LEARNING SYSTEMS; MATHEMATICAL MODELS; NONLINEAR CONTROL SYSTEMS; OPTIMIZATION; ROBUSTNESS (CONTROL SYSTEMS); SIGNAL FILTERING AND PREDICTION; WAVEFORM ANALYSIS;

ADJUSTABLE ACTIVATION FUNCTIONS; MULTIRESOLUTION LEARNING; NONLINEAR SIGNAL PREDICTION; WAVELET ANALYSIS;

NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPARATIVE STUDY; FORECASTING; INFORMATION RETRIEVAL; METHODOLOGY; NONLINEAR SYSTEM; SIGNAL PROCESSING; TIME;

ALGORITHMS; FORECASTING; INFORMATION STORAGE AND RETRIEVAL; NEURAL NETWORKS (COMPUTER); NONLINEAR DYNAMICS; PATTERN RECOGNITION, AUTOMATED; SIGNAL PROCESSING, COMPUTER-ASSISTED; TIME FACTORS;

EID: 33645001792     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCB.2005.857092     Document Type: Article

References (57)

1. B. L. Bowerman and R. T. O'Connell, Time Series Forecasting. New York: PWS, 1987.
2. G. E. P. Box and G. M. Jenkins, Time Series Analysis, Forecasting, and Control. San Francisco, CA: Holden-Day, 1976.
3. A. Cichocki and R. Unbehauen, Neural Networks for Optimization and Signal Processing. New York: Wiley, 1993.
4. A. S. Weigend and N. A. Gershenfeld, Eds., Time Series Prediction: Forecasting the Future and Understanding the Past. Reading, MA: Addison-Wesley, 1994.
5. A. Lapedes and R. Farber, "Nonlinear Signal Processing Using Neural Network: Prediction and System Modeling," Los Alamos Nat. Lab., Tech. Rep. LA-UR-87-2662, 1987.
6. K. Hornik, "Approximation capabilities of multilayer feedforward networks," Neural Netw., vol. 4, pp. 251-257, 1991.
7. M. Leshno, V. Y. Lin, A. Pinkus, and S. Schocken, "Multilayer feedforward networks with a nonpolynomial activation function can approximate any function," Neural Netw., vol. 6, pp. 861-867, 1993.
8. K. J. Lang, A. H. Waibel, and G. E. Hinton, "A time-delay neural network architecture for isolated word recognition," Neural Netw., vol. 3, pp. 23-43, 1990.
9. E. A. Wan, "Time series prediction by using a connectionist network with internal delay lines," in Time Series Prediction: Forecasting the Future and Understanding the Past. Reading, MA: Addison-Wesley, 1994, pp. 195-218.
10. A. Krogh and J. A. Hertz, "A simple weight decay can improve generalization," Adv. Neural Inf. Process. Syst., vol. 4, pp. 950-957, 1992.
11. A. S. Weigend, D. E. Rumelhart, and B. A. Huberman, "Backpropagation, weight-elimination and time series prediction," in Proc. Connectionist Models Summer School, 1990, pp. 105-116.
12. A. S. Weigend, B. A. Huberman, and D. E. Rumelhart, "Predicting the future: A connectionist approach," Int. J. Neural Syst., vol. 1, no. 3, pp. 193-209, 1990.
13. M. Cottrell, B. Girard, Y. Girard, M. Mangeas, and C. Muller, "Neural modeling for time series: A statistical stepwise method for weight elimination," IEEE Trans. Neural Networks, vol. 6, no. 12, pp. 1355-1364, Dec. 1995.
14. J. Hall and P. Mars, "Limitations of artificial neural networks for traffic prediction in broadband networks," Proc. Inst. Elect. Eng., vol. 147, no. 2, pp. 114-118, Apr. 2000.
15. P. Chang and J. Hu, "Optimal nonlinear adaptive prediction and modeling of MPEG video in ATM networks using pipelined recurrent neural networks," IEEE J. Sel. Areas Commun., vol. 15, no. 6, pp. 1087-1100, Aug. 1997.
16. S. Chong and S. Li, "Predictive dynamic bandwidth allocation for efficient transport of real-time VBR video over ATM," IEEE J. Sel. Areas Commun., vol. 13, no. 1, pp. 12-23, Jan. 1995.
17. Y. Liang and E. W. Page, "Multiresolution learning paradigm and signal prediction," IEEE Trans. Signal Process., vol. 45, no. 11, pp. 2858-2864, Nov. 1997.
18. Y. Liang, "Multiresolutiion Learning Paradigm and High-Speed Network Traffic Prediction," Ph.D. dissertation, Dept. Comp. Sci., Clemson Univ., Clemson, SC, 1997.
19. Y. Liang and E. W. Page, "VBR video traffic prediction using neural networks with multiresolution learning," in Advances in Neural Networks and Applications. Athens, Greece: WSES, 2001, pp. 98-103.
20. R. Koenen, "MPEG-4 multimedia for our time," IEEE Spectrum, vol. 36, no. 2, pp. 26-33, Feb. 1999.
21. F. H. P. Fitzek and M. Reisslein, "MPEG-4 and H.263 video traces for network performance evaluation," IEEE Network, pp. 40-54, Nov./Dec. 2001.
22. A. B. Geva, "ScaleNet - multiscale neural-network architecture for time series prediction," IEEE Trans. Neural Netw., vol. 9, no. 5, pp. 1471-1482, Oct. 1998.
23. B. L. Zhang et al., "Multiresolution forecasting for future trading using wavelet decompositions:' IEEE Trans. Neural Netw., vol. 12, no. 4, pp. 765-775, Aug. 2001.
24. S. G. Mallat, "A theory for multiresolution signal decomposition: The wavelet representation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 11, no. 7, pp. 674-693, Jul. 1989.
25. A. Cohen and J. Kovacevic, "Wavelets: The mathematical background," Proc. IEEE, vol. 84, no. 4, pp. 514-522, Apr. 1996.
26. T. L. Fine, Feedforward Neural Network Methodology. New York: Springer-Verlag, 1999.
27. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning internal representations by error propagation," Parallel Distrib. Process., vol. 1, pp. 318-362, 1986.
28. P. J. Werbos, "Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences," Ph.D. dissertation, Dept. Appl. Math., Harvard Univ., Cambrige, MA, 1974.
29. S. Ma and C. Ji, "Performance and efficiency: Recent advances in supervised learning," Proc. IEEE, vol. 87, no. 9, pp. 1519-1535, Sep./Oct. 1999.
30. A. Sperduti and A. Starita, "Speed up learning and network optimization with extended back propagation," Neural Netw., vol. 6, pp. 365-383, 1993.
31. J. K. Kruschke and L. R. Movellan, "Benefits of gain: Speeded learning and minimal hidden layers in back-propagation networks," IEEE Trans. Syst., Man, Cybern., vol. 21, no. 2, pp. 273-280, Apr. 1991.
32. D. L. Gall, "MPEG: A video compression standard for multimedia applications," Commun. ACM, vol. 34, no. 4, pp. 46-58,1991.
33. O. Rose, "Statistical Properties of MPEG Video Traffic and Their Impact on Traffic Modeling in ATM Systems," Univ. Wurzburg, Wurzburg, Germany, Tech. Rep. 101, 1995.
34. M. B. Priestley, Non-Linear and Non-Stationary Time Series Analysis. New York: Academic, 1988.
35. Y. R. Park, T. J. Murray, and C. Chen, "Predicting sun spots using a layered perceptron neural network," IEEE Trans. Neural Netw., vol. 7, no. 3, pp. 501-505, Mar. 1996.
36. NOAA National Geophysical Dada Center. [Online]. Available: http:// www.ngdc.noaa.gov/
37. A. G. Parlos, O. T. Rais, and A. F. Atiya, "Multi-step-ahead prediction using dynamic recurrent neural networks," Neural Netw., vol. 13, pp. 765-786, Sep. 2000.
38. X. Liang, D. P. Lettenmaier, E. F. Wood, and S. J. Burges, "A simple hydrologically based model of land surface water and energy fluxes for general circulation models," J. Geophys. Res., vol. 99, no. D7, pp. 14415-14,428, 1994.
39. X. Liang, E. F. Wood, and D. P. Lettenmaier, "Surface soil moisture parameterization of the VIC-2L model: Evaluation and modifications," Global Planetary Change, vol. 13, pp. 195-206, 1996.
40. X. Liang, D. P. Lettenmaier, and E. F. Wood, "A one-dimensional statistical dynamic representation of subgrid spatial variability of precipitation in the two-layer variable infiltration capacity model," J. Geophys. Res., vol. 101, no. D16, pp. 21 403-21 422, 1996.
41. X. Liang and Z. Xie, "A new surface runoff parameterization with subgrid-scale soil heterogeneity for land surface models," Adv. Water Resour., vol. 24, pp. 1173-1193, 2001.
42. K. Hsu, H. V. Gupta, and S. Sorooshian, "Artificial neural network modeling of the rainfall-runoff process," Water Resour. Res., vol. 31, no. 10, pp. 2517-2530, 1995.
43. F. Wood, D. P. Lettenmaier, X. Liang, B. Nijssen, and S. W. Wetzel, "Hydrological modeling of continental-scale basins," Annu. Rev. Earth Planet. Sci., vol. 25, pp. 279-300, 1997.
44. Q. Duan, V. K. Gupta, and S. Sorooshian, "Effective and efficient global optimization for conceptual rainfall-runoff models," Water Resour. Res., vol. 28, no. 4, pp. 1015-1031, 1992.
45. C. Ji, "Generalization error and the expected network complexity," in Advances in Neural Information Processing Systems 5. San Mateo, CA: Morgan Kaufmann, 1993.
46. A. Atiya and C. Ji, "How initial conditions affect generalization performance in large networks," IEEE Trans. Neural Netw., vol. 8, no. 2, pp. 448-451, Apr. 1997.
47. Y. Liang, "Real-Time VBR video traffic prediction for dynamic band-width allocation," IEEE Trans. Syst., Man, Cybern. C, Appl. Rev, vol. 34, no. 1, pp. 32-47, Feb. 2004.
48. A. Bhattacharya, A. G. Parlos, and A. F. Atiya, "Prediction of MPEG-coded video source traffic using recurrent neural networks," IEEE Trans. Signal Process., vol. 51, no. 8, pp. 2177-2190, Aug. 2003.
49. A. M. Adas, "Using adaptive linear prediction to support real-time VBR video under RCBR network service model," IEEE/ACM Trans. Netw., vol. 6, pp. 635-644, Oct. 1998.
50. S. J. Yoo, "Efficient traffic prediction scheme for real-time VBR MPEG video transmission over high-speed networks," IEEE Trans. Broadcasting, vol. 48, no. 1, pp. 10-18, Mar. 2002.
51. P. Coulibaly, F. Anctil, P. Rasmussen, and B. Bobee, "A recurrent neural networks approach using indices of low-frequency climatic variability to forecast regional annual runoff," Hydrol. Processes, vol. 14, pp. 2755-2777, 2000.
52. C. E. Imrie, S. Durucan, and A. Korre, "River flow prediction using artifical neural networks: Generalization beyond the calibration range," J. Hydrol., vol. 233, pp. 138-153, 2000.
53. S. Birikundavyi, R. Labib, H. T. Trung, and J. Rousselle, "Performance of neural networks in daily streamflow forecasting," J. Hydrol. Eng., vol. 7, no. 5, pp. 392-398, 2002.
54. F. Anctil and D. G. Tape, "An exploration of artificial neural network rainfall-runoff forecasting combined with wavelet decomposition," J. Environ. Eng. Sci., vol. 3, pp. S121-S128, 2004.
55. F. Anctil, C. Michel, C. Perrin, and V. Andreassian, "A soil moisture index as an auxiliary ANN input for stream flow forecasting," J. Hydrol., vol. 286, pp. 155-167, 2004.
56. H. Moradkhani, K.-L. Hsu, H. V. Gupta, and S. Sorooshian, "Improved streamflow forecasting using self-organizing radial basis function artificial neural networks," J. Hydrol., vol. 295, pp. 246-262, 2004.
57. V. Alarcón-Aquino and J. A. Barria, "Multiresolution FIR neural network based learning algorithm applied to network traffic prediction," IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., to be published.