Universal piecewise linear prediction via context trees

IEEE Transactions on Signal Processing

Volumn 55, Issue 7 II, 2007, Pages 3730-3745

  Kozat, Suleyman S ^a   Singer, Andrew C ^b   Zeitler, Georg Christoph ^b  

^a IBM T J WATSON RESEARCH CENTER   (United States)

^b United States of America   (United States)

Author keywords

Context tree; Piecewise linear; Prediction; Universal

Indexed keywords

ERROR ANALYSIS; FORECASTING; LINEAR REGRESSION; TREES (MATHEMATICS);

COMPETITIVE ALGORITHM APPROACH; CONTEXT TREES; LINEAR PREDICTION; PIECEWISE LINEAR PREDICTOR;

ALGORITHMS;

EID: 34347403006     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2007.894235     Document Type: Article

References (30)

1. J. Makhoul, "Linear prediction: A tutorial review," Proc. IEEE, vol. 63, pp. 561-80, 1975.
2. A. Singer, G. Wornell, and A. Oppenheim, "Nonlinear autoregressive modeling and estimation in the presence of noise," Digital Signal Process., vol. 4, no. 4, pp. 207-221, Oct. 1994.
3. P. Djuric, J. Kotecha, J. Zhang, Y. Huang, T. Ghirmai, M. Bugallo, and J. Miguez, "Particle filtering," IEEE Signal Process. Mag., vol. 20, no. 5, pp. 19-38, Sep. 2003.
4. V. Vovk, "Aggregating strategies," COLT, pp. 371-383, 1990.
5. V. Vovk, "Competitive on-line linear regression," in Proc. Conf. Processing Systems, 1998, pp. 364-370.
6. N. Cesa-Bianchi, P. M. Long, and M. K. Warmuth, "Worst-case quadratic loss bounds for prediction using linear functions and gradient descent," IEEE Trans. Neur, Netw., vol. 7, no. 3, pp. 604-619, May 1996.
7. A. C. Singer, S. S. Kozat, and M. Feder, "Universal linear least squares prediction: Upper and lower bounds," IEEE Trans. Inf. Theory vol. 48, no. 8, pp. 2354-2362, Aug. 2002.
8. H. Tong, Non-Linear Time Series: A Dynamical System Approach. Oxford, U.K.: Oxford Univ. Press, 1990.
9. R.S. Tsay, "Testing and modeling threshold autoregressive processes," J. Amer. Statist. Assoc., vol. 84, no. 405, pp. 231-240, 1989.
10. T. Coulson, E. A. Catchpole, S. D. Albon, B. J. T. Morgan, J. M. Pemberton, T. H. C. Brock, M. J. Crawley, and B. T. Grenfell, "Age, sex, density, winter weather, and population crashes in soay sheep," Science, vol. 292, no. 5521, pp. 1528-1531, May 2001.
11. M. P. Clements and J. Smith, "A Monte Carlo study of the forecasting performance of empirical setar models," J. Appl. Econom., vol. 14, no. 2, pp. 123-41, Mar.-Apr. 1999, published by Wiley.
12. J. Schoentgen, "Modelling the glottal pulse with a self-excited threshold auto-regressive model," in Proc. EUROSPEECH'93, 1993, pp. 107-110.
13. A. C. Singer and M. Feder, "Universal linear prediction by model order weighting," IEEE Trans. Signal Process., vol. 47, no. 10, pp. 2685-2699, Oct. 1999.
14. N. Merhav and M. Feder, "Universal schemes for sequential decision from individual data sequences," IEEE Trans. Inf. Theory, vol. 39, no. 4, pp. 1280-1292, Jul. 1993.
15. S. R. Kulkarni and S. E. Posner, "Universal prediction of nonlinear systems," in Proc. 34th Conf. Decision and Control, Dec. 1995, pp. 4024-4029.
16. M. Feder, N. Merhav, and M. Gutman, "Universal prediction of individual sequences," IEEE Trans. Inf. Theory, vol. 38, no. 4, pp. 1258-1270, Jul. 1992.
17. F. M. J. Willems, Y. M. Shtarkov, and T. J. Tjalkens, "The context-tree weighting method: Basic properties," IEEE Trans. Inf. Theory, vol. 41, no. 3, pp. 653-664, May 1995.
18. N. J. A. Sloane, The On-Line Encyclopedia of Integer Sequences Sequence A003095 (formerly M1544).
19. A.V. Aho and N. J. A. Sloane, "Some doubly exponential sequences," Fibonacci Quart., vol. 11, pp. 429-437, 1970.
20. D. P. Helmbold and R. E. Schapire, "Predicting nearly as well as the best pruning of a decision tree," Mach. Learn., vol. 27, no. 1, pp. 51-68, 1997.
21. E. Takimoto, A. Maruoka, and V. Vovk, "Predicting nearly as well as the best pruning of a decision tree through dyanamic programming scheme," Theor. Comp. Sci., vol. 261, pp. 179-209, 2001.
22. E. Takimoto and M. K. Warmuth, "Predicting nearly as well as the best pruning of a planar decision graph," Theor. Comp. Sci., vol. 288, pp. 217-235, 2002.
23. G. I. Shamir and N. Merhav, "Low-complexity sequential lossless coding for piecewise-stationary memoryless sources," IEEE Trans. Inf. Theory, vol. 45, no. 5, pp. 1498-1519, Jul. 1999.
24. F. M. J. Willems, "Coding for a binary independent piecewise-identically-distributed source," IEEE Trans. Inf. Theory vol. 42, no. 6, pp. 2210-2217, Nov. 1996.
25. D. S. Modha and E. Masry, "Universal, nonlinear, mean-square prediction of markov processes," in Proc. IEEE Int. Symp. Information Theory Whistler, BC, Canada, Sep. 1995, p. 259.
26. T. M. Cover, "Estimation by the nearest neighbor rule," IEEE Trans. Inf. Theory, vol. IT-14, no. 1, pp. 50-55, Jan. 1968.
27. O. J. J. Michel, A. O. Hero III, and A. E. Badel, "Tree-structured nonlinear signal modeling and prediction," IEEE Trans. Signal Process., vol. 47, no. 11, pp. 3027-3041, Nov. 1999.
28. D. Luengo, S. S. Kozat, and A. C. Singer, "Universal piecewise linear least squares prediction: Upper and lower bounds," in Proc. IEEE Int. Symp. Information Theory, Chicago, IL, Jun./Jul. 2004, p. 198.
29. R. E. Krichevsky and V. K. Trofimov, "The performance of universal encoding," IEEE Trans. Inf. Theory, vol. IT-27, no. 2, pp. 190-207, Mar. 1981.
30. B. Ya. Ryabko, "Twice-universal coding," Probl. Inf. Transm., vol. 20, no. 3, pp. 173-7, 1984.