Universal switching linear least squares prediction

IEEE Transactions on Signal Processing

Volumn 56, Issue 1, 2008, Pages 189-204

  Kozat, Suleyman S ^a   Singer, Andrew C ^b  

^a KOÇ UNIVERSITY   (Turkey)

^b University of Illinois at Urbana Champaign   (United States)

Author keywords

Piecewise continuous; Prediction; Transition diagram; Universal

Indexed keywords

ALGORITHMS; COMPUTATIONAL COMPLEXITY; LINEAR REGRESSION; PIECEWISE LINEAR TECHNIQUES;

SQUARE-ERROR LOSS; TRANSITION DIAGRAM; UNIVERSAL SWITCHING LINEAR PREDICTOR;

IMAGE SEGMENTATION;

EID: 37749021616     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2007.901161     Document Type: Article

References (48)

1. F. M. J. Willems, "Coding for a binary independent piecewise-identically-distributed source," IEEE Trans. Inf. Theory vol. 42, no. 6, pt. 2, pp. 2210-2217, Nov. 1996.
2. V. Vovk, "Competitive on-line linear regression," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 1998, pp. 364-370.
3. N. Cesa-Bianchi, Y. Freund, D. Haussler, D. P. Helmbold, R. E. Schapire, and M. K. Warmuth, "How to use expert advice," J. Assoc. Comput. Mach., vol. 44, no. 3, pp. 427-485, 1997.
4. N. Littlestone and M. K. Warmuth, "The weighted majority algorithm," Inf. Comput., vol. 108, no. 2, pp. 212-261, 1994.
5. S. S. Kozat, A. C. Singer, and G. Zeitler, "Universal piecewise linear prediction via context trees," IEEE Trans. Signal Process., vol. 55, no. 7, pt. 2, pp. 3730-3745, Jul. 2007.
6. T. Cover and E. Ordentlich, "Universal portfolios with side-information," IEEE Trans. Inf. Theory, vol. 42, no. 2, pp. 348-363, Mar. 1996.
7. T. Linder and G. Lugosi, "A zero-delay sequential scheme for lossy coding of individual sequences," IEEE Trans. Inf. Theory, vol. 46, no. 1, pp. 190-207, Jan. 2001.
8. J. Hannan, Approximation to Bayes Risk in Repeated Play. Princeton, NJ: Princeton Univ. Press, 1957, vol. III, Contributions to the Theory of Games, pp. 97-139.
9. B. Fittingoff, "Universal methods of coding for the case of unknown statistics," in Proc. 5th Symp. Inf. Theory, Moscow/Gorky, U.S.S.R., 1972, pp. 129-135.
10. L. D. Davisson, "Universal noiseless coding," IEEE Trans. Inf. Theory, vol. IT-19, no. 6, pp. 783-795, Nov. 1973.
11. P. Elias, "Universal codeword sets and representations of the integers," IEEE Trans. Inf. Theory, vol. IT-21, no. 2, pp. 194-203, Mar. 1975.
12. J. Ziv, "Coding of sources with unknown statistics, Part I: Probability of encoding error," IEEE Trans. Inf. Theory, vol. IT-18, no. 3, pp. 373-343, May 1972.
13. J. Ziv and A. Lempel, "Compression of individual sequences via variable-rate coding," IEEE Trans. Inf. Theory, vol. IT-24, no. 5, pp. 530-536, Sep. 1978.
14. R. E. Krichevsky and V. K. Trofimov, "The performance of universal encoding," IEEE Trans. Inf. Theory, vol. IT-27, no. 2, pp. 199-207, Mar. 1981.
15. J. Rissanen and G. G. Langdon, "Universal modeling and coding," IEEE Trans. Inf. Theory, vol. IT-27, no. 1, pp. 12-23, Jan. 1984.
16. J. Rissanen, "A universal data compression system," IEEE Trans. Inf. Theory, vol. IT-29, no. 5, pp. 656-664, Sep. 1983.
17. B. Y. Ryabko, "Twice-universal coding," Prob. Inf. Trans., vol. 20, no. 3, pp. 173-177, Jul.-Sep. 1984.
18. P. A. Chou, M. Effros, and R. M. Gray, "Universal quantization of parametric sources has redundancy k/2 logn/n," in Proc. IEEE Int. Symp. Inf. Theory, 1995, pp. 371-371.
19. M. Effros, P. A. Chou, and R. M. Gray, "Rates of convergence in adaptive universal vector quantization," in Proc. IEEE Int. Symp. Inf. Theory, 1994, pp. 456-456.
20. R. Zamir and M. Feder, "On universal quantization by randomized uniform/lattice quantizers," IEEE Trans. Inf. Theory, vol. 38, no. 2, pp. 428-36, Mar. 1992.
21. T. M. Cover, "Universal gambling schemes and the complexity measures of Kolmogorov and Chaitin," Dept. Statist., Stanford Univ., Stanford, CA, Tech. Rep. 12, 1974.
22. J. Rissanen, "Universal coding, information, prediction, and estimation," IEEE Trans. Inf. Theory, vol. IT-30, no. 4, pp. 629-636, Jul. 1984.
23. B. Ya Ryabko, "Prediction of random sequences and universal coding," Prob. Inf. Trans., vol. 24, no. 2, pp. 87-96, Apr.-Jun. 1988.
24. N. Merhav and M. Feder, "Universal prediction," IEEE Trans. Inf. Theory, vol. 44, no. 10, pp. 2124-2147, Oct. 1998.
25. P. Algoet, "Universal schemes for prediction, gambling, and portfolio selection," Ann. Probab., pp. 901-941, 1992.
26. V. G. Vovk, M. Fulk and J. Case, Eds., "Aggregating strategies (learning)," in Proc. 3rd Annu. Workshop Comput. Learn. Theory, San Mateo, CA, 1990, pp. 371-383.
27. D. Foster, "Prediction in the worst case," Annu. Statist., vol. 19, no. 2, pp. 1084-1090, 1991.
28. N. Cesa-Bianchi, Y. Freund, D. Helmbold, D. Haussler, R. Schapire, and M. Warmuth, "How to use expert advice," in Proc. Annu. Assoc. Comput. Mach. Symp. Theory Comput., 1993, pp. 382-391.
29. D. Haussler, J. Kivinen, and M. K. Warmuth, P. Vitanyi, Ed., "Tight worst-case loss bounds for predicting with expert advice," in Proc. 2nd Eur. Conf. Comput. Learn. Theory (EuroCOLT), Mar. 1995, pp. 69-83.
30. A. C. Singer, S. S. Kozat, and M. Feder, "Universal linear least squares prediction: Upper and lower bounds," IEEE Trans. Inf. Theory vol. 48, no. 8, pp. 2354-2362, Aug. 2002.
31. V. Vovk, "A game of prediction with expert advice," J. Comput. System Sci., vol. 56, pp. 153-173, 1998.
32. A. C. Singer and M. Feder, "UNiversal linear prediction by model order weighting," IEEE Trans. Signal Process., vol. 47, no. 10, pp. 2685-2699, Oct. 1999.
33. V. Vovk, "Aggregating strategies," in Proc. Comput. Learn. Theory 1990,pp.371-383.
34. D. Haussler, J. Kivinen, and M. K. Warmuth, "Sequential prediction of individual sequences under general loss functions," IEEE Trans. Inf. Theory, vol. 44, no. 5, pp. 1906-1925, Sep. 1998.
35. J. Kivinen and D. Haussler, "Additive versus exponentiated gradient updates for linear prediction," J. Inf. Comput., vol. 132, no. 1, pp. 1-64, 1997.
36. M. Feder, N. Merhav, and M. Gutman, "Universal prediction of individual sequences," IEEE Trans. Inf. Theory, vol. 38, no. 4, pp. 1258-1270, Jul. 1992.
37. D. Luengo, S. S. Kozat, and A. C. Singer, "Universal piecewise linear least squares prediction," in Proc. Int. Symp. Inf. Theory, 2004, p. 198.
38. M. Herbster and M. K. Warmuth, "Tracking the best regressor," in Proc. 11th Annu. Conf. Comput. Learn. Theory, 1998, pp. 24-31.
39. G. I. Shamir and N. Merhav, "Low-complexity sequential lossless coding for piecewise-stationary memoryless sources," IEEE Trans. Inf. Theory, vol. 45, no. 5, pp. 1498-1519, Jul. 1999.
40. M. Herbster and M. K. Warmuth, "Tracking the best linear predictor," J. Mach. Learn. Res., pp. 281-309, 2001.
41. V. Vovk, "Derandomizing stochastic prediction strategies," Mach. Learn., vol. 35, pp. 247-282, 1999.
42. M. Herbster and M. K. Warmuth, "Tracking the best expert," in Proc. Int. Conf. Mach. Learn., 1995, pp. 286-294.
43. O. Bousquet and M. K. Warmuth, "Tracking a small set of experts by mixing past posteriors," J. Mach. Learn. Res., vol. 3, pp. 363-396, 2000.
44. A. Gyorgy, T. Linder, and G. Lugosi, "Tracking the best of many expert," in Proc. Comput. Learn. Theory, 2005, pp. 204-216.
45. P. Auer and M. K. Warmuth, "Tracking the best disjuction," J. Mach. Learn., vol. 32, no. 2, pp. 127-150, 1998.
46. Y. Singer, "Switching portfolios," in Proc. 14th Conf. Uncertainty Artif. Intell., 1998, pp. 1498-1519.
47. R. E. Krichevsky and V. K. Trofimov, "The performance of universal encoding," IEEE Trans. Inf. Theory, vol. IT-27, no. 2, pp. 190-207, Mar. 1981.
48. P. Wessa, Free Statistics Software. ver. 1.1.22-rl, Office for Research Wevelopment and Education, 2007 [Online]. Available: http:www.wessa.net