PPCA-based missing data imputation for traffic flow volume: A systematical approach

IEEE Transactions on Intelligent Transportation Systems

Volumn 10, Issue 3, 2009, Pages 512-522

  Qu, Li ^a   Li, Li ^a   Zhang, Yi ^a   Hu, Jianming ^a  

^a TSINGHUA UNIVERSITY   (China)

Author keywords

Missing data; Probabilistic principal component analysis (PPCA); Traffic flow volume

Indexed keywords

CONVENTIONAL METHODS; FLOW VOLUMES; GAUSSIANS; HISTORICAL DATA; IMPUTATION ALGORITHM; IMPUTATION METHODS; IMPUTATION PROCESS; MISSING DATA; MISSING DATA PROBLEM; PROBABILISTIC PRINCIPAL COMPONENT ANALYSIS; PROBABILISTIC PRINCIPAL COMPONENT ANALYSIS (PPCA); ROBUST PCA; ROOT MEAN SQUARES; STATISTICAL PROPERTIES; TRAFFIC ANALYSIS; TRAFFIC FLOW; TRAFFIC FLOW VOLUME;

DATA FLOW ANALYSIS; TIME SERIES; TRAFFIC SURVEYS;

PRINCIPAL COMPONENT ANALYSIS;

EID: 70349166727     PISSN: 15249050     EISSN: None     Source Type: Journal    
DOI: 10.1109/TITS.2009.2026312     Document Type: Article

References (48)

1. S. Turner, L. Albert, B. Gajewski, and W. Eisele, "Archived intelligent transportation system data quality: Preliminary analyses of San Antonio TransGuide data," Trans. Res. Rec., no.1719, pp. 77-84, 2000.
2. C. Chen, J. Kwon, J. Rice, A. Skabardonis, and P. Varaiya, "Detecting errors and imputing missing data for single loop surveillance systems," Trans. Res. Rec., vol.1855, pp. 160-167, 2002.
3. J. H. Conklin and B. L. Smith, "The use of local lane distribution patterns for the estimation of missing data in transportation management systems," Trans. Res. Rec., no.1811, pp. 50-56, 2002.
4. H. S. Zhang, Y. Zhang, Z. H. Li, and D. Hu, "Spatial-temporal traffic data analysis based on global data management using MAS," IEEE Trans. Intell. Transp. Syst., vol.5, no.4, pp. 268-275, Dec. 2004.
5. A. Stathopoulos and T. Tsekeris, "Methodology for processing archived ITS data for reliability analysis in urban networks," Proc. Inst. Elect. Eng.-Intell. Transp. Syst., vol.153, no.1, pp. 105-112, Mar. 2006.
6. D. Ni, J. D. Leonard, II, A. Guin, and C. Feng, "Multiple imputation scheme for overcoming the missing values and variability issues in ITS data," J. Transp. Eng., vol.131, no.12, pp. 931-938, Dec. 2005.
7. B. L. Smith, W. T. Scherer, and J. H. Conklin, "Exploring imputation techniques for missing data in transportation management systems," Trans. Res. Rec., no.1836, pp. 132-142, 2003.
8. M. Treibera and D. Helbing, "Reconstructing the spatio-temporal traffic dynamics from stationary detector data," Coop. Transp. Dyn.,vol.1, pp. 3.1-3.24, 2002.
9. A. Afifi and R. Elashoff, "Missing observations in multivariate statistics I: Review of the literature," J. Amer. Stat. Assoc., vol.61, no.315, pp. 595- 604, 1966.
10. R. J. A. Little and D. B. Rubin, Statistical Analysis With Missing Data. New York: Wiley, 1987.
11. J. L. Schafer, Analysis of Incomplete Multivariate Data, ser. Monographs on Statistics and Applied Probability, no.72. London, U.K.: Chapman & Hall, 1997.
12. J. Chen and J. Shao, "Nearest neighbour imputation for survey data," J. Off. Stat., vol.16, no.2, pp. 113-131, 2000.
13. P. D. Allison, Missing Data. Thousand Oaks, CA: Sage, 2001.
14. Y. Chen, J. Hu, Y. Zhang, and X. Li, "Traffic data analysis using Kernel PCA and self-organizing map," in Proc. IEEE Intell. Vehicles Symp., Tokyo, Japan, 2006, pp. 472-477.
15. L. Qu, J. Hu, and Y. Zhang, "A fiow volumes data compression approach for traffic network based on principal component analysis," in Proc. IEEE Intell. Transp. Syst. Conf., Seattle, WA, 2007, pp. 125-130.
16. X. Jin, Y. Zhang, and D. Yao, "Simultaneously prediction of network traffic fiow based on PCA-SVR," in Proc. 4th Int. Symp. Neural Netw., 2007, vol.4493, pp. 1022-1031.
17. L. Qu, Y. Zhang, J. Hu, L. Jia, and L. Li, "A BPCA based missing value imputing method for traffic flow volume data," in Proc. IEEE Conf. Intell. Vehicle Symp., Eindhoven, The Netherlands, 2008, pp. 985-990.
18. Z. Zhao, Y. Zhang, J. Hu, and L. Li, "A comparison study of PCA and ICA based traffic flow compression," J. Highway Transp. Res. Develop., vol.25, no.11, pp. 109-113, 2008 (in Chinese).
19. X. Jin, Y. Zhang, L. Li, and J. Hu, "Robust PCA based abnormal traffic flow pattern isolation and loop detector fault detection," Tsinghua Sci. Technol., vol.13, no.6, pp. 829-835, Dec. 2008.
20. I. T. Jolliffe, Principal Component Analysis, 2nd ed. New York: Springer-Verlag, 2002.
21. H. Hartley, "Maximum likelihood estimation from incomplete data," Biometrics, vol.14, pp. 174-194, 1958.
22. A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," J. R. Stat. Soc., Ser. B, vol.39, no.1, pp. 1-38, 1977.
23. G. McLachlan and T. Krishnan, The EM Algorithm and Extensions, ser. Wiley Series in Probability and Statistics. Hoboken, NJ: Wiley, 1997.
24. TDAD: Traffic Data Acquisition and Distribution, ITS Research Program, UW. [Online]. Available: http://www.its.washington.edu/tdad/
25. H. Hotelling, "Analysis of a complex of statistical variables into principal components," J. Educ. Psychol., vol.24, pp. 417-441, 1933.
26. T. Wiberg, "Computation of principal components when data is missing," in Proc. 2nd Symp. Comput. Stat., Berlin, Germany, 1976, pp. 229-236.
27. H.-Y. Shum, K. Ikeuchi, and R. Reddy, "Principal component analysis with missing data and its application to polyhedral object modeling," IEEE Trans. Pattern Anal. Mach. Intell., vol.17, no.9, pp. 854-867, Sep. 1995.
28. S. Roweis and Z. Ghahramani, "A unifying review of linear Gaussian models," Neural Comput., vol.11, no.2, pp. 305-345, Feb. 1999.
29. M. E. Tipping and C. M. Bishop, "Probabilistic principal component analysis," J. R. Stat. Soc., Ser. B, vol.21, no.3, pp. 611-622, 1999.
30. C. M. Bishop, "Bayesian PCA," in Proc. Conf. Adv. Neural Inform. Process. Syst., 1998, pp. 382-388.
31. T. P. Minka, "Automatic choice of dimensionality for PCA," in Advances in Neural Information Processing Systems, vol.13. Cambridge, MA: MIT Press, 2000, pp. 598-604.
32. S. Oba, M. Sato, I. Takemasa, M. Monden, K. Matsubara, and S. Ishii, "Bayesian missing value estimation method for gene expression profile data," Bioinformatics, vol.19, no.16, pp. 2088-2096, Nov. 2003.
33. V. Smidl and A. Quinn, "On Bayesian principal component analysis," Comput. Stat. Data Anal., vol.51, no.9, pp. 4101-4123, May 2007.
34. P. J. Huber, "Projection pursuit," Ann. Stat., vol.13, no.2, pp. 435-475, 1985.
35. M. L. Shyu, S. C. Chen, K. Sarinnapakorn, and L.-W. Chang, "A novel anomaly detection scheme based on principal component classifier," in Proc. IEEE Found. New Directions Data Mining, Melbourne, FL, 2003, pp. 171-179.
36. Y.-L. Xie, J.-H. Wang, Y.-Z. Liang, L.-X. Sun, X.-H. Song, and R.-Q. Yu, "Robust principal component analysis by projection pursuit," J. Chemom., vol.7, no.6, pp. 527-541, 1993.
37. C. Croux and A. Ruiz-Gazen, "High breakdown estimators for principal components: The projection-pursuit approach revisited," J. Multivariate Anal., vol.95, no.1, pp. 206-226, Jul. 2005.
38. M. Hubert, P. Rousseeuw, and K. V. Branden, "Robpca: A new approach to robust principal components analysis," Technometrics, vol.47, no.1, pp. 64-79, 2005.
39. S. Engelen, M. Hubert, and K. V. Branden, "A comparison of three procedures for robust PCA in high dimensions," Aust. J. Stat., vol.34, no.2, pp. 117-126, 2005.
40. S. Serneels and T. Verdonck, "Principal component analysis for data containing outliers and missing elements," Comput. Stat. Data Anal., vol.52, no.3, pp. 1712-1727, Jan. 2008.
41. Y. Chen, Y. Zhang, J. Hu, and L. Li, "Mining for similarities in urban traffic flow using wavelets," in Proc. IEEE Conf. Intell. Transp. Syst., 2007, pp. 119-124.
42. P. I. Good, Resampling Methods: A Practical Guide to Data Analysis, 2nd ed. Boston, MA: Birkhäuser, 2001.
43. I. H. Witten and E. Frank, Data Mining: Practical Machine Learning Tools and Techniques, 2nd ed. San Mateo, CA: Morgan Kaufmann, 2005.
44. J. Han and M. Kamber, Data Mining: Concepts and Techniques, 2nd ed. San Mateo, CA: Morgan Kaufmann, 2006.
45. C. de Boor, A Practical Guide to Splines. New York: Springer-Verlag, 1978.
46. D. Kahaner, C. Moler, and S. Nash, Numerical Methods and Software. Englewood Cliffs, NJ: Prentice-Hall, 1989.
47. C. Moler, Numerical Computing With MATLAB. Philadelphia, PA: SIAM, 2004, ch. 3 Interpolation. [Online]. Available: http://www. mathworks.com/moler/
48. S. Sun, C. Zhang, and G. Yu, "A Bayesian network approach to traffic flow forecasting," IEEE Trans. Intell. Transp. Syst., vol.7, no.1, pp. 124-132, Mar. 2006.