Anomaly detection using proximity graph and PageRank algorithm

IEEE Transactions on Information Forensics and Security

Volumn 7, Issue 4, 2012, Pages 1288-1300

  Yao, Zhe ^a   Mark, Philip ^a,b   Rabbat, Michael ^a  

^a MCGILL UNIVERSITY   (Canada)

^b ERICSSON RESEARCH   (Sweden)

Author keywords

Anomaly detection; personalized PageRank; proximity graph

Indexed keywords

ANOMALY DETECTION; COMPUTATIONAL EFFORT; DATA POINTS; DENSITY ESTIMATES; DETECTION PERFORMANCE; HIGH DIMENSIONAL DATA; KERNEL METHODS; LARGE HIGH-DIMENSIONAL DATA; PAGERANK; PAGERANK ALGORITHM; PROBABILITY DENSITIES; PROXIMITY GRAPHS; REAL WORLD DATA; RUNNING TIME; TRAINING DATA;

VIRTUAL REALITY;

ALGORITHMS;

EID: 84863895638     PISSN: 15566013     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIFS.2012.2191963     Document Type: Article

References (56)

1. V. Chandola, A. Banerjee, and V. Kumar, "Anomaly detection: A survey," ACM Comput. Surveys, vol. 41, no. 3, pp. 15:1-15:58, 2009.
2. L. Page, S. Brin, R. Motwani, and T. Winograd, The PageRank Citation Ranking: Bringing Order to the Web Stanford InforLab, 1999, pp. 1999-1966, Tech. Rep..
3. B. Schölkopf, J. Platt, J. Shawe-Taylor, A. Smola, and R. Williamson, "Estimating the support of a high-dimensional distribution," Neural Computation, vol. 13, no. 7, pp. 1443-1471, 2001. (Pubitemid 33595028)
4. C. Scott and R. Nowak, "A Neyman-Pearson approach to statistical learning," IEEE Trans. Inf. Theory, vol. 51, no. 11, pp. 3806-3819, Nov. 2005. (Pubitemid 41622582)
5. C. Scott and R. Nowak, "Learning minimum volume sets," J. Mach. Learning Res., vol. 7, pp. 665-704, 2006. (Pubitemid 43668110)
6. C. Scott and E. Kolaczyk, "Nonparametric assessment of contamination in multivariate data using generalized quantile sets and fdr," J. Computat. Graph. Statist., vol. 19, no. 2, pp. 439-456, 2010.
7. A. Hero, III, "Geometric entropy minimization (GEM) for anomaly detection and localization," in Proc. Advances in Neural Information Processing Systems, Vancouver, BC, Canada, 2006, vol. 19, pp. 585-592.
8. S. Byers and A. Raftery, "Nearest-neighbor clutter removal for estimating features in spatial point processes," J. Amer. Statist. Assoc., vol. 93, no. 442, pp. 577-584, 1998.
9. S. Ramaswamy, R. Rastogi, and K. Shim, "Efficient algorithms for mining outliers from large data sets," ACM SIGMOD Record, vol. 29, no. 2, pp. 427-438, 2000.
10. M. Breunig, H. Kriegel, R. Ng, and J. Sander, "OPTICS-OF: Identifying local outliers," in Proc. Eur. Conf. Principles of Data Mining and Knowledge Discovery, Prague, Czech Republic, 1999, pp. 262-270.
11. M. Breunig, H. Kriegel, R. Ng, and J. Sander, "LOF: Identifying density-based local outliers," ACM SIGMOD Record, vol. 29, no. 2, pp. 93-104, 2000.
12. D. Pokrajac, A. Lazarevic, and L. Latecki, "Incremental local outlier detection for data streams," in Proc. IEEE Symp. Computational Intelligence and Data Mining, Honolulu, HI, 2007, pp. 504-515. (Pubitemid 47431481)
13. M. Zhao and V. Saligrama, "Anomaly detection with score functions based on nearest neighbor graphs," in Proc. Advances in Neural Information Processing Systems, Vancouver, BC, Canada, 2009, vol. 22, pp. 2250-2258.
14. V. Janeja and V. Atluri, "Random walks to identify anomalous freeform spatial scan windows," IEEE Trans. Knowl. Data Eng., vol. 20, no. 10, pp. 1378-1392, Oct. 2008.
15. J. He, Y. Liu, and R. Lawrence, "Graph-based rare category detection," in Proc. IEEE Int.Conf.DataMining,Houston, TX, 2005, pp. 418-425.
16. J. He, J. Carbonell, and Y. Liu, "Graph-based semi-supervised learning as a generative model," in Proc. Int. Joint Conf. Artificial Intelligence, Hyderabad, India, 2007, pp. 2429-2497.
17. H. Cheng, P. Tan, C. Potter, and S. Klooster, "Detection and characterization of anomalies in multivariate time series," in Proc. SIAM Int. Conf. Data Mining, Sparks, NV, 2009, pp. 413-424.
18. J. Sun, H. Qu, D. Chakrabarti, and C. Faloutsos, "Neighborhood formation and anomaly detection in bipartite graphs," in Proc. IEEE Int. Conf. Data Mining, Houston, TX, 2005, pp. 418-425. (Pubitemid 47385721)
19. C. Noble and D. Cook, "Graph-based anomaly detection," in Proc. ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, Washington, DC, 2003, pp. 631-636.
20. D. Chakrabarti, "AutoPart: Parameter-free graph partitioning and outlier detection," in Proc. Eur. Conf. Principles and Practice of Knowledge Discovery in Databases, Pisa, Italy, 2004, pp. 112-124. (Pubitemid 39740396)
21. G. Nunez, Z. Kutalik, K. Cho, and O. Wolkenhauer, "Level sets and minimum volume sets of probability density functions," Int. J. Approximate Reasoning, vol. 34, no. 1, pp. 25-47, 2003.
22. M. Carreira-Perpinán and R. Zemel, "Proximity graphs for clustering and manifold learning," in Proc. Advances in Neural Information Processing Systems, Vancouver, BC, Canada, 2004, vol. 17, pp. 225-232.
23. J. Shi and J. Malik, "Normalized cuts and image segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 888-905, Aug. 2000.
24. S. Roweis and L. Saul, "Nonlinear dimensionality reduction by locally linear embedding," Science, vol. 290, no. 5500, pp. 2323-2326, 2000. (Pubitemid 32041578)
25. X. Zhu, Semi-Supervised Learning Literature Survey Computer Sciences, University of Wisconsin-Madison, Tech. Rep. 1530, 2005.
26. D. O. Loftsgaarden and C. P. Quesenberry, "A nonparametric estimate of a multivariate density function," Ann. Math. Statist., vol. 36, no. 3, pp. 1049-1051, 1965.
27. P. Balister, B. Bollobás, A. Sarkar, and M. Walters, "Connectivity of random k-nearest-neighbour graphs," Adv. Appl. Probability, vol. 37, no. 1, pp. 1-24, 2005. (Pubitemid 40573777)
28. S. Brin and L. Page, "The anatomy of a large-scale hypertextual web search engine," Comput. Netw. ISDN Syst., vol. 30, no. 1-7, pp. 107-117, 1998.
29. A. Langville and C.Meyer, "Deeper inside PageRank," Internet Math., vol. 1, no. 3, pp. 335-380, 2004.
30. T. Haveliwala, "Topic-sensitive PageRank: A context-sensitive ranking algorithm for web search," IEEE Trans. Knowl. Data Eng., vol. 15, no. 4, pp. 784-796, Jul./Aug. 2003.
31. G. Jeh and J. Widom, "Scaling personalized web search," in Proc. Int. World Wide Web Conf., Budapest, Hungary, 2003, pp. 271-279.
32. S. Kamvar, T. Haveliwala, C. Manning, and G. Golub, Exploiting the Block Structure of the Web for Computing PageRank Stanford InfoLab, 2003, pp. 2003-2017, Tech. Rep..
33. J. Tenenbaum, V. Silva, and J. Langford, "A global geometric framework for nonlinear dimensionality reduction," Science, vol. 290, no. 5500, pp. 2319-2323, 2000. (Pubitemid 32041577)
34. M. Penrose, Random Geometric Graphs. London, U.K.: Oxford Univ. Press, 2003.
35. P. Gupta and P. Kumar, "The capacity of wireless networks," IEEE Trans. Inf. Theory, vol. 46, no. 2, pp. 388-404, Mar. 2000.
36. A. Goel, S. Rai, and B. Krishnamachari, "Sharp thresholds for monotone properties in random geometric graphs," in Proc. ACM Symp. Theory of Computing, Chicago, IL, 2004, pp. 580-586.
37. F. Rohlf, "Generalization of the gap test for the detection of multivariate outliers," Biometrics, vol. 31, no. 1, pp. 93-101, 1975.
38. W. Dixon, "Analysis of extreme values," Ann. Math. Statist., vol. 21, no. 4, pp. 488-506, 1950.
39. E. Grinspun, A First Look at DDG: Discrete Curves Columbia Univeristy [Online]. Available: http://ddg.cs.columbia.edu/SIGGRAPH05/ Didactic.pdf
40. S. Ross, Introduction to ProbabilityModels, 10th ed. NewYork:Academic Press, 2009.
41. U. von Luxburg, A. Radl, and M. Hein, Hitting, commute times in large graphs are often misleading ACM Computing Research Repository, arXiv:1003.1266v2 [cs.DS], 2011 [Online]. Available: http://arxiv.org/ abs/1003.1266v2
42. M. Penrose, "A strong law for the longest edge of theminimal spanning tree," Ann. Probability, vol. 27, no. 1, pp. 246-260, 1999.
43. D. Karger, P. Klein, and R. Tarjan, "A randomized linear-time algorithm to find minimum spanning trees," J. ACM, vol. 42, no. 2, pp. 321-328, 1995.
44. E. Kreyszig, Advanced Engineering Mathematics, 9th ed. Hoboken, NJ: Wiley, 2007.
45. H. Tong, S. Papadimitriou, P.Yu, and C. Faloutsos, "Proximity tracking on time-evolving bipartite graphs," in SIAM Conf. Data Mining, Atlanta, GA, 2008, pp. 704-715.
46. H. Tong, C. Faloutsos, and J.-Y. Pan, "Random walk with restart: Fast solutions and applications," Knowl. Inf. Syst., vol. 14, no. 3, pp. 327-346, 2008.
47. J. Tang, Z. Chen, A. Fu, and D. Cheung, "Enhancing effectiveness of outlier detections for low density patterns," in Proc. Pacific-Asia Conf. Advances in Knowledge Discovery and Data Mining, Taipei, Taiwan, 2002, pp. 535-548.
48. H. Ringberg,M. Roughan, and J. Rexford, "The need for simulation in evaluating anomaly detectors," ACM SIGCOMM Comput. Commun. Rev., vol. 38, no. 1, pp. 55-59, 2008.
49. M. Langaas, B. Lindqvist, and E. Ferkingstad, "Estimating the proportion of true null hypotheses, with application to dna microarray data," J. Royal Statist. Soc., Series B (Statistical Methodology), vol. 67, no. 4, pp. 555-572, 2005. (Pubitemid 41351405)
50. D. Gleich, MatlabBGL [Online]. Available: http://www.stanford.edu/ dgleich/programs/matlab-bgl/
51. G. Lee and C. Scott, "The one class support vector machine solution path," in Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing, Honolulu, HI, 2007, vol. 2, pp. 521-524.
52. A. Lakhina, M. Crovella, and C. Diot, "Diagnosing network-wide traffic anomalies," in Proc. ACM SIGCOMM, Portland, OR, 2004, pp. 219-230. (Pubitemid 40954882)
53. A. Frank and A. Asuncion, UCI Machine Learning Repository 2010 [Online]. Available: http://archive.ics.uci.edu/ml/
54. H. Hoffmann, "Kernel PCA for novelty detection," Pattern Recognit., vol. 40, pp. 863-874, 2007. (Pubitemid 44667761)
55. T. Ahmed, M. Coates, and A. Lakhina, "Multivariate online anomaly detection using kernel recursive least squares," in Proc. IEEE Int. Conf. Computer Communications (INFOCOM2007), Anchorage, AK, 2007, pp. 625-633. (Pubitemid 47334279)
56. P. Simard, Y. LeCun, J. Denker, and B. Victorri, "Transformation invariance in pattern recognition-Tangent distance and tangent propagation," in Neural Networks: Tricks of the Trade. London, U.K.: Springer, 1998, pp. 239-274. (Pubitemid 128162419)