TEXT: Automatic template extraction from heterogeneous web pages

IEEE Transactions on Knowledge and Data Engineering

Volumn 23, Issue 4, 2011, Pages 612-626

  Kim, Chulyun ^a   Shim, Kyuseok ^a  

^a Seoul National University   (South Korea)

Author keywords

clustering; MinHash; minimum description length principle; Template extraction

Indexed keywords

CLUSTERING; COMPREHENSIVE ANALYSIS; FAST APPROXIMATION; HIGH PRODUCTIVITY; MINHASH; MINIMUM DESCRIPTION LENGTH PRINCIPLE; NOVEL ALGORITHM; REAL LIFE DATASETS; STATE OF THE ART; TEMPLATE DETECTION; TEMPLATE EXTRACTION; TEMPLATE STRUCTURES; WEB APPLICATION; WEB DOCUMENT; WEB PAGE;

APPROXIMATION ALGORITHMS; CLUSTER ANALYSIS; INFORMATION RETRIEVAL SYSTEMS; INFORMATION THEORY; SEARCH ENGINES; WEBSITES;

CLUSTERING ALGORITHMS;

EID: 79951930082     PISSN: 10414347     EISSN: None     Source Type: Journal    
DOI: 10.1109/TKDE.2010.140     Document Type: Article

References (26)

1. Document Object Model (dom) Level 1 Specification Version 1.0, http://www.w3.org/TR/REC-DOM-Level-1, 2010.
2. Xpath Specification, http://www.w3.org/TR/xpath, 2010.
3. A. Arasu and H. Garcia-Molina, "Extracting Structured Data from Web Pages," Proc. ACM SIGMOD, 2003.
4. Z. Bar-Yossef and S. Rajagopalan, "Template Detection via Data Mining and Its Applications," Proc. 11th Int'l Conf. World Wide Web (WWW), 2002.
5. A.Z. Broder, M. Charikar, A.M. Frieze, and M. Mitzenmacher, "Min-Wise Independent Permutations," J. Computer and System Sciences, vol. 60, no. 3, pp. 630-659, 2000.
6. D. Chakrabarti, R. Kumar, and K. Punera, "Page-Level Template Detection via Isotonic Smoothing," Proc. 16th Int'l Conf. World Wide Web (WWW), 2007.
7. Z. Chen, F. Korn, N. Koudas, and S. Muithukrishnan, "Selectivity Estimation for Boolean Queries," Proc. ACM SIGMOD-SIGACTSIGART Symp. Principles of Database Systems (PODS), 2000.
8. J. Cho and U. Schonfeld, "Rankmass Crawler: A Crawler with High Personalized Pagerank Coverage Guarantee," Proc. Int'l Conf. Very Large Data Bases (VLDB), 2007.
9. T.M. Cover and J.A. Thomas, Elements of Information Theory. Wiley Interscience, 1991.
10. V. Crescenzi, G. Mecca, and P. Merialdo, "Roadrunner: Towards Automatic Data Extraction from Large Web Sites," Proc. 27th Int'l Conf. Very Large Data Bases (VLDB), 2001.
11. V. Crescenzi, P. Merialdo, and P. Missier, "Clustering (Pubitemid 40683780)
12. M. De Castro Reis, P.B. Golgher, A.S. Da Silva, and A.H.F. Laender, "Automatic Web News Extraction Using Tree Edit Distance," Proc. 13th Int'l Conf. World Wide Web (WWW), 2004.
13. I.S. Dhillon, S. Mallela, and D.S. Modha, "Information-Theoretic Co-Clustering," Proc. ACM SIGKDD, 2003.
14. M.N. Garofalakis, A. Gionis, R. Rastogi, S. Seshadri, and K. Shim, "Xtract: A System for Extracting Document Type Descriptors from Xml Documents," Proc. ACM SIGMOD, 2000.
15. D. Gibson, K. Punera, and A. Tomkins, "The Volume and Evolution of Web Page Templates," Proc. 14th Int'l Conf. World Wide Web (WWW), 2005.
16. K. Lerman, L. Getoor, S. Minton, and C. Knoblock, "Using the Structure of Web Sites for Automatic Segmentation of Tables," Proc. ACM SIGMOD, 2004.
17. B. Long, Z. Zhang, and P.S. Yu, "Co-Clustering by Block Value Decomposition," Proc. ACM SIGKDD, 2005.
18. F. Pan, X. Zhang, and W. Wang, "Crd: Fast Co-Clustering on Large Data Sets Utilizing Sampling-Based Matrix Decomposition," Proc. ACM SIGMOD, 2008.
19. M.D. Plumbley, "Clustering of Sparse Binary Data Using a Minimum Description Length Approach," http://www.elec. qmul.ac.uk/staffinfo/markp/, 2002.
20. J. Rissanen, "Modeling by Shortest Data Description," Automatica, vol. 14, pp. 465-471, 1978.
21. J. Rissanen, Stochastic Complexity in Statistical Inquiry. World Scientific, 1989.
22. K. Vieira, A.S. Da Silva, N. Pinto, E.S. De Moura, J.M.B. Cavalcanti, and J. Freire, "A Fast and Robust Method for Web Page Template Detection and Removal," Proc. 15th ACM Int'l Conf. Information and Knowledge Management (CIKM), 2006.
23. Y. Zhai and B. Liu, "Web Data Extraction Based on Partial Tree Alignment," Proc. 14th Int'l Conf. World Wide Web (WWW), 2005.
24. H. Zhao, W. Meng, Z. Wu, V. Raghavan, and C. Yu, "Fully Automatic Wrapper Generation for Search Engines," Proc. 14th Int'l Conf. World Wide Web (WWW), 2005.
25. H. Zhao, W. Meng, and C. Yu, "Automatic Extraction of Dynamic Record Sections from Search Engine Result Pages," Proc. 32nd Int'l Conf. Very Large Data Bases (VLDB), 2006.
26. S. Zheng, D. Wu, R. Song, and J.-R. Wen, "Joint Optimization of Wrapper Generation and Template Detection," Proc. ACM SIGKDD, 2007.