Optimal subspace dimensionality for k-nearest-neighbor queries on clustered and dimensionality reduced datasets with SVD

Multimedia Tools and Applications

Volumn 40, Issue 2, 2008, Pages 241-259

  Thomasian, Alexander ^a   Li, Yue ^b   Zhang, Lijuan ^c  

^a Thomasian and Associates   (United States)

^b AIG Software   (United States)

^c AMICAS Inc   (United States)

Author keywords

Clustering; Content based retrieval; Dimensionality reduction; High dimensional data; Multimedia databases; Nearest neighbor search; Singular value decomposition

Indexed keywords

CHLORINE COMPOUNDS; CLUSTER ANALYSIS; CLUSTERING ALGORITHMS; DATABASE SYSTEMS; FLOW OF SOLIDS; IMAGE ENHANCEMENT; IMAGE RETRIEVAL; INDEXING (OF INFORMATION); NUMBER THEORY; SINGULAR VALUE DECOMPOSITION; VECTORS;

BACKGROUND INFORMATION; CLUSTERING; CLUSTERING METHODS; CONTENT-BASED IMAGE RETRIEVAL; CONTENT-BASED RETRIEVAL; DATA-SETS; DIMENSIONALITY REDUCTION; DR METHOD; FEATURE VECTOR; FEATURE VECTORS; GLOBAL CORRELATIONS; HIGH DIMENSIONAL DATA; HYBRID METHODS; K-MEANS CLUSTERING METHODS; K-NEAREST NEIGHBOR QUERIES; K-NEAREST NEIGHBORS; LOWER COST; MINIMUM COSTS; MINIMUM INFORMATION LOSS; MULTIMEDIA DATABASES; NEAREST NEIGHBOR SEARCH; NORMALIZED MEAN SQUARE ERROR; OPTIMAL SUBSPACE; REAL-WORLD DATASETS; REDUCED DATA; RETRIEVAL ACCURACY; SEARCH SPACES; SINGULAR VALUE DECOMPOSITION METHODS; SR-TREE; TARGET IMAGING;

DATA REDUCTION;

EID: 51849154226     PISSN: 13807501     EISSN: 15737721     Source Type: Journal    
DOI: 10.1007/s11042-008-0206-3     Document Type: Article

References (41)

1. Agrawal R, Gehrke J, Gunopulos D, Raghavan P (1998) Automatic subspace clustering of high dimensional data for data mining applications. In: Proc. ACM SIGMOD int'l conf. on management of data, Seattle, WA, June 1998, pp 94-105
2. W Barbara W DuMouchel C Faloutsos PJ Haas JM Hellerstein Y Ioannidis HV Jagadish T Johnson R Ng V Poosala KA Ross KC Sevcik 1997 The New Jersey data reduction report Data Eng Bull 20 4 3 42 December
3. Beckmann N, Kriegel H-P, Schneider R, Seeger B (1990) The R*-tree: an efficient and robust access method for points and rectangles. In: Proc. ACM SIGMOD int'l conf. on management of data, Atlantic City, NJ, May 1990, pp 322-331
4. LD Bergman V Castelli C-S Li JR Smith 2000 SPIRE: a digital library for scientific information Int J Digit Libr 3 1 85 99
5. C Bohm 2000 A cost model for query processing in high-dimensional data space ACM Trans Database Syst 25 2 129 178
6. C Bohm S Berchtold DA Keim 2001 Searching in high-dimensional spaces - index structures for improving the performance of multimedia databases ACM Comput Surv 33 3 322 373 September
7. Castelli V (2002) Multidimensional indexing structures for content-based retrieval. In: Castelli V, Bergman LD (eds) Image databases: search and retrieval of digital imagery. Wiley-Interscience, New York, pp 373-434
8. Castelli V, Bergman LD (eds) (2002) Image databases: search and retrieval of digital imagery. Wiley-Interscience, New York
9. V Castelli A Thomasian CS Li 2003 CSVD: clustering and singular value decomposition for approximate similarity search in high dimensional spaces IEEE Trans Knowl Data Eng 14 3 671 685 June
10. Chakrabarti K, Mehrotra S (1999) The hybrid tree: an index structure for high dimensional feature spaces. In: Proc. 15th IEEE int'l conf. on data eng. - ICDE, Sidney, Australia, March 1999, pp 440-447
11. Chakrabarti K, Mehrotra S (2000) Local dimensionality reduction: a new approach to indexing high dimensional space. In: Proc. int'l conf. on very large data bases - VLDB, Cairo, Egypt, August 2000, pp 89-100
12. Faloutsos C (1996) Searching multimedia databases by content. Kluwer Academic, Boston, MA
13. Faloutsos C, Kamel I (1994) Beyond uniformity and independence: analysis of the R-tree using the concept of fractal dimension. In: Proc. ACM symp. on principles of database systems - PODS, Minneapolis, MN, June 1994, pp 4-13
14. F Farnstrom J Lewis C Elkan 2000 Scalability for clustering algorithms "revisited" ACM SIGKDD Explor Newslett 2 1 51 57
15. V Gaede O Gunther 1998 Multidimensional access methods ACM Comput Surv 30 2 170 231
16. TF Gonzalez 1985 Clustering to minimize the maximum intercluster distance Theor Comp Sci 38 293 306
17. Han J, Kamber M (2006) Data mining: concepts and techniques. Morgan-Kaufmann, San Francisco
18. Hjaltason GR, Samet H (1995) Ranking in spatial databases. In: Proc. 4th symp. advances in spatial databases, lecture notes in computer science 951, Springer, Berlin Heidelberg New York, pp 83-95
19. Jin H, Ooi BC, Shen H, Yu C (2003) An adaptive and efficient dimensionality reduction algorithm for high-dimensional indexing. In: Proc. 19th IEEE int'l conf. on data engineering - ICDE, Bangalore, India, March 2003, pp 87-100
20. Jolliffe IT (2002) Principal component analysis. Springer, Berlin Heidelberg New York
21. Katayama N, Satoh S (1997) The SR-tree: an index structure for high dimensional nearest neighbor queries. In: Proc. ACM SIGMOD int'l conf. on management of data, Tucson, AZ, May 1997, pp 369-380
22. B Kim S Park 1986 A fast k-nearest-neighbor finding algorithm based on the ordered partition IEEE Trans Pattern Anal Mach Intell 8 6 761 766
23. Korn F, Jagadish HV, Faloutsos C (1997) Efficiently supporting ad hoc queries in large datasets of time sequences. In: Proc. ACM SIGMOD int'l conf. on management of data, Tucson, AZ, May 1997, pp 289-300
24. F Korn B Pagel C Faloutsos 2001 On the "dimensionality curse" and the "self-similarity blessing" IEEE Trans Knowl Data Eng 13 1 96 111
25. Korn F, Sidiropoulos N, Faloutsos C, Siegel E, Protopapas Z (1996) Fast nearest neighbor search in medical image databases. In: Proc. 22nd int'l conf. on very large data bases - VLDB, Mumbai, India, September 1996, pp 215-226
26. F Korn N Sidiropoulos C Faloutsos E Siegel Z Protopapas 1998 Fast and effective retrieval of medical tumor shapes: nearest neighbor search in medical image databases IEEE Trans Knowl Data Eng 10 6 889 904
27. Li Y (2004) Efficient similarity search in high-dimensional data spaces. Ph.D. dissertation, Computer Science Department, New Jersey Institute of Technology - NJIT, Newark, NJ, May
28. McCallum A, Nigam K, Unger LH (2000) Efficient clustering of high-dimensional data sets with applications to reference matching. In: Proc. 6th ACM SIGKDD int'l conf. on knowledge discovery and data mining, Boston, MA, 20-23 August 2000, pp 169-178
29. Niblack W, Barber R, Equitz W, Flickner M, Glasman EH, Petkovic D, Yanker P, Faloutsos C, Taubin G (1993) The QBIC project: querying images by content, using color, texture, and shape. In: Niblack W (ed) Proc. SPIE vol. 1908: storage and retrieval for image and video databases, SPIE, Bellingham, pp 173-187
30. L Parsons E Haque H Liu 2004 Subspace clustering for high dimensional data: a review ACM SIGKDD Explor Newslett 6 1 90 105
31. Roussopoulos N, Kelley S, Vincent F (1995) Nearest neighbor queries. In: Proc. ACM SIGMOD int'l conf. on management of data, San Jose, CA, June 1995, pp 71-79
32. Samet H (2007) Fundamentals of multidimensional and metric data structure. Morgan- Kaufmann, San Francisco
33. Singh AK, Lang CA (2001) Modeling high-dimensional index structures using sampling. In: Proc. ACM SIGMOD int'l conf. on management of data, Santa Barbara, CA, June 2001, pp 389-400
34. Theodoridis Y, Sellis T (1996) A model for the prediction of R-tree performance. In: Proc. ACM symp. on principles of database systems - PODS, Montreal, Quebec, Canada, June 1996, pp 161-171
35. Thomasian A, Castelli V, Li CS (1998) RCSVD: recursive clustering and singular value decomposition for approximate high-dimensionality indexing. In: Proc. conf. on information and knowledge management - CIKM, Baltimore, MD, November 1998, pp 201-207
36. Thomasian A, Li Y, Zhang L (2003) Performance comparison of local dimensionality reduction methods. Technical report ISL-2003-01, Integrated Systems Lab, Computer Science Dept., New Jersey Institute of Technology
37. A Thomasian Y Li L Zhang 2005 Exact k-NN queries on clustered SVD datasets Inf Process Lett 94 6 247 252 July
38. A Thomasian L Zhang 2006 Persistent semi-dynamic ordered partition index Comput J 49 6 670 684 June
39. White DA, Jain R (1996) Similarity indexing: algorithms and performance. In: Storage and retrieval for image and video databases SPIE, vol 2670. SPIE, San Jose, CA, pp 62-73
40. Yi BK, Faloutsos C (2000) Fast time sequence indexing for arbitrary Lp norms. In: Proc. 26th int'l conf. on very large data bases - VLDB, Cairo, Egypt, September 2000, pp 385-394
41. Zhang L (2005) High-dimensional indexing methods utilizing clustering and dimensionality reduction. Ph.D. dissertation, Computer Science Department, New Jersey Institute of Technology - NJIT, Newark, NJ, May