Application-specific disk I/O optimisation for a search engine

Parallel and Distributed Computing, Applications and Technologies, PDCAT Proceedings

Volumn , Issue , 2008, Pages 399-404

  Jia, Xiangfei ^a   Trotman, Andrew ^a   O'Keefe, Richard ^a   Huang, Zhiyi ^a  

^a UNIVERSITY OF OTAGO   (New Zealand)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER OPERATING SYSTEMS; COMPUTER SOFTWARE; DISKS (STRUCTURAL COMPONENTS); INFORMATION RETRIEVAL; INTERNET; SEARCH ENGINES; TELECOMMUNICATION NETWORKS; WORLD WIDE WEB;

APPLICATION LEVELS; APPLICATION SPECIFICS; OPERATING SYSTEMS; OPTIMISATION;

APPLICATIONS;

EID: 58249117167     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/PDCAT.2008.61     Document Type: Conference Paper

References (28)

1. http://en.wikipedia.org/wiki/SMART-Information-Retrieval -System, July 2008.
2. J. Allan, B. Carterette, J. Aslam, V. Pavlu, B. Dachev, and E. Kanoulas. Million query track 2007 overview. In TREC, 2008.
3. V. N. Anh, O. de Kretser, and A. Moffat. Vector-space ranking with effective early termination. In SIGIR '01, 2001.
4. V. N. Anh and A. Moffat. Inverted index compression using word-aligned binary codes. Inf. Retr., 8(1):151-166, 2005.
5. V. N. Anh and A. Moffat. Improved word-aligned binary compression for text indexing. IEEE T Knowl Data En, 18(6):857-861, 2006.
6. V. N. Anh and A. Moffat. Pruned query evaluation using pre-computed impacts. In SIGIR '06, pages 372-379, 2006.
7. R. Baeza-Yates, A. Gionis, F. Junqueira, V. Murdock, V. Plachouras, and F. Silvestri. The impact of caching on search engines. In SIGIR '07, pages 183-190, 2007.
8. D. P. Bovet and M. E. Cesati. Understanding the Linux Kernel, 3rd Edition. O'Reilly, November 2005.
9. A. R. Butt, C. Gniady, and Y. C. Hu. The performance impact of kernel prefetching on buffer cache replacement algorithms. IEEE Trans. Comput., 56(7):889-908, 2007.
10. N. Craswell and D. Hawking. Overview of the TREC-2002 web track. In TREC, 2002.
11. X. Ding, S. Jiang, and F. Chen. A buffer cache management scheme exploiting both temporal and spatial localities. Trans. Storage, 3(2):5, 2007.
12. T. Fagni, R. Perego, F. Silvestri, and S. Orlando. Boosting the performance of web search engines: Caching and prefetching query results by exploiting historical usage data. ACM Trans. Inf. Syst., 24(1):51.-78, 2006.
13. S. Jiang, X. Ding, F. Chen, E. Tan, and X. Zhang. DULO: An effective buffer cache management scheme to exploit both temporal and spatial locality. 2005.
14. T. Johnson and D. Shasha. 2Q: A low overhead high performance buffer management replacement algorithm. In VLDB '94, pages 439-450, 1994.
15. R. Lempel and S. Moran. Predictive caching and prefetching of query results in search engines. In WWW '03, 2003.
16. E. P. Markatos. On caching search engine query results. Computer Communications, 24(2): 137-143, 2001.
17. A. Moffat and J. Zobel. Self-indexing inverted files for fast text retrieval. ACM Trans. Inf. Syst., 14(4):349-379, 1996.
18. A. Moffat, J. Zobel, and R. Sacks-Davis. Memory efficient ranking. Inf. Process. Manage., 30(6):733-744, 1994.
19. R. Pai, B. Pulavarty, and M. Cao. Linux 2.6 performance improvement through readahead optimization. In Proceedings of the Linux Symposium, 2004.
20. S. L. Pratt and D. A. Heger. Workload dependent performance evaluation of the linux 2.6 I/O schedulers. In Proceedings of the Linux Symposium, 2004.
21. S. E. Robertson, S. Walker, M. Hancock-Beaulieu, A. Gull, and M. Lau. Okapi at TREC-4. In TREC-4, 1992.
22. Seagate. Barracuda 7200.10 PATA, August 2007.
23. S. Seelam, R. Romero, and P. Teller. Enhancements to linux I/O scheduling. In Proceedings of the Linux Symposium, 2005.
24. T. Strohman and W. B. Croft. Efficient document retrieval in main memory. In SIGIR '07, pages 175-182, 2007.
25. A. Trotman. Compressing inverted files. Inf. Retr., 6(1):5-19, 2003.
26. H. E. Williams and J. Zobel. Compressing integers for fast file access. The Computer Journal, 42(3): 193-201, 1999.
27. F. Wu, H. Xi, and J. Li. Linux readahead: less tricks fo more. In Proceedings of the Linux Symposium, 2007.
28. J. Zobel, A. Moffat, and K. Ramamohanarao. Inverted files versus signature files for text indexing. ACM Trans. Database Syst., 23(4):453-490, 1998.