Towards standard test collections for the empirical evaluation of semantic web service approaches

International Journal of Semantic Computing

Volumn 2, Issue 3, 2008, Pages 381-402

  Küster, Ulrich ^a   König Ries, Birgitta ^a  

^a FRIEDRICH SCHILLER UNIVERSITY JENA   (Germany)

Author keywords

evaluation; Semantic web services; test collections

Indexed keywords

SEMANTIC WEB; WEBSITES;

EMPIRICAL EVALUATIONS; EVALUATION; STANDARD TESTS; STATE OF THE ART; TEST COLLECTION;

WEB SERVICES;

EID: 77954398720     PISSN: 1793351X     EISSN: 17937108     Source Type: Journal    
DOI: 10.1142/S1793351X0800052X     Document Type: Article

References (19)

1. R. Akkiraju, J. Farrell, J. Miller, M. Nagarajan, M.-T. Schmidt, A. Sheth and K. Verma, Web service semantics-WSDL-S, W3CMember Submission, 7 November 2005.
2. J. de Bruijn, C. Bussler, J. Domingue, D. Fensel, M. Hepp, U. Keller, M. Kifer, B. Konig-Ries, J. Kopecky, R. Lara, H. Lausen, E. Oren, A. Polleres, D. Roman, J. Scicluna and M. Stollberg, Web service modeling ontology (WSMO), W3C Member Submission, 3 June 2005.
3. J. Farrell and H. Lausen (eds.), Semantic annotations for WSDL and XML Schema, W3C Recommendation, 28 August 2007.
4. Y. Ganjisaffar, H. Abolhassani, M. Neshati and M. Jamali, A similarity measure for OWL-S annotated web services, Proc. of 2006 IEEE/WIC/ACM Int. Conf. on Web Intelligence, 2006, pp. 621-624.
5. S. Grimm, B. Motik and C. Preist, Matching semantic service descriptions with local closed-world reasoning, Proc. of 3rd European Semantic Web Conference (ESWC 2006), Budva, Montenegro, June 2006.
6. D. Harman, Overview of the first Text REtrieval Conference (TREC-1), Proceedings of the first Text REtrieval Conference, Gaithersbury, MD, USA, November 1992.
7. F. Kaufer and M. Klusch, Performance of hybrid WSML service matching with WSMO-MX: Preliminary results, Proc. of First International Joint Workshop SMR2 on Service Matchmaking and Resource Retrieval in the Semantic Web at the 6th International Semantic Web Conference, Busan, South Korea, November 2007.
8. M. Klein, B. Konig-Ries and M. Mussig, What is needed for semantic service descriptions-A proposal for suitable language constructs, Int. J. Web & Grid Services 1(3/4) (2005) 328-364.
9. M. Klusch, Semantic web service coordination, in M. Schumacher and H. Helin (eds.), CASCOM-Intelligent Service Coordination in the Semantic Web, chapter 4, Springer, 2008.
10. M. Klusch, B. Fries, M. Khalid and K. Sycara, OWLS-MX: Hybrid OWL-S service matchmaking, Proc. of First International AAAI Fall Symposium on Agents and the Semantic Web, Arlington, Vriginia, USA, November 2005.
11. M. Klusch, P. Kapahnke and B. Fries, Hybrid semantic web service retrieval: A case study with OWLS-MX, Proc. of 2nd IEEE International Conference on Semantic Computing, Santa Clara, CA, USA, August 2008.
12. M. Klusch and X. Zhing, Deployed semantic services for the common user of the web: A reality check, Proc. of 2nd IEEE International Conference on Semantic Computing, Santa Clara, CA, USA, August 2008.
13. U. Kuster and B. Konig-Ries, Evaluating semantic web service matchmaking effectiveness based on graded relevance, Proc. of 2nd International Workshop on Service Matchmaking and Resource Retrieval in the Semantic Web at the 7th International Semantic Web Conference, Karlsruhe, Germany, October 2008.
14. U. Kuster, H. Lausen and B. Konig-Ries, Evaluation of semantic service discovery-a survey and directions for future research, Postproceedings of the 2nd Workshop on Emerging Web Services Technology at the 5th IEEE European Conference on Web Services, Halle (Saale), Germany, November 2007.
15. D. Martin, M. Burstein, J. Hobbs, O. Lassila, D. McDermott, S. McIlraith, S. Narayanan, M. Paolucci, B. Parsia, T. Payne, E. Sirin, N. Srinivasan and K. Sycara, OWL-S: Semantic markup for web services, W3C Member Submission, 22 November 2004.
16. S. A. McIlraith, T. C. Son and H. Zeng, Semantic web services, IEEE Intelligent Systems 16(2) (2001) 46-53.
17. C. Petrie, U. Kuster and T. Margaria-Steffen, W3C SWS challenge testbed incubator methodology report, W3c incubator report, W3C, March 2008. available online at http://www.w3.org/2005/Incubator/swsc/XGR-SWSC/.
18. W. F. Tichy, Should computer scientists experiment more? IEEE Computer 31(5) (1998) 32-40.
19. W. F. Tichy, P. Lukowicz, L. Prechelt and E. A. Heinz, Experimental evaluation in computer science: A quantitative study, Journal of Systems and Software 28(1) (1995) 9-18.