Evaluating a collaborative IT based research and development project

Evaluation and Program Planning

Volumn 40, Issue , 2013, Pages 27-41

  Khan, Zaheer ^a   Ludlow, David ^a   Caceres, Santiago ^b  

^a UNIVERSITY OF THE WEST OF ENGLAND   (United Kingdom)

^b ETRA I+D   (Spain)

Author keywords

Collaborative research project evaluation; Integrated project research and development; Participatory evaluation methodology; Questionnaire and user based evaluation; User and technical evaluation

Indexed keywords

DEVELOPMENT PROJECT; INFORMATION TECHNOLOGY; INTEGRATED APPROACH; METHODOLOGY; PARTICIPATORY APPROACH; PLANNING METHOD; RESEARCH AND DEVELOPMENT; STAKEHOLDER;

ARTICLE; COLLABORATIVE RESEARCH PROJECT EVALUATION; COOPERATION; EUROPE; HEALTH CARE QUALITY; HUMAN; INTEGRATED PROJECT RESEARCH AND DEVELOPMENT; MEDICAL INFORMATICS; ORGANIZATION AND MANAGEMENT; PARTICIPATORY EVALUATION METHODOLOGY; PROGRAM DEVELOPMENT; PROGRAM EVALUATION; PUBLIC-PRIVATE PARTNERSHIP; QUESTIONNAIRE AND USER-BASED EVALUATION; RESEARCH; USER AND TECHNICAL EVALUATION;

COLLABORATIVE RESEARCH PROJECT EVALUATION; INTEGRATED PROJECT RESEARCH AND DEVELOPMENT; PARTICIPATORY EVALUATION METHODOLOGY; QUESTIONNAIRE AND USER-BASED EVALUATION; USER AND TECHNICAL EVALUATION;

COOPERATIVE BEHAVIOR; EUROPE; HUMANS; MEDICAL INFORMATICS; PROGRAM DEVELOPMENT; PROGRAM EVALUATION; PUBLIC-PRIVATE SECTOR PARTNERSHIPS; QUALITY INDICATORS, HEALTH CARE; RESEARCH;

EID: 84879541258     PISSN: 01497189     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.evalprogplan.2013.04.004     Document Type: Article

References (38)

1. Babar M.A., Zhu L., Jeffery R. A framework for classifying and comparing software architecture evaluation methods. Proceedings of the 15th Australian software engineering conference (ASWEC), IEEE computer society 2004, 309-319.
2. Basili V., Caldiera G., Rombach D. Goal question metric approach, encyclopedia of software engineering 1994, 528-532. John Wiley & Sons, Inc., New York.
3. Basili V., Donzelli P., Asgari S. A unified model of dependability: Capturing dependability in context. IEEE Software 2004, 21(6):19-25.
4. Briand L., Morasca S., Basili V. An operational process for goal-driven definition of measures. IEEE Transactions on Software Engineering 2002, 28(12):1106-1125.
5. Brooke J. SUS - a quick and dirty usability scale. Usability evaluation in industry 1996, 189-194. Taylor and Francis, London, UK. P. Jordan, B. Thomas, B. Weerdmeester, A.L. McClelland (Eds.).
6. Bryson J., Patton M., Bowman R. Working with evaluation stakeholders: A rationale, step-wise approach and toolkit. Evaluation and Program Planning 2011, 34(1):1-12. 0149-7189, doi:10.1016/j.evalprogplan.2010.07.001.
7. CASCADOSS Consortium (Ed.) (2007). Evaluation criteria for Open Source GIS and RS software [Online], URL access: Last accessed 04.12.12. http://www.cascadoss.eu/en/index.php%3Foption=com_content%26task=view%26id=9%26Itemid=11.
8. Dobrica L., Niemela E. A survey on software architecture analysis methods. IEEE Transactions on Software Engineering 2002, 28(7):638-653.
9. Falessi D., Babar M.A., Cantone G., Kruchten P. Applying empirical software engineering to software architecture: challenges and lessons learned. Empirical Software Engineering 2009, 15(3):250-276.
10. Fichtinger A., Rix J., Schäffler U., Michi I., Gone M., Reitz Th. Data Harmonisation Put into Practice by the HUMBOLDT Project. International Journal of Spatial Data Infrastructures Research 2011, 6:234-260.
11. Finstad K. The usability metric for user experience. Interacting with Computers 2010, 22(2010):232-327.
12. GMES. (2012). Global monitoring for environment and security, URL access: Last accessed 04.12.12. http://www.gmes.info/.
13. Gustedt, J., Jeannot, E., & Quinson, M. (2009). Experimental validation in large-scale systems: a survey of methodologies, INRIA NANCY - GRAND EST Report: inria-00364180, version 1, 25 February 2009, p. 16.
14. HUDSON. (2010). Extensible continuous integration server, URL Access: Last accessed 04.12.12. http://hudson-ci.org/.
15. HUMBOLDT Framework Tools. (2010). URL Access: Last accessed 04.12.12. http://www.esdi-humboldt.eu/files/0982-a5_2-d3__3_0__a_lightweight_introduction-fhg-igd-004-final.pdf.
16. HUMBOLDT Handbook of Standards. (2010). URL Access: Last accessed 04.12.12. http://www.esdi-humboldt.eu/files/0996-a3_6-d4_handbook_of_standards_-_final_version-ethz-002-final.pdf.
17. HUMBOLDT Scenarios. (2010). URL Access: Last accessed 04.12.12. http://www.esdi-humboldt.eu/scenarios.html.
18. HUMBOLDT Technical Evaluation. (2010). URL Access: Last accessed 04.12.12. http://www.esdi-humboldt.eu/files/1109-a10_4-d1_report_on_end-user_or_developer_software-fomi-001-final.pdf.
19. INSPIRE. (2007). URL Accessed Last accessed 04.12.12. http://inspire.jrc.ec.europa.eu/.
20. ISO/IEC 9126-1. (2001). Software engineering - Product quality - Part 1: Quality model.
21. Juristo N. Structuring the knowledge-based systems evaluation process. Proceedings of Intelligent Information Systems 1997, 97:266-270.
22. Juristo N., Morant J. Common framework for the evaluation process of KBS and conventional software. Elsevier Knowledge-Based Systems 1998, 11(2):145-159.
23. Khan Z., Odeh M., McClatchey R. Bridging the gap between business process models and service-oriented architectures with reference to the grid environment. International Journal of Grid and Utility Computing 2011, 2(4):253-283.
24. Khan Z., Ludlow D., Rix J. Applying the criteria indicators and metrics evaluation method on ICT research: The HUMBOLDT project. Research Evaluation 2013, 22(1):30-40.
25. Kotonya G., Sommerville I. Requirements engineering: Processes and techniques, ISBN # 0-471-97208-8 2002, John Wiley & Sons Inc., Chichester, UK.
26. Lindvall M., Rus I., Donzelli P., Memon A., Zelkowitz M., Betin-can A., et al. Experimenting with software testbeds for evaluating new technologies. Empirical Software Engineering 2007, 12(4):417-444. 10.1007/s10664-006-9034-0.
27. OGC. (2012). Open Geospatial Consortium, URL Access: Last accessed 04.12.12. http://www.opengeospatial.org/.
28. Ostreika, A. (2010). Evaluation of benefit analysis - The HUMBOLDT project deliverable A10.5, URL Access: Last accessed 04.12.12. http://www.esdi-humboldt.eu/files/0885-a10_5_evaluation_of_benefit_analysis-ktc-004-final.pdf.
29. RESPECT Consortium (Ed.) (1998). User-centred requirements handbook. Version 3.3. URL Access: Last accessed 04.12.12. http://www.idemployee.id.tue.nl/g.w.m.rauterberg/lecturenotes/UserCenteredRequirementsHandbook.pdf.
30. RISE (Ed.) (2007). Report on the operation of the RISE testing system. URL Access: Last accessed 04.12.12. http://www.eurogeographics.org/eng/documents/RISE36_Testing_System_Report_V1.0.pdf.
31. Sommerville I. Software engineering, ISBN # 9780321313799 2006, Addison-Wesley, Wokingham, UK.
32. Sonar. (2010). Code Analysis server, URL Access: Last accessed 01.12.12. http://docs.codehaus.org/display/SONAR/Documentation.
33. The HUMBOLDT Project. (2006-2011). Sixth Framework Programme, European Commission, URL Access: Last accessed 04.12.12. http://www.esdi-humboldt.eu/home.html.
34. The HUMBOLDT Project Community. (2006-2011). URL Access: Last accessed 01.12.12. http://community.esdi-humboldt.eu/.
35. The HUMBOLDT training platform. (2006-2011). URL Access: or Last accessed 01.11.12. http://www.gisig.it/humboldt/training/.
36. Tiits, K. (2003). Usability of Geographic Information Systems in Internet. A case study of Journey Planners. Master thesis at Tartu University, Institute of Geography. URL Access: Last accessed 04.12.12. http://www.hot.ee/kyllitiits/Kylli_Tiits_MSc_thesis.pdf.
37. Villa P., Molina R., Gomarasca M., Roccatagliata E. Harmonisation requirements and capabilities towards a European spatial data infrastructure (ESDI): The HUMBOLDT protected areas scenario. International Journal of Digital Earth 2012, 5/5(417):38.
38. WebPark consortium (Ed.) (2002). Usability testing template, D2.2.3, V05. URL: Last accessed 04.12.12. http://mp.mountaintrip.eu/uploads/media/deliverable/webpark_d3.pdf.