Guidelines for empirical evaluations of conceptual modeling grammars

Journal of the Association for Information Systems

Volumn 10, Issue 6, 2009, Pages 495-532

  Burton Jones, Andrew ^a   Wand, Yair ^a   Weber, Ron ^b  

^a UNIVERSITY OF BRITISH COLUMBIA   (Canada)

^b MONASH UNIVERSITY   (Australia)

Author keywords

Conceptual modeling grammars; Experimental design; Grammar performance; Grammar quality; Ontology; Script creation; Script interpretation

Indexed keywords

DESIGN OF EXPERIMENTS; SEMANTICS;

CONCEPTUAL MODEL; DOMAIN SEMANTICS; EFFECTIVENESS AND EFFICIENCIES; EMPIRICAL EVALUATIONS; EMPIRICAL RESEARCH; GRAMMAR PERFORMANCE; SCRIPT CREATION; SCRIPT INTERPRETATION;

ONTOLOGY;

EID: 70749130748     PISSN: None     EISSN: 15369323     Source Type: Journal    
DOI: 10.17705/1jais.00201     Document Type: Article

References (52)

1. Agarwal, R., De, P., and Sinha, A. "Comprehending object and process models: An empirical study,"IEEE Transactions on Software Engineering (25:4) 1999, pp. 541-556.
2. Aguirre-Urreta, M.I. and Marakas, G. "Comparing conceptual modeling techniques: A critical review of the EER vs. OO empirical literature," The DATA BASE for Advances in Information Systems (39:2) May 2008, pp. 9-32.
3. Allen, G. and March, S.T. "The effects of state-based and event-based data representation on user performance in query formulation tasks," MIS Quarterly (30:2) 2006, pp. 269-290.
4. Allen, G. and Parsons, J. "Is query reuse potentially harmful? Anchoring and adjustment in adaptingexisting database queries," Information Systems Research in press 2008.
5. Angeles, P. A. Dictionary of Philosophy. Harper Perennial, New York, NY,: 1981.
6. Becker, J., Niehaves, B. and Pfeiffer, D. "Ontological evaluation of conceptual models: A linguistic interpretivist approach," Scandinavian Journal of Information Systems (20:2) 2008, pp. 83-110.
7. Berners-Lee, T., Hendler, J., and Lassila, O. "The semantic web," Scientific American (284:5) 2001,pp. 34-43.
8. Bodart, F., Patel, A., Sim, M., and Weber, R. "Should optional properties be used in conceptualmodeling? A theory and three empirical tests," Information Systems Research (12:4) Dec 2001, pp. 384-405.
9. Bowen, P.L., O'Farrell, R.A., and Rohde, F.H. "Analysis of competing data structures: Doesontological clarity produce better end user query performance," Journal of the Association for Information Systems (7:8) 2006, pp. 514-544.
10. Burton-Jones, A. and Meso, P. "Conceptualizing systems for understanding: An empirical test ofdecomposition principles in object-oriented analysis," Information Systems Research (17:1) 2006, pp. 38-60.
11. Burton-Jones, A. and Meso, P. "The effects of decomposition quality and multiple forms ofinformation on novices' understanding of a domain from a conceptual model," Journal of the Association for Information Systems (9:12) 2008, pp. 748-802.
12. Burton-Jones, A., and Weber, R. "Understanding relationships with attributes in entity-relationship diagrams," in Proceedings of the 20th International Conference on Information Systems, P. De and J. I. DeGross (eds.), Charlotte, NC, 1999, pp. 214-228.
13. Corral, K., Schuff, D., and St. Louis, R.D. "The impact of alternative diagrams on the accuracy of recall: A comparison of star-schema diagrams and entity-relationship diagrams," Decision Support Systems (42) 2006, pp. 450-468.
14. Gemino, A., and Wand, Y. "A framework for empirical evaluation of conceptual modeling techniques," Requirements Engineering (9:4) 2004, pp. 248-260.
15. Gemino, A., and Wand Y. "Complexity and clarity in conceptual modeling: Comparison of mandatory and optional properties," Data & Knowledge Engineering (55:3) 2005 pp. 301-326.
16. Gomez-Perez, A., Fernandez-Lopez, M., and Corcho, O. Ontological engineering, Springer-Verlag, London, 2004.
17. Hadar, I., and Soffer, P. "Variations in conceptual modeling: Classification and ontological analysis,"Journal of the Association for Information Systems (7:8) 2006, pp. 568-592.
18. Hirschheim, R., Klein, H., and Lyytinen, K. Information systems development and data modeling: Conceptual foundations and philosophical foundations. Cambridge, UK: Cambridge University Press, 1995.
19. Irwin, G., and Turk, D. "An ontological analysis of use case modeling grammar," Journal of theAssociation for Information Systems (6:1) 2005, pp. 1-36.
20. Kent, W. Data and reality, 1st Books Library, 2000 (originally published 1978).
21. Khatri, V., Vessey, I., Ramesh, V., Clay, P., and Park, S.-J. "Understanding conceptual schemas: Exploring the role of application and IS domain knowledge," Information Systems Research (17:1) 2006, pp. 81-99.
22. Kim, J., Hahn, J., and Hahn, H. "How do we understand a system with (so) many diagrams?Cognitive integration processes in diagrammatic reasoning," Information Systems Research (11:3) 2000, pp. 284-303.
23. Krogstie, J., Sindre, G., and Jørgensen, H. "Process models representing knowledge for action: Arevised quality framework," European Journal of Information Systems (15) 2006, pp. 91-102.
24. Kung, C.H., and Sølvberg, A. "Activity modelling and behavior modelling," in: Information systems design methodologies: Improving the practice, T.W. Olle, H.G. Sol and A.A. Verrijn-Stuart (eds.), IFIP, Amsterdam, North-Holland, 1986, pp. 145-171.
25. Larkin, J., and Simon, H. "Why a diagram is (sometimes) worth ten thousand words," CognitiveScience (11) 1987, pp. 65-99.
26. Lee, A.S. and Baskerville, R.L. "Generalizing generalizability in information systems research,"Information Systems Research (14: 3) September 2003, pp. 221-243.
27. Lindland, O.I., Sindre, G. and Sølvberg, A. "Understanding quality in conceptual modeling." IEEESoftware (11) 1994, pp. 42-49.
28. Maes, A. and Poels, G. "Evaluating quality of conceptual modeling scripts based on userperceptions," Data & Knowledge Engineering (63) 2007, pp. 701-724.
29. th International Conference on Information Systems, Seattle WA, 2003, pp. 207-221.
30. Nordbotten, J.C. and Crosby, M.E. "The effect of graphic style on data model interpretation,"Information Systems Journal (9) 1999, pp. 139-155.
31. Norman, D. "Cognitive Engineering," in User Centered Design: New Perspectives on HumanComputer Interaction. Lawrence Erlbaum Associates, Hillsdale, 1986, pp. 31-61.
32. Parker, F. and Riley, K. Linguistics for non-linguists: A primer with exercises, 4th ed. Pearson/Allyn and Bacon, Boston, 2005.
33. Parsons, J. "Effects of local versus global schema diagrams on verification and communication inconceptual data modeling," Journal of Management Information Systems (19:3) 2003, pp.155-184.
34. Parsons, J. and Cole, L. "What do the pictures mean? Guidelines for experimental evaluation ofrepresentation fidelity in diagrammatical conceptual modeling techniques," Data & KnowledgeEngineering (55) 2005, pp. 327-342.
35. Parsons, J. and Wand, Y. "Using cognitive principles to guide classification in information systemsmodeling," MIS Quarterly (32:4) 2008, pp. 839-868.
36. Patel, V.L., Allen, V.G., Arocha, J.F., and Shortliffe, E.H. "Representing clinical guidelines in GLIF: Individual and collaborative expertise," Journal of the American Medical Informatics Association (5:5) 1998, pp. 467-483.
37. Poels, G., Maes, A., Gailly, F., and Paemeleire, R. "The pragmatic quality of Resources-Events- Agents diagrams: An experimental evaluation," Information Systems Journal, in press, Published online: Nov 13 2007, 27 pp.
38. Recker, J. Understanding Process Modelling Grammar Continuance: A Study of the Consequences of Representational Capabilities, Unpublished Doctoral Dissertation, Queensland University of Technology, Australia, 2008.
39. Rumbaugh, J., Blaha, M., Premerlani, W., Eddy, F., and Lorensen, W. Object-oriented modeling and design, Prentice Hall, Englewood Cliffs NJ, 1991.
40. Rumbaugh, J., Jacobson, I., and Booch, G. Unified Modeling Language reference manual, 2nd ed., Addison-Wesley, Boston, 2005.
41. Scheer, A.W. ARIS - Business Process Frameworks, 3rd Edition, Springer-Verlag, 1999.
42. Shanks, G., Tansley, E., Nuredini, J., Tobin, D., and Weber, R. "Representing part-whole relations inconceptual modeling: An empirical evaluation," MIS Quarterly (32:3) 2008, pp. 553-573.
43. Siau, K. "Informational and computational equivalence in comparing conceptual modeling methods,"Journal of Database Management (15:1) 2004, pp. 73-86.
44. Siau, K. and Rossi, M. "Evaluation techniques for systems analysis and design modelling methods -A review and comparative analysis," Information Systems Journal, in press (PublishedOnline: Dec 21 2007), 20 pp.
45. Siau, K. and Tian, Y. "A semiotic analysis of unified modeling language graphical notations," Requirements Engineering (14) 2009, pp. 15-26.
46. Siau, K., Wand, Y., and Benbasat, I. "The relative importance of structural constraints and surfacesemantics in information modeling," Information Systems (22:2&3) 1997, pp. 155-170.
47. Soffer, P. and Hadar, I. "Applying ontology-based rules to conceptual modeling: A reflection onmodeling decision making," European Journal of Information Systems (16:5) 2007, pp. 599-611.
48. Vessey, I. "Cognitive fit: A theory-based analysis of the graphs versus tables literature," Decision Sciences (22:2) 1991, pp. 219-240.
49. Wand, Y. and Weber, R. "An ontological model of an information system," IEEE Transactions onSoftware Engineering (16:11) 1990, pp. 1282-1292.
50. Wand, Y. and Weber, R. "On the ontological expressiveness of information systems analysis anddesign grammars," Journal of Information Systems (3) 1993, pp. 217-237.
51. Wand, Y. and Weber R. "Information systems and conceptual modeling - A research agenda," Information Systems Research (13:4) 2002, pp. 363-376.
52. Yourdon, E. Modern structured analysis, Prentice Hall, Englewood Cliffs, NJ, 1989.