STORYEVAL: An empirical evaluation framework for narrative generation

AAAI Spring Symposium - Technical Report

Volumn SS-09-06, Issue , 2009, Pages 103-110

  Rowe, Jonathan P ^a   Mcquiggan, Scott W ^b   Robison, Jennifer L ^a   Marcey, Derrick R ^a   Lester, James C ^a  

^a North Carolina State University   (United States)

^b SAS INSTITUTE INC   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COGNITIVE SCIENCE; COMPLEMENTARY TOOLS; COMPUTATIONAL MODEL; EMPIRICAL EVALUATIONS; EVALUATION FRAMEWORK; EVALUATION METHODOLOGIES; NATURAL LANGUAGE PROCESSING; NON-INTERACTIVE;

COMPUTATIONAL LINGUISTICS; HUMAN COMPUTER INTERACTION; KNOWLEDGE MANAGEMENT; NATURAL LANGUAGE PROCESSING SYSTEMS;

TECHNOLOGY;

EID: 70350543766     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (38)

1. Aylett, R. and Louchart, S. 2008. If I were you: Double appraisal in affective agents. Proc. of the 7th International Conference on AAMAS, 1233-1236, Estoril, Portugal.
2. Bailey, P. 1999. Searching for storiness: Story-generation from a reader's perspective. Working Notes of the AAAI Fall Symposium on Narrative Intelligence, 157-163, Cape Cod, MA.
3. th Conf. ofEACL, 313-320, Trento, Italy.
4. Callaway, C. and Lester, J. 2001. Evaluating the effects of natural language generation techniques on reader satisfaction. Proc. of the 23rd Annual Conference of the Cognitive Science Society, 164-169, Edinburgh, UK.
5. Cassell, J., Sullivan, J., Prevost, S., and Churchill, E. (Eds.) 2000. Embodied Conversational Agents. Boston, MA.: MIT Press.
6. Cavazza, M., Charles, F. and Mead, S.J., 2002. Planning characters' behaviour in interactive storytelling. Journal of Visualization and Computer Animation 13: 121-131. (Pubitemid 135714641)
7. Dix, A., Finlay, J., Abowd, G., and Beale, R. 2004. Human-Computer Interaction. Harlow, England: Pearson Education, Inc.
8. Dow, S., Mehta, M., Harmon, E., Maclntyre, B., and Mateas, M. 2007. Presence and engagement in an interactive drama. Proc. of CHI, 1475-1484, San Jose, CA.
9. Ekman, P. 2003. Emotions Revealed. New York: Henry Holt.
10. Gerrig, R. 1993. Experiencing Narrative Worlds: On the Psychological Activities of Reading. New Haven: Yale University Press.
11. Graesser, A.C., Lang, K.L., and Roberts, R.M. 1991. Question answering in the context of stories. Journal of Experimental Psychology: General, 120(3), 254-277.
12. Green, M., and Brock, T. 2000. The role of transportation in the persuasiveness of public narratives. Journal of Personality and Social Psychology, 79(5), 701-721.
13. Insko, B. E. 2003. Measuring presence: Subjective, behavioral and physiological methods. In G. Riva, F. Davide, & W. A. Usselsteijn (Eds.), Being There: Concepts, Effects and Measurements of User Presence in Synthetic Environments. Amsterdam: IOS Press. 109-119.
14. th Intl Conference on Artificial Intelligence in Education, 290-297, Amsterdam, The Netherlands.
15. Jurafsky, D. and Martin, J. 2008. Speech and Language Processing. Upper Saddle River, NJ.: Pearson Education, Inc.
16. Kelso, M., Weyhrauch, P., and Bates, J. 1993. Dramatic presence. Presence: The Journal of Teleoperators and Virtual Environments, 2(1), 1-15.
17. th Intl. Conf. on User Modeling, 380-384, Corfu, Greece.
18. Magerko, B. 2007. Evaluating Preemptive Story Direction in the Interactive Drama Architecture. Journal of Game Development, 2(3).
19. Mateas, M. and Stern, A. 2005. Structuring content in the Facade interactive drama architecture. Proc. of AIIDE, 93-98, Marina del Rey, CA.
20. McQuiggan, S., Robison, J., and Lester, J. 2008. Affective transitions in narrative-centered learning environments. Proc of the 9th Intl Conf on ITS, 490-499, Montreal, CAN.
21. McQuiggan, S., Rowe, J. and Lester, J. 2008. The effects of empathetic virtual characters on presence in narrative-centered learning environments. Proc. of CHI, 1511-1520, Florence, Italy.
22. McQuiggan, S., Rowe, J., Lee, S. and Lester, J. 2008. Story-based learning: the impact of narrative on learning experiences and outcomes. Proc. of the 9th Intl Conference on ITSs, 530-539, Montreal, Canada.
23. Mehta, M., Dow, S., Mateas, M. and Maclntyre, B. 2007. Evaluating a conversation-centered interactive drama. Proc. of the 6th Intl Conf on AAMAS, 1-8, Honolulu, HI.
24. Mott, B. and Lester, J. 2006. U-Director: A decision-theoretic narrative planning architecture for storytelling environments. In Proc. of the 5th Intl Conf on AAMAS, 977-984, Hakodate, Japan.
25. Nelson, M., Mateas, M., Roberts, D., and Isbell, C. 2006. Declarative optimization-based drama management in theinteractive fiction Anchorhead. IEEE Computer Graphics and Applications, 26(3): 32-41. (Pubitemid 46404982)
26. Prince, G. 2003. Dictionary of Narratology (Revised Edition). Lincoln, NE.: University of Nebraska Press.
27. Riedl, M. and Young, R. M. 2005. An objective character believability evaluation procedure for multi-agent story generation systems. Proc. ofPVA, 278-291, Kos, Greece.
28. Riedl, M., Saretto, C. J., and Young, R.M. 2003. Managing interaction between users and agents in a multi-agent storytelling environment. Proc. of the 2nd Intl Conf. on AAMAS, 741-748, Melbourne, Australia.
29. st AAAI, Boston, MA.
30. Rowe, J., McQuiggan, S., and Lester, J. 2007. Narrative presence in intelligent learning environments. Working Notes of the 2007 AAAI Fall Symposium on Intelligent Narrative Technologies, 126-133, Washington D.C.
31. Russell, S., and Norvig, P. 2003. Artificial Intelligence -A Modern Approach. Upper Saddle River, NJ.: Pearson Education, Inc.
32. Schubert, T., Friedmann, F., and Regenbrecht, H. 1999. Embodied presence in virtual environments. In Ray Paton & Irene Neilson (Eds.), Visual Representations and Interpretations. London: Springer. 269-278.
33. th Intl Conf. on Artificial Intelligence in Education, 21-28, Amsterdam, The Netherlands.
34. Turner, S. 1994. The Creative Process: A Computer Model of Storytelling and Creativity. Hillsdale, NJ.: Lawrence Erlbaum Associates.
35. Walker, M., Litman, D., Kamm, C, and Abella, A. 1997. PARADISE: A framework for evaluating spoken dialogue agents. Proc. of ACL, 271-280, Madrid, Spain.
36. Weyhrauch, P. 1997. Guiding interactive drama. Ph.D. diss., Dept. of Computer Science, Carnegie Mellon Univ., Pittsburgh, PA.
37. Witmer, B. and Singer, M. 1998. Measuring presence in virtual environments: A presence questionnaire. Presence: Teleoperators and Virtual Environments, 7(3), 225-240.
38. Zoll, C, Enz, S., Schaub, H., Aylett, R., and Paiva, A. 2006. Fighting bullying with the help of autonomous agents in a virtual school environment. Proc. of the 7th Intl Conf. on Cognitive Modeling, Trieste, Italy.