Observing the observer: Non-intrusive verbalisations using the Concurrent Observer Narrative Technique

Cognition, Technology and Work

Volumn 13, Issue 2, 2011, Pages 135-149

  McIlroy, Rich C ^a   Stanton, Neville A ^a  

^a UNIVERSITY OF SOUTHAMPTON   (United Kingdom)

Author keywords

Information acquisition; Knowledge elicitation; Observation; Verbal protocols; Video research

Indexed keywords

INFORMATION ACQUISITIONS; KNOWLEDGE ELICITATION; OBSERVATION; VERBAL PROTOCOLS; VIDEO RESEARCH;

HUMAN ENGINEERING;

ENGINEERING RESEARCH;

EID: 79955789405     PISSN: 14355558     EISSN: 14355566     Source Type: Journal    
DOI: 10.1007/s10111-010-0160-5     Document Type: Article

References (42)

1. Bainbridge L (1979) Verbal reports as evidence of the process operator's knowledge. Int J Man Mach Stud 11: 411-436.
2. Bartl C, Dörner D (1998) Speechless when thinking: about the role of speech for problem solving. Sprache und Kognition 17: 224-238.
3. Bisantz AM, Drury CG (2005) Applications of archival and observational data. In: Wilson JR, Corlett N (eds) Evaluation of human work. Taylor and Francis, London, pp 61-82.
4. Bloom BS (1976) Human characteristics and school learning. McGraw-Hill Book Company, New York.
5. Braun V, Clarke V (2006) Using thematic analysis in psychology. Qual Res Psychol 3: 77-101.
6. Büttner OB, Silberer G (2007) Research spoilt the shopping trip: comparing the reactivity of thinking aloud and video-cued thought protocols at the point of purchase. In: Flexible marketing in an unpredictable world, Proceedings of the 36th EMAC Conference, 22-25 May, Reykjavik, Iceland.
7. Cooke NJ (1999) Knowledge elicitation. In: Durso F, Nickerson RS, Schvaneveldt RW, Dumais ST, Lindsay DS, Chi MTH (eds) Handbook of applied cognition. New York, Wiley, pp 479-509.
8. Dickson J, McLennan J, Omodei MM (2000) Effects of concurrent verbalisation on a time-critical, dynamic decision-making task. J Gen Psychol 127: 217-228.
9. Duval S, Wicklund RA (1972) A theory of objective self-awareness. Academic Press, New York.
10. Ericsson KA, Simon HA (1980) Verbal reports as data. Psychol Rev 87: 215-251.
11. Ericsson KA, Simon HA (1993) Protocol analysis: verbal reports as data. The MIT Press, Cambridge Revised ed.
12. Erlandsson M, Jansson A (2007) Collegial verbalisation-a case study on a new method of information acquisition. Behav Inf Technol 26: 535-543.
13. Farrington-Darby T, Wilson JR (2009) Understanding social interactions in a complex world: a video ethnography. Cogn Tech Work 11: 1-15.
14. Farrington-Darby T, Wilson JR, Norris BJ, Clarke T (2006) A naturalistic study of railway controllers. Ergonomics 49: 1370-1394.
15. Fleiss JL (1981) Statistical methods for rates and proportions. Wiley, New York.
16. Gibbons FX (1983) Self-attention and self-report: the "veridicality" hypothesis. J Pers 51: 517-554.
17. Glaser B, Strauss A (1967) The discovery of grounded theory. Aldine, Chicago.
18. Hammersley M, Atkinson P (1995) Ethnography: principles in practice. Routledge, London.
19. Jansson A, Olsson E, Erlandsson M (2006) Bridging the gap between analysis and design: Improving existing driver interfaces with tools from the framework of cognitive work analysis. Cogn Tech Work 8: 41-49.
20. Jenkins DP, Stanton NA, Salmon PM, Walker GH (2009) Cognitive work analysis: coping with complexity. Ashgate Publishing Limited, Farnham.
21. Kirwan B, Ainsworth LK (1992) A guide to task analysis. Taylor Francis, London.
22. Klein G, Calderwood R, MacGregor D (1983) Critical decision method for knowledge elicitation. IEEE Trans Syst Man Cybern 19: 462-472.
23. Kuusela H, Paul P (2000) A comparison of concurrent and retrospective verbal protocol analysis. Am J Psychol 113: 387-404.
24. Lomax H, Casey N (1998) Recording social life: reflexivity and video methodology. Sociological Research Online, 3, retrieved from http://www. socresonline. org. uk/3/lomax/lomax_doc. html on the 21st of April 2010.
25. McNeese MD (2004) How video informs cognitive systems engineering: making experience count. Cogn Tech Work 6: 186-196.
26. Miller JE, Patterson ES, Woods DD (2006) Elicitation by critiquing as a cognitive task analysis methodology. Cogn Tech Work 8: 90-102.
27. Omodei MM, McLennan JP (1994) Complex decision making in natural settings: using a head mounted video camera to study competitive orienteering. Percept Mot Skills 79: 1411-1425.
28. Omodei MM, McLennan J, Whitford P (1998) Improving decision making in complex natural settings: using a head-mounted video camera to improve performance of competitive orienteers. Int J Sport Psychol 29: 115-131.
29. Omodei MM, McLennan J, Wearing AJ (2002) Head-mounted video and cued recall: a methodology for detecting, understanding and minimising error in the control of complex Systems. In: Johnson CW (ed) Proceedings of the 21st European Annual Conference of Human Decision Making and Control, GIST technical report G2002. 1, Department of Computer Science, University of Glasgow, pp 72-80.
30. Omodei MM, McLennan JP, Wearing AJ (2004) How expertise is applied in real-world dynamic environments: head mounted video and cued recall as a methodology for studying routines of decision making. In: Betsch T, Haberstroh S (eds) The routines of decision making. Lawrence Erlbaum and Associates, Mahwah, pp 271-288.
31. Øvergård KI, Bjørkli CA, Røed BK, Hoff T (2010) Control strategies used by experienced marine navigators: observation of verbal conversations during navigation training. Cogn Tech Work. doi: 10. 1007/s10111-009-0132-9.
32. Patterson KA, McDonlad N, Engen OA (2010) Rethinking the role of social theory in socio-technical analysis: a critical realist approach to aircraft maintenance. Cogn Tech Work. doi: 10. 1007/s10111-009-0133-8.
33. Rasmussen J, Pejtersen A, Goodstein LP (1994) Cognitive systems engineering. Wiley, New York.
34. Russo JE, Johnson EJ, Stephens D (1989) The validity of verbal protocols. Mem Cognit 17: 759-769.
35. Seagull FJ, Xiao Y (2001) Using eye-tracking video data to augment knowledge elicitation in cognitive task analysis. In: Proceedings of the Human Factors and Ergonomics Society 45th annual meeting. HFES, Minneapolis.
36. Stanton NA, Salmon PM, Walker GH, Baber C, Jenkins DP (2005) Human factors methods: a practical guide for engineering and design. Ashgate, Aldershot.
37. Swann WB, Griffin JJ, Predmore SC, Gaines B (1987) The cognitive-affective crossfire: when self-consistency confronts self-enhancement. J Pers Soc Psychol 52: 881-889.
38. van den Haak MJ, de Jong MDT, Schellens JP (2003) Retrospective vs. concurrent think-aloud protocols: testing the usability of an online library catalogue. Behav Inf Technol 22: 339-351.
39. van Gog T, Kester L, Nievelstein F, Geisbers B, Paas F (2009) Uncovering cognitive processes: different techniques that contribute to cognitive load research and instruction. Comput Hum Behav 25: 325-331.
40. Vicente KJ (1999) Cognitive work analysis: toward safe, productive, and healthy computer-based work. Lawrence Erlbaum Associates, Mahwah.
41. Walker GH, Stanton NA, Salmon PM, Jenkins DP (2009) Command and control: the sociotechnical perspective. Ashgate, Aldershot.
42. Xiao Y, Seagull FJ, Mackenzie CF, Klein K (2004) Adaptive leadership in trauma resuscitation teams: a grounded theory approach to video analysis. Cogn Tech Work 6: 158-164.