Operations analysis of behavioral observation procedures: A taxonomy for modeling in an expert training system

Behavior Research Methods

Volumn 43, Issue 3, 2011, Pages 616-634

  Ray, Roger D ^a   Ray, Jessica M ^a,b   Eckerman, David A ^c   Milkosky, Laura M ^a   Gillins, LaDarien J ^a  

^a ROLLINS COLLEGE   (United States)

^b UNIVERSITY OF CENTRAL FLORIDA   (United States)

^c UNIVERSITY OF NORTH CAROLINA   (United States)

Author keywords

Behavioral observation; Computer based training; Expert training systems; Observation procedures; Observer agreement; Observer training

Indexed keywords

ARTICLE; BEHAVIOR; COMPUTER INTERFACE; EXPERT SYSTEM; HUMAN; METHODOLOGY; OBSERVATION; TASK PERFORMANCE;

BEHAVIOR; EXPERT SYSTEMS; HUMANS; OBSERVATION; RESEARCH DESIGN; TASK PERFORMANCE AND ANALYSIS; USER-COMPUTER INTERFACE;

EID: 80052836207     PISSN: 1554351X     EISSN: 15543528     Source Type: Journal    
DOI: 10.3758/s13428-011-0140-6     Document Type: Article

References (47)

1. Altmann, J. (1974). Observational study of behavior: Sampling methods. Behaviour, 49, 227-267.
2. Astley, C. A., Smith, O. A., Ray, R. D., Golanov, E. V., Chesney, M., Chalyan, V. G., Bowden, D. M. (1991). Integrating behavior and cardiovascular responses: The code. American Journal of Physiology: Regulatory, Integrative, and Comparative Physiology, 261, R172-R181.
3. Baer, D. M. (1977). Perhaps it would be better not to know everything. Journal of Applied Behavior Analysis, 10, 167-172.
4. Bakeman, R., & Gottman, J. M. (1987). Applying observational methods: A systematic view. In J. D. Osofsky (Ed.), Handbook of infant development (pp. 818-854). New York: Wiley.
5. Bakeman, R., & Gottman, J. M. (1997). Observing interaction: An introduction to sequential analysis (2nd ed.). Cambridge: Cambridge University Press.
6. Boykin, R. A., & Nelson, R. O. (1981). The effect of instructions and calculation procedures on observers' accuracy, agreement, and calculation correctness. Journal of Applied Behavior Analysis, 14, 479-489.
7. Campbell, J. T. (1976). Psychometric theory. In M. D. Dunnette (Ed.), Handbook of industrial and organizational psychology (pp. 185-222). Chicago: Rand McdNally.
8. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37-46.
9. Collins, J. J., & Stewart, I. N. (1993). Coupled nonlinear oscillators and the symmetries of animal gaits. Journal of Nonlinear Science, 3, 349-392.
10. Cone, J. D. (1982). Validity of direct observation assessment procedures. In D. P. Hartmann (Ed.), Using observers to study behavior (New Directions for Methodology of Social and Behavioral Science, Vol. 14, pp. 67-79). San Francisco: Jossey-Bass.
11. Cronbach, L. S., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral mesurements. New York: Wiley.
12. Cunado, D., Nixon, M. S., & Carter, J. N. (2003). Automatic extraction and description of human gait models for recognition purposes. Computer Vision and Image Understanding, 90, 1-41.
13. Friman, P. C. (2009). Behavior assessment. In D. H. Barlow, M. K. Nock, & M. Herson (Eds.), Single case experimental designs: Strategies for studying behavior change (3rd ed., pp. 99-134). Boston: Pearson.
14. Grove, W. M., Andreasen, N. C., McDonald-Scott, P., Keller, M. B., & Shapiro, R. W. (1981). Reliability studies of psychiatric diagnosis. Archives of General Psychiatry, 38, 408-413.
15. Guggenmoos-Holzmann, I. (1993). How reliable are chance-corrected measures of agreement? Statistics in Medicine, 12, 2191-2205.
16. Hartmann, D. P. (1977). Considerations in the use of interobserver reliability estimates. Journal of Applied Behavior Analysis, 10, 103-116.
17. Hartmann, D. P., & Wood, D. D. (1992). Observational methods. In A. S. Bellack, M. Hersen, & A. E. Akzdin (Eds.), International handbook of behavior modification and therapy (pp. 107-138). New York: Plenum.
18. Hawkins, A., Sharpe, T. L., & Ray, R. D. (1994). Toward instructional process measurability: An interbehavioral field systems perspective. In R. Gardner, D. Sainatok, T. Cooper, T. Heron, W. Heward, J. Eshleman, & T. Grossi (Eds.), Behavior analysis in education: Focus on measurably superior instruction. (pp. 241-255). Pacific Grove, CA: Brooks/Cole.
19. Herrnstein, R. J. (1961). Relative and absolute strength of response as a function of frequency of reinforcement. Journal of the Experimental Analysis of Behavior, 4, 267-272.
20. Holland, J. G., & Skinner, B. F. (1961). The analysis of behavior: A program for self-instruction. New York: McGraw-Hill.
21. House, A. E., Farber, J., & Nier, L. L. (1983). Differences in computational accuracy and speed of calculation between three measures of interobserver agreement. Child Study Journal, 13, 95-201.
22. Kelleher, R. T., & Gollub, L. R. (1962). A review of positive conditioned reinforcement. Journal of the Experimental Analysis of Behavior, 5, 543-597.
23. Kelly, M. B. (1977). A review of the observational data-collection and reliability procedures reported in The Journal of Applied Behavior Analysis. Journal of Applied Behavior Analysis, 10, 97-101.
24. Lee, M. D., & Del Fabbro, P. H. (2002). A Bayesian coefficient of agreement for binary decisions. Retrieved from http://www. psychology. adelaide. edu. au/members/staff/michaellee/homepage/bayeskappa. pdf.
25. Mann, J., Have, T. T., Plunkett, J. W., & Meisels, S. J. (1991). Time sampling: A methodological critique. Child Development, 62, 227-241.
26. Martin, P., & Bateson, P. (1993). Measuring behavior: An introductory guide. London: Cambridge University Press.
27. Mitchell, S. K. (1979). Interobserver agreement, reliability, and generalizability of data collected in observational studies. Psychological Bulletin, 86, 376-390.
28. Mudford, O. C., Martin, N. T., Hui, J. K. Y., & Taylor, S. A. (2009). Assessing observer accuracy in continuous recording of rate and duration: Three algorithms compared. Journal of Applied Behavior Analysis, 42, 527-539.
29. Mudford, O. C., Taylor, S. A., & Martin, N. T. (2009). Continuous recording and interobserver agreement algorithms reported in the Journal of Applied Behavior Analysis (1995-2005). Journal of Applied Behavior Analysis, 42, 165-169.
30. Purton, A. C. (1978). Ethological categories of behavior and some consequences of their conflation. Animal Behavior, 26, 653-670.
31. Ray, R. D. (1977). Physiological-behavioral coupling research in the Soviet Science of Higher Nervous Activity: A visitation report. Pavlovian Journal of Biological Sciences, 12, 41-50.
32. Ray, R. D., & Delprato, D. J. (1989). Behavioral systems analysis: Methodological strategies and tactics. Behavioral Science, 34, 81-127.
33. Ray, R. D., & Ray, M. R. (1976). A systems approach to behavior II: The ecological description and analysis of human behavior dynamics. Psychological Record, 26, 147-180.
34. Ray, J. M., & Ray, R. D. (2008). Train-to-code: An adaptive expert system for training systematic observation and coding skills. Behavior Research Methods, 40, 673-693.
35. Sanford, E. C. (1888). Personal equation. American Journal of Psychology, II, 3-38.
36. Sharpe, T., & Koperwas, J. (2003). Behavioral and sequential analysis: Principles and practice. Thousand Oaks, CA: Sage.
37. Skinner, B. F. (1938). The behavior of organisms. New York: Appleton-Century-Crofts.
38. Spitznagel, E. L., & Helzer, J. E. (1985). A proposed solution to the base rate problem in the kappa statistic. Archives of General Psychiatry, 42, 725-728.
39. Suen, H. K., & Ary, D. (1989). Analyzing quantitative behavioural observation data. Hillsdale, NJ: Erlbaum.
40. Suen, H. K., & Lee, P. S. C. (1985). Effects of the use of percentage agreement on behavioral observation reliabilities: A reassessment. Journal of Psychopathology and Behavioral Assessment, 7, 221-234.
41. Terrace, H. S. (1963). Discrimination learning with and without "errors." Journal of the Experimental Analysis of Behavior, 6, 1-27.
42. Thorndike, E. L. (1898). Animal intelligence: An experimental study of the associative processes in animals. Psychological Monographs, 2(4), Whole No. 8.
43. Thorndike, E. L. (1899). A reply to "The nature of animal intelligence and the methods of investigating it." Psychological Review, 6, 412-420.
44. Verplanck, W. S. (1957) A glossary of terms. Psychological Review, 64(Suppl.), 42 and i.
45. Verplanck, W. S. (1996). From 1924 to 1996 and into the future: Operation analytic behaviorism. Mexican Journal of Behavior Analysis, 22, 19-60.
46. Vollmer, T. R., Soloman, K. N., & St. Peter Pipkin, C. (2008). Practical implications of data reliability and treatment integrity monitoring. Behavior Analysis in Practice, 1, 4-11.
47. Wallace, B., & Ross, A. J. (2006). Beyond human error: Taxonomies and safety science. Boca Raton, FL: CRC Press.