Evolution of the SCORE framework to enhance field-based performance evaluations of emerging technologies

Performance Metrics for Intelligent Systems (PerMIS) Workshop

Volumn , Issue , 2008, Pages 1-8

  Weiss, Brian A ^a   Schlenoff, Craig ^a  

^a NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY   (United States)

Author keywords

ASSIST; DARPA; elemental tests; performance evaluation; SCORE; soldier worn sensor; speech translation; task tests; TRANSTAC; vignette tests

Indexed keywords

ASSIST; BUILDING BLOCKES; DESIGN CRITERION; EMERGING TECHNOLOGIES; END USERS; EVALUATION DESIGN; OPERATING ENVIRONMENT; PERFORMANCE EVALUATION; REALISTIC ENVIRONMENTS; SENSOR SYSTEMS; SPEECH TRANSLATION; SPOKEN LANGUAGE TRANSLATION; TECHNICAL PERFORMANCE;

INTELLIGENT SYSTEMS; SENSORS;

TESTING;

EID: 77954518623     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1774674.1774676     Document Type: Conference Paper

References (8)

1. th Hawaii International Conference of System Sciences (HICSS-38), Hawaii.
2. Schlenoff, C., Steves, M., Weiss, B.A., Shneier, M., and Virts, A. 2006. Applying SCORE to Field-based Performance Evaluations of Soldier Worn Sensor Technologies. Journal of Field Robotics, Volume 24, (8-9), pp. 671-698. (Pubitemid 47570561)
3. Schlenoff, C., Weiss, B.A., Steves, M., Virts, A., Shneier, M. and Linegang, M. 2006. Overview of the First Advanced Technology Evaluations for ASSIST, In Proceedings of the 2006 Performance Metrics for Intelligent Systems (PerMIS) Conference, Gaithersburg, Maryland.
4. Weiss, B.A., Schlenoff, C., Shneier, M., and Virts, A. 2006. ASSIST: Technology Evaluations and Performance Metrics for Soldier-Worn Sensor Systems. In Proceedings of the 2006 Performance Metrics for Intelligent Systems (PerMIS) Conference, Gaithersburg, Maryland.
5. CONOPS, http://en.wiktionary.org/wiki/CONOPS, (Definition retrieved on 07/10/08)
6. Steves, M. P. 2006. Utility Assessments for ASSIST Systems. In Proceedings of the 2006 Performance Metrics for Intelligent Systems (PerMIS) Conference, Gaithersburg, Maryland.
7. Weiss, B.A., Schlenoff, C., Sanders, G., Steves, M., Condon, S., Phillips, J., and Parvaz, D. 2008. Performance Evaluation of Speech Translation Systems, In Proceedings of the 6th edition of the Language Resources and Evaluation Conference. Marrakech, Morocco.
8. Sanders, G., Bronsart, S., Condon, S., and C. Schlenoff. 2008. Odds of successful transfer of low-level concepts: A key metric for bidirectional speech-to-speech machine translation in DARPA's TRANSTAC program. In Proceedings of LREC 2008.