Supervised automatic evaluation for summarization with voted regression model

Information Processing and Management

Volumn 43, Issue 6, 2007, Pages 1521-1535

  Hirao, Tsutomu ^a   Okumura, Manabu ^b   Yasuda, Norihito ^a   Isozaki, Hideki ^a  

^a NTT Communication Science Laboratories   (Japan)

^b TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

Author keywords

Automatic evaluation; Automatic summarization; Document understanding conference; Regression model; Text summarization challenge

Indexed keywords

AUTOMATION; COMPUTER SIMULATION; ERROR ANALYSIS; MATHEMATICAL MODELS; PROBLEM SOLVING; REGRESSION ANALYSIS;

AUTOMATIC SUMMARIZATION; DOCUMENT UNDERSTANDING CONFERENCE; REGRESSION MODELS; SUPERVISED AUTOMATIC EVALUATION; TEXT SUMMARIZATION CHALLENGE;

INFORMATION SYSTEMS;

EID: 34547983771     PISSN: 03064573     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ipm.2007.01.012     Document Type: Article

References (20)

1. Akaike, H. (1973). Information theory as an extension of the maximum likelihood principle. In Proceedings of the 2nd international symposium on information theory (pp. 267-281).
2. Burnham K., and Anderson D. Model selection and multimodel inference (2002), Springer
3. Cancedda N., Gaussier E., Goutte C., and Renders J.-M. Word sequence kernels. Journal of Machine Learning Research 3 (2003) 1059-1082
4. Daumé III, H., & Marcu, D. (2005). Bayesian multi-document summarization at MSE. In Proceedings of the workshop on multilingual summarization evaluation (MSE).
5. Doddington, G. (2002). Automatic evaluation of machine translation quality using N-gram co-occurrence statistics. In Proceedings of the human language technology (pp. 138-145).
6. Hirao, T., Sasaki, Y., & Isozaki, H. (2001). An extrinsic evaluation for question-biased text summarization on QA tasks. In Proceedings of the NAACL workshop on text summarization (pp. 61-68).
7. Hirao, T., Okumura, M., Fukushima, T., & Nanba, H. (2004). Text summarization challenge 3 - text summarization evaluation at NTCIR workshop 4. In Working Notes of the Fourth NTCIR Workhop Meeting (pp. 407-411).
8. Hirao, T., Okumura, M., & Isozaki, H. (2005). Kernel-based approach for automatic evaluation of natural language generation technologies: Application to automatic summarization. In Proceedings of the human language technology and empirical methods for natural language processing (pp. 145-152).
9. Hoerl A.E., and Kennard R. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 12 (1970) 56-69
10. Lin, C.-Y. (2004). ROUGE: A package for automatic evaluation of summaries. In Proceedings of workshop on text summarization branches out, post conference workshop of ACL 2004 (pp. 74-81).
11. Lin, C.-Y., & Hovy, E. (2003). Automatic evaluation of summaries using N-gram co-occurrence statistics. In Proceedings of the fourth meeting of the north American chapter of the association for computational linguistics and human language technology (pp. 150-157).
12. Lin, C.-Y., & Och, F. J. (2004). Automatic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics. In Proceedings of the 42nd annual meeting of the association for computational linguistics (pp. 606-613).
13. Mani I., Klein G., House D., Hirschman L., Firman T., and Sundheim B. SUMMAC: A text summarization evaluation. Natural Language Engineering 8 1 (2002) 43-68
14. Morris A.H., Kasper G.M., and Adams D. The effects and limitations of automatic text condensing on reading comprehension. Information System Research 3 1 (1992) 17-35
15. Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2002). BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the association for computational linguistics (pp. 311-318).
16. Schwarz G. Estimating the dimension of a model. Annals of Statistics 6 (1978) 461-464
17. Soricut, R., & Brill, E. (2004). A unified framework for automatic evaluation using N-gram co-occurrence statistics. In Proceedings of the 42nd annual meeting of the association for computational linguistics (pp. 614-621).
18. Sparck Jones K., and Galliers J. Evaluating natural language processing systems: An analysis and review (1996), Springer
19. Sugiura N. Further analysis of the data by Akaike's information criterion and the finite corrections. Theory and Methods A7 1 (1978) 13-26
20. Tibshirani R. Regression shrinkage and selection Via the Lasso. Journal of Royal Statistical Society B 58 (1996) 267-288