Active learning and visual analytics for stance classification with ALVA

ACM Transactions on Interactive Intelligent Systems

Volumn 7, Issue 3, 2017, Pages

  Kucher, Kostiantyn ^a   Paradis, Carita ^b   Sahlgren, Magnus ^c   Kerren, Andreas ^a  

^a LINNAEUS UNIVERSITY   (Sweden)

^b LUND UNIVERSITY   (Sweden)

^c SWEDISH INSTITUTE OF COMPUTER SCIENCE   (Sweden)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; CLASSIFIERS; DATA VISUALIZATION; LEARNING ALGORITHMS; LEARNING SYSTEMS; LINGUISTICS; NATURAL LANGUAGE PROCESSING SYSTEMS; TEXT PROCESSING; USER INTERFACES; VECTOR SPACES; VISUALIZATION;

ACTIVE LEARNING STRATEGIES; AUTOMATIC DETECTION; CLASSIFIER TRAINING; MACHINE LEARNING METHODS; SOCIAL MEDIA MONITORING; VECTOR SPACE MODELS; VISUAL REPRESENTATIONS; VISUALIZATION METHOD;

CLASSIFICATION (OF INFORMATION);

EID: 85032958347     PISSN: 21606455     EISSN: 21606463     Source Type: Journal    
DOI: 10.1145/3132169     Document Type: Article

References (58)

1. Bandar Alhumaidi A. Almutairi. 2013. Visualizing patterns of appraisal in texts and corpora. Text Talk 33, 4-5 (2013), 691-723. DOI:http://dx.doi.org/10.1515/text-2013-0031
2. Ron Artstein and Massimo Poesio. 2008. Inter-coder agreement for computational linguistics. Comput. Linguist. 34, 4 (Dec. 2008), 555-596. DOI:http://dx.doi.org/10.1162/coli.07-034-R2
3. Rita Borgo, Johannes Kehrer, David H. S. Chung, Eamonn Maguire, Robert S. Laramee, Helwig Hauser, Matthew Ward, and Min Chen. 2013. Glyph-based visualization: Foundations, design guidelines, techniques and applications. In Eurographics 2013-State of the Art Reports, M. Sbertand L. Szirmay-Kalos (Eds.). The Eurographics Association. DOI:http://dx.doi.org/10.2312/conf/EG2013/stars/039-063
4. Michael Brooks, Katie Kuksenok, Megan K. Torkildson, Daniel Perry, John J. Robinson, Taylor J. Scott, Ona Anicello, Ariana Zukowski, Paul Harris, and Cecilia R. Aragon. 2013. Statistical affect detection in collaborative chat. In Proceedings of the 2013 Conference on Computer Supported CooperativeWork (CSCW'13). ACM, 317-328. DOI:http://dx.doi.org/10.1145/2441776.2441813
5. John W. Du Bois. 2007. The stance triangle. In Stancetaking in Discourse: Subjectivity, Evaluation, Interaction, Robert Englebretson (Ed.). Pragmatics & Beyond New Series, Vol. 164. John Benjamins, 139-182. DOI:http://dx.doi.org/10. 1075/pbns.164
6. Mennatallah El-Assady, Valentin Gold, Carmela Acevedo, Christopher Collins, and Daniel Keim. 2016. ConToVi: Multi-party conversation exploration using topic-space views. Comput. Graph. Forum 35, 3 (2016), 431-440. DOI: http://dx.doi.org/10.1111/cgf.12919
7. Geoffrey Ellis and Alan Dix. 2007. A taxonomy of clutter reduction for information visualisation. IEEE Trans. Visual. Comput. Graph. 13, 6 (Nov. 2007), 1216-1223. DOI:http://dx.doi.org/10.1109/TVCG.2007.70535
8. Robert Englebretson (Ed.). 2007. Stancetaking in Discourse: Subjectivity, Evaluation, Interaction. Pragmatics & Beyond New Series, Vol. 164. John Benjamins. DOI:http://dx.doi.org/10.1075/pbns.164
9. Florian Heimerl, Steffen Koch, Harald Bosch, and Thomas Ertl. 2012. Visual classifier training for text document retrieval. IEEE Trans. Visual. Comput. Graph. 18, 12 (Dec. 2012), 2839-2848. DOI:http://dx.doi.org/10.1109/TVCG.2012. 277
10. Benjamin Hoferlin, Rudolf Netzel, Markus Hoferlin, Daniel Weiskopf, and Gunther Heidemann. 2012. Inter-active learning of ad-hoc classifiers for video visual analytics. In Proceedings of the IEEE Conference on Visual Analytics Science and Technology (VAST'12). 23-32. DOI:http://dx.doi.org/10.1109/VAST.2012.6400492
11. Shih-Wen Huang, Pei-Fen Tu, Wai-Tat Fu, and Mohammad Amanzadeh. 2013. Leveraging the crowd to improve feature-sentiment analysis of user reviews. In Proceedings of the International Conference on Intelligent User Interfaces (IUI'13). ACM, 3-14. DOI:http://dx.doi.org/10.1145/2449396.2449400
12. Andreas Kerren and Falk Schreiber. 2012. Toward the role of interaction in visual analytics. In Proceedings of the Winter Simulation Conference (WSC'12). Winter Simulation Conference, Article 420, 13 pages.
13. Janez Kranjc, Jasmina Smailović, Vid Podpečan, Miha Grčar, Martin Žnidaršič, and Nada Lavrač. 2015. Active learning for sentiment analysis on data streams: Methodology and workflow implementation in the ClowdFlows platform. Inform. Process. Manage. 51, 2 (2015), 187-203. DOI:http://dx.doi.org/10.1016/j.ipm.2014.04.001
14. Kostiantyn Kucher and Andreas Kerren. 2015. Text visualization techniques: Taxonomy, visual survey, and community insights. In Proceedings of the 8th IEEE Pacific Visualization Symposium (PacificVis'15). IEEE, 117-121. DOI:http://dx.doi.org/10.1109/PACIFICVIS.2015.7156366
15. Kostiantyn Kucher, Andreas Kerren, Carita Paradis, and Magnus Sahlgren. 2016. Visual analysis of text annotations for stance classification with ALVA. In Poster Abstracts of the EG/VGTC Conference on Visualization (EuroVis'16). The Eurographics Association, 49-51. DOI:http://dx.doi.org/10.2312/eurp.20161139
16. Kostiantyn Kucher, Carita Paradis, and Andreas Kerren. 2017. The state of the art in sentiment visualization. Comput. Graph. Forum (2017). DOI:http://dx.doi.org/10.1111/cgf.13217, Published electronically before print.
17. Kostiantyn Kucher, Teri Schamp-Bjerede, Andreas Kerren, Carita Paradis, and Magnus Sahlgren. 2016. Visual analysis of online social media to open up the investigation of stance phenomena. Inform. Visual. 15, 2 (2016), 93-116. DOI: http://dx.doi.org/10.1177/1473871615575079
18. Katie Kuksenok, Michael Brooks, John J. Robinson, Daniel Perry, Megan K. Torkildson, and Cecilia Aragon. 2012. Automating large-scale annotation for analysis of social media content. In Proceedings of the 2nd IEEE Workshop on Interactive Visual Text Analytics "Task-Driven Analysis of Social Media" (TextVis'12).
19. Hanchuan Li, Haichen Shen, Shengliang Xu, and Congle Zhang. 2015. Visualizing NLP annotations for crowdsourcing. CoRR abs/1508.06044 (2015). http://arxiv.org/abs/1508.06044.
20. Shusen Liu, Dan Maljovec, Bei Wang, Peer-Timo Bremer, and Valerio Pascucci. 2015. Visualizing high-dimensional data: Advances in the past decade. In Proceedings of the Eurographics Conference on Visualization (EuroVis)-STARs (EuroVis'15), R. Borgo, F. Ganovelli, and I. Viola (Eds.). The Eurographics Association. DOI:http://dx.doi.org/10.2312/eurovisstar.20151115
21. Yafeng Lu, Robert Kruger, Dennis Thom, Feng Wang, Steffen Koch, Thomas Ertl, and Ross Maciejewski. 2014. Integrating predictive analytics and social media. In Proceedings of the IEEE Conference on Visual Analytics Science and Technology (VAST'14). IEEE, 193-202. DOI:http://dx.doi.org/10.1109/VAST.2014.7042495
22. Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. J. Mach. Learn. Res. 9, Nov. (2008), 2579-2605.
23. Bernardo Magnini, Anne-Lyse Minard, Mohammed R. H. Qwaider, and Manuela Speranza. 2016. TextPro-AL: An active learning platform for flexible and efficient production of training data for NLP tasks. In Proceedings of the 26th International Conference on Computational Linguistics-System Demonstrations (COLING'16). Association for Computational Linguistics, 131-135.
24. Raheleh Makki, Stephen Brooks, and Evangelos E. Milios. 2014. Context-specific sentiment lexicon expansion via minimal user interaction. In Proceedings of the International Conference on Information Visualization Theory and Applications (IVAPP'14). SciTePress, 178-186. DOI:http://dx.doi.org/10.5220/0004745301780186
25. Christopher D.Manning and Hinrich Schutze. 1999. Foundations of Statistical Natural Language Processing.MIT Press.
26. Saif M. Mohammad. 2016. Sentiment analysis: Detecting valence, emotions, and other affectual states from text. In Emotion Measurement, Herbert L. Meiselman (Ed.). Woodhead Publishing, 201-237. DOI:http://dx.doi.org/10.1016/B978-0-08-100508-8.00009-6
27. Saif M. Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016. SemEval-2016 task 6: Detecting stance in tweets. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval'16). Association for Computational Linguistics, 31-41.
28. Saif M. Mohammad, Parinaz Sobhani, and Svetlana Kiritchenko. 2017. Stance and sentiment in tweets. ACM Trans. Internet Technol. 17, 3, Article 26 (June 2017), 23 pages. DOI:http://dx.doi.org/10.1145/3003433
29. Thomas Morton and Jeremy LaCivita. 2003. WordFreak: An open tool for linguistic annotation. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology: Demonstrations-Volume 4 (NAACL-Demonstrations'03). Association for Computational Linguistics, 17-18. DOI:http://dx.doi.org/10.3115/1073427.1073436
30. Christoph Muller and Michael Strube. 2003. Multi-level annotation in MMAX. In Proceedings of the Fourth SIGdial Workshop on Discourse and Dialogue (SIGdial'03), Akira Kurematsu, Alexander Rudnicky, and Syun Tutiya (Eds.). Association for Computational Linguistics, 198-207.
31. Myriam Munezero, Calkin Suero Montero, Erkki Sutinen, and John Pajunen. 2014. Are they different? Affect, feeling, emotion, sentiment, and opinion detection in text. IEEE Trans. Affect. Comput. 5, 2 (April 2014), 101-111. DOI:http://dx.doi.org/10.1109/TAFFC.2014.2317187
32. Philip V. Ogren. 2006. Knowtator: A Protege plug-in for annotated corpus construction. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology- Companion Volume: Demonstrations (NAACL-Demonstrations'06). Association for Computational Linguistics, 273-275. DOI:http://dx.doi.org/10.3115/1225785.1225791
33. Bo Pang and Lillian Lee. 2008. Opinion mining and sentiment analysis. Found. Trends Info. Retrieval 2, 1-2 (2008), 1-135. DOI:http://dx.doi.org/10.1561/1500000011
34. Fabian Pedregosa, Gael Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher,Matthieu Perrot, and Edouard Duchesnay. 2011. Scikit-learn:Machine learning in Python. J. Mach. Learn. Res. 12 (Oct. 2011), 2825-2830.
35. Martin Perez-Perez, Daniel Glez-Pena, Florentino Fdez-Riverola, andAnalia Lourenco. 2015. Marky:Atool supporting annotation consistency in multi-user and iterative document annotation projects. Comput. Methods Progr. Biomed. 118, 2 (2015), 242-251. DOI:http://dx.doi.org/10.1016/j.cmpb.2014.11.005
36. Forough Poursabzi-Sangdeh, Jordan Boyd-Graber, Leah Findlater, and Kevin Seppi. 2016. ALTO: Active learning with topic overviews for speeding label induction and document labeling. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL'16). Association for Computational Linguistics, 1158-1169.
37. Ramik Sadana, Timothy Major, Alistair Dove, and John Stasko. 2014. OnSet: A visualization technique for large-scale binary set data. IEEE Trans. Visual. Comput. Graph. 20, 12 (Dec. 2014), 1993-2002. DOI:http://dx.doi.org/10.1109/TVCG.2014.2346249
38. Michael Sedlmair, Tamara Munzner, and Melanie Tory. 2013. Empirical guidance on scatterplot and dimension reduction technique choices. IEEE Trans. Visual. Comput. Graph. 19, 12 (Dec. 2013), 2634-2643. DOI:http://dx.doi.org/10.1109/TVCG.2013.153
39. Burr Settles. 2011. Closing the loop: Fast, interactive semi-supervised annotation with queries on features and instances. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP'11). Association for Computational Linguistics, 1467-1478.
40. Burr Settles. 2012. Active learning. Synth. Lect. Artific. Intell. Mach. Learn. 6, 1 (2012), 1-114. DOI:http://dx.doi.org/10.2200/S00429ED1V01Y201207AIM018
41. Azra Shamim, Vimala Balakrishnan, and Muhammad Tahir. 2015. Evaluation of opinion visualization techniques. Info. Visual. 14, 4 (2015), 339-358. DOI:http://dx.doi.org/10.1177/1473871614550537
42. Ben Shneiderman and Aleks Aris. 2006. Network visualization by semantic substrates. IEEE Trans. Visual. Comput. Graph. 12, 5 (Sept. 2006), 733-740. DOI:http://dx.doi.org/10.1109/TVCG.2006.166
43. Vasiliki Simaki, Carita Paradis, and Andreas Kerren. 2017. Stance classification in texts from blogs on the 2016 British referendum. In Proceedings of the International Conference on Speech and Computer (SPECOM'17). Springer, 700-709. DOI:10.1007/978-3-319-66429-3-70
44. Maria Skeppstedt, Carita Paradis, Andreas Kerren, and Magnus Sahlgren. 2015. Finding infrequent phenomena in large corpora using distributional semantics. In Poster Abstracts of the Symposium on Methods and Linguistic Theories (MaLT'15).
45. Maria Skeppstedt, Magnus Sahlgren, Carita Paradis, and Andreas Kerren. 2016. Active learning for detection of stance components. In Proceedings of the Workshop on Computational Modeling of People's Opinions, Personality, and Emotions in Social Media at COLING 2016 (PEOPLES'16). Association for Computational Linguistics, 50-59.
46. Maria Skeppstedt, Magnus Sahlgren, Carita Paradis, and Andreas Kerren. 2016. Unshared task: (Dis)agreement in online debates. In Proceedings of the 3rd Workshop on Argument Mining at ACL'16, short papers track (ArgMining'16). Association for Computational Linguistics, 154-159.
47. Maria Skeppstedt, Teri Schamp-Bjerede, Magnus Sahlgren, Carita Paradis, and Andreas Kerren. 2015. Detecting speculations, contrasts and conditionals in consumer reviews. In Proceedings of the 6th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA'15). Association for Computational Linguistics, 162-168.
48. Maria Skeppstedt, Vasiliki Simaki, Carita Paradis, and Andreas Kerren. 2017. Detection of stance and sentiment modifiers in political blogs. In Proceedings of the International Conference on Speech and Computer (SPECOM'17). Springer, 302-311. DOI:10.1007/978-3-319-66429-3-70
49. Henry Small. 2011. Interpreting maps of science using citation context sentiments: A preliminary investigation. Scientometrics 87, 2 (2011), 373-388. DOI:http://dx.doi.org/10.1007/s11192-011-0349-2
50. Pontus Stenetorp, Sampo Pyysalo, Goran Topić, Tomoko Ohta, Sophia Ananiadou, and Jun'ichi Tsujii. 2012. BRAT: A web-based tool for NLP-assisted text annotation. In Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics (EACL'12). Association for Computational Linguistics, 102-107.
51. Katrin Tomanek, JoachimWermter, and Udo Hahn. 2007. Efficient annotation with the Jena ANnotation Environment (JANE). In Proceedings of the Linguistic Annotation Workshop (LAW'07). Association for Computational Linguistics, 9-16.
52. Simon Tong and Daphne Koller. 2001. Support vector machine active learning with applications to text classification. J. Mach. Learn. Res. 2, Nov. (2001), 45-66.
53. Megan K. Torkildson. 2013. Visualizing the performance of classification algorithms with additional re-annotated data. In Proceedings of the Extended Abstracts on Human Factors in Computing Systems (CHI EA'13). ACM, 2767-2772. DOI:http://dx.doi.org/10.1145/2468356.2479507
54. Edward R. Tufte. 2006. Beautiful Evidence. Graphics Press.
55. Peter D. Turney and Patrick Pantel. 2010. From frequency to meaning: Vector space models of semantics. J. Artific. Intell. Res. 37 (2010), 141-188. DOI:http://dx.doi.org/10.1613/jair.2934
56. MatthewWard. 2002. A taxonomy of glyph placement strategies for multidimensional data visualization. Info. Visual. 1, 3-4 (2002), 194-210. DOI:http://dx.doi.org/10.1057/palgrave.ivs.9500025
57. Malcolm Ware, Eibe Frank, Geoffrey Holmes, Mark Hall, and Ian H. Witten. 2001. Interactive machine learning: Letting users build classifiers. Int. J. Human-Comput. Studies 55, 3 (2001), 281-292. DOI:http://dx.doi.org/10.1006/ijhc.2001.0499
58. Seid Muhie Yimam, Iryna Gurevych, Richard Eckart de Castilho, and Chris Biemann. 2013. WebAnno: A flexible, web-based and visually supported system for distributed annotations. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics-Conference System Demonstrations (ACL'13). Association for Computational Linguistics, 1-6.