Document Open Access Logo

Towards a Corpus of Historical German Plays with Emotion Annotations

Authors Thomas Schmidt, Katrin Dennerlein, Christian Wolff

PDF
Thumbnail PDF

File

OASIcs.LDK.2021.9.pdf
  • Filesize: 0.7 MB
  • 11 pages

Document Identifiers

Author Details

Thomas Schmidt
  • Media Informatics Group, University of Regensburg, Germany
Katrin Dennerlein
  • German Literary Studies and Computational Literary Studies, University of Würzburg, Germany
Christian Wolff
  • Media Informatics Group, University of Regensburg, Germany

Funding

This research is part of the project “Emotions in Drama” (Emotionen im Drama), is funded by the German Research Foundation (DFG) and part of the priority programme SPP 2207 Computational Literary Studies (CLS).

Acknowledgements

We want to thank the following student annotators for their contributions to this project: Viola Hipler, Julia Jäger, Emma Ruß, and Leon Sautter.

Cite AsGet BibTex

Thomas Schmidt, Katrin Dennerlein, and Christian Wolff. Towards a Corpus of Historical German Plays with Emotion Annotations. In 3rd Conference on Language, Data and Knowledge (LDK 2021). Open Access Series in Informatics (OASIcs), Volume 93, pp. 9:1-9:11, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2021)
https://doi.org/10.4230/OASIcs.LDK.2021.9

Abstract

In this paper, we present first work-in-progress annotation results of a project investigating computational methods of emotion analysis for historical German plays around 1800. We report on the development of an annotation scheme focussing on the annotation of emotions that are important from a literary studies perspective for this time span as well as on the annotation process we have developed. We annotate emotions expressed or attributed by characters of the plays in the written texts. The scheme consists of 13 hierarchically structured emotion concepts as well as the source (who experiences or attributes the emotion) and target (who or what is the emotion directed towards). We have conducted the annotation of five example plays of our corpus with two annotators per play and report on annotation distributions and agreement statistics. We were able to collect over 6,500 emotion annotations and identified a fair agreement for most concepts around a κ-value of 0.4. We discuss how we plan to improve annotator consistency and continue our work. The results also have implications for similar projects in the context of Digital Humanities.

Subject Classification

ACM Subject Classification
  • Applied computing → Arts and humanities
  • Computing methodologies → Machine learning
Keywords
  • Emotion
  • Annotation
  • Digital Humanities
  • Computational Literary Studies
  • German Drama
  • Sentiment Analysis
  • Emotion Analysis
  • Corpus

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

References

  1. Cecilia Ovesdotter Alm and Richard Sproat. Emotional Sequencing and Development in Fairy Tales. In Jianhua Tao, Tieniu Tan, and Rosalind W. Picard, editors, Affective Computing and Intelligent Interaction, Lecture Notes in Computer Science, pages 668-674, Berlin, Heidelberg, 2005. Springer. URL: https://doi.org/10.1007/11573548_86.
  2. Thomas Anz. Todesszenarien - Literarische Techniken zur Evokation von Angst, Trauer und anderen Gefühlen. In Emotionale Grenzgänge, pages 113-129. Königshausen & Neumann (Würzburg), 2011. Google Scholar
  3. Frank Fischer, Ingo Börner, Mathias Göbel, Angelika Hechtl, Christopher Kittel, Carsten Milling, and Peer Trilcke. Programmable Corpora: Introducing DraCor, an Infrastructure for the Research on European Drama, July 2019. Conference Name: Digital Humanities 2019: "Complexities" (DH2019) Publisher: Zenodo. URL: https://doi.org/10.5281/zenodo.4284002.
  4. Evelyn Gius, Jan Christoph Meister, Marco Petris, Malte Meister, Christian Bruck, Janina Jacke, Mareike Schuhmacher, Marie Flüh, and Jan Horstmann. CATMA, 2020. URL: https://doi.org/10.5281/zenodo.4353618.
  5. Kaisla Kajava, Emily Öhman, Piao Hui, and Jörg Tiedemann. Emotion Preservation in Translation: Evaluating Datasets for Annotation Projection. In Proceedings of Digital Humanities in Nordic Countries (DHN 2020), pages 38-50. CEUR, 2020. Google Scholar
  6. Tuomo Kakkonen and Gordana Galić Kakkonen. SentiProfiler: Creating Comparable Visual Profiles of Sentimental Content in Texts. In Proceedings of the Workshop on Language Technologies for Digital Humanities and Cultural Heritage, pages 62-69, Hissar, Bulgaria, 2011. Association for Computational Linguistics. Google Scholar
  7. Evgeny Kim and Roman Klinger. Who Feels What and Why? Annotation of a Literature Corpus with Semantic Roles of Emotions. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1345-1359, Santa Fe, New Mexico, USA, 2018. Association for Computational Linguistics. URL: https://www.aclweb.org/anthology/C18-1114.
  8. Evgeny Kim and Roman Klinger. An Analysis of Emotion Communication Channels in Fan-Fiction: Towards Emotional Storytelling. In Proceedings of the Second Workshop on Storytelling, pages 56-64, Florence, Italy, August 2019. Association for Computational Linguistics. Google Scholar
  9. Evgeny Kim and Roman Klinger. A Survey on Sentiment and Emotion Analysis for Computational Literary Studies. Zeitschrift für digitale Geisteswissenschaften, 2019. arXiv: 1808.03137. URL: https://doi.org/10.17175/2019_008.
  10. Andrei P. Kirilenko and Svetlana Stepchenkova. Inter-Coder Agreement in One-to-Many Classification: Fuzzy Kappa. PLOS ONE, 11(3):e0149787, 2016. Publisher: Public Library of Science. URL: https://doi.org/10.1371/journal.pone.0149787.
  11. J. Richard Landis and Gary G. Koch. The Measurement of Observer Agreement for Categorical Data. Biometrics, 33(1):159-174, 1977. Publisher: [Wiley, International Biometric Society]. Google Scholar
  12. Saif Mohammad. From Once Upon a Time to Happily Ever After: Tracking Emotions in Novels and Fairy Tales. In Proceedings of the 5th ACL-HLT Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities, pages 105-114, Portland, OR, USA, 2011. Association for Computational Linguistics. Google Scholar
  13. Saif Mohammad. A Practical Guide to Sentiment Annotation: Challenges and Solutions. In Proceedings of the 7th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, pages 174-179, San Diego, California, June 2016. Association for Computational Linguistics. URL: https://doi.org/10.18653/v1/W16-0429.
  14. Luis Moßburger, Felix Wende, Kay Brinkmann, and Thomas Schmidt. Exploring online depression forums via text mining: A comparison of Reddit and a curated online forum. In Proceedings of the Fifth Social Media Mining for Health Applications Workshop & Shared Task, pages 70-81, Barcelona, Spain (Online), December 2020. Association for Computational Linguistics. URL: https://www.aclweb.org/anthology/2020.smm4h-1.11.
  15. Kevin Mulligan and Klaus R. Scherer. Toward a Working Definition of Emotion. Emotion Review, 4(4):345-357, 2012. Publisher: SAGE Publications. URL: https://doi.org/10.1177/1754073912445818.
  16. Mika V. Mäntylä, Daniel Graziotin, and Miikka Kuutila. The evolution of sentiment analysis - A review of research topics, venues, and top cited papers. Computer Science Review, 27:16-32, February 2018. URL: https://doi.org/10.1016/j.cosrev.2017.10.002.
  17. Eric T. Nalisnick and Henry S. Baird. Character-to-Character Sentiment Analysis in Shakespeare’s Plays. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 479-483, Sofia, Bulgaria, August 2013. Association for Computational Linguistics. URL: https://www.aclweb.org/anthology/P13-2085.
  18. Winfried Nolting. Die Dialektik der Empfindung: Lessings Trauerspiele "Miss Sara Sampson" und "Emilia Galotti": mit einer Einleitung, gemischte Gefühle: zur Problematik eines explikativen Verstehens des Empfindung. Number 1 in Studien zu einer Geschichte der literarischen Empfindung / Winfried Nolting. F. Steiner Verlag Wiesbaden, Stuttgart, 1986. Google Scholar
  19. Federico Pianzola, Simone Rebora, and Gerhard Lauer. Wattpad as a resource for literary studies. Quantitative and qualitative examples of the importance of digital social reading and readers’ comments in the margins. PLOS ONE, 15(1):e0226708, 2020. Publisher: Public Library of Science. URL: https://doi.org/10.1371/journal.pone.0226708.
  20. Robert Plutchik. Emotion, a Psychoevolutionary Synthesis. Harper & Row, 1980. Google-Books-ID: G5t9AAAAMAAJ. Google Scholar
  21. Andrew J. Reagan, Lewis Mitchell, Dilan Kiley, Christopher M. Danforth, and Peter Sheridan Dodds. The emotional arcs of stories are dominated by six basic shapes. EPJ Data Science, 5(1):31, 2016. arXiv: 1606.07772. URL: https://doi.org/10.1140/epjds/s13688-016-0093-1.
  22. Nils Reiter. Anleitung zur Erstellung von Annotationsrichtlinien. In Reflektierte algorithmische Textanalyse, pages 193-202. De Gruyter, 2020. URL: https://doi.org/10.1515/9783110693973-009.
  23. Thomas Schmidt. Distant reading sentiments and emotions in historic german plays. In Abstract Booklet, DH_Budapest_2019, pages 57-60. Budapest, Hungary, September 2019. URL: https://epub.uni-regensburg.de/43592/.
  24. Thomas Schmidt, Marlene Bauer, Florian Habler, Hannes Heuberger, Florian Pilsl, and Christian Wolff. Der einsatz von distant reading auf einem korpus deutschsprachiger songtexte. In Christof Schöch, editor, DHd 2020: Spielräume; Digital Humanities zwischen Modellierung und Interpretation. Konferenzabstracts; Universität Paderborn, 02. bis 06. März 2020, pages 296-300. Paderborn, Germany, 2020. URL: https://epub.uni-regensburg.de/43704/.
  25. Thomas Schmidt and Manuel Burghardt. An Evaluation of Lexicon-based Sentiment Analysis Techniques for the Plays of Gotthold Ephraim Lessing. In Proceedings of the Second Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature, pages 139-149, Santa Fe, New Mexico, August 2018. Association for Computational Linguistics. URL: https://www.aclweb.org/anthology/W18-4516.
  26. Thomas Schmidt and Manuel Burghardt. Toward a Tool for Sentiment Analysis for German Historic Plays. In Michael Piotrowski, editor, COMHUM 2018: Book of Abstracts for the Workshop on Computational Methods in the Humanities 2018, pages 46-48, Lausanne, Switzerland, June 2018. Laboratoire laussannois d'informatique et statistique textuelle. URL: https://zenodo.org/record/1312779.
  27. Thomas Schmidt, Manuel Burghardt, and Katrin Dennerlein. "Kann man denn auch nicht lachend sehr ernsthaft sein?" – Zum Einsatz von Sentiment Analyse-Verfahren für die quantitative Untersuchung von Lessings Dramen. In Georg Vogeler, editor, Kritik der digitalen Vernunft. Abstracts zur Jahrestagung des Verbandes Digital Humanities im deutschsprachigen Raum, 26.02. - 02.03.2018 an der Universität zu Köln, pages 244-249, Köln, 2018. Universitäts- und Stadtbibliothek Köln. Google Scholar
  28. Thomas Schmidt, Manuel Burghardt, and Katrin Dennerlein. Sentiment annotation of historic german plays: An empirical study on annotation behavior. In Sandra Kübler and Heike Zinsmeister, editors, annDH 2018, Proceedings of the Workshop on Annotation in Digital Humanities 2018 (annDH 2018), Sofia, Bulgaria, August 6-10, 2018, pages 47-52. RWTH Aachen, Aachen, August 2018. URL: https://epub.uni-regensburg.de/43701/.
  29. Thomas Schmidt, Manuel Burghardt, Katrin Dennerlein, and Christian Wolff. Katharsis - a tool for computational drametrics. In Book of Abstracts, Digital Humanities Conference 2019 (DH 2019). Utrecht, Netherlands, 2019. URL: https://epub.uni-regensburg.de/43579/.
  30. Thomas Schmidt, Manuel Burghardt, Katrin Dennerlein, and Christian Wolff. Sentiment annotation for lessing’s plays: Towards a language resource for sentiment analysis on german literary texts. In Thierry Declerck and John P. McCrae, editors, 2nd Conference on Language, Data and Knowledge (LDK 2019), pages 45-50. RWTH Aachen, Aachen, May 2019. URL: https://epub.uni-regensburg.de/43569/.
  31. Thomas Schmidt, Manuel Burghardt, and Christian Wolff. Herausforderungen für Sentiment Analysis-Verfahren bei literarischen Texten. In Manuel Burghardt and Claudia Müller-Birn, editors, INF-DH-2018, Berlin, Germany, September 2018. Gesellschaft für Informatik e.V. URL: https://doi.org/10.18420/infdh2018-16.
  32. Thomas Schmidt, Manuel Burghardt, and Christian Wolff. Toward Multimodal Sentiment Analysis of Historic Plays: A Case Study with Text and Audio for Lessing’s Emilia Galotti. In Costanza Navarretta, Manex Agirrezabal, and Bente Maegaard, editors, Proceedings of the Digital Humanities in the Nordic Countries 4th Conference, volume 2364 of CEUR Workshop Proceedings, pages 405-414, Copenhagen, Denmark, March 2019. CEUR-WS.org. URL: http://ceur-ws.org/Vol-2364/37_paper.pdf.
  33. Thomas Schmidt, Johanna Dangel, and Christian Wolff. Senttext: A tool for lexicon-based sentiment analysis in digital humanities. In Thomas Schmidt and Christian Wolff, editors, Information Science and its Neighbors from Data Science to Digital Humanities. Proceedings of the 16th International Symposium of Information Science (ISI 2021), volume 74, pages 156-172. Werner Hülsbusch, Glückstadt, 2021. URL: https://epub.uni-regensburg.de/44943/.
  34. Thomas Schmidt, Marco Jakob, and Christian Wolff. Annotator-centered design: Towards a tool for sentiment and emotion annotation. In Claude Draude, Martin Lange, and Bernhard Sick, editors, INFORMATIK 2019: 50 Jahre Gesellschaft für Informatik – Informatik für Gesellschaft (Workshop-Beiträge), pages 77-85, Bonn, 2019. Gesellschaft für Informatik e.V. URL: https://doi.org/10.18420/inf2019_ws08.
  35. Thomas Schmidt, Florian Kaindl, and Christian Wolff. Distant reading of religious online communities: A case study for three religious forums on reddit. In DHN, pages 157-172, Riga, Latvia, 2020. Google Scholar
  36. Thomas Schmidt, Brigitte Winterl, Milena Maul, Alina Schark, Andrea Vlad, and Christian Wolff. Inter-rater agreement and usability: A comparative evaluation of annotation tools for sentiment annotation. In Claude Draude, Martin Lange, and Bernhard Sick, editors, INFORMATIK 2019: 50 Jahre Gesellschaft für Informatik – Informatik für Gesellschaft (Workshop-Beiträge), pages 121-133, Bonn, 2019. Gesellschaft für Informatik e.V. URL: https://doi.org/10.18420/inf2019_ws12.
  37. Anja Schonlau. Emotionen im Dramentext: eine methodische Grundlegung mit exemplarischer Analyse zu Neid und Intrige 1750-1800. Number Band 25 in Deutsche Literatur. De Gruyter, Berlin Boston, 2017. OCLC: 978262308. Google Scholar
  38. Rachele Sprugnoli, Sara Tonelli, Alessandro Marchetti, and Giovanni Moretti. Towards sentiment analysis for historical texts. Digital Scholarship in the Humanities, 31:762-772, 2015. Publisher: Oxford : Oxford University Press. URL: https://doi.org/10.1093/llc/fqv027.
  39. Hermann Wiegmann, editor. Die ästhetische Leidenschaft: Texte zur Affektenlehre im 17. und 18. Jahrhundert. Number 27 in Germanistische Texte und Studien. Olms, Hildesheim, 1987. OCLC: 15741918. Google Scholar
  40. Mehmet Can Yavuz. Analyses of Character Emotions in Dramatic Works by Using EmoLex Unigrams. In Proceedings of the Seventh Italian Conference on Computational Linguistics, CLiC-it’20. Bologna, Italy, 2021. Google Scholar
  41. Emily Öhman. Challenges in Annotation: Annotator Experiences from a Crowdsourced Emotion Annotation Task. In Proceedings of the Digital Humanities in the Nordic Countries 5th Conference, pages 293-301. CEUR Workshop Proceedings, 2020. Google Scholar
  42. Emily Sofi Öhman and Kaisla S. A. Kajava. Sentimentator: Gamifying Fine-grained Sentiment Annotation. In Digital Humanities in the Nordic Countries 2018, pages 98-110. CEUR Workshop Proceedings, 2018. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH Imprint Privacy Contact