On Homophony and Rényi Entropy

Tiago Pimentel, Clara Meister, Simone Teufel, Ryan Cotterell


Abstract
Homophony’s widespread presence in natural languages is a controversial topic. Recent theories of language optimality have tried to justify its prevalence, despite its negative effects on cognitive processing time, e.g., Piantadosi et al. (2012) argued homophony enables the reuse of efficient wordforms and is thus beneficial for languages. This hypothesis has recently been challenged by Trott and Bergen (2020), who posit that good wordforms are more often homophonous simply because they are more phonotactically probable. In this paper, we join in on the debate. We first propose a new information-theoretic quantification of a language’s homophony: the sample Rényi entropy. Then, we use this quantification to revisit Trott and Bergen’s claims. While their point is theoretically sound, a specific methodological issue in their experiments raises doubts about their results. After addressing this issue, we find no clear pressure either towards or against homophony—a much more nuanced result than either Piantadosi et al.’s or Trott and Bergen’s findings.
Anthology ID:
2021.emnlp-main.653
Volume:
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2021
Address:
Online and Punta Cana, Dominican Republic
Editors:
Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
8284–8293
Language:
URL:
https://aclanthology.org/2021.emnlp-main.653
DOI:
10.18653/v1/2021.emnlp-main.653
Bibkey:
Cite (ACL):
Tiago Pimentel, Clara Meister, Simone Teufel, and Ryan Cotterell. 2021. On Homophony and Rényi Entropy. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8284–8293, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
On Homophony and Rényi Entropy (Pimentel et al., EMNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.emnlp-main.653.pdf
Video:
 https://aclanthology.org/2021.emnlp-main.653.mp4
Data
CELEX