Multi-Granular Sequence Encoding via Dilated Compositional Units for Reading Comprehension

Yi Tay, Anh Tuan Luu, Siu Cheung Hui


Abstract
Sequence encoders are crucial components in many neural architectures for learning to read and comprehend. This paper presents a new compositional encoder for reading comprehension (RC). Our proposed encoder is not only aimed at being fast but also expressive. Specifically, the key novelty behind our encoder is that it explicitly models across multiple granularities using a new dilated composition mechanism. In our approach, gating functions are learned by modeling relationships and reasoning over multi-granular sequence information, enabling compositional learning that is aware of both long and short term information. We conduct experiments on three RC datasets, showing that our proposed encoder demonstrates very promising results both as a standalone encoder as well as a complementary building block. Empirical results show that simple Bi-Attentive architectures augmented with our proposed encoder not only achieves state-of-the-art / highly competitive results but is also considerably faster than other published works.
Anthology ID:
D18-1238
Volume:
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
Month:
October-November
Year:
2018
Address:
Brussels, Belgium
Editors:
Ellen Riloff, David Chiang, Julia Hockenmaier, Jun’ichi Tsujii
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
2141–2151
Language:
URL:
https://aclanthology.org/D18-1238
DOI:
10.18653/v1/D18-1238
Bibkey:
Cite (ACL):
Yi Tay, Anh Tuan Luu, and Siu Cheung Hui. 2018. Multi-Granular Sequence Encoding via Dilated Compositional Units for Reading Comprehension. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2141–2151, Brussels, Belgium. Association for Computational Linguistics.
Cite (Informal):
Multi-Granular Sequence Encoding via Dilated Compositional Units for Reading Comprehension (Tay et al., EMNLP 2018)
Copy Citation:
PDF:
https://aclanthology.org/D18-1238.pdf
Data
NarrativeQARACESQuADSearchQA