An Empirical Analysis of Leveraging Knowledge for Low-Resource Task-Oriented Semantic Parsing

Mayank Kulkarni, Aoxiao Zhong, Nicolas Guenon des mesnards, Sahar Movaghati, Mukund Sridhar, He Xie, Jianhua Lu


Abstract
Task-oriented semantic parsing has drawn a lot of interest from the NLP community, and especially the voice assistant industry as it enables representing the meaning of user requests with arbitrarily nested semantics, including multiple intents and compound entities. SOTA models are large seq2seq transformers and require hundreds of thousands of annotated examples to be trained. However annotating such data to bootstrap new domains or languages is expensive and error-prone, especially for requests made of nested semantics. In addition large models easily break the tight latency constraints imposed in a user-facing production environment. As part of this work we explore leveraging external knowledge to improve model accuracy in low-resource and low-compute settings. We demonstrate that using knowledge-enhanced encoders inside seq2seq models does not result in performance gains by itself, but jointly learning to uncover entities in addition to the parse generation is a simple yet effective way of improving performance across the board. We show this is especially true in the low-compute scarce-data setting and for entity-rich domains, with relative gains up to 74.48% on the TOPv2 dataset.
Anthology ID:
2023.findings-acl.123
Volume:
Findings of the Association for Computational Linguistics: ACL 2023
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1955–1969
Language:
URL:
https://aclanthology.org/2023.findings-acl.123
DOI:
10.18653/v1/2023.findings-acl.123
Bibkey:
Cite (ACL):
Mayank Kulkarni, Aoxiao Zhong, Nicolas Guenon des mesnards, Sahar Movaghati, Mukund Sridhar, He Xie, and Jianhua Lu. 2023. An Empirical Analysis of Leveraging Knowledge for Low-Resource Task-Oriented Semantic Parsing. In Findings of the Association for Computational Linguistics: ACL 2023, pages 1955–1969, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
An Empirical Analysis of Leveraging Knowledge for Low-Resource Task-Oriented Semantic Parsing (Kulkarni et al., Findings 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.findings-acl.123.pdf