Annotating FrameNet via Structure-Conditioned Language Generation

Xinyue Cui, Swabha Swayamdipta


Abstract
Despite the remarkable generative capabilities of language models in producing naturalistic language, their effectiveness on explicit manipulation and generation of linguistic structures remain understudied. In this paper, we investigate the task of generating new sentences preserving a given semantic structure, following the FrameNet formalism. We propose a framework to produce novel frame-semantically annotated sentences following an overgenerate-and-filter approach. Our results show that conditioning on rich, explicit semantic information tends to produce generations with high human acceptance, under both prompting and finetuning. Our generated frame-semantic structured annotations are effective at training data augmentation for frame-semantic role labeling in low-resource settings; however, we do not see benefits under higher resource settings. Our study concludes that while generating high-quality, semantically rich data might be within reach, the downstream utility of such generations remains to be seen, highlighting the outstanding challenges with automating linguistic annotation tasks.
Anthology ID:
2024.acl-short.63
Volume:
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Month:
August
Year:
2024
Address:
Bangkok, Thailand
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
681–692
Language:
URL:
https://aclanthology.org/2024.acl-short.63
DOI:
10.18653/v1/2024.acl-short.63
Bibkey:
Cite (ACL):
Xinyue Cui and Swabha Swayamdipta. 2024. Annotating FrameNet via Structure-Conditioned Language Generation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 681–692, Bangkok, Thailand. Association for Computational Linguistics.
Cite (Informal):
Annotating FrameNet via Structure-Conditioned Language Generation (Cui & Swayamdipta, ACL 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.acl-short.63.pdf