<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="margatina-etal-2019-attention">
    <titleInfo>
        <title>Attention-based Conditioning Methods for External Knowledge Integration</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Katerina</namePart>
        <namePart type="family">Margatina</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Christos</namePart>
        <namePart type="family">Baziotis</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Alexandros</namePart>
        <namePart type="family">Potamianos</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2019-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Korhonen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">David</namePart>
            <namePart type="family">Traum</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Lluís</namePart>
            <namePart type="family">Màrquez</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Florence, Italy</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>In this paper, we present a novel approach for incorporating external knowledge in Recurrent Neural Networks (RNNs). We propose the integration of lexicon features into the self-attention mechanism of RNN-based architectures. This form of conditioning on the attention distribution, enforces the contribution of the most salient words for the task at hand. We introduce three methods, namely attentional concatenation, feature-based gating and affine transformation. Experiments on six benchmark datasets show the effectiveness of our methods. Attentional feature-based gating yields consistent performance improvement across tasks. Our approach is implemented as a simple add-on module for RNN-based models with minimal computational overhead and can be adapted to any deep neural architecture.</abstract>
    <identifier type="citekey">margatina-etal-2019-attention</identifier>
    <identifier type="doi">10.18653/v1/P19-1385</identifier>
    <location>
        <url>https://aclanthology.org/P19-1385</url>
    </location>
    <part>
        <date>2019-07</date>
        <extent unit="page">
            <start>3944</start>
            <end>3951</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Attention-based Conditioning Methods for External Knowledge Integration
%A Margatina, Katerina
%A Baziotis, Christos
%A Potamianos, Alexandros
%Y Korhonen, Anna
%Y Traum, David
%Y Màrquez, Lluís
%S Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
%D 2019
%8 July
%I Association for Computational Linguistics
%C Florence, Italy
%F margatina-etal-2019-attention
%X In this paper, we present a novel approach for incorporating external knowledge in Recurrent Neural Networks (RNNs). We propose the integration of lexicon features into the self-attention mechanism of RNN-based architectures. This form of conditioning on the attention distribution, enforces the contribution of the most salient words for the task at hand. We introduce three methods, namely attentional concatenation, feature-based gating and affine transformation. Experiments on six benchmark datasets show the effectiveness of our methods. Attentional feature-based gating yields consistent performance improvement across tasks. Our approach is implemented as a simple add-on module for RNN-based models with minimal computational overhead and can be adapted to any deep neural architecture.
%R 10.18653/v1/P19-1385
%U https://aclanthology.org/P19-1385
%U https://doi.org/10.18653/v1/P19-1385
%P 3944-3951

Markdown (Informal)

[Attention-based Conditioning Methods for External Knowledge Integration](https://aclanthology.org/P19-1385) (Margatina et al., ACL 2019)

• Attention-based Conditioning Methods for External Knowledge Integration (Margatina et al., ACL 2019)

ACL

• Katerina Margatina, Christos Baziotis, and Alexandros Potamianos. 2019. Attention-based Conditioning Methods for External Knowledge Integration. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3944–3951, Florence, Italy. Association for Computational Linguistics.