<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="misra-etal-2020-exploring">
    <titleInfo>
        <title>Exploring BERT’s Sensitivity to Lexical Cues using Tests from Semantic Priming</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Kanishka</namePart>
        <namePart type="family">Misra</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Allyson</namePart>
        <namePart type="family">Ettinger</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Julia</namePart>
        <namePart type="family">Rayz</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2020-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2020</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Trevor</namePart>
            <namePart type="family">Cohn</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yulan</namePart>
            <namePart type="family">He</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yang</namePart>
            <namePart type="family">Liu</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Online</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Models trained to estimate word probabilities in context have become ubiquitous in natural language processing. How do these models use lexical cues in context to inform their word probabilities? To answer this question, we present a case study analyzing the pre-trained BERT model with tests informed by semantic priming. Using English lexical stimuli that show priming in humans, we find that BERT too shows “priming”, predicting a word with greater probability when the context includes a related word versus an unrelated one. This effect decreases as the amount of information provided by the context increases. Follow-up analysis shows BERT to be increasingly distracted by related prime words as context becomes more informative, assigning lower probabilities to related words. Our findings highlight the importance of considering contextual constraint effects when studying word prediction in these models, and highlight possible parallels with human processing.</abstract>
    <identifier type="citekey">misra-etal-2020-exploring</identifier>
    <identifier type="doi">10.18653/v1/2020.findings-emnlp.415</identifier>
    <location>
        <url>https://aclanthology.org/2020.findings-emnlp.415</url>
    </location>
    <part>
        <date>2020-11</date>
        <extent unit="page">
            <start>4625</start>
            <end>4635</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Exploring BERT’s Sensitivity to Lexical Cues using Tests from Semantic Priming
%A Misra, Kanishka
%A Ettinger, Allyson
%A Rayz, Julia
%Y Cohn, Trevor
%Y He, Yulan
%Y Liu, Yang
%S Findings of the Association for Computational Linguistics: EMNLP 2020
%D 2020
%8 November
%I Association for Computational Linguistics
%C Online
%F misra-etal-2020-exploring
%X Models trained to estimate word probabilities in context have become ubiquitous in natural language processing. How do these models use lexical cues in context to inform their word probabilities? To answer this question, we present a case study analyzing the pre-trained BERT model with tests informed by semantic priming. Using English lexical stimuli that show priming in humans, we find that BERT too shows “priming”, predicting a word with greater probability when the context includes a related word versus an unrelated one. This effect decreases as the amount of information provided by the context increases. Follow-up analysis shows BERT to be increasingly distracted by related prime words as context becomes more informative, assigning lower probabilities to related words. Our findings highlight the importance of considering contextual constraint effects when studying word prediction in these models, and highlight possible parallels with human processing.
%R 10.18653/v1/2020.findings-emnlp.415
%U https://aclanthology.org/2020.findings-emnlp.415
%U https://doi.org/10.18653/v1/2020.findings-emnlp.415
%P 4625-4635

Markdown (Informal)

[Exploring BERT’s Sensitivity to Lexical Cues using Tests from Semantic Priming](https://aclanthology.org/2020.findings-emnlp.415) (Misra et al., Findings 2020)

• Exploring BERT’s Sensitivity to Lexical Cues using Tests from Semantic Priming (Misra et al., Findings 2020)

ACL

• Kanishka Misra, Allyson Ettinger, and Julia Rayz. 2020. Exploring BERT’s Sensitivity to Lexical Cues using Tests from Semantic Priming. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4625–4635, Online. Association for Computational Linguistics.