<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="knuples-etal-2024-gender">
    <titleInfo>
        <title>Gender Identity in Pretrained Language Models: An Inclusive Approach to Data Creation and Probing</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Urban</namePart>
        <namePart type="family">Knupleš</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Agnieszka</namePart>
        <namePart type="family">Falenska</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Filip</namePart>
        <namePart type="family">Miletić</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2024</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yaser</namePart>
            <namePart type="family">Al-Onaizan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Mohit</namePart>
            <namePart type="family">Bansal</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yun-Nung</namePart>
            <namePart type="family">Chen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Miami, Florida, USA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Pretrained language models (PLMs) have been shown to encode binary gender information of text authors, raising the risk of skewed representations and downstream harms. This effect is yet to be examined for transgender and non-binary identities, whose frequent marginalization may exacerbate harmful system behaviors. Addressing this gap, we first create TRANsCRIPT, a corpus of YouTube transcripts from transgender, cisgender, and non-binary speakers. Using this dataset, we probe various PLMs to assess if they encode the gender identity information, examining both frozen and fine-tuned representations as well as representations for inputs with author-specific words removed. Our findings reveal that PLM representations encode information for all gender identities but to different extents. The divergence is most pronounced for cis women and non-binary individuals, underscoring the critical need for gender-inclusive approaches to NLP systems.</abstract>
    <identifier type="citekey">knuples-etal-2024-gender</identifier>
    <identifier type="doi">10.18653/v1/2024.findings-emnlp.680</identifier>
    <location>
        <url>https://aclanthology.org/2024.findings-emnlp.680</url>
    </location>
    <part>
        <date>2024-11</date>
        <extent unit="page">
            <start>11612</start>
            <end>11631</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Gender Identity in Pretrained Language Models: An Inclusive Approach to Data Creation and Probing
%A Knupleš, Urban
%A Falenska, Agnieszka
%A Miletić, Filip
%Y Al-Onaizan, Yaser
%Y Bansal, Mohit
%Y Chen, Yun-Nung
%S Findings of the Association for Computational Linguistics: EMNLP 2024
%D 2024
%8 November
%I Association for Computational Linguistics
%C Miami, Florida, USA
%F knuples-etal-2024-gender
%X Pretrained language models (PLMs) have been shown to encode binary gender information of text authors, raising the risk of skewed representations and downstream harms. This effect is yet to be examined for transgender and non-binary identities, whose frequent marginalization may exacerbate harmful system behaviors. Addressing this gap, we first create TRANsCRIPT, a corpus of YouTube transcripts from transgender, cisgender, and non-binary speakers. Using this dataset, we probe various PLMs to assess if they encode the gender identity information, examining both frozen and fine-tuned representations as well as representations for inputs with author-specific words removed. Our findings reveal that PLM representations encode information for all gender identities but to different extents. The divergence is most pronounced for cis women and non-binary individuals, underscoring the critical need for gender-inclusive approaches to NLP systems.
%R 10.18653/v1/2024.findings-emnlp.680
%U https://aclanthology.org/2024.findings-emnlp.680
%U https://doi.org/10.18653/v1/2024.findings-emnlp.680
%P 11612-11631

Markdown (Informal)

[Gender Identity in Pretrained Language Models: An Inclusive Approach to Data Creation and Probing](https://aclanthology.org/2024.findings-emnlp.680) (Knupleš et al., Findings 2024)

• Gender Identity in Pretrained Language Models: An Inclusive Approach to Data Creation and Probing (Knupleš et al., Findings 2024)

ACL

• Urban Knupleš, Agnieszka Falenska, and Filip Miletić. 2024. Gender Identity in Pretrained Language Models: An Inclusive Approach to Data Creation and Probing. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 11612–11631, Miami, Florida, USA. Association for Computational Linguistics.