See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/377300345

A Content Overlap Analysis of bilingualism questionnaires: Considering

diversity

Article in Bilingualism: Language and Cognition · January 2024

DOI: 10.1017/S1366728923000767

CITATION READS

1 57

6 authors, including:

Irina Smirnova-Godoy Olivia McColl

Carleton University Carleton University
1 PUBLICATION 1 CITATION 1 PUBLICATION 1 CITATION

SEE PROFILE SEE PROFILE

John G Grundy John A. E. Anderson

Iowa State University Carleton University
48 PUBLICATIONS 1,131 CITATIONS 74 PUBLICATIONS 2,480 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by John G Grundy on 10 January 2024.

The user has requested enhancement of the downloaded file.

 Bilingualism: Language and A Content Overlap Analysis of bilingualism
Cognition
questionnaires: Considering diversity
cambridge.org/bil
Ronessa Dass1, Irina Smirnova-Godoy2, Olivia McColl2, John G. Grundy3,
Gigi Luk4 and John A. E. Anderson2
1
Research Notes McMaster University, Hamilton, Ontario, Canada; 2Carleton University, Ottawa, Ontario, Canada; 3Iowa State
University, Ames, IA, USA and 4McGill University, Montreal, Quebec, Canada
Cite this article: Dass, R., Smirnova-Godoy,
I., McColl, O., Grundy, J.G., Luk, G., & Abstract
Anderson, J.A.E. (2024). A Content Overlap
Analysis of bilingualism questionnaires: Bilingualism is a multifaceted experience that researchers have examined using various ques-
Considering diversity. Bilingualism: Language tionnaires to gain insights and characterize the experience. However, there are several issues
and Cognition, 1–7. https://doi.org/10.1017/ related to questionnaire choice. To address this, we applied CONTENT OVERLAP ANALYSIS to
S1366728923000767
seven prevalent bilingualism questionnaires, assessing their affinity. We found little overlap
Received: 22 February 2023 in these questionnaires; most had fewer than 15% of items in common, suggesting they cap-
Revised: 10 September 2023 ture different aspects of the bilingual experience and provide complementary rather than
Accepted: 11 October 2023 redundant data for researchers. Our investigation highlights the importance of choosing a
Keywords:
bilingualism assessment tool to carefully fit research questions and sample language
bilingualism; questionnaires; content-overlap; experiences.
Jaccard; consistency

Corresponding author:
John A. E. Anderson; Introduction
Email: [email protected]
When researchers say they study “bilingualism,” there is an assumption that they study the
same phenomenon. Yet, bilingualism encompasses various life experiences that are complex,
interacting, and dynamic across the lifespan. To document and describe this experience,
researchers design questionnaires and definitions focusing on different aspects of bilingual
experiences (e.g., age of acquisition, usage, proficiency, dominance, preference). The piecemeal
approach is not concordant with how “bilingualism” is broadly used to describe the ability to
use multiple languages (Grosjean & Li, 2013). In this paper, we assess the tools that are preva-
lently used to document adult bilingual experiences. Importantly, we apply a new method to
evaluate the magnitude of consensus across these tools. The purpose is to highlight the
strengths and diversity of the tools for researchers to evaluate and choose the appropriate
tool for their research.
Debates around measuring bilingualism have been ongoing since the field’s inception. For
example, Grosjean (1989) critiqued the prevailing ratio-based method employed by neurolin-
guists and psychologists and argued that there are contextual differences in how and when
multiple languages are used (also see Complementarity Principles described in Grosjean,
2016). Similarly, Green and Abutalebi (2013) pointed out multiple language contexts for lin-
guistically diverse individuals – suggesting many different sides to bi/multilingual identities
beyond pure linguistic knowledge ratios. Building on these ideas, Titone and Tiv (2022)
have recently suggested that Bronfenbrenner’s (1992) ecological systems approach, which
examines the bilingual experience from microscale to large-scale temporal levels, should be
implemented in bilingualism research. Though some current questionnaires collect relevant
information on a person’s bilingual experience or context, questionnaires designed for differ-
ent studies or locales may highlight diverse experiences.
There has been renewed interest in defining bilingualism more precisely and reaching a
consilience among bilingualism questionnaires and researchers. Kašćelan et al. (2022) identi-
fied differences in operationalization, components, scale types, and precision in a recent con-
tent analysis of bilingualism questionnaires in children. A Delphi census surveying researchers
and practitioners also noted the inconsistency in this area (De Cat et al., 2023). While it is
© The Author(s), 2024. Published by logical to propose a new questionnaire that is as comprehensive as possible, creating another
Cambridge University Press. This is an Open ultimate bilingualism questionnaire is perhaps not as helpful as identifying shared and unique
Access article, distributed under the terms of
aspects of existing questionnaires, so researchers can make informed choices about which
the Creative Commons Attribution licence
(http://creativecommons.org/licenses/by/4.0/), instrument to use for their research questions (see, for example, this editorial to a special
which permits unrestricted re-use, distribution issue in Luk & Esposito, 2020).
and reproduction, provided the original article To identify the commonalities and uniqueness of existing questionnaires, we apply a quan-
is properly cited. titative content overlap analysis of existing bilingualism questionnaires. This approach was first
used by Fried (2017), who examined widely used measures of depression and concluded that

This article has earned badges for transparent research practices: Open Data and Open Materials. For details see the
Data Availability Statement.

https://doi.org/10.1017/S1366728923000767 Published online by Cambridge University Press

 2 Ronessa Dass et al.

most varied considerably in how they measure the same under- seven papers included for final extraction (see Figure 1 for full
lying constructs. Our goal was to identify common items across Prisma diagram). The full dataset and keywords can be found on
bilingualism questionnaires to assess their coverage and breadth the OSF: https://osf.io/s4qug/?view_only=9e6efe3eca434446a503
as indications of consensus and diversity. In our overlap analysis, b1d7ef32cf20.
we identified 50 unique categories of bilingualism items in the Eligibility criteria were adapted from Kašćelan et al. (2022).
seven questionnaires. We calculated Jaccard indices for each ques- Questionnaires were included if they were in English and men-
tionnaire pair to quantify the similarity between two sets of items tioned bilingualism. Questionnaires were excluded if they: (1)
to calculate an overlap score for each bilingualism questionnaire were parents’ reports of children; (2) concerned foreign language
relative to all others (Jaccard, 1912). Fried (2017) states that ques- learning; (3) concerned speech and language disorders; (4) were
tionnaires are similar if they have overlapping items. If there is lit- about bilingual education; (5) had six or fewer questions about
tle overlap across questionnaires, then it is likely that they are not bilingualism; (6) not focused on language; (7) duplicates; and
quantifying constructs similarly. He notes that this is problematic (8) if the full questionnaire could not be obtained. We identified
since it decreases the potential for cross-comparisons and general- seven questionnaires that are commonly used to capture and
izations across populations, such as meta-analyses. quantify bilingualism in adults: the Bilingual Dominance Scale
(BDS; Dunn & Fox Tree, 2009), the Bilingualism and Emotion
Questionnaire (BEQ; Dewaele & Pavlenko, n.d.), the Bilingual
Methods Language Profile (BLP; Birdsong et al., 2012; Gertken et al.,
2014), the Bilingualism Switching Questionnaire (BSWQ;
Search strategy
Rodriguez-Fornells et al., 2011), the Language Experience and
Similar to Fried (2017) and Kašćelan et al. (2022), studies were Proficiency Questionnaire (LEAP-Q; Marian et al., 2007), the
identified through a formal search facilitated by Covidence Language History Questionnaire (LHQ3: Li et al., 2020), and
(Veritas Health Innovation, 2022). The search strategy was adapted the Language and Social Background Questionnaire (LSBQ;
from Kašćelan et al. (2022); however, terms regarding pediatric Anderson et al., 2018; Luk & Bialystok, 2013)).
populations were excluded. The search was performed in
PsycINFO, ERIC, Web of Science, and Scopus. Keywords were
Content analysis
combined through the Boolean logic term “AND.” Searches were
limited to items published by May 3rd, 2022. All references were Following Fried (2017), we performed a content analysis on all
imported to Covidence. The search yielded 8866 papers, with seven questionnaires. Questions from each questionnaire were
prepared in three spreadsheets for different procedures: 1) cat-
egorization, 2) within-scale content analysis, and 3) across-scale
content analysis.
For categorization, all questions from each questionnaire were
placed into one spreadsheet and were organized by category and
subcategory (see OSF link above for all categories). For each ques-
tionnaire, if a single question contained multiple sub-questions, the
sub-questions were treated as separate items and assigned to the
appropriate category and subcategory. A total of 222 items across
the seven questionnaires were identified and organized into five
global categories ( production, switching, exposure, subjective state-
ments, identity, history/acquisition). These category labels were
assigned based on the current understanding of bilingualism com-
ponents in the existing literature and the aim of the questionnaires
(see Table 1 for a description of the questionnaires).
Production referred to questions about how participants used
language, including time frequency and the reason for which they
used the language (e.g., “If you have children, what language do
you speak to them in?; BEQ” or “Estimate how many hours per
week you speak language one;” LHQ). Next, switching included
questions that explored participants switching or mixing multiple
languages (e.g., do you switch between languages with your
friends?; “LHQ). Subjective statements included questions that
referred to individuals’ unique perceptions of their language use
(e.g., “how comfortable are you speaking language one?;” LHQ,
“what language do you feel dominant in?”; BDS). The identity cat-
egory is distinct from subjective statements as it only includes
questions related to the relationship between one’s identity and
language use (e.g., “I feel like myself when I speak language
one;” BLP or “Which culture/language do you identify with
more?”; LHQ), rather than their general feelings about language
use. Further, exposure and history/acquisition are similar categor-
ies. We elected to categorize questions that were centered around
Figure 1. Prisma Diagram. duration and frequency of language exposure as “exposure” – for

https://doi.org/10.1017/S1366728923000767 Published online by Cambridge University Press

 Bilingualism: Language and Cognition 3

Table 1. Questionnaire descriptions

The practical difficulty of

Questionnaire Purpose Structure Length administering

Language History To obtain an overview of one’s Fixed scheme (Multiple 27 distinct Comprehensive, however,
Questionnaire (LHQ 3.0) language acquisition and current choice + Likert scale) questions time consuming
use.
Language Experience and To categorize different Fixed scheme (Multiple 6 distinct Good practicality
Proficiency Questionnaire bilingualism proficiencies choice + Likert scale) & questions
(LEAP-Q) open ended
BIlingualism Switching To assess language dominance Fixed scheme (Multiple 7 distinct Good practicality
Questionnaire choice + Likert scale) questions
BIlingualism Emotion To understand the relationship Fixed scheme (Multiple 25 distinct Comprehensive, however,
Questionnaire between language and emotion choice + Likert scale) questions time consuming
Language and Social To understand bilingualism within Fixed scheme (Multiple 22 distinct Comprehensive, however,
Background Questionnaire a socio-cultural context choice + Likert scale) questions time consuming
BIlingual Language Profile To assess language dominance Fixed scheme (Multiple 19 distinct Good practicality
choice + Likert scale) questions
Bilingual Dominance Scale To assess language dominance Fixed scheme (Multiple 8 distinct
choice + Likert scale) questions

example, “Please list the percentage of time you are currently formula, as described by Fried, is shared/(unique1 + unique2 +
exposed to each language?” (LHQ) – whereas questions focusing shared), where “shared” refers to items shared between the two
on historical language acquisition such as “at what age did you questionnaires, and unique1 and unique2 refer to the items that
learn the following languages?” (BEQ) were categorized as “his- are unique to each questionnaire, respectively. We follow Fried’s
tory/acquisition.” Therefore the difference is about the kind of (2017) interpretation of Jaccard index strength which he adapted
information that would be elicited – in the first case, the question from Evans (1996) for correlation coefficients very weak 0.00–
would focus on the duration or amount of language exposure, 0.19, weak 0.20–0.39, moderate 0.40–0.59, strong 0.60–0.79, and
while in the second case, the question would have pointed to a very strong 0.80–1.0.
single date of language acquisition. Where there were overlaps
in categories across questionnaires, these were resolved through
Vignettes
consensus among all authors.
All seven questionnaires were coded in Qualtrics and adminis-
After initial categorization was complete, a within-questionnaires
tered to lab members in JAEA’s research group. The order of
overlap analysis was conducted to determine whether questionnaires
the questionnaire presentation was randomized. Vignettes were
contained multiple questions assessing the same constructs.
first analyzed within each questionnaire. As scoring manuals for
Following Fried (2017), if questions were worded similarly or in
questionnaires were unavailable, the authors devised a standar-
reverse, they were categorized as one item. This reduced the
dized scoring sheet across all questions (see OSF link for full
Bilingual Language Profile (BLP) from 19 items to 12, the
explanation). All scores were set to fit within the Likert scale
Language Experience and Proficiency Questionnaire (LEAP-Q)
values in a given questionnaire. Additionally, some items within
from 6 to 5, and the BDS from 11 to 8. Overall, this reduced the
a questionnaire were not accounted for if they provided context
total number of items from 222 to 211. Next, we conducted the
rather than objectively measuring bilingualism. All scores were
across-scale content analysis. Taking the established list of categories
re-scaled from original values to 0 (least bilingual) to 1 (most
and subcategories, we organized all questions from each question-
bilingual) to aid visual comparison. Vignette analyses were
naire into one master spreadsheet to identify how frequently each
included to illustrate the similarities and differences in categoriza-
category appeared in each questionnaire. Using information from
tion across distinct categories. Given the small sample size (see
the across-scales analysis, two construct tables were used to produce
results for a description of participant demographics), this ana-
two matrices (A and B). Matrix A had three codes: an item was
lysis was meant to serve solely as an example of our findings
coded “2” if it was specifically featured in the scale, “1” if it was gen-
from the content analysis rather than a comprehensive sample
erally featured in the scale, or “0” if it was not featured in the scale.
of the bilingual population.
The second matrix, Matrix B, had two codes: an item was coded “1”
if it was featured in the scale or “0” if it was not. Twenty-seven idio-
syncratic categories were identified. Results
Content overlap
Statistical analyses
Two hundred and eleven items from seven bilingualism question-
Following Fried (2017), Jaccard similarity indices were calculated naires were organized into 51 categories. On average, categories
in R (R Core Team, 2022), adapting the code Fried shared (see appeared 1.607 times in the seven questionnaires (mode=1,
Fried, 2017 for his OSF link). For each pair of questionnaires, median=1). None of the established categories appeared in all
the Jaccard index was calculated by computing the ratio of the questionnaires. The most common categories were production
number of shared items to unique items and shared items. The (math) and subjective statements (speaking), which both

https://doi.org/10.1017/S1366728923000767 Published online by Cambridge University Press

 4 Ronessa Dass et al.

Table 2. Category breakdown by Questionnaire

Categories captured Adjusted questionnaire length Idiosyncratic categories Specific categories Compound categories
Scale (No.) (No.) (%) (%) (%)

LEAP-Q 2 5 0 4 0
BLP 7 12 2 16 0
BEQ 10 55 4 33 0
LHQ 27 8 29 53 4
BSQ 8 13 2 4 0
BDS 9 8 4 16 0
LSBQ 17 28 12 53 2

appeared in four questionnaires (production (math): LHQ, BEQ, recomputed the Jaccard Index, which rose to 0.42. However,
BDS, and BLP; subjective statements (speaking): LHQ, BEQ, BDS, such a broad approach is different from how this measure has
LEAPQ). The LHQ was present across the largest number of cat- traditionally been applied and does not capture item-wise agree-
egories (27), having the most coverage. See Table 2 and Figure 2A ment of the questionnaires. Thus, our initial results are the most
for representations of how many times each questionnaire cap- reliable.
tured a certain category.
Table 2 and Figure 2A summarize the amount of idiosyncratic, Vignette overview
compounded, and specific categories, as well as the total adjusted Six participants (M= 2, F= 4), aged 22-64, completed the ques-
number of items per questionnaire. The LHQ has the largest tionnaires. Participants were primarily students, with education
number of idiosyncratic questions, mainly because it contributes levels ranging from bachelor’s to doctoral degrees and parental
to the MOST categories (27) and contains the most items (88), education levels from high school to doctoral. Participants mainly
meaning that it covers constructs that other questionnaires do resided in the Ottawa region, with some having immigrated to
not. The LSBQ has the second largest number of idiosyncratic cat- Canada from 1989-2009. The sample was diverse, with ethnicities
egories (12); however, it is also a longer questionnaire (28 items including White, South Asian, Middle Eastern, and
and 17 categories). The remainder of the questionnaires fall Indo-Caribbean.
within the range of 2-4 idiosyncratic categories, except for the We first present each participant’s performance within each
LEAP-Q, which contained no idiosyncratic categories; however, questionnaire. The purple line represents the participant categor-
it is also a shorter questionnaire (5 items), meaning it covers ized as “most bilingual,” and the green line represents the “least
fewer categories (2). bilingual” individual among the six. As shown in Figure 3, “the
The Jaccard Index was next used to estimate the overlap most’’ and “least” bilingual participant differs between question-
between questionnaires. The average overlap was 0.09, which indi- naires. The most bilingual participant in the LHQ and BEQ is
cates minimal content overlap across questionnaires (individual participant 6, and in the BDS, it is participant 4. Conversely,
and mean overlap across questionnaires are presented in the most bilingual person on the LEAP-Q, LSBQ, BSQ, and
Table 3). Notable elements from the overlap table include: BLP is participant 1. The least bilingual participant in the LHQ,
LSBQ, BSQ, and BEQ is participant 3; for the BDS and
1. The LENGTH of questionnaires affected the amount of overlap LEAP-Q, this is participant 6; and for the BLP, it is participant
across questionnaires. For example, the LEAP-Q has the fewest 5. Interestingly, participant 6 was categorized as the least (BDS)
number of items (5) and a low overlap (0.04). This could indi- and most (BLP) bilingual participant in different questionnaires.
cate that questionnaires with too few items may need more cat- It should be acknowledged that 3 out of 6 participants scored
egories to capture all concepts in the field adequately. as the least bilingual on BDS, very likely due to the binary nature
2. The PURPOSE of a questionnaire can affect its overlap with other of scoring items in that questionnaire.
questionnaires. For instance, the BSQ was explicitly created to
look at switching in bilingualism and has a mean overlap of 0.03.
3. Interestingly, the BEQ was also created for a specific purpose Discussion
(to investigate emotion in bilingualism), yet it has the highest Given the diversity of tools characterizing bilingual experiences in
mean overlap of 0.15. It has many items (55); however, it does adults, we identified the commonality and uniqueness across
not appear to have the same low overlap rates as the LSBQ (28 seven prevalently used language questionnaires. The most prom-
items, 0.10 overlap) or the LHQ (88 items, 0.10 overlap). inent finding is that the mean overlap amongst all questionnaires
4. The highest Jaccard index for any two questionnaires was was quite low (0.09). However, differences in length and purpose
shared between the BEQ and BDS, which had an overlap of questionnaires may have accounted for this effect. This low
score of 0.25, which still constitutes weak overlap. overlap is evidence that researchers in the field of bilingualism
5. The LEAP-Q and BSQ had the lowest individual overlap with should not use questionnaires interchangeably and must carefully
other questionnaires. select the most appropriate questionnaire for their selected
research questions and the multilingual experiences of their sam-
To get a sense of how much questionnaires agreed, at least on ples. Additionally, the low overlap suggests that it may be
conceptual categories, we eliminated subcategories and inappropriate to use meta-analysis to synthesize data across

https://doi.org/10.1017/S1366728923000767 Published online by Cambridge University Press

 Bilingualism: Language and Cognition 5

Figure 2. Content Overlap figures for A) language items and B) demographic items from language background questionnaires. LEAP-Q = Language Experience and
Proficiency Questionnaire, BLP = Bilingual Language Profile, BEQ = Bilingualism and Emotion Questionnaire.

multiple studies that have characterized bilinguals using different consideration in these types of scales (Surrain & Luk, 2017).
questionnaires. The lack of overlap in questionnaires was also Researchers currently have to try and glean this information
demonstrated in the vignette analysis, as participants were classi- from questions designed to measure something else (e.g., con-
fied differently in individual questionnaires. struct: switching, questionnaire: LHQ, item: “if you used mixed
Titone and Tiv (2022) proposed a Bilingualism Systems language in daily life, please indicate the languages that you
Framework that encompasses three layers of bilingualism that mix and estimate the frequency of mixing in normal conversation
account for interactions, interpersonal, and social dynamics. with the following groups of people…”).
However, questions that elicit information about these layers Overall, the BEQ appears better poised to address more aspects
and contexts are rarely included in bilingualism questionnaires. of Titone and Tiv’s proposed framework than other question-
Indeed, the investigation of context is often a secondary naires. Systems one and two are directly addressed in multiple

https://doi.org/10.1017/S1366728923000767 Published online by Cambridge University Press

 6 Ronessa Dass et al.

Table 3. Jaccard Index of seven bilingualism questionnaires

BSQ LHQ BEQ BLP LEAP-Q BDS LSBQ

BSQ 1.00 0.03 0.05 0 0 0.09 0

LHQ 0.03 1.00 0.13 0.17 0 0.06 0.18
BEQ 0.05 0.13 1.00 0.14 0.12 0.25 0.21
BLP 0 0.17 0.14 1.00 0 0.14 0.13
LEAP-Q 0 0 0.12 0 1.00 0.11 0.05
BDS 0.09 0.06 0.25 0.14 0.11 1.00 0.04
LSBQ 0 0.18 0.21 0.13 0.04 0.05 1.00
Mean overlap 0.03 0.10 0.15 0.10 0.05 0.12 0.10

items, specifically asking about emotional language use for differ- the least bilingual participant in different questionnaires (BLP
ent types of social relationships, including parent-child, friends, and BDS).
and colleagues. However, layers three (societal values, beliefs, Given the findings, we provide four suggestions when choos-
and policies) and four (change over time) do not appear to be dir- ing a questionnaire to characterize bilingual experiences in adults.
ectly addressed. First, the BSWQ is the most appropriate when language switching
Our vignette analysis revealed that using different question- is the research focus. Second, BEQ provides the most information
naires can yield different conclusions about how bilingual an indi- about the interaction between an individual and the social envir-
vidual is. Which participant was “most bilingual” or “least onment where bilingualism occurs. Third, the BEQ and the LSBQ
bilingual” was inconsistent across questionnaires. For example, provide the most coverage of categories and overlap to character-
in one instance, an individual was classified as the most and ize bilingualism more holistically. Finally, for studies aiming to

Figure 3. Vignette responses to each of the seven language questionnaires. BDS = Bilingual Dominance Scale, BEQ = Bilingualism and Emotion Questionnaire, BLP =
Bilingual Language Profile, BSQ = Bilingualism Switching Quotient, LEAP-Q = Language Experience and Proficiency Questionnaire, LHQ = Language History
Questionnaire, LSBQ = Language and Social Background Questionnaire. All numeric items from the above questionnaires were rescaled so that “0” represented
monolingualism, and “1” represented higher bilingualism. The x-axis indicates the number of items from each scale, and the lines indicate individual responses.
The plots are split so individual item responses can be seen on the left-hand plot, and the average across questions within participants is shown on the right-hand
plot. Three of the responses are highlighted in blue, red, and yellow to showcase individuals who are more bilingual (blue) and monolingual (red), as well as an
individual who obtained the highest and lowest scores on different measures (yellow).

https://doi.org/10.1017/S1366728923000767 Published online by Cambridge University Press

 Bilingualism: Language and Cognition 7

categorize participants into groups, the BLP provides the most Dewaele, J. M., & Pavlenko, A. (n.d.). Web questionnaire bilingualism and
efficient information. Two of the most popular questionnaires emotions. University of London.
are the LEAP-Q and the LHQ. The LHQ3 (Li et al., 2020) offers Dunn, A. L., & Fox Tree, J. E. (2009). A quick, gradient Bilingual Dominance
Scale*. Bilingualism: Language and Cognition, 12(03)), 273–289. https://doi.
wide coverage compared to most other questionnaires since its
org/10.1017/S1366728909990113
mandate is to incorporate most existing concepts and questions.
Evans, R. H. (1996). An Analysis of Criterion Variable Reliability in Conjoint
This breadth may reduce its similarity with more restricted mea- Analysis. Perceptual and Motor Skills, 82(3), 988–990. https://doi.org/10.
sures, but it remains the most comprehensive questionnaire on 2466/pms.1996.82.3.988
our list. The LEAP-Q offers the benefit of being a shorter ques- Fried, E. I. (2017). The 52 symptoms of major depression: Lack of content
tionnaire. However, this brevity also limits the LEAP-Q’s similar- overlap among seven common depression scales. Journal of Affective
ity to the other questionnaires in our study. Disorders, 208, 191–197. https://doi.org/10.1016/j.jad.2016.10.019
The findings of this study must be considered in light of its lim- Gertken, L. M., Amengual, M., & Birdsong, D. (2014). Assessing language
itations. A fair critique of these findings is that the amount of agree- dominance with the bilingual language profile. Measuring L2 Proficiency:
ment will naturally decrease as a function of increasing categories. Perspectives from SLA, 208, 225. https://books.google.com/books?hl=en
&lr=&id=xCLpAwAAQBAJ&oi=fnd&pg=PA208&dq=Bilingual+language+
The parcellation we arrived at is only one perspective on the data.
profile+(BLP)&ots=eMKNyf2WVt&sig=eZET4l9PdEugFwzxs0wXbNrkGo8
Thus, one way to force more agreement between questionnaires
Green, D. W., & Abutalebi, J. (2013). Language Control in Bilinguals: The
would be to lower the number of categories to the concept level. Adaptive Control Hypothesis. Journal of Cognitive Psychology, 25(5), 515–
These categories might, for example, be derived from the Delphi 530. https://doi.org/10.1080/20445911.2013.796377
Consensus, which includes: “language exposure and use, language Grosjean, F. (1989). Neurolinguists, beware! The bilingual is not two monolin-
difficulties, proficiency […], education and literacy, input quality, guals in one person. Brain and Language, 36(1), 3–15. https://doi.org/10.
language mixing practices, and attitudes (towards languages and 1016/0093-934x(89)90048-5
language mixing)” (De Cat et al., 2023). Re-categorizing and analyz- Grosjean, F. (2016). The Complementarity Principle and its impact on pro-
ing the data from the questionnaires can easily be done using the cessing, acquisition, and dominance. Language Dominance in Bilinguals:
data we have provided in the OSF. We encourage researchers to Issues of Measurement and Operationalization, 66–84. https://www.
francoisgrosjean.ch/bilin_bicult/12%20Grosjean.pdf
explore this dataset from various angles.
Grosjean, F., & Li, P. (2013). The Psycholinguistics of Bilingualism. John Wiley
The findings of this study highlight some of the issues
& Sons. https://play.google.com/store/books/details?id=3IumBMf-DJAC
researchers may have when trying to operationalize bilingualism. Jaccard, P. (1912). The distribution of the flora in the alpine zone.1. The
Bilingualism is a complex construct, and the existence of multiple New Phytologist, 11(2), 37–50. https://doi.org/10.1111/j.1469-8137.
tools for capturing various aspects of the experience is not neces- 1912.tb05611.x
sarily evidence of a lack of understanding or consensus but rather Kašćelan, D., Prévost, P., Serratrice, L., Tuller, L., Unsworth, S., & De Cat, C.
evidence of the diversity of experience that requires multiple tools (2022). A review of questionnaires quantifying bilingual experience in chil-
to assess properly. Research teams should examine and administer dren: Do they document the same constructs? Bilingualism: Language and
multiple scales to better understand the variations in emphasis and Cognition, 25(1), 29–41. https://doi.org/10.1017/S1366728921000390
intended use. As such, we caution researchers to carefully choose Li, P., Zhang, F., Yu, A., & Zhao, X. (2020). Language History Questionnaire
(LHQ3): An Enhanced Tool for Assessing Multilingual Experience.
and supplement tools best suited for their research questions.
Bilingualism: Language and Cognition, 23(5), 938–944. https://doi.org/10.
Data availability. The data that support the findings of this study are openly 1017/S1366728918001153
available via the Open Science Foundation (OSF) and can be found at https:// Luk, G., & Bialystok, E. (2013). Bilingualism Is Not a Categorical Variable:
osf.io/s4qug/?view_only=9e6efe3eca434446a503b1d7ef32cf20. Interaction between Language Proficiency and Usage. Journal of Cognitive
Psychology, 25(5), 605–621. https://doi.org/10.1080/20445911.2013.795574
Funding statement. This research was funded by grants from the Natural Luk, G., & Esposito, A. G. (2020). BLC Mini-Series: Tools to Document
Sciences and Engineering Research Council of Canada (NSERC), and a Bilingual Experiences. Bilingualism: Language and Cognition, 23(5), 927–
Canada Research Chair (CRC, Tier II) to JAEA. R.D. was supported by an 928. https://doi.org/10.1017/S1366728920000632
NSERC Undergraduate Summer Research Award (USRA). Marian, V., Blumenfeld, H. K., & Kaushanskaya, M. (2007). The Language
Experience and Proficiency Questionnaire (LEAP-Q): Assessing Language
Profiles in Bilinguals and Multilinguals. Journal of Speech, Language, and
References
Hearing Research: JSLHR, 50(4), 940–967. https://doi.org/10.1044/1092-
Anderson, J. A. E., Mak, L., Keyvani Chahi, A., & Bialystok, E. (2018). The lan- 4388(2007/067)
guage and social background questionnaire: Assessing degree of bilingual- R Core Team. (2022). R: A Language and Environment for Statistical Computing.
ism in a diverse population. Behavior Research Methods, 50(1), 250–263. R Foundation for Statistical Computing. https://www.R-project.org/
https://doi.org/10.3758/s13428-017-0867-9 Rodriguez-Fornells, A., Krämer, U. M., Lorenzo-Seva, U., Festman, J., &
Birdsong, D., Gertken, L. M., & Amengual, M. (2012). Bilingual Language Münte, T. F. (2011). Self-assessment of individual differences in language
Profile: An easy-to-use instrument to assess bilingualism. COERLL, switching. Frontiers in Psychology, 2, 388. https://doi.org/10.3389/fpsyg.
University of Texas at Austin. Web. 20 Jan. 2012. 2011.00388
Bronfenbrenner, U. (1992). Ecological systems theory. In R. Vasta (Ed.), Six Surrain, S., & Luk, G. (2017). Describing bilinguals: A systematic review of
theories of child development: Revised formulations and current issues, (pp labels and descriptions used in the literature between 2005–2015.
(Vol. 285, pp. 187–249). Jessica Kingsley Publishers. https://psycnet.apa. Bilingualism: Language and Cognition, 22(2), 401–415.
org/fulltext/1992-98662-005.pdf Titone, D. A., & Tiv, M. (2022). Rethinking multilingual experience through a
De Cat, C., Kašćelan, D., Prévost, P., Serratrice, L., Tuller, L., Unsworth, S., & Systems Framework of Bilingualism. Bilingualism: Language and Cognition,
The Q-BEx Consortium. (2023). How to quantify bilingual experience? 1–16. https://doi.org/10.1017/S1366728921001127
Findings from a Delphi consensus survey. Bilingualism: Language and Veritas Health Innovation. (2022). Covidence systematic review software. www.
Cognition, 26(1), 112–124. https://doi.org/10.1017/S1366728922000359 covidence.org

https://doi.org/10.1017/S1366728923000767
View publication stats
Published online by Cambridge University Press