Computer Assisted Language Learning Electronic Journal (CALL-EJ), 23(1), 187-205, 2022

Syntactic Complexity in EFL Writing: Within-Genre Topic and

Writing Quality1

Myung Jeong Ha ([email protected])

English Language and Regional Studies, Sangmyung University, South Korea

Abstract
This study examined the relationship between the syntactic complexity of EFL writing
and writing quality as judged by human raters. It also explored the role of topics in the
relationship. The data set used was 320 argumentative essays produced by EFL learners
taken from the International Corpus Network of Asian Learners of English (ICNALE).
These essays were analyzed using eight syntactic complexity measures with the L2
Syntactic Complexity Analyzer. The complexity indices and writing scores of the essays
were quantitatively analyzed. The result indicated strong topic effects on the majority of
syntactic complexity measures. There were significant changes across different
proficiency levels in phrasal-level measures but not in clause-level measures. In
comparison to essays on the smoking topic, essays on the part-time job topic showed a
significantly greater overall T-unit complexity, particularly at more advanced proficiency
levels. However, there were no statistical differences in overall sentence complexity.
Concerning the relationship between syntactic complexity and writing quality, global
features including Mean Length of Sentence (MLS) and Mean Length of T-unit (MLT)
were found to have a significant, positive relationship with writing scores across both
topics. At the local level, though, the correlations varied considerably between syntactic
measures.

Keywords: second language writing, corpus analysis, syntactic complexity, topic

effect, computational linguistics

Introduction

Syntactic complexity is accepted widely as the range and sophistication of

grammatical resources exhibited in language production (Ortega, 2015). A commonly
held belief is that L2 writers obtain more syntactic complexity as their language develops
(Crossley & McNamara, 2014). It has long been recognized as an essential construct in
EFL/ESL writing, as evidenced in writing studies that have examined the relationship of
syntactic complexity to L2 proficiency or the quality of L2 writing (Lu, 2011; Stockwell,
& Harrington, 2003). These studies have demonstrated that some indices of syntactic

1
This research was funded by a 2021 research Grant from Sangmyung University.
 187

complexity may be used to differentiate L2 proficiency levels and other indices used to
predict the quality of ESL/EFL writing.
Meanwhile, several studies have tapped on different issues associated with
syntactic complexity, including learners' development of syntactic complexity (e.g.,
Bult'e & Housen, 2014; Lu, 2011; Polat et al., 2019), and effects of various learner-, task-
and context-related factors on syntactic complexity such as L1 backgrounds, genre, topic,
planning time, and instructional setting (e.g., Lu & Ai, 2015; Ortega, 2003; Staples &
Reppen, 2016; Yang et al., 2015; Yoon & Polio, 2017). These studies have provided deep
insights into the growth of a learner's syntactic repertoire as an integral part of his or her
development in the target language.
However, most L2 writing studies have some weaknesses, such as small sample
sizes and homogeneity of learner proficiency, yielding conflicting findings. Also, it has
been questioned what the construct in syntactic complexity is and what syntactic
measures are relevant. Since Norris and Ortega (2009) proposed the examination of
syntactic complexity as a multidimensional construct, to date, the research that adopted
this proposal has been scarce (Byrnes et al., 2010). As Yang et al. (2015) point out, it is
not easy to assume that the relationship between syntactic complexity and writing quality
is the same across the various measures of syntactic complexity.
Regarding topic effects, most studies in L2 writing have highlighted the effect of
different genres on syntactic complexity, such as argumentation or narration (e.g., Beers
& Nagy, 2009; Lu, 2011). However, few studies are available on the effect of different
topics within the same genre (e.g., Yang et al., 2015; Yoon, 2017). Additionally, insights
from the related literature into the influence of within topic-related factors on L2 writing
point to the need for scrutiny of within topic-related differences in the syntactic
complexity in L2 writing.
To circumvent the limitations of previous research, this study attempts to measure
syntactic complexity as a multidimensional construct by adopting a well-controlled, large
corpus with standardized proficiency information. While addressing the gaps mentioned
above, this study attends to the issue of topic effects in L2 writing and the relationship
between syntactic complexity and L2 writing quality.
This study aims to address the following research questions:

What is the effect of topics on the syntactic complexity of EFL students' writing?
What is the relationship between syntactic complexity and the quality of EFL
students' writing?

Literature Review

Syntactic Complexity and Second Language Writing

Syntactic complexity is generally defined as "the range of forms that surface in

language production and the degree of sophistication of such forms" (Ortega, 2003, p.
492). As a sub-dimension of the larger construct of linguistic complexity, syntactic
complexity has been viewed as a valuable indicator of language proficiency, language
development, and the writing quality of L2 writers.
 188

One of the most significant research strands is identifying appropriate and

effective syntactic complexity measures regarding syntactic complexity. For example,
Taguchi et al. (2013) investigated the syntactical features that can predict the writing
quality of argumentative essays produced by college students. They found that the
frequencies of subordinating conjunctions, attributive adjectives, and postnoun-
modifying prepositional phrases could distinguish across different writing proficiency
groups. As Yin et al. (2021) point out, different measurements have been proposed for
detecting L2 writing syntactic complexity, which leads to the recognition that syntactic
complexity as a multidimensional construct should be evaluated using indices tapping
into complexity at phrasal, clausal, and global levels (e.g., Lu, 2017; Norris & Ortega,
2009).
Traditionally, the notion of syntactic complexity has been restricted to several
global or clause-level measures of linguistic complexity (e.g., unit-length, subordination,
and coordination) in L2 writing research (Ortega, 2003). Previous related literature
involved a small number of measures and a relatively small amount of data (Lu, 2017).
With the advancement in corpus linguistics that enables large-scale quantitative analysis
of linguistic features, the validity of using subordination or other clause-level complexity
measures to estimate writing proficiency development has been challenged and
questioned. Numerous corpus-based studies have contributed significantly to advancing
our understanding of variations in linguistic complexity across modalities and registers
(e.g., Biber & Conrad, 2019; Biber et al.,2011).
For example, Biber et al. (2011) suggested phrase-level sophistication (e.g.,
complex noun phrases) as more valid measures for academic writing development.
Further evidence in other studies below supported the validity of phrase-level complexity
as an integral part of syntactic complexity. For example, by analyzing a written corpus of
Chinese learners of English, Lu (2011) reported a significant increase of complex
nominals across proficiency levels. Similarly, Bulté and Housen (2014), in their L2
developmental study, examined the utility of the complexity construct for studying L2
writing development. They found a significant development in phrasal complexity and a
lack of development in clausal subordination.
Further evidence has been provided by Mazgutova and Kormos's (2015) short-
term longitudinal data. Mazgutova and Kormos were interested in the instructed
development of syntactic complexity within a short time frame of a 4-week course. They
found that intermediate-level L2 learners developed phrase-level syntactic complexity
over such a short period.
In sum, it appears that phrasal complexity measures are capable of exploring L2
writing development more in-depth than clausal measures. However, the concurrent
validity of different dimensions of complexity is not fully validated, necessitating further
empirical evidence from more extensive sampling data.
Based on Norris and Ortega's (2009) conceptualization of syntactic complexity as
a multidimensional construct, this study thus adopts Yang et al.'s (2015)
conceptualization of this multidimensional construct and the hierarchical relationships
among the sub-constructs with a minor adaptation (Figure 1). In this study, the term
clause refers only to finite clauses, which aligns with Hunt's (1965) definition in the
tradition of writing research. The term non-finite element is employed to refer to non-
finite clauses.
 189

Figure 1
A multidimensional representation of syntactic complexity

The Role of Writing Topic on Syntactic Complexity

Previous studies mainly explored the effect of different genres on students'

syntactic complexity. These studies revealed consistently that L2 writers often achieved
higher syntactic complexity in argumentative or expository essays than in narrative essays
(e.g., Lu, 2011; Way et al., 2000). For example, Lu (2011) examined the linguistic
complexity in argumentative and narrative essays written by Chinese EFL students and
reported greater linguistic complexity in argumentative essays. Way et al. (2000), by
employing four different evaluation methods (holistic scoring, length of the product,
mean length of T-units, and percentage of correct T-units), investigated the effect of three
different writing types (descriptive, narrative, and expository) on 937 writing samples
from 330 novice learners. Their analyses indicated that descriptive writing was the
easiest and expository writing the most difficult. Similarly, Yoon and Polio (2017)
examined genre differences in linguistic complexity involving syntactic measures, lexical
measures, fluency, and accuracy. They reported a significant genre effect on the phrasal
level but not on the clausal level, but no genre effect on accuracy and fluency.
While the effect of different genres on students' syntactic complexity has received
due attention in the L2 writing literature (e.g., Beers & Nagy, 2009; Lu, 2011), the effect
of the topic as a within-genre prompt variable is not yet fully explored. Furthermore, there
has been little research on the relationship between syntactic complexity and different
topics within the same genre.
 190

Thus far, a limited number of L2 writing studies have examined topic effects on
syntactic or lexical complexity and how topics play a role in the relationship between
linguistic complexity and writing scores. It was found that topic familiarity influences
learners' choice of lexical measures (Yang and Kim, 2020), as different topics require
different reasoning demands, thus generating different complexity measures. Ruiz-funes
(2015) examined task complexity in essay writing and measures of syntactic complexity
by analyzing the data from two separate studies she conducted with undergraduate FL
learners of Spanish in an American university. Findings indicated that the familiarity of
topics and genre determines task complexity in L2 writing. Yang et al. (2015) also
examined the role of topics in syntactic complexity in ESL argumentative writing. In their
study, 190 ESL graduate students produced argumentative essays on two different topics.
The difference of the two writing topics they adopted was that the future topic was
associated with causal reasoning while the appearance topic was not (i.e., Topic 1:
people's excessive emphasis on personal appearance - lower reasoning; Topic 2:
possibility of having a good future with careful planning at a young age - higher
reasoning).
They reported that the writing topic had a significant effect on the syntactic
complexity features. They concluded that "specific topics may naturally elicit more use
of certain syntactic complexity features, that is, topic-intrinsic complexity features" (p.
62). Their study contributes to the syntactic complexity literature by focusing on a
relationship between increased causal reasoning and increased syntactic complexity by
subordination.
As Yoon (2017) strongly argued, however, it is not easy to exclude another
possibility that participating students with diverse backgrounds, including ten major
fields and 38 L1s, might not have felt both the future topic and the appearance topic as
initially intended by the researchers. Informed by previous research on linguistic
complexity in L2 writing, writing topics produced by relatively more homogeneous
participating groups would usefully complement Yang et al.'s (2015) comparison on
different levels of causal reasoning between two topics.
Similarly, Yoon (2017), examining within-genre topic effects on syntactic
complexity, investigated a corpus of argumentative essays on two different topics written
by college-level Chinese EFL learners. Yoon (2017) found that there were main effects
of topics on syntactic complexity. Regarding topic familiarity, Yoon (2017) reported that
the part-time job topic elicited more linguistically complex language than the smoking
topic. While Yoon's work contributes to the growing literature on within-genre topic
effects in L2 writing, it has some concerns. One primary concern is that there are many
ungrammatical sentences in her learner corpus, which might have affected the accuracy
of automated syntactic complexity measurement. More recently, Yoon (2021), exploring
topic effects on metadiscourse features that reflect context and development in EFL
argumentative writing, indicated a large-scale quantitative analysis of interactional
metadiscourse by focusing on the topics of writings is essentially required. The result
showed significant differences in EFL students' use of metadiscourse features across
topics.
Meanwhile, Atak and Saricaoglu (2021) examined complexity developmental
stages of 90 intermediate level L1 Turkish learners in their argumentative essays on three
different topics, including cell phones, online learning, and the death penalty. Their
findings illustrated a significant effect of the death penalty topic on learners' complexity
 191

development, which is caused by its greater cognitive demands as a more impersonal

topic than the others.
In sum, findings of topic effect studies have offered helpful information on how
to approach the contribution of syntactic and lexical features to writing quality validly in
consideration of topics. Also, these studies have offered some valuable insights into the
multidimensionality of complexity measures and the within-genre effect of writing
quality and linguistic complexity. Meanwhile, they also leave much room for further
research.

Methods

Corpus Data

The dataset used in this study was an updated ICNALE module, the ICNALE
Edited Essays of the International Corpus Network of Asian Learners of English
(INCALE) (Ishikawa, 2013). Among the ICNALE Edited essays samples, essays
produced by college students from Indonesia and Thailand were excluded because the
number of essays was insufficient. In this study, thus the INCALE edited essays produced
by non-native speakers (NNS) English as a Foreign Language (EFL) learners of four
nationalities (i.e., Korea, Japan, China, and Taiwan) were adopted (see Ishikawa, 2018).
In addition, EFL writers were divided into four groups in terms of the four different
proficiency levels.
The original ICNALE is characterized as a reliable database for sophisticated
international contrastive interlanguage analysis (CIA) and studies of the World Englishes
in Asia. Unlike other corpora, the INCALE has the following principles: (1) a focus on
Asia, (2) consideration of linguistic modes, (3) condition control, (4) proficiency control,
(5) learner background survey, (6) native speakers' reference data collection, and (7) open
distribution (Ishikawa, 2018).
Although the original INCALE has been widely used in publications, these essays
are not rated. As a result, it was impossible to examine the type of error patterns and the
degree of writing quality. This study thus adopted the INCALE Edited Essays to make a
deeper analysis of writing quality possible. The INCALE Edited Essays include learners'
original essays, their edited versions, and rubric-based evaluation scores.
The INCALE was designed with strict criteria, including writing conditions and
L2 proficiency. College students wrote two argumentative essays on different topics
under time constraints. The two topics assigned were as follows:

Topic 1. It is important for college students to have a part-time job (hereafter PTJ
topic).
Topic 2. Smoking should be completely banned at all the restaurants in the country
(hereafter SMK topic).

Each essay is expected to be between 200 to 300 words in length. Students were
required to write two essays within 80 minutes. The EFL writers were classified into one
of the 4 levels of the Common European Framework of Reference for Languages (i.e.,
A2: waystage; B1.1: threshold, lower; B1.2: threshold, upper; and B2: vantage or higher),
 192

based on their scores on a proficiency test (e.g., TOEIC or TOEFL) or a receptive

vocabulary size test.
Table 1 summarizes the detailed composition of the final dataset.

Table 1
Size of the corpus for this study
Total Words Rate
Topic Proficiency Essays words Mean SD Mean SD
PTJ A2 40 8832 220.80 26.16 60.29 11.31
B1.1 40 9184 229.60 26.91 67.42 12.36
B1.2 40 9062 226.55 25.36 69.09 12.44
B2 40 9752 243.80 36.65 75.74 10.01
SMK A2 40 8855 221.37 34.57 63.63 12.06
B1.1 40 8973 224.32 27.30 68.21 11.38
B1.2 40 8922 223.05 26.88 70.49 12.68
B2 40 9532 238.30 31.30 73.07 15.52
All 320 73,112
Note. PTJ = Topic 1; SMK = Topic 2

Syntactic Complexity Measurement

This study examined the measures of syntactic complexity obtained from the L2
Syntactic Complexity Analyzer (L2SCA), which is available for free. L2SCA consists of
14 indices representing five dimensions of syntactic complexity, including the amount of
subordination, amount of coordination, overall sentence complexity, length of a
production unit, and phrasal complexity. L2SCA has been successfully applied in L2
writing syntactic complexity studies to examine issues of cross-proficiency differences,
learner development, effects of learner- or task-related variables, and writing quality (e.g.,
Jiang et al., 2019; Lu, 2011; Lu & Ai, 2015; Yang et al., 2015; Yoon & Polio, 2017).
L2SCA has been reported to have a good level of accuracy in calculating complexity
values, with the relevant correlations between scores computed by human raters and the
system (Lu, 2010). In addition, L2SCA generates syntactic complexity indices both at the
clause level and the phrase level.
With a note of caution that some measures may be redundant from each other (e.g.,
Norris & Ortega, 2009), the syntactic complexity of each essay was assessed using eight
different measures representing the eight interconnected sub-constructs laid out in the
Literature Review. Following Lu (2011) and Yoon (2017), selecting the eight syntactic
complexity measures was grounded in the criteria of redundancy, validity, and construct
distinctiveness.
These measures include the mean length of sentence (MLS), T-units per sentence
(T/S), mean length of T-unit (MLT), mean length of clause (MLC), clauses per T-unit
(C/T), dependent clauses per T-unit (DC/T), coordinate phrases per clause (CP/C), and
complex noun phrases per clause (CN/C). The definitions of the eight measures and the
sub-constructs they represent are summarized in Table 2. Following Lu's (2010)
definitions for the index's linguistic units and the eight measures - MLS, MLT, MLC, T/S,
C/T, DC/T, CP/C, and CNP/C- were computed.
 193

Table 2
Target complexity measures
Sub-construct Measure Definition
Overall sentence complexity Mean length of sentence Number of words divided by the
(MLS) number of sentences
Clausal coordination T-units per sentence (T/S) Number of T-units divided by the
number of sentences
Overall T-unit complexity Mean length of T-unit (MLT) Number of words divided by the
number of T-units
Amount of subordination Clauses per T-unit (C/T) Number of clauses divided by the
number of T-units
Finite clausal subordination Dependent clauses per T-unit Number of dependent clauses divided
(DC/T) by the number of T-units
Elaboration at clause level Mean length of clause (MLC) Number of words divided by the
number of clauses
Phrasal coordination Coordinate phrases per clause Number of coordinate phrases divided
(CP/C) by the number of clauses
Noun phrase complexity Complex NPs per clause Number of complex NPs divided by
(CNP/C) the number of clauses

Statistical Analysis

First, the target corpus was analyzed using the L2SCA, a freely-available
computer program designed to examine the syntactic complexity of English written texts
discussed above. For each of the essay samples, L2SCA generates frequency counts for
nine structural units: words, sentences, verb phrases, clauses, dependent clauses, T-units,
complex T-units, coordinate phrases, and complex nominals. Moreover, it also produces
eight indices of target syntactic complexity calculated using the frequency counts.
After the target complexity indices had been obtained for each writing sample,
SPSS version 20.0 was used to analyze the data. First, dependent samples t-tests were
conducted to examine the effect of writing topics on the syntactic complexity of the
writing samples. Pearson's product-moment correlations between syntactic complexity
indices and writing scores were then run for each topic to identify the relationship
between syntactic complexity and the quality of the essays.

Data Analysis and Findings

Research Question 1: Effect of Topic on Syntactic Complexity

Tables 3, 4, and 5 summarize the t statistics and the p values. The t statistics and
the p values present the statistical testing results for the topic comparison for each feature.
Table 3 shows the descriptive results for Mean length of sentence (MLS), T-units
per sentence (T/S), and Mean length of T-unit (MLT) used in the essays for the PTJ topic
and the SMK topic across four proficiencies. Compared to essays on the SMK topic,
essays on the PTJ topic showed a significantly greater overall T-unit complexity, as can
be observed in the significantly higher values for MLT at B1.2 and B2 levels. There were,
however, no statistical differences in overall sentence complexity, as measured by MLS.
 194

Tables 4 and 5 display the descriptive statistics for Clauses per T-unit (C/T),
Dependent clauses per T-unit (DC/T), Mean length of clause (MLC), Coordinate phrases
per clause (CP/C), and Complex nominals per clause (CNP/C). In general, essays on the
PTJ topic utilized a significantly higher amount of elaboration at the finite clause level,
as observed in the significantly higher values for MLC, CP/C, and CNP/C. On the other
hand, essays on the SMK topic tended to show a higher amount of elaboration in clausal
subordination, as seen in the higher values for C/T.

Table 3
Mean values, standard deviations, and t statistics of MLS, T/S, and MLT by writing topic
across four proficiencies
MLS T/S MLT
PTJ SMK t p PTJ SMK t p PTJ SMK t p
Mean Mean Mean Mean Mean Mean
(SD) (SD) (SD) (SD) (SD) (SD)
A2 17.43 14.94 1.26 .213 1.15 1.06 2.09 .05 14.61 13.36 .87 .386
(12.52) (10.33) (0.23) (0.20) (8.51) (7.49)
B1. 15.39 14.75 1.35 .182 1.10 1.11 -0.21 .833 13.93 13.34 1.14 .261
1 (3.75) (3.20) (0.10) (0.12) (3.35) (2.52)
B1. 15.82 14.91 1.67 .101 1.09 1.11 -0.61 .544 14.43 13.41 2.23 .031*
2 (3.54) (3.24) (0.10) (0.12) (2.96) (2.51)
B2 16.80 15.70 1.83 .075 1.08 1.09 -1.23 .225 15.55 14.39 2.07 .044*
(4.35) (4.71) (0.09) (0.11) (4.00) (4.95)
*
p<0.05
Note. MLS = Mean length of sentence; T/S = T-units per sentence; MLT = Mean length of T-unit

Table 4
Mean values, standard deviations, and t statistics of C/T, DC/T, and MLC by writing topic
across four proficiencies
C/T DC/T MLC
PTJ SMK t p PTJ SMK t p PTJ SMK t p
Mean Mean Mean Mean Mean Mean
(SD) (SD) (SD) (SD) (SD) (SD)
A2 7.02 7.80 -1.54 .130 0.56 0.59 -0.34 .731 8.70 7.88 1.90 .064
(2.23) (2.60) (0.32) (0.29) (2.12) (2.34)
B1. 7.67 8.12 -1.09 .279 0.58 0.66 -1.42 .163 8.30 8.24 0.25 .801
1 (2.45) (2.06) (0.29) (0.31) (1.13) (1.46)
B1. 7.30 8.80 -2.58 .025* 0.57 0.62 -1.28 .207 9.71 8.30 4.82 .031*
2 (1.99) (2.46) (0.26) (0.22) (1.31) (1.15)
B2 7.72 8.95 -2.59 .013* 0.64 0.72 -1.85 .072 9.42 8.15 5.57 .000**
(2.50) (3.24) (0.25) (0.26) (1.90) (1.71)
*
p<0.05; **p<0.01
Note. C/T = Clauses per T-unit; DC/T = Dependent clauses per T-unit; MLC = Mean length of clause
 195

Table 5
Mean values, standard deviations, and t statistics of CP/C and CNP/C by writing topic
across four proficiencies
CP/C CNP/C
PTJ SMK t p PTJ SMK t p
Mean Mean Mean Mean
(SD) (SD) (SD) (SD)
A2 0.12 0.11 0.48 .633 0.87 0.81 1.21 .233
(0.07) (0.06) (0.27) (0.27)
B1.1 0.16 0.11 2.28 .021* 1.14 0.81 3.54 .002*
(0.10) (0.08) (0.25) (0.23)
B1.2 0.17 0.13 2.43 .031* 0.99 0.80 2.38 .025*
(0.10) (0.06) (0.26) (0.24)
B2 0.16 0.12 2.31 .026* 1.04 0.87 3.14 .003*
(0.14) (0.08) (0.35) (0.35)
*
p<0.05
Note. CP/C = Coordinate phrases per clause; CNP/C = Complex NPs per clause

Research Question 2: Relationship Between Syntactic Complexity and Writing

Quality

Table 6 shows the correlations between each of the four syntactic complexity
indices (i.e., MLS, T/S, MLT, and C/T) and writing scores for the PTJ topic and the SMK
topic across proficiency levels.
First, MLS, indicating overall sentence complexity, significantly positively
correlated with writing scores for both topics at B1.1, B1.2, and B2 proficiency levels.
Second, MLT, indicating overall T-unit complexity, also significantly positively
correlated with writing scores for both topics at B1.1, B1.2, and B2 proficiency levels.
Third, T/S, showing clausal coordination, and C/T, showing the amount of subordination,
did not correlate with writing scores for either topic.

Table 6
Correlations between syntactic complexity indices (MLS, T/S, MLT, C/T) and writing
scores across four proficiencies
MLS T/S MLT C/T
PTJ SMK PTJ SMK PTJ SMK PTJ SMK
A2 0.25 0.29 -0.28 -0.08 0.28 0.22 0.18 0.14
B1.1 0.36* 0.46** -0.16 -0.15 0.39* 0.57** 0.75 -0.08
B1.2 0.32* 0.37* -0.14 0.11 0.33* 0.37* 0.22 -0.08
B2 0.31** 0.40* 0.19 0.18 0.32* 0.35* -0.08 -0.22
*
p<0.05; **p<0.01
Note. MLS = Mean length of sentence, T/S = T-units per sentence, MLT = Mean length of T-unit, C/T =
Clauses per T-unit

Table 7 summarizes the correlations between the remaining four of the eight
syntactic complexity indices (i.e., DC/T, MLC, CPC, and CNP/C) and writing scores for
the PTJ topic and the SMK topic across proficiency levels.
First, MLC, pertaining to elaboration at the finite clause level, significantly
positively correlated with writing scores for the PTJ topic at B2 level and the SMK topic
at B1.1, B1.2, and B2 levels. Second, CP/C, indicating phrasal coordination, significantly
 196

positively correlated with writing scores for the PTJ topic at B1.2 and B2 proficiency
levels and the SMK topic at the B2 level. Third, CNP/C, indicating noun phrase
complexity at the finite clause, significantly positively correlated with writing scores for
the PTJ topic at B1.2 and B2 proficiency levels and writing scores for the SMK topic at
B1.1, B1.2, and B2 levels. Fourth, DC/T, measuring finite clausal subordination, did not
significantly correlate with writing scores for either topic.

Table 7
Correlations between syntactic complexity indices (DCT, MLC, CP/C, and CNP/C) and
writing scores across four proficiencies
DC/T MLC CP/C CNP/C
PTJ SMK PTJ SMK PTJ SMK PTJ SMK
A2 -0.14 -0.17 -0.24 -0.04 0.00 -0.09 0.15 0.23
B1.1 0.07 0.22 0.09 0.50** 0.06 0.23 0.12 0.40*
B1.2 0.13 0.09 0.10 0.34* 0.31* 0.09 0.32* 0.47**
B2 0.06 -0.03 0.40** 0.43** 0.35* 0.37* 0.41** 0.45**
*
p<0.05; **p<0.01
Note. DC/T = Dependent clauses per T-unit; MLC = Mean length of clause; CP/C = Coordinate phrases
per clause; CNP/C = Complex NPs per clause

In summary, three measures indicating elaboration at the finite clause level –

MLC, CP/C, and CNP/C- tended to exhibit some degree of positive correlation with the
PTJ topic and the SMK topic. It should be noted that its correlation for the SMK topic, in
general, had a tendency to be higher than that for the PTJ topic, and the strength of the
relationships for significant findings is overall moderate to high, ranging from 0.31 to
0.57.

Discussion
Given that syntactic complexity has been viewed as a multidimensional construct
(Norris & Ortega, 2009) with different levels of sub-constructs, the present study revealed
complex findings of the effect of within-genre topic on syntactic complexity and the
relationship between syntactic complexity and writing quality.
Regarding topic effects, the results of the dependent sample t-tests indicated that
five out of eight complexity measures were significantly influenced, particularly at more
advanced proficiency levels (i.e., B1.2 and B2). Furthermore, most of the complexity
measures with statistical significance showed higher values in the PTJ topic than the SMK
topic (except for C/T).
This result may reflect that a personal issue (a part-time job) is more situated
explicitly within the everyday lives of college students in Asian countries than a social
issue (a public smoking ban). It is highly possible that college students often have
occasions in need of money and are employed as part-timers. This result is congruent with
Yoon (2017), who suggested that the PTJ topic is more closely related to EFL writers'
experience as college students than the SMK topic.
In line with Yoon's findings, this finding also echoes Hinkel's (2002)
comprehensive study of topic relevance effects on linguistic features. In her study,
prompts affected writings produced by both NNS and NS students. Hinkel (2002)
 197

considered the prompt about choosing one's academic major as most relevant and the
prompt on celebrity wealth as least relevant. When NNS and NS students worked on
choosing academic majors, they used more nominalizations, infinitives, and conjunctions,
which increased syntactic complexity.
Additionally, Ishikawa (2018), in his case study, demonstrated that the essay
evaluation scores and the number of edits have a middle-level correlation and that they
may be influenced by the essay topic, implying the topic effect on the argumentative
essays' evaluation and editing. In summary, it is clear that a topic on college students'
part-time jobs, which is believed to be more relevant to college-level EFL writers, tends
to elicit more complex language.
The dependent sample t-tests also showed three measures with no significant
effects: DC/T, T/S, and MLS (i.e., dependent clauses per T-unit, T-units per sentence,
and mean length of sentence). This result gives partial support for a widely accepted
statement that academic writing is featured by phrasal sophistication rather than clausal
elaboration (Biber et al., 2011). The finding is consistent with Ortega's (2003) study
suggesting that advanced writers express their complex ideas in argumentation through
increased phrasal density. This study thus suggests that advanced college-level EFL
writers would rely on a higher amount of complex noun phrases rather than greater clause-
level elaboration.
Regarding the relationship between syntactic complexity and writing quality,
global features including MLS and MLT were found to have a significant positive
relationship with writing scores across both topics. At the local level, though, the
correlations varied considerably between measures. For example, T/S, C/T, and DC/T
were not significantly correlated with writing scores, whereas MLC, CP/C, and CNP/C
tended to correlate with writing scores significantly. Therefore, these findings suggest
that either of the two global measures (MLS and MLT) can operate well as a generic
syntactic complexity measure since both were found to be significantly and consistently
correlated with writing scores across topics.
The combined results indicate that the EFL students who were able to use both
topic-related linguistic features and other types of local-level complexity features were
rated with higher scores. Furthermore, a manual re-examination of the syntactic
complexity features in the learner data showed how a higher level of syntactic complexity
and variation are identified in the more highly rated essays for each topic.
Appendix A displays the use of the syntactic complexity features in essays for the
two topics at two different score points. The first two samples, P1 and P2, are excerpts
from two essays on the PTJ topic rated at 7.8 and 12, respectively. In both essays,
subordinate clauses and complex noun phrases were frequently used, providing
descriptions and lengthening the clauses.
However, in the lower-scored essay sample, the structure of complex nominals
was much simpler than that in the higher-scored essay sample. The lower-scored essay
often replied on two-word combinations: future job, many jobs, other jobs, good way,
hard experiences, department store, interesting time, and so on. In contrast, the higher-
scored essay sample contained more elaborated complex-noun phrases, making the essay
highly propositional as well as descriptive. The instances of the longer complex-noun
phrases included their own parents' money, some type of other activity, a regular, paying
job, life outside the classroom, the service industry off campus, and an important element
 198

to their future resume, and so on. The higher-scored essay demonstrated much greater
syntactic complexity and variation
The second two samples, S1 and S2, illustrate how both essays for the SMK topic
utilized finite and non-finite subordination and complex nominals, making both essays
propositional. However, the lower-scored essay sample showed very few coordinate
phrases. On the other hand, the higher-scored essay sample contained more use of
complex noun phrases and several coordinate phrases that make the essay highly
propositional. Similarly, the higher-scored essay showed higher syntactic complexity and
variation.
In a further exploration of the sample essays, it was found that various viewpoints
were put forward into EFL writing supported by different main points and specific
examples (Appendix B). However, a similar argumentative approach was applied to
banning indoor smoking with a mere focus on the adverse effects of smoking on the health
of smokers and others nearby.
For example, in the PTJ topic, writers who supported the statement in favor of
having a part-time job had the following different reasons: 1) gaining hands-on
experiences, 2) becoming an asset for the future, 3) keeping students from becoming lazy,
4) realizing the value of money, and 5) learning time management skills.
In the SMK topic, most writers argued for the statement of banning indoor
smoking with a reason to avoid second-hand smoke exposure among non-smokers.
Relying on Hinkel's (2002) discussion on the topic relevance, it is reasonable to argue
that while having sufficient content knowledge helps construct well-developed ideas in
writing, less complex language features in writing may be attributable to topics with
limited relevance to writers' own experience.

Conclusion
The present study confirmed the indices of phrase-level syntactic complexity as
effective measures of L2 proficiency development while also indicating little change in
clause-level syntactic complexity across proficiency levels. Finally, it can be concluded
that phrase-level complexity measures are capable of detecting second language writing
development more sensitively than clause-level measures. This outcome also calls for
more specific measures to detect clause-level development in L2 writing.
Nevertheless, this does not necessarily suggest that clausal complexity is not
essential. For example, a lack of differences in the statistical analysis does not indicate a
lack of complexity. Instead, this means that EFL writers of texts rated as higher or lower
quality do not produce such structures more or less frequently.
In addition, the findings can illuminate measurement choices for syntactic
complexity. While investigating the relationship between syntactic complexity and
writing quality, the present study suggests that either of the two global syntactic features
(MLS and MLT) can be valid as a generic syntactic complexity measure since both
significantly correlate with writing scores across topics.
The present study can also conclude that a personal and concrete topic may elicit
better writings. This is an important implication for designers of writing tasks. For
example, writing topics designed for assessments with time limits need to be relevant to
EFL learners' experiences to elicit more complex language. Furthermore, writing
 199

instructors are expected to have a clear understanding of the constructs of syntactic

complexity measures to enhance the development of learners' linguistic repertoire more
efficiently.
Conclusively this study has shown the effects within-genre topics have on the
writing quality. However, this study is not without its weaknesses; the two writing tasks
under examination probably differ not just in topic relevance. Therefore, future studies
on writing-topic effects must develop a clearer-cut classification and definition of all the
topic-related variables. At the same time, it is crucial to identify which local-level
complexity features are of great use in predicting writing scores.

Acknowledgment
I would like to thank the editor and the anonymous reviewers for their insightful
comments and suggestions on this article.

References
Atak, N., & Saricaoglu, A. (2021). Syntactic complexity in L2 learners' argumentative
writing: Developmental stages and the within-genre topic effect. Assessing
Writing, 47. https://doi.org/10.1016/j.asw.2020.100506
Beers, S. F., & Nagy, W. E. (2009). Syntactic complexity as a predictor of adolescent
writing quality: Which measures? Which genre? Reading and Writing, 22(2), 185-
200. http://dx.doi.org/10.1007/s11145-007-9107-5
Biber, D., & Conrad, S. (2019). Register, genre, and style. Cambridge University Press.
Biber, D., Gray, B., & Poonpon, K. (2011). Should we use characteristics of conversation
to measure grammatical complexity in L2 writing development? TESOL Quarterly,
45(1), 5–35. https://www.jstor.org/stable/41307614
Bulté, B., & Housen, A. (2014). Conceptualizing and measuring short-term changes in
L2 writing complexity. Journal of second language writing, 26, 42-65.
https://doi.org/10.1016/j.jslw.2014.09.005
Byrnes, H., Maxim, H. H., & Norris, J. M. (2010). Realizing advanced foreign language
writing development in collegiate education: Curricular design, pedagogy,
assessment. The Modern Language Journal, 94, i-235. http://www.jstor.org/stable/
40985261
Crossley, S. A., & McNamara, D. S. (2014). Does writing development equal writing
quality? A computational investigation of syntactic complexity in L2
learners. Journal of Second Language Writing, 26, 66-79. https://doi.org/10.1016/
j.jslw.2014.09.006
Hinkel, E. (2002). Second language writers' text: Linguistic and rhetorical features.
Routledge.
Hunt, K. W. (1965). Grammatical structures written at three grade levels. National
Council of Teachers of English.
Ishikawa, S. (2013). The ICNALE and sophisticated contrastive interlanguage analysis
of Asian learners of English. Learner corpus studies in Asia and the world, 1, 91-118.
http://jairo.nii.ac.jp/0006/00018294
 200

Ishikawa, S. (2018). The ICNALE Edited Essays: A dataset for analysis of L2 English
learner essays based on a new integrative viewpoint. English Corpus Studies, 25, 117-
130. https://www.researchgate.net/publication/341343658_The_ICNALE_Edited_
Essays_A_Dataset_for_Analysis_of_L2_English_Learner_Essays_Based_on_a_Ne
w_Integrative_Viewpoint
Jiang, J., Bi, P., & Liu, H. (2019). Syntactic complexity development in the writings of
EFL learners: Insights from a dependency syntactically-annotated corpus. Journal of
Second Language Writing, 46. https://doi.org/10.1016/j.jslw.2019.100666
Lu, X. (2010). Automatic analysis of syntactic complexity in second language
writing. International journal of corpus linguistics, 15(4), 474–496. http://lps3.doi.
org.libproxy.smu.ac.kr/10.1075/ijcl.15.4.02lu
Lu, X. (2011). A corpus‐based evaluation of syntactic complexity measures as indices of
college‐level ESL writers' language development. TESOL Quarterly, 45(1), 36-62.
http://lps3.doi.org.libproxy.smu.ac.kr/10.5054/tq.2011.240859
Lu, X. (2017). Automated measurement of syntactic complexity in corpus-based L2
writing research and implications for writing assessment. Language Testing, 34(4),
493-511. https://doi.org/10.1177/0265532217710675
Mazgutova, D., & Kormos, J. (2015). Syntactic and lexical development in an intensive
English for Academic Purposes programme. Journal of Second Language Writing, 29,
3-15. https://doi.org/10.1016/j.jslw.2015.06.004
Norris, J. M., & Ortega, L. (2009). Towards an organic approach to investigating CAF in
instructed SLA: The case of complexity. Applied linguistics, 30(4), 555-578.
http://lps3.doi.org.libproxy.smu.ac.kr/10.1093/applin/amp044
Ortega, L. (2003). Syntactic complexity measures and their relationship to L2
proficiency: A research synthesis of college‐level L2 writing. Applied
linguistics, 24(4), 492-518. https://doi.org/10.1093/applin/24.4.492
Ortega, L. (2015). Syntactic complexity in L2 writing: Progress and expansion. Journal
of Second Language Writing, 29, 82-94. http://dx.doi.org/10.1016/j.jslw. 2015.06.008
Polat, N., Mahalingappa, L., & Mancilla, R. L. (2020). Longitudinal growth trajectories
of written syntactic complexity: The case of Turkish learners in an intensive English
program. Applied Linguistics, 41(5), 688-711.https://doi.org/10.1093/applin/amz034
Ruiz-funes, M. (2015). Exploring the potential of second/foreign language writing for
language learning : The effects of task factors and learner variables. Journal of
Second Language Writing, 28, 1–19. https://doi.org/10.1016/j.jslw.2015.02.001
Staples, S., & Reppen, R. (2016). Understanding first-year L2 writing: A lexico-
grammatical analysis across L1s, genres, and language ratings. Journal of Second
Language Writing, 32, 17-35. https://doi.org/10.1016/j.jslw.2016.02.002
Stockwell, G., & Harrington, M. (2003). The Incidental Development of L2 Proficiency
in NS-NNS Email Interactions. CALICO Journal, 20(2), 337-359.
https://www.proquest.com/scholarly-journals/incidental-development-l2-proficiency
-ns-nns/docview/750475265/se-2?accountid=16907
Taguchi, N., Crawford, W., & Wetzel, D. Z. (2013). What linguistic features are
indicative of writing quality? A case of argumentative essays in a college composition
program. Tesol Quarterly, 47(2), 420-430. http://lps3.doi.org.libproxysmu.ac.kr
/10.1002/tesq.91
Way, D. P., Joiner, E. G., & Seaman, M. A. (2000). Writing in the secondary foreign
language classroom: The effects of prompts and tasks on novice learners of
 201

French. The Modern Language Journal, 84(2), 171-184. http://www.jstor.org/

stable/330483
Yang, W. & Kim, Y. (2020). The effect of topic familiarity on the complexity, accuracy,
and fluency of second language writing. Applied Linguistics Review, 11(1), 79-
108. http://lps3.doi.org.libproxy.smu.ac.kr/10.1515/applirev-2017-0017
Yang, W., Lu, X., & Weigle, S. C. (2015). Different topics, different discourse:
Relationships among writing topic, measures of syntactic complexity, and
judgments of writing quality. Journal of Second Language Writing, 28, 53–67.
https://doi.org/10.1016/j.jslw.2015.02.00
Yin, S., Gao, Y., & Lu, X. (2021). Syntactic complexity of research article part-genres:
Differences between emerging and expert international publication
writers. System, 97. https://doi.org/10.1016/j.system.2020.102427
Yoon, H. J. (2017). Linguistic complexity in L2 writing revisited: Issues of topic,
proficiency, and construct multidimensionality. System, 66, 130-141. https://doi.org/
10.1016/j.system.2017.03.007
Yoon, H. J., & Polio, C. (2017). The linguistic development of students of English as a second
language in two written genres. Tesol Quarterly, 51(2), 275-301. http://lps3.doi.org.
libproxy.smu.ac.kr/10.1002/tesq.296

Appendix A
Essay samples
Annotation symbols:
underlined: subordinate clause or element
italicized: complex noun phrases
bold: coordinate phrases
italicized and bold: coordinate phrases as modifiers of nouns, or complex noun
phrases in a coordinate noun phrase

PTJ topic (Full score: 12)

P1. Score: 7.8

A part-time job is a way to look for future job. We can also experience many jobs
through part-time jobs. It is very important to experience other jobs because we don't
know that what we are interested in and what we want to be. For these reasons, I think
it is a good way to gain experience. I have had part-time jobs. They were hard experiences,
but it was a great time for me. When I needed money I sold clothes last year. I worked
in a department store. It was very hard and I really wanted quit right away, but I needed
money to travel abroad so I couldn't do that. When I was 21, I worked at a theater. It was
a interesting time, but I stressed about money. Because I checked money and calculated
money accurately, the theater earned money. I also worked at a buffet restaurant. The
work was very hard but eating delicious food was very good. Sometimes people gave me
tips, so I thought it was a cool job. A few months ago, I taught middle school students. It
was hard work too, because middle school students did not listen to my lessons. I think
 202

that the many jobs that I have experienced are going to be good sources in my future.
Someday, you can experience many part-time jobs too.

P2. Score: 12
I'm also a current college student and I agree, the students who work even just a
small part-time job at school are more motivated to get work done. Now, some students
do live off of their own parents' money, which is fine if they are that fortunate. However,
if they don't have a need to work, these students should be involved in a club or some
type of other activity. There's a lot of responsibility that goes into running an organization
on campus and it can help students remained grounded in a similar way that a regular,
paying job would. Ultimately, anything that keeps students from becoming lazy is a good
thing. Holding down a part-time job during college is important for many different
reasons. Students will enjoy their work, learn some personal and financial
responsibility and experience life outside the classroom. Having to focus on their studies
is not a valid excuse, because there are many options available for part-time work, ranging
from a few hours per week at an on-campus job to 20-30 hours per week in the service
industry off campus. Some of the most successful students are successful because their
part-time job allowed them to learn an important element of success: time management.
If students keep their priorities straight and find a part-time job that is at least tolerable,
students will be adding an important element to their future resume.

SMK topic (Full score: 12)

S1. Score: 5.4

In my childhood, my father always smoked near me. I thought that smoking was
not a bad thing, but in middle school I saw a video that showed that smoking is so bad for
people's bodies. I realized that smoking is a very bad thing. I do not smoke, but I am sure
that my body is so bad. Because of my father, I experienced second-hand smoking. I don't
care about people smoking, but smokers must not smoke near non-smokers. When I wait
for a bus at the bus stop, I smell smokers' smoke. It is so bad, smoking near non-smokers.
When I am an adult, I will ban smoking for my children, because it is so bad for a mom
with a baby. Smoking is also banned in playgrounds and kindergarten. This gives
health benefits. I saw on television that a famous comedian died because of smoking. I
was so scared when I saw his death. When I go to a computer fee room, I always feel
smoking smell. Smoking should be removed forever.

S2. Score: 8.8

I think smoking should be banned in public places like restaurants, because it
causes more than 1000 deaths from lung cancer. Second hand smoke affects pregnant
women, children and elderly people more. Smoking is also setting bad examples for
teens and young adults. Also, scientists agree that smoking is dangerous. Tobacco smoke
can cause cancer, strokes and heart disease. Smoking does not just harm the smoker;
it also harms people nearby who breathe in the smoke. This is called passive smoking.
Smokers choose to smoke, but people nearby do not choose to smoke passively. People
should only be exposed to harm if they understand the risks and choose to accept them.
 203

A complete ban on smoking in public is needed to protect people from passive smoking.
Society accepts that adults can decide to harm themselves to some extent, so long as they
do not harm others. This is why the proposition is not arguing that people should be
banned from smoking in private. Passive smokers do not choose to breathe in other
people's smoke. If they do not want to smoke passively, they do not need to go to places
where smoking is allowed. There is therefore strong reason to ban smoking in public.

Appendix B
Main points underlined
PTJ topic
P1. Score: 7.8
A part-time job is a way to look for my future job. We can also experience many
jobs through part-time jobs. It is very important to experience other jobs because we don't
know that what we are interested in and what we want to be. For these reasons, I think it
is a very good way to gain experience. I have had several part-time jobs. They were very
hard experiences, but it was a great time for me. When I needed money I sold clothes last
year. I worked in a department store. It was very hard and I really wanted quit right away,
but I needed money to travel abroad so I couldn't do that. When I was 21, I worked at a
theater. It was a very interesting time, but I stressed about money. Because I checked
money and calculated money accurately, the theater earned money. I also worked at a
buffet restaurant. The work was very hard but eating delicious food was very good.
Sometimes people gave me tips, so I thought it was a cool job. A few months ago, I taught
middle school students. It was hard work too, because middle school students did not
listen to my lessons. I think that the many jobs that I have experienced are going to be
good sources in my future. Someday, you can experience many part-time jobs too

P2. Score: 12
I'm also a current college student and I agree, the students who work even just a
small part-time job at school are more motivated to get work done. Now, some students
do live off of their own parents' money, which is fine if they are that fortunate. However,
if they don't have a need to work, these students should be involved in a club or some
type of other activity. There's a lot of responsibility that goes into running an organization
on campus and it can help students remained grounded in a similar way that a regular,
paying job would. Ultimately, anything that keeps students from becoming lazy is a good
thing. Holding down a part-time job during college is important for many different
reasons. Students will enjoy their work, learn some personal and financial responsibility
and experience life outside the classroom. Having to focus on their studies is not a valid
excuse, because there are many options available for part-time work, ranging from a few
hours per week at an on-campus job to 20-30 hours per week in the service industry off
campus. Some of the most successful students are successful because their part-time job
allowed them to learn an important element of success: time management. If students
keep their priorities straight and find a part-time job that is at least tolerable, students will
be adding an important element to their future resume.
 204

SMK topic

S1. Score: 5.4

In my childhood, my father always smoked near me. I thought that smoking was
not a bad thing, but in middle school I saw a video that showed that smoking is so bad for
people's bodies. I realized that smoking is a very bad thing. I do not smoke, but I am sure
that my body is so bad. Because of my father, I experienced second-hand smoking. I don't
care about people smoking, but smokers must not smoke near non-smokers. When I wait
for a bus at the bus stop, I smell smokers' smoke. It is so bad, smoking near non-smokers.
When I am an adult, I will ban smoking for my children, because it is so bad for a mom
with a baby. Smoking is also banned in playgrounds and kindergarten. This gives health
benefits. I saw on television that a famous comedian died because of smoking. I was so
scared when I saw his death. When I go to a computer fee room, I always feel smoking
smell. Smoking should be removed forever.

S2. Score: 8.8

I think smoking should be banned in public places like restaurants, because it
causes more than 1000 deaths from lung cancer. Second hand smoke affects pregnant
women, children and elderly people more. Smoking is also setting bad examples for teens
and young adults. Also, scientists agree that smoking is dangerous. Tobacco smoke can
cause cancer, strokes and heart disease. Smoking does not just harm the smoker; it also
harms people nearby who breathe in the smoke. This is called passive smoking. Smokers
choose to smoke, but people nearby do not choose to smoke passively. People should
only be exposed to harm if they understand the risks and choose to accept them. A
complete ban on smoking in public is needed to protect people from passive smoking.
Society accepts that adults can decide to harm themselves to some extent, so long as they
do not harm others. This is why the proposition is not arguing that people should be
banned from smoking in private. Passive smokers do not choose to breathe in other
people's smoke. If they do not want to smoke passively, they do not need to go to places
where smoking is allowed. There is therefore strong reason to ban smoking in public.