Towards Greater Conceptual Clarity in Complexity and Difficulty: A Commentary on “Complexity and Difficulty in Second Language Acquisition: A Theoretical and Methodological Overview”

Abstract

The conceptual review article by Bulté, Housen, and Pallotti constitutes a much-needed effort to disentangle the conceptual and methodological messiness surrounding complexity and difficulty, two key constructs in second language (L2) acquisition research. Seeking clear conceptual distinction between these constructs, Bulté et al. interpret complexity as structural properties of linguistic items and difficulty as the cognitive costs associated with acquiring and using such items. They further propose a small set of core measures for researchers to routinely use so as to promote replicability and knowledge accumulation. In this commentary, I offer some thoughts on their discussion of the definitions and measurements of these constructs.

The inclusion and exclusion of “sophistication” in different definitions of “complexity” (e.g., Kyle et al., 2021; Ortega, 2003), along with the introduction of the terms “absolute complexity” and “relative complexity” (e.g., Housen & Simoens, 2016), contributed to the terminological confusion surrounding complexity. I concur with Bulté et al.’s narrow interpretation of complexity as equivalent to absolute complexity that excludes sophistication. Meanwhile, an explicit discussion of (a) the precise relationship among the terms “difficulty,” “sophistication,” and “relative complexity” and (b) the future status of the terms “absolute complexity” and “relative complexity” would provide greater terminological clarity for the field.

Bulté et al. touch upon two important debates on the definition and measurement of complexity. First, they define complexity independently from notions of register-/genre-based adequacy. Register/genre variation research has yielded valuable insights into the linguistic characteristics of different registers/genres. However, the relative frequencies of linguistic items in different registers/genres should not form the basis for including or excluding them in analyzing the complexity of texts of a specific register/genre. This is because while a linguistic item may be less frequent in one register/genre than in others, it may nevertheless play an indispensable role in that register/genre, and this specific role should be analyzed register- or genre-internally. Second, they believe that the fine-grained approach to complexity analysis is complementary with a more holistic approach. In my view, all complexity measures are on a scale of granularity, and the criticism that holistic measures (e.g., dependent clauses per clause) lack specificity and interpretability, which is sometimes cited without thorough consideration, similarly applies to most fine-grained measures (e.g., relative clauses per 1,000 words). For example, just as there are different types of dependent clauses, there are also different types of relative clauses, each of which may in turn have subtypes. In fact, granularity could go all the way down to lexicalization. Holistic measures provide an efficient way to assess high-level complexity (e.g., the emergence or overall level of finite subordination or noun modification). As L2 writing teachers and researchers may be interested in complexity at different levels of granularity in different contexts and for different purposes, it is important to recognize the complementarity of holistic and fine-grained approaches to complexity analysis, similar to that of holistic and analytical rating scales used in writing assessments.

Bulté et al. do not consider the sentence as a unit of analysis but consider the T-unit or AS-unit as the largest syntactic unit of analysis. This recommendation opens up an interesting discussion. First, the sentence is an intuitively useful unit of information organization in writing. Second, complexification by clausal coordination is known to be especially important for beginning learners (e.g., Ortega, 2003). Third, by disregarding punctuated sentence fragments in our analysis, we may miss the opportunity to capture intentional, appropriate uses of such fragments and potentially unintended, erroneous uses in learner texts. Fourth, the T-unit may not be an appropriate or the largest unit of analysis in all languages. For example, the topic–comment unit has been argued to be more appropriate than the T-unit for analyzing complexity in Chinese, and a terminal topic–comment unit could correspond to two or more T-units (e.g., Hu et al., 2022; Yu, 2021).

For syntactic complexity measurement, Bulté et al. recommend using mean words per phrase and mean phrases per clause as two of the six core constitutional measures. To promote replicability, it would seem necessary to agree on a consistent approach to identifying and counting phrases (e.g., how many phrases are there in read a book about ancient China?). For lexical complexity measurement, Bulté et al. recommend lemmatizing inflectional word forms but leave the treatment of derivational forms open. They propose mean word length as the core constitutional measure of this construct. However, without defining lexical complexity as word family complexity, mean morphemes per word and mean length of (root) morphemes might better align with their effort to capture constitutional complexity in syntactic complexity measurement, as words are constituted by morphemes, which are then represented by letters or characters in written form.

Bulté et al. recognize the normalized rate of occurrence of linguistic forms as a valid approach to calculating text-level complexity. Measures of this type are known to facilitate the comparison of the rate of occurrence of linguistic forms across texts. However, they prioritize the frequency of occurrence of linguistic forms over their internal complexity. More importantly, they do not consider the distribution of linguistic forms across the larger units in which they occur and thus cannot account for the structural organization of the text. Texts are not organized by n-word chunks and rarely display homogeneous dispersion of linguistic forms. Rather, specific linguistic forms occur within larger units, both syntactic units immediately containing them and larger units realizing discursive acts/functions, that contribute to the overall text structure. Therefore, measures based on normalized rate of occurrence are best used in combination with other types of measures that capture the internal complexity of linguistic forms and their distribution in larger units.

Bulté et al. argue against the consideration of word meanings in analyzing the complexity or difficulty of L2 production based on the concern that it is difficult to determine the meanings of a linguistic item in learner texts. While this concern is warranted, the consideration of word meanings may be necessary in measuring difficulty. For example, based on production and interview data, Liu and Lu (2020) reported that their Chinese-speaking L2 English learners frequently used some but not other meanings of specific words (e.g., composition meaning a piece of writing vs. the way in which something is formed by its parts) due to their knowledge gaps. Lu and Hu (2022) further argued that different meanings of polysemous words may represent different levels of sophistication. They showed that it is possible to pinpoint the meanings of most polysemous words in learner texts and that sense-aware lexical sophistication indices can better predict learner proficiency than form-based ones. The cases in which the meanings of polysemous words are unclear (e.g., due to incorrect or creative usage) could constitute another useful dimension of analysis, as learner development involves learning not only to express more sophisticated meanings but also to express meanings unambiguously. Regarding the concern for accurate word sense disambiguation, large language models could now be fine-tuned to improve performance, and manual analysis can also be involved.

In sum, I am thankful for Bulté et al.’s contribution towards resolving the conceptual and methodological controversies around the constructs of complexity and difficulty, and I have raised a few further issues to be considered. Importantly, the complex and multidimensional nature of these constructs necessitates a set of complementary measures of different types, at different levels of granularity, and with sensitivity to features of different languages.