International Journal of Artificial Intelligence in Education最新文献

Recent investigations in automated essay scoring research imply that hybrid models, which combine feature engineering and the powerful tools of deep neural networks (DNNs), reach state-of-the-art performance. However, most of these findings are from holistic scoring tasks. In the present study, we use a total of four prompts from two different corpora consisting of both L1 and L2 learner essays annotated with trait scores (e.g., content, organization, and language quality). In our main experiments, we compare three variants of trait-specific models using different inputs: (1) models based on 220 linguistic features, (2) models using essay-level contextual embeddings from the distilled version of the pre-trained transformer BERT (DistilBERT), and (3) a hybrid model using both types of features. Results imply that when trait-specific models are trained based on a single resource, the feature-based models slightly outperform the embedding-based models. These differences are most prominent for the organization traits. The hybrid models outperform the single-resource models, indicating that linguistic features and embeddings indeed capture partially different aspects relevant for the assessment of essay traits. To gain more insights into the interplay between both feature types, we run addition and ablation tests for individual feature groups. Trait-specific addition tests across prompts indicate that the embedding-based models can most consistently be enhanced in content assessment when combined with morphological complexity features. Most consistent performance gains in the organization traits are achieved when embeddings are combined with length features, and most consistent performance gains in the assessment of the language traits when combined with lexical complexity, error, and occurrence features. Cross-prompt scoring again reveals slight advantages for the feature-based models.

Although the ability to comprehend what one is reading is one of the most fundamental necessities to function within society, the reading comprehension skills of students have recently been on the decline in many countries. An essential prerequisite to reading comprehension is the ability to read fluently, which is defined as the ability to read (aloud) with accuracy, speed, automaticity and prosody. Current oral reading fluency assessment instruments seldom provide detailed diagnostics however, and bestow a heavy testing burden on practitioners. Recent developments in Artificial Intelligence-based assessment methodology might provide a solution to current assessment issues, but thorough validations of such procedures have proven scarce. This study evaluates whether valid word decoding and passage reading measures (accuracy, speed and automaticity) can be generated for a semi-transparent language, using an automatic speech recognition (ASR) based oral reading fluency assessment instrument. A validation study was conducted, using the Argument-Based Approach to Validation. Data concerned 176 h of speech data, and the results of 569 and 622 oral word- and passage reading tests that are currently administered in primary schools, from 653 children attending the second- or third grade of Dutch primary education. The results of the validation indicate that it is possible to generate fluency metrics for a semi-transparent language, using an ASR-based oral reading fluency assessment instrument. Future researchers are advised to further optimize the ASR, evaluate its errors, and realize a prosody component, completing the envisioned reading fluency assessment instrument, thereby improving reading fluency assessment throughout primary education.

Supplementary information: The online version contains supplementary material available at 10.1007/s40593-025-00480-y.