Automatic extraction of formal features from Word, Excel, and PowerPoint productions in a diagnostic-assessment perspective

Abstract

The large-scale evaluation of documents produced by hundreds or thousands of students poses a challenge in terms of human and time resources and equity of treatment. In this paper, we discuss the development and use of methods for automatic extraction of formal properties of Microsoft Word, Excel, and PowerPoint documents saved in the Office Open XML (OOXML) format for educational purposes. We briefly describe the structural ideas underlying the OOXML format and show examples of how certain document features are represented. Building on this, we describe the tool we developed for automatic evaluation of students' productions in the context of entrance examinations at our institution. The exam tests the participants' word-processing and spreadsheet abilities. We report the results of a case study comparing manual and automatic evaluation. We discuss the benefits (speed, cost, accuracy) and limitations (for situations where we lack precise formal conditions to test) of our approach.