Interobserver Reliability of Remote Observations

Abstract

ABSTRACT Following the pandemic-related school shutdowns in spring 2020, direct observations continued to be a necessary component of special education evaluations even when students were not present at school. As students began learning at home instead of in classrooms, the continued need for observational data likely compelled most educators to use video conferencing for collecting data even though little was known about the reliability of observing students remotely. Consequently, the following is an exploratory examination of the interobserver agreement (IOA) that can occur between remote and in-person observers. Within an alternating observers design, overall IOA of 97.2% was achieved between observers who simultaneously used Ecobehavioral Assessment Systems Software (EBASS) to measure the behavior of elementary students and their instructional environments during reading tutoring. This study indicates that EBASS may be a reliable tool for observing students remotely and, in turn, emphasizes the importance of replication and experimental studies to determine whether these findings are typical, especially if millions of students continue to be taught at home.