Discovering Internal Validity Threats and Operational Concerns in Single-Case Experimental Designs Through Directed Acyclic Graphs

Abstract

Single-case experimental designs (SCEDs) have a long history in clinical and educational disciplines. One underdeveloped area in advancing SCED design and analysis is understanding the process of how internal validity threats and operational concerns are avoided or mitigated. Two strategies to ameliorate such issues in SCED involve replication and randomization. Although replication and randomization are indispensable tools in improving the internal validity of SCEDs, little attention has been paid to (a) why this is the case; or (b) the ways in which these design features are not immune from internal validity threats and operational concerns. In the current paper, we describe the use of directed acyclic graphs (DAGs) to better understand, discover, and mitigate internal validity threats and operational concerns in SCEDs. DAGs are a tool for visualizing causal relations among variables and can help researchers identify both causal and noncausal relations among their variables according to specific algorithms. We introduce the use of DAGs in SCEDs to prompt applied researchers to conceptualize internal validity threats and operational concerns, even when an SCED includes replication and randomization in the design structure. We discuss the general principles of causal inference in conventional “group” designs and in SCEDs, the unique factors impacting SCEDs, and how DAGs can be incorporated into SCEDs. We also discuss the limitations of DAGs applied to SCEDs, as well as future directions for this area of work.