P12 How to build the ‘right’ directed acyclic graph (DAG): a systematic, transparent and accessible method for evidence synthesis

Abstract

Background Causal inference methods are increasingly popular in health research, with directed acyclic graphs (DAGs) being notably prominent. Theoretically, DAGs are powerful tools for minimising bias in quantitative analysis, however their transition into practice has been problematic. Lack of guidelines for generating the ‘right’ DAG for research questions have been cited as a central reason. This study presents a solution in the form of ‘evidence synthesis for constructing directed acyclic graphs’ (ESC-DAGs). The approach embeds DAGs in a procedural evidence synthesis method which focuses on how to derive and integrate DAGs from research evidence in a transparent and systematic fashion. Methods For studies meeting inclusion criteria: 1) Appraisal of study quality with split focus on the degree of explicit causal thinking employed and on more generic study quality issues such as study design, sample size, etc; 2) Mapping of conclusions for each study using causal inference theory to produce an ‘implied graph’; 3) Translation of implied graphs into DAGs through procedural application of four ‘causal criteria’ to each relationship in the implied graph (temporality, plausibility, recourse to theory, counterfactual thought experiment); 4) Integration of DAGs, starting with those with the highest appraisal scores until all DAGs are integrated. The output is an ‘integrated-DAG’. ESC-DAGs is demonstrated on the exposure-outcome relationship of parental influences on adolescent alcohol harm. Results 30 studies were included. Study appraisal produces a scale with scores ranging from 0 to 5 (median=2). The DAGs produced for individual studies are substantially less comprehensive than the integrated-DAG (covering between 5% and 40% of causal pathways). Over 90% of the implied graphs were changed during the translation process. The most common changes reflect a strong tendency in research to either mistakenly control for mediation or for unjustified control of parallel risk factors. Conclusion As a methodological contribution to an increasingly popular form of health research, ESC-DAGs has broad relevance to population health. Through its systematic treatment of research evidence, ESC-DAGs is a reproducible and transparent process that is suitable for use by researchers with only minimal training on the causal inference methods. Compared to how DAGs have been constructed elsewhere, those generated from ESC-DAGs are more comprehensive and have greater potential to reduce bias. In meeting the need for guidelines on generating DAGs in such a way, ESC-DAGs represents an important step towards realising the potential of DAGs to improve the practice of health research.