The development of a novel, standardized, norm-referenced Arabic Discourse Assessment Tool (ADAT), including an examination of psychometric properties of discourse measures in aphasia

Abstract

Background

People with aphasia (PWA) typically exhibit deficits in spoken discourse. Discourse analysis is the gold standard approach to assess language deficits beyond sentence level. However, the available discourse assessment tools are biased towards English and European languages and Western culture. Additionally, there is a lack of consensus on which discourse measures to use and limited evidence of the psychometric properties of published discourse measurements.

Aims

(1) To develop a standardized, norm-referenced, culturally and linguistically appropriate Arabic Discourse Assessment Tool (ADAT); and (2) to examine the psychometric properties of content and construct validity and interrater reliability of different discourse measures elicited using three discourse genres (descriptive, narrative and procedural) in neurotypical control adults and matched PWA.

Methods & Procedures

Discourse samples were collected using three novel discourse stimuli that are sensitive to the Arabic language and culture from 70 neurotypical control adults and a matched group of 50 PWA. Transcription agreement was assessed. A standard approach was used to evaluate construct validity and interrater reliability for 16 discourse measures that assess fluency, language productivity, information content, lexical–semantics, lexical diversity, grammatical category, grammatical structure and syntactic complexity. Strong measures were identified based on their psychometric properties, and normative data were established on these measures. Discourse performance of PWA was then examined using the newly developed tool (ADAT).

Outcomes & Results

Transcription agreement was extremely high for all discourse stimuli in both groups. Eight discourse measures were proven to have consistently very high construct validity and consistently very good to excellent reliability across the three stimuli in both neurotypical control and aphasia groups: lexical information units, content information units, words per minute, discourse duration, number of different words, number of complete sentences and proportion of open and closed class words. Norms were established on these measures, and cut-off scores of impairments were determined. Other measures showed low construct validity and variable or poor reliability across the two groups.

Conclusions & Implications

The newly developed, standardized, and norm-referenced tool (ADAT) consist of three discourse stimuli and eight high-quality discourse measures that assess multiple aspects of spoken discourse and were able to differentiate PWA from neurotypical adults consistently. ADAT also includes normative data and cut-off impairment scores. The tool has great potential to enhance clinical practice and research with Arabic speakers. Evidence was provided that not all discourse measures are of high quality, as some are vulnerable to differences between raters, discourse stimuli and groups. Clinicians and researchers can use ADAT for accurate aphasia assessments, better management plans and to monitor therapy effectiveness. ADAT can be further validated in other clinical populations with language impairments.

WHAT THIS PAPER ADDS

What is already known on the subject

• Discourse analysis is the gold standard approach to assess language deficits beyond sentence level. However, existing discourse assessment tools are biased towards English and European languages and Western culture. Additionally, there is a lack of consensus on which discourse measures to use in aphasia, and limited evidence of the psychometric properties of published discourse measurements.

What this paper adds to existing knowledge

• A novel, standardized, norm-referenced Arabic Discourse Assessment Tool (ADAT) was developed and validated in this study. ADAT was further validated among PWA. The study provides evidence that not all discourse measures are of high quality and thus should not be used with confidence. Specific measures are vulnerable to the type of stimuli, the rater and/or the tested group. On the other hand, eight discourse measures were identified to be reliable between different raters and across different stimuli for the two groups, and they were able to differentiate the discourse performance of PWA from neurotypical control adults. Normative data derived from neurotypical control adults were established on these strong measures, and the performance of PWA was classified as impaired based on these norms.

What are the potential or actual clinical implications of this work?

• The present study provides a novel, standardized, norm-referenced, validated discourse assessment tool that is culturally and linguistically appropriate for use by Arabic speakers (ADAT). ADAT holds immense potential to enhance clinical practice and research with Arabic speakers. The study also identified strong discourse measures that can be used to assess language productivity, information content, lexical–semantics, lexical diversity, grammatical category, and syntactic complexity for accurate and comprehensive assessments. This will lead to better rehabilitation management by guiding the development of tailored client-centred interventions. ADAT can be utilized in clinical and research settings in PWA and has the potential to be further validated with other clinical populations.