N. Midgley, Antonella Cirasola, Eva A. Sprecher, S. Redfern, Hannah Wright, Beth Rider, Peter Martin
{"title":"The Reflective Fostering Programme fidelity rating scale: development and inter-rater reliability","authors":"N. Midgley, Antonella Cirasola, Eva A. Sprecher, S. Redfern, Hannah Wright, Beth Rider, Peter Martin","doi":"10.1108/jcs-01-2022-0002","DOIUrl":null,"url":null,"abstract":"\nPurpose\nThe purpose of this study is to describe the development of the 14-item reflective fostering fidelity rating (RFFR), an observational rating system to evaluate model fidelity of group facilitators in the Reflective Fostering Programme (RFP), a mentalisation-based psychoeducation programme to support foster carers. The authors assess usability, dimensionality, inter-rater reliability and discriminative ability of the RFFR.\n\n\nDesign/methodology/approach\nEighty video clip extracts documenting 20 RFP sessions were independently rated by four raters using the RFFR. The dimensionality of the RFFR was assessed using principal components analysis. Inter-rater agreement was assessed using the intra-class correlation coefficient.\n\n\nFindings\nThe proportion of missing ratings was low at 2.8%. A single principal component summarised over 90% of the variation in ratings for each rater. The inter-rater reliability of individual item ratings was poor-to-moderate, but a summary score had acceptable inter-rater reliability. The authors present evidence that the RFFR can distinguish RFP sessions that differ in treatment fidelity.\n\n\nOriginality/value\nTo the best of the authors’ knowledge, this is the first investigation and report of the RFFR’s validity in assessing the programme fidelity of the RFP. The paper concludes that the RFFR is an appropriate rating measure for treatment fidelity of the RFP and useful for the purposes of both quality control and supervision.\n","PeriodicalId":45244,"journal":{"name":"Journal of Childrens Services","volume":null,"pages":null},"PeriodicalIF":1.4000,"publicationDate":"2023-02-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Childrens Services","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1108/jcs-01-2022-0002","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"SOCIAL WORK","Score":null,"Total":0}
引用次数: 0
Abstract
Purpose
The purpose of this study is to describe the development of the 14-item reflective fostering fidelity rating (RFFR), an observational rating system to evaluate model fidelity of group facilitators in the Reflective Fostering Programme (RFP), a mentalisation-based psychoeducation programme to support foster carers. The authors assess usability, dimensionality, inter-rater reliability and discriminative ability of the RFFR.
Design/methodology/approach
Eighty video clip extracts documenting 20 RFP sessions were independently rated by four raters using the RFFR. The dimensionality of the RFFR was assessed using principal components analysis. Inter-rater agreement was assessed using the intra-class correlation coefficient.
Findings
The proportion of missing ratings was low at 2.8%. A single principal component summarised over 90% of the variation in ratings for each rater. The inter-rater reliability of individual item ratings was poor-to-moderate, but a summary score had acceptable inter-rater reliability. The authors present evidence that the RFFR can distinguish RFP sessions that differ in treatment fidelity.
Originality/value
To the best of the authors’ knowledge, this is the first investigation and report of the RFFR’s validity in assessing the programme fidelity of the RFP. The paper concludes that the RFFR is an appropriate rating measure for treatment fidelity of the RFP and useful for the purposes of both quality control and supervision.