{"title":"Observer agreement on the morphology of porous cranial lesions: Results from a workshop at the 2019 meeting of the Paleopathology Association","authors":"Amy S. Anderson","doi":"10.1016/j.ijpp.2023.09.006","DOIUrl":null,"url":null,"abstract":"<div><h3>Objective</h3><p>This paper presents the results of a 2019 Paleopathology Association workshop that tested observer agreement on porous cranial lesion morphology and presence using multiple sets of existing guidelines for data collection.</p></div><div><h3>Materials</h3><p>Sixteen conference attendees of varying osteological experience served as observers. Three crania were assigned to each of four published guidelines for identifying and categorizing lesion morphology, for a total of twelve well-preserved human crania from the National Museum<span> of Natural History Biological Anthropology Collections.</span></p></div><div><h3>Methods</h3><p>Observers assessed each cranium macroscopically according to its assigned set of guidelines.</p></div><div><h3>Results</h3><p>Observer concordance was higher using scoring guidelines with higher-quality photographs, such as the 2019 guidelines from Rinaldo and colleagues.</p></div><div><h3>Conclusions</h3><p>Data collection guidelines with high-quality color photos may support greater reliability of researcher-generated data on macroscopic skeletal features.</p></div><div><h3>Significance</h3><p>The conclusions of any research study are only as reliable as the data on which they are based. This work highlights the need for ongoing practices of quality control in a field in which much data results from individual judgement calls.</p></div><div><h3>Limitations</h3><p>Observer concordance is not a measure of observer accuracy. Sample size is insufficient to draw broadly generalizable conclusions on the reliability of data collected using the guidelines tested, and conference environments are not a facsimile of research settings.</p></div><div><h3>Suggestions for further research</h3><p>Iterative testing of methodological consistency using larger sample sizes and more non-pathological crania is advised to identify the factors that influence observer discordance and to improve guidelines for qualitative assessments.</p></div>","PeriodicalId":48817,"journal":{"name":"International Journal of Paleopathology","volume":null,"pages":null},"PeriodicalIF":1.3000,"publicationDate":"2023-10-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Paleopathology","FirstCategoryId":"89","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1879981723000542","RegionNum":3,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"PALEONTOLOGY","Score":null,"Total":0}
引用次数: 0
Abstract
Objective
This paper presents the results of a 2019 Paleopathology Association workshop that tested observer agreement on porous cranial lesion morphology and presence using multiple sets of existing guidelines for data collection.
Materials
Sixteen conference attendees of varying osteological experience served as observers. Three crania were assigned to each of four published guidelines for identifying and categorizing lesion morphology, for a total of twelve well-preserved human crania from the National Museum of Natural History Biological Anthropology Collections.
Methods
Observers assessed each cranium macroscopically according to its assigned set of guidelines.
Results
Observer concordance was higher using scoring guidelines with higher-quality photographs, such as the 2019 guidelines from Rinaldo and colleagues.
Conclusions
Data collection guidelines with high-quality color photos may support greater reliability of researcher-generated data on macroscopic skeletal features.
Significance
The conclusions of any research study are only as reliable as the data on which they are based. This work highlights the need for ongoing practices of quality control in a field in which much data results from individual judgement calls.
Limitations
Observer concordance is not a measure of observer accuracy. Sample size is insufficient to draw broadly generalizable conclusions on the reliability of data collected using the guidelines tested, and conference environments are not a facsimile of research settings.
Suggestions for further research
Iterative testing of methodological consistency using larger sample sizes and more non-pathological crania is advised to identify the factors that influence observer discordance and to improve guidelines for qualitative assessments.
期刊介绍:
Paleopathology is the study and application of methods and techniques for investigating diseases and related conditions from skeletal and soft tissue remains. The International Journal of Paleopathology (IJPP) will publish original and significant articles on human and animal (including hominids) disease, based upon the study of physical remains, including osseous, dental, and preserved soft tissues at a range of methodological levels, from direct observation to molecular, chemical, histological and radiographic analysis. Discussion of ways in which these methods can be applied to the reconstruction of health, disease and life histories in the past is central to the discipline, so the journal would also encourage papers covering interpretive and theoretical issues, and those that place the study of disease at the centre of a bioarchaeological or biocultural approach. Papers dealing with historical evidence relating to disease in the past (rather than history of medicine) will also be published. The journal will also accept significant studies that applied previously developed techniques to new materials, setting the research in the context of current debates on past human and animal health.