Inter-rater reliability of symptom repertorisation: a pragmatic empirical study

Objective: To determine the extent to which two homeopaths agree on whether symptoms reported by patients in a proving are possibly associated with Mercurius solubilis.

Design: Blinded, inter-rater reliability study.

Participants: 104 subjects in a randomised, double-blind mercury proving.

Outcome measures: 557 symptom episodes spontaneously reported by subjects were classified as ‘mercury’ or ‘not mercury’ by two homeopaths working blind to each other's conclusions and to patient allocation.

Results: Initial agreement between homeopaths was 70.2%, a kappa of 0.39, (95% CI 0.31, 0.47). Some disagreements appear to have resulted from differing interpretations of the study instructions. After suitable correction, agreement was 76.5% and kappa 0.56 (95% CI 0.49, 0.63).

Conclusions: The study homeopaths had only a moderate level degree of agreement greater than that expected by chance. The main factor seems to have been differences between data from different sources. There is an urgent need for more research on the methods of choosing homoeopathic medicines in order to improve the reliability and validity of homoeopathic diagnoses.