Interobserver Agreement on “Assessment of Debris” During Endoscopic Ultrasound-guided Drainage of Walled-off Pancreatic Necrosis

Background and Aims

Estimation of the solid component in walled-off pancreatic necrosis (WOPN) during endoscopic ultrasound (EUS) is an important parameter to guide the selection of stent for drainage but is often not objectively quantified.

Methods

We aimed to study the interobserver agreement (IOA) among endosonographers assessing debris in the pancreatic fluid collection and their choice of stent. EUS videos of 15 patients having pancreatic fluid collection with concurrent magnetic resonance imaging (MRI) assessment of debris were independently reviewed by 40 endosonographers for the percentage of debris (10% increments) and their choice of stent (plastic or metal). The Fleiss kappa (κ) coefficient was used to assess IOA. Post-hoc analysis was done using wider debris intervals (20% or greater increments).

Results

Poor agreement was observed for the percentage of debris (κ = 0.188), which did not improve (κ = 0.196) even after including only expert endosonographers with more than 10 years of experience (n = 33). There was fair agreement (κ = 0.266) with reference MRI on the percentage of debris. On post-hoc analysis, the IOA (κ = 0.56, substantial agreement) and agreement with MRI (κ = 0.59, substantial agreement) improved as the debris intervals widened (<10%, 10%-50%, >50%). The agreement for stent selection (plastic vs metal) was poor (κ = 0.174) and did not improve with case volume (κ = 0.153 among respondents with >25 EUS-guided drainage/year; n = 21) or years of experience (κ = 0.195 for >10-year experience; n = 33).

Conclusion

IOA between endosonographers regarding estimation of debris in WOPN and subsequent stent choice for drainage is poor. The experience of endosonographers did not improve IOA. Studies to standardize the EUS criteria for debris assessment in WOPN and subsequent therapeutic approaches are warranted.