Interplay of Documents' Readability, Comprehension and Consumer Health Search Performance Across Query Terminology

Abstract

Because of terminology mismatches, health consumers frequently face difficulties while searching the Web for health information. Difficulties arise in query formulation but also in understanding the retrieved documents. In this work we analyze how documents' readability affects users' comprehension and how both affect the retrieval performance, measured in different ways. In addition, we analyze how performance measures relate with each other. For this purpose we have conducted a laboratory user study with 40 participants. We found that readability is essential for a document to be at least partially relevant and that it becomes even more important if the document has medico-scientific terminology. Moreover, the relevance of a document to a specific user highly depends on its comprehension. In lay queries we found the medical accuracy of users' answers is related to the session's relevance assessments. This shows that users can, at least in part, relate their relevance assessments with the medical accuracy of the documents. On the other hand, this relationship does not exist with medico-scientific queries.