Evaluating medical student searches of MEDLINE for evidence-based information: process and application of results.

Objective: To evaluate the adequacy of the MEDLINE instruction routinely given to all entering medical students at the University of Miami School of Medicine and the ability of students to search effectively for and retrieve evidence-based information for clinical decision making by the end of their third-year.

Methodology: The authors developed and implemented a strategy for evaluating the search strategies and articles selected by third-year students, who participated in the Objective Structured Clinical Examination (OSCE) in June 1996, 1997, and 1998, and reviewed the literature on evidence-based medicine and evaluation of medical student searches.

Results: A mean of 5% of the students' search strategies and a mean of 26% of articles selected were ranked "excellent" or "good"; a mean of 26% of search strategies were ranked "fair" and a mean of 69% were ranked "poor"; and a mean of 22% of articles selected were ranked "fair" and a mean of 52% were ranked "poor" based on the strategy developed to evaluate student searches.

Conclusions: Evaluating medical student searches for evidence-based information is an effective way of evaluating students' searching proficiency, and, in turn, the adequacy of the instruction they receive. Based on the results of the OSCE test, the school of medicine expanded the library's educational role and the library implemented major changes in the training program. Information on evidence-based medicine is now incorporated into the MEDLINE instruction. Library faculty evaluate the required searches performed by students for evidence-based information during their first and second years; 30% of students are identified for follow-up, individualized instruction based on the evaluation; and a new case-based curriculum has been proposed with a fourteen-week problem-based learning (PBL) block. These developments are timely in light of the evidence-based competencies recently published by the Association of American Medical Colleges.