Measuring the Accuracy of Information Retrieval Based Bug Localization Techniques

Bug localization involves using information about a bug to locate affected code sections. Several automated bug localization techniques based on information retrieval (IR) models have been constructed recently. The "gold standard" of measuring an IR technique's accuracy considers the technique's ability to locate a "first relevant method." However, the question remains -- does finding this single method enable the location of a complete set of affected methods? Previous arguments assume this to be true, however, few analyses of this assumption have been performed. In this paper, we perform a case study to test the reliability of this "gold standard" assumption. To further measure IR accuracy in the context of bug localization, we analyze the relevance of the IR model's "first method returned." We use various structural analysis techniques to extend relevant methods located by IR techniques and determine accuracy and reliability of these assumptions.