Compaction techniques for nextword indexes

Abstract

Most queries to text search engines are ranked or Boolean. Phrase querying is a powerful technique for rejning searches, but is expensive to implement on conventional indexes. In previous work we introduced the nextword index, a structure specifically designed for phrase queries, which however is relatively large. In this paper we introduce new compaction techniques for nextword indexes. In contrast to most index compression schemes, these techniques are lossy, yet as we show allow full resolution ofphrase queries without false match checking. We show experimentally that our novel techniques lead to significant savings in index size.