Combined Coverage, Attention and Pointer Networks for Improving Slot Filling in Spoken Language Understanding

Abstract

Sequence to sequence (Seq2Seq) model together with pointer network (Ptr-Net) has recently show promising results in slot filling task, in the situation where only sentence-level annotations are available, while the model's prediction contains repetition of slot values. In this paper, we add a coverage mechanism to alleviate issues of repeating prediction in slot filling task. We use a coverage vector to record attention history, and then add to the computation of attention, which can force model to consider more about un-predicted slot values. Experiments show that the proposed model significantly improves slot value prediction F1 with 8.5% relative improvement compare to the baseline models on benchmark DSTC2 (Dialog State Tracking Challenge 2) datasets.