Reading times and temporo-parietal BOLD activity encode the semantic hierarchy of language prediction

Abstract

When poor acoustics challenge speech comprehension, listeners are thought to increasingly draw on semantic context to predict upcoming speech. However, previous research focused mostly on speech material with short timescales of context (e.g., isolated sentences). In an fMRI experiment, 30 participants listened to a one-hour narrative incorporating a multitude of timescales while confronted with competing resynthesized natural sounds. We modeled semantic predictability at five timescales of increasing context length by computing the similarity between word embeddings. An encoding model revealed that short informative timescales are coupled to increased activity in the posterior portion of superior temporal gyrus, whereas long informative timescales are coupled to increased activity in parietal regions like the angular gyrus. In a second experiment, we probed the behavioral relevance of semantic timescales in language prediction: 11 participants performed a self-paced reading task on a text version of the narrative. Reading times sped up for the shortest informative timescale, but also tended to speed up for the longest informative timescales. Our results suggest that short-term dependencies as well as the gist of a story drive behavioral processing fluency and engage a temporo-parietal processing hierarchy.