Speech provides a rich behavioral signal of psychosis, yet its diagnostic use remains limited because speech patterns vary widely across individuals and contexts. We model this variability as uncertainty, capturing how consistently speech features indicate symptom expression. We introduce a multimodal model that integrates acoustic and linguistic information to predict symptom severity and psychosis-related traits across the spectrum, from high schizotypy to clinical psychosis. By estimating uncertainty for each modality, the model learns when to rely on specific signals, adapting to speech quality and task context to improve accuracy and interpretability. Using speech from 114 participants-32 with early psychosis and 82 with low or high schizotypy-recorded in German across structured and narrative tasks, the model achieved an F1-score of 83% (ECE = 0.045), demonstrating robust and well-calibrated performance. Uncertainty estimation further revealed which speech markers most reliably indicated symptoms, including pitch variability, fluency disruptions, and spectral instability.
Variation in medical practices and reporting standards across healthcare systems limits the transferability of prediction models based on structured electronic health record data. Prior studies have demonstrated that embedding medical codes into a shared semantic space can help address these discrepancies, but real-world applications remain limited. Here, we show that leveraging embeddings from a large language model alongside a transformer-based prediction model provides an effective and scalable solution to enhance generalizability. We call this approach GRASP and apply it to predict the onset of 21 diseases and all-cause mortality in over one million individuals. Trained on the UK Biobank (UK) and evaluated in FinnGen (Finland) and Mount Sinai (USA), GRASP achieved an average ΔC-index that was 88% and 47% higher than language-unaware models, respectively. GRASP also showed significantly higher correlations with polygenic risk scores for 62% of diseases, and maintained robust performance even when datasets were not harmonized to the same data model.
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：