Natural language processing and information systems : ... International Conference on Applications of Natural Language to Information Systems, NLDB ... revised papers. International Conference on Applications of Natural Language to Info...最新文献

How important is domain-specific language model pretraining and instruction finetuning for biomedical relation extraction? 特定领域的语言模型预训练和指令微调对生物医学关系提取有多重要？

Natural language processing and information systems : ... International Conference on Applications of Natural Language to Information Systems, NLDB ... revised papers. International Conference on Applications of Natural Language to Info...

Pub Date : 2026-01-01 Epub Date: 2025-07-01 DOI: 10.1007/978-3-031-97141-9_6

Aviv Brokman, Ramakanth Kavuluru

Major technical advances in the general NLP domain are often subsequently applied to the high-value, data-rich biomedical domain. The past few years have seen generative language models (LMs), instruction finetuning, and few-shot learning become foci of NLP research. As such, generative LMs pretrained on biomedical corpora have proliferated and biomedical instruction finetuning has been attempted as well, all with the hope that domain specificity improves performance on downstream tasks. Given the nontrivial effort in training such models, we investigate what, if any, benefits they have in the key biomedical NLP task of relation extraction. Specifically, we address two questions: (1) Do LMs trained on biomedical corpora outperform those trained on general domain corpora? (2) Do models instruction finetuned on biomedical datasets outperform those finetuned on assorted datasets or those simply pretrained? We tackle these questions using existing LMs, testing across four datasets. In a surprising result, general-domain models typically outperformed biomedical-domain models. However, biomedical instruction finetuning improved performance to a similar degree as general instruction finetuning, despite having orders of magnitude fewer instructions. Our findings suggest it may be more fruitful to focus research effort on larger-scale biomedical instruction finetuning of general LMs over building domain-specific biomedical LMs.

一般自然语言处理领域的主要技术进步通常随后应用于高价值、数据丰富的生物医学领域。在过去的几年里，生成语言模型（LMs）、指令微调和少量学习成为NLP研究的焦点。因此，在生物医学语料库上预训练的生成式LMs已经激增，生物医学指令微调也已经尝试过，所有这些都希望领域特异性可以提高下游任务的性能。考虑到在训练这些模型方面的重要努力，我们调查了它们在关系提取的关键生物医学NLP任务中有什么好处，如果有的话。具体来说，我们解决了两个问题：(1)在生物医学语料库上训练的LMs是否优于在一般领域语料库上训练的LMs ？(2)在生物医学数据集上微调的模型指令是否优于在各种数据集上微调的模型指令或简单预训练的模型指令？我们使用现有的LMs解决这些问题，跨四个数据集进行测试。令人惊讶的结果是，一般领域模型通常优于生物医学领域模型。然而，生物医学指令微调提高性能的程度与一般指令微调相似，尽管有数量级较少的指令。我们的研究结果表明，将研究重点放在一般LMs的大规模生物医学教学微调上可能比构建特定领域的生物医学LMs更有成效。

{"title":"How important is domain-specific language model pretraining and instruction finetuning for biomedical relation extraction?","authors":"Aviv Brokman, Ramakanth Kavuluru","doi":"10.1007/978-3-031-97141-9_6","DOIUrl":"https://doi.org/10.1007/978-3-031-97141-9_6","url":null,"abstract":"Major technical advances in the general NLP domain are often subsequently applied to the high-value, data-rich biomedical domain. The past few years have seen generative language models (LMs), instruction finetuning, and few-shot learning become foci of NLP research. As such, generative LMs pretrained on biomedical corpora have proliferated and biomedical instruction finetuning has been attempted as well, all with the hope that domain specificity improves performance on downstream tasks. Given the nontrivial effort in training such models, we investigate what, if any, benefits they have in the key biomedical NLP task of relation extraction. Specifically, we address two questions: (1) Do LMs trained on biomedical corpora outperform those trained on general domain corpora? (2) Do models instruction finetuned on biomedical datasets outperform those finetuned on assorted datasets or those simply pretrained? We tackle these questions using existing LMs, testing across four datasets. In a surprising result, general-domain models typically outperformed biomedical-domain models. However, biomedical instruction finetuning improved performance to a similar degree as general instruction finetuning, despite having orders of magnitude fewer instructions. Our findings suggest it may be more fruitful to focus research effort on larger-scale biomedical instruction finetuning of general LMs over building domain-specific biomedical LMs.","PeriodicalId":92107,"journal":{"name":"Natural language processing and information systems : ... International Conference on Applications of Natural Language to Information Systems, NLDB ... revised papers. International Conference on Applications of Natural Language to Info...","volume":"15836 ","pages":"80-94"},"PeriodicalIF":0.0,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12367199/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144981967","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Comparison of pipelines, seq2seq models, and LLMs for rare disease information extraction. 罕见病信息提取管道、seq2seq模型和llm的比较

Natural language processing and information systems : ... International Conference on Applications of Natural Language to Information Systems, NLDB ... revised papers. International Conference on Applications of Natural Language to Info...

Pub Date : 2026-01-01 Epub Date: 2025-07-01 DOI: 10.1007/978-3-031-97141-9_4

Shashank Gupta, Xuguang Ai, Yuhang Jiang, Ramakanth Kavuluru

End-to-end relation extraction (E2ERE) is an important application of natural language processing (NLP) in biomedicine. The extracted relations populate knowledge graphs and drive more high level applications in knowledge discovery and information retrieval. E2ERE is frequently handled at the sentence level involving continuous entities. A more complex setting is document level E2ERE with discontinuous and overlapping/nested entities. We identified a recently introduced RE dataset for rare diseases (RareDis) that has these complex traits. Among current E2ERE methods, we see three well-known paradigms: (1) pipeline based approaches where a named entity recognition (NER) model's output is input to a relation classification (RC) model; (2) joint sequence-to-sequence style models where the raw input text is directly transformed into relations through linearization schemas; and (3) generative large language models (LLMs), where prompts, fine-tuning, and in-context learning are being leveraged for RE. While LLMs are becoming popular because of tools such as ChatGPT, the biomedical NLP community needs to carefully evaluate which paradigm is more suitable for E2ERE. In this effort, using the RareDis dataset as a complex use-case, we evaluate the best representative models from each of the three paradigms for E2ERE. Our findings reveal that pipeline models are still the best, while sequence-to-sequence models are not far behind. We verify these findings on a second E2ERE dataset for chemical-protein interactions. Although LLMs are more suitable for zero-shot settings, our results show that it is better to work with more conventional models trained and tailored for E2ERE when training data is available. Our contribution is also the first to conduct E2ERE for the RareDis dataset.

端到端关系提取（E2ERE）是自然语言处理（NLP）在生物医学中的重要应用。提取的关系填充知识图，并推动知识发现和信息检索的高级应用。E2ERE通常在涉及连续实体的句子级处理。更复杂的设置是具有不连续和重叠/嵌套实体的文档级E2ERE。我们确定了一个最近引入的罕见病（RareDis）的RE数据集，该数据集具有这些复杂的特征。在当前的E2ERE方法中，我们看到了三种众所周知的范例：(1)基于管道的方法，其中命名实体识别（NER）模型的输出输入到关系分类（RC）模型；(2)联合序列到序列样式模型，通过线性化模式将原始输入文本直接转换为关系；(3)生成式大型语言模型（llm），其中提示、微调和上下文学习被用于RE。虽然llm由于ChatGPT等工具而变得流行，但生物医学NLP社区需要仔细评估哪种范式更适合E2ERE。在这项工作中，使用RareDis数据集作为一个复杂的用例，我们从E2ERE的三个范例中评估了最具代表性的模型。我们的研究结果表明，管道模型仍然是最好的，而序列到序列模型也不落后。我们在第二个E2ERE化学-蛋白质相互作用数据集上验证了这些发现。虽然llm更适合零射击设置，但我们的研究结果表明，当训练数据可用时，更好地使用为E2ERE训练和定制的更传统的模型。我们的贡献也是第一个对RareDis数据集进行E2ERE的人。

{"title":"Comparison of pipelines, seq2seq models, and LLMs for rare disease information extraction.","authors":"Shashank Gupta, Xuguang Ai, Yuhang Jiang, Ramakanth Kavuluru","doi":"10.1007/978-3-031-97141-9_4","DOIUrl":"https://doi.org/10.1007/978-3-031-97141-9_4","url":null,"abstract":"End-to-end relation extraction (E2ERE) is an important application of natural language processing (NLP) in biomedicine. The extracted relations populate knowledge graphs and drive more high level applications in knowledge discovery and information retrieval. E2ERE is frequently handled at the sentence level involving continuous entities. A more complex setting is document level E2ERE with discontinuous and overlapping/nested entities. We identified a recently introduced RE dataset for rare diseases (RareDis) that has these complex traits. Among current E2ERE methods, we see three well-known paradigms: (1) pipeline based approaches where a named entity recognition (NER) model's output is input to a relation classification (RC) model; (2) joint sequence-to-sequence style models where the raw input text is directly transformed into relations through linearization schemas; and (3) generative large language models (LLMs), where prompts, fine-tuning, and in-context learning are being leveraged for RE. While LLMs are becoming popular because of tools such as ChatGPT, the biomedical NLP community needs to carefully evaluate which paradigm is more suitable for E2ERE. In this effort, using the RareDis dataset as a complex use-case, we evaluate the best representative models from each of the three paradigms for E2ERE. Our findings reveal that pipeline models are still the best, while sequence-to-sequence models are not far behind. We verify these findings on a second E2ERE dataset for chemical-protein interactions. Although LLMs are more suitable for zero-shot settings, our results show that it is better to work with more conventional models trained and tailored for E2ERE when training data is available. Our contribution is also the first to conduct E2ERE for the RareDis dataset.","PeriodicalId":92107,"journal":{"name":"Natural language processing and information systems : ... International Conference on Applications of Natural Language to Information Systems, NLDB ... revised papers. International Conference on Applications of Natural Language to Info...","volume":"15836 ","pages":"49-63"},"PeriodicalIF":0.0,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12367198/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144982006","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Natural Language Processing and Information Systems: 28th International Conference on Applications of Natural Language to Information Systems, NLDB 2023, Derby, UK, June 21–23, 2023, Proceedings 自然语言处理和信息系统:第28届自然语言在信息系统中的应用国际会议，NLDB 2023，德比，英国，2023年6月21-23日，论文集

Natural language processing and information systems : ... International Conference on Applications of Natural Language to Information Systems, NLDB ... revised papers. International Conference on Applications of Natural Language to Info...

Pub Date : 2023-01-01 DOI: 10.1007/978-3-031-35320-8