Large Language Model Agent-Based Framework for automated Treatment Prescription in Patients with Chronic Hepatitis C Virus Infection

Background

Large Language Models (LLMs) may be useful for clinical tasks that require reasoning over different information sources. LLMs could be regarded as intelligent "agents" with internal planning abilities, enabling them to engage in multi-step reasoning and interact with other agents or external users. Hepatitis C Virus (HCV) management is a potential area where LLM-enabled agents could be useful, since treatment decisions require consideration of genotype, treatment history, liver fibrosis extent, and drug-drug interactions.

Aim

To evaluate the performance of different LLM agent-based configurations in automating HCV treatment decisions and to determine whether specialized multi-agent architectures improve prescription accuracy compared to single-agent approaches.

Material and Methods

We developed 50 clinical cases focusing on therapeutic regimen prescription. Cases included genotype, prior treatment history, fibrosis status, and concurrent medications. We compared multiple configurations using OpenAI's GPT-3.5 and GPT-4o, fine-tuned with HCV treatment guidelines. Different agent architectures were tested: single agent (one LLM extracting all data), multi-agent (three specialized LLMs for data extraction plus prescriber), and specialized multi-agent (four specialized extraction agents plus prescriber). Each agent accessed specific guideline sections relevant to its task. Performance was compared to baseline fine-tuned models.

Results

Using GPT-3.5, the baseline model achieved 24% prescription accuracy. The single agent configuration reached 50% (p=0.007), multi-agent 76% (p<0.001), and specialized multi-agent 89% (p<0.001). With GPT-4, performance improved significantly: baseline 35% accuracy, single agent 65% (p=0.005), multi-agent 88% (p<0.001), and specialized multi-agent 94% (p<0.001).

Conclusions

Specialized multi-agent LLM frameworks significantly improve HCV treatment recommendation accuracy, with GPT-4 showing superior performance. The agent-based approach demonstrates the potential for complex clinical decision-making. Future work should validate these findings in real-world settings and explore integration with clinical decision-support systems.