Investigating deep-learning NLP for automating the extraction of oncology efficacy endpoints from scientific literature

Objective

Benchmarking drug efficacy is a critical step in clinical trial design and planning. The challenge is that much of the data on efficacy endpoints is stored in scientific papers in free text form, so extraction of such data is currently a largely manual task. Our objective is to automate this task as much as possible.

Methods

In this study we have developed and optimised a framework to extract efficacy endpoints from text in scientific papers, using a machine learning approach.

Results

Our machine learning model predicts 25 classes associated with efficacy endpoints and leads to high F1 scores (harmonic mean of precision and recall) of 96.4 % on the test set, and 93.9 % and 93.7 % on two case studies.

Conclusion

These methods were evaluated against – and showed strong agreement with – subject matter experts and show significant promise in the future of automating the extraction of clinical endpoints from free text.

Significance

Clinical information extraction from text data is currently a laborious manual task which scales poorly and is prone to human error. Demonstrating the ability to extract efficacy endpoints automatically shows great promise for accelerating clinical trial design moving forwards.