Nucleotide-resolution DNA foundation models of prokaryotic genomes

Abstract

Machine learning models have recently generated excitement for their potential in a broad range of domain applications, including genomics. However, owing to their complexity, they are prohibitively expensive to train for the large genomic contexts of DNA language models, resulting in limited receptive fields and/or n-mer sequence tokenization. Nguyen et al. present a step forward for the field with Evo, a foundation model that applies the efficient, hybrid StripedHyena architecture trained on 80,000 prokaryotic and millions of phage and plasmid sequences at single-nucleotide resolution. In benchmarking, Evo shows equivalent or improved performance against state-of-the-art nucleotide and language models for variant fitness, promoter activity and protein expression prediction. Impressively, Evo can be used to generate novel, experimentally validated CRISPR–Cas and transposon systems and predict gene essentiality by premature stop codon insertion; it also shows some promise for generating synthetic whole genomes. Foundation DNA language models that are applicable to many tasks would be of broad utility and Evo underlines their promise. However, it must be noted that the training datasets are small compared to the genomes of eukaryotes, and the still-limited 131-kb context and next-token prediction will need to be further adapted for the increased complexity of multicellular life, showing there is still much to do.

Original reference: Science 386, eado9336 (2024)