Combining diffusion and transformer models for enhanced promoter synthesis and strength prediction in deep learning.

Abstract

In the field of synthetic biology, the engineering of synthetic promoters that outperform their natural counterparts is of paramount importance, which can optimize the expression of exogenous genes, enhance the efficiency of metabolic pathways, and possess substantial commercial value. Research indicates that some synthetic promoters have higher transcriptional activity compared to strong natural promoters. However, with the exponential increase in complexity due to the 4ⁿ potential combinations in a promoter sequence of length n, identifying effective synthetic promoters remains a formidable challenge. Deep learning models, by adaptively learning from extensive data sets, have become instrumental in analyzing biological data. This study introduces a diffusion model-based approach for designing promoters viable in model bacteria such as Escherichia coli and cyanobacteria. This model proficiently assimilates and utilizes inherent biological features from natural promoter sequences to engineer synthetic variants. Additionally, we employed a transformer model to evaluate the efficacy of these synthetic promoters, aiming at screening those with high performance. The experimental findings suggest that the synthetic promoters by the diffusion model not only share key biological features with their natural counterparts but also demonstrate greater similarity to natural promoters than those generated by a variational autoencoder. In predicting promoter strength, the transformer model demonstrated improved performance over the convolutional neural network. Finally, we developed an integrated platform for generating promoters and predicting their strength.

Importance: We demonstrated that diffusion models are superior in accomplishing the promoter synthesis task compared to other state-of-the-art deep learning models. The effectiveness of our method was validated using data sets of Escherichia coli and cyanobacteria promoters, showing more stable and prompt convergence and more natural-like promoters than the variational autoencoder model. We extracted sequence information, dimer information, and position information from promoters and combined them with a transformer model to predict promoter strength. Our prediction results were more accurate than those obtained with a convolutional neural network model. Our in silico experiments systematically introduced mutations in promoter sequences and explored their contribution to promoter strength, highlighting the depth of learning in our model.