Gene Ontology based protein functional annotation using pretrained embeddings

Abstract

The Gene Ontology (GO) database contains approximately 40,000 classes of terms arranged in a hierarchical relationship. These terms mainly define protein functions and are used in bioinformatics to automatically predict protein functions using their sequences. Recently, several models have been studied, such as ProtBert and ProteinBERT, which predict protein functions by fine-tuning a pretrained model of the nucleotide sequence using a self-supervised deep method. We proposed two models to predict GO using protein features extracted by the ProtBert model to annotate proteins with their GO terms. Additionally, we customized the ProteinBERT model and fine-tuned it to predict GO terms. The experiment showed that protein embeddings created using pretrained transformer models can be used as a source of data for tasks involving sequence prediction, with a focus on protein functions. The suggested models allow flexible sequence lengths and provide improved performance compared to other comparison methods.