Improving generalization through self-supervised learning using generative pre-training transformer for natural gas segmentation

Abstract

Seismic reflection is an essential geophysical method for subsurface mapping in the oil and gas industry, used to deduce the position and extension of potential gas accumulations through the processing and interpretation of data. However, interpreting seismic reflection data presents inherent ambiguity, as similar signatures may arise from geological scenarios with distinct physical properties. Furthermore, seismic interpretation is costly and labor-intensive, complicating the generation of extensive labeled datasets useful for conventional AI-assisted solutions. The emergence of self-supervised learning techniques has gained significant attention in the AI community, enabling the pre-training of deep neural networks without annotated data. This paper proposes a two-stage method for segmenting natural gas regions in seismic reflection images by using a generative pre-training transformer model for feature extraction and a recurrent neural network for per-seismic trace analysis. In the first stage, we pre-train the feature extractor in a self-supervised fashion under a seismic image reconstruction task. In the second stage, using the pre-trained feature extractor, we train the recurrent neural network for the segmentation of natural gas accumulations in 1D seismic image traces. We conducted quantitative and qualitative experiments on a private dataset to demonstrate how our self-supervised learning methodology helps achieve better gains in model generalization. The increase, which can reach up to 20%, in the F1-Score metric corroborates the latter. Thus, an improvement in generalization corresponds to an increase in success rates in drilling new wells.