Ultrasonic testing (UT) is a commonly used method to detect internal damage in composite materials, and the test data are commonly analyzed by manual determination, relying on a priori knowledge to assess the status of the specimen. In this work, A method for the automatic detection of delamination defects based on improved EfficientDet was proposed. The Swin Transformer block was adopted in the Backbone part of the network to capture the global information of the feature map and improve the feature extraction capability of the whole model. Meanwhile, a custom block was added to prompt the model to extract object features from different receptive fields, which enriches the feature information. In the Neck part of the network, the adaptive weighting was used to keep the features that were more conductive to the prediction object, and desert or give smaller weights to those features that were not desirable for the prediction object. Two kinds of specimens were prepared with embedded artificial delamination defects and delamination damage caused by low-velocity impacts. Ultrasonic phased array technology was employed to investigate the specimens and the amount of data was increased by the sliding window approach. The object detection model proposed in this work was evaluated on the obtained dataset and delamination in the composites was effectively detected. The proposed model achieved 98.97% of mean average precision, which is more accurate compared to ultrasonic testing methods.