In China, wild Asian elephants (Elephas maximus) are primarily distributed in three prefectures in Southwest Yunnan, along the border with Laos and Myanmar. These elephants occur in small, fragmented populations and face significant threats from habitat loss and fragmentation. Here, we successfully retrieved 48 mitochondrial genomes, including those from 35 wild Asian elephants in China and those from 13 captive Asian elephants, based on whole genome sequencing data to analyze their maternal population structure and genetic diversity. In addition, we extracted approximately 600 kb of non-coding genomic regions for a comparative analysis of the genetic structure between the nuclear and mitochondrial DNA. Wild Asian elephants in China exhibited extremely low genetic diversity compared to global populations, with only two haplotypes detected in the Chinese population. Despite limited mitochondrial haplotypes, the Xishuangbanna population maintains gene flow with external populations. In contrast, the genetic diversity in the Cangyuan population was even more severely limited, with no evidence of gene flow with the nearest populations in Myanmar. Given the close genetic relationship between the Cangyuan population and populations in other countries, the most promising strategy for introducing genetic diversity to rescue the Cangyuan population may involve translocating Asian elephants from other countries. This study provides a deeper understanding of the genetic status of wild Asian elephants in China and offers important insights for future conservation efforts in China and elsewhere.