China has the most numerous Danxia and Karst landscapes, which serve as special terrestrial islands harboring ample endemic species, though how did these endemic species spread among those isolated sites is still an unresolved issue. To address this question, we explored the phylogeographical structure and demographic history of Firmiana danxiaensis, a tree species endemic to Danxia and Karst landscapes. We collected 295 samples (28 populations) of F. danxiaensis. Plastid genomes were assembled for 25 representative samples. Sanger sequencing of four plastid regions and restriction-site-associated DNA sequencing were performed on the 28 populations. The phylogenetic tree constructed from plastid genomes and restriction site-associated DNA sequencing (RAD-seq) data supported that F. danxiaensis originated from Mount Danxia and Nanxiong Basin, spread to Karst landscapes near Yingde City, and then back to Danxia Mountain and the Nanxiong Basin. In the Nanxiong Basin, the latter arrivals captured the plastid of the former. Population analyses revealed strong population structure among and within Danxia and Karst landscapes, possibly due to low seed and pollen dispersal abilities of the species. The demographic and ecological niche modeling approaches suggested that F. danxiaensis have widely occurred in the southeast of China during the last glacial period, and later retreated to the cliffs of Danxia and Karst landscapes due to temperature rising and competition failure. The declining of the effective population size of the species throughout the postglacial period suggested that global warming, agriculture, and industrial civilizations could have affected the survival of this species, and more measures should be taken to conserve these species.