With global warming accelerating, the heavily populated and rapidly urbanized coastal regions of the Pearl River Delta (PRD) and the Yangtze River Delta (YRD) stand as representative areas with mounting concerns about extreme heat stress. This study analyzes differentiated effects of temperature (TAS) and relative humidity (RH) on human heat stress measured by wet-bulb globe temperature (WBGT) in those urban regions based on machine learning and mathematical derivation, while also examining the impacts of global warming and urbanization on prospective heat risks. To generate fine-scale climate projections targeted at the PRD and YRD, two global projections forced by Representative Concentration Pathway (RCP) 8.5 scenario are dynamically downscaled using non-hydrostatic Regional Climate Model version 4.7 (RegCM4), with the urban density and extent updated every year based on Shared Socioeconomic Pathways 5-8.5 (SSP5) scenario, thereby incorporating the transient urban growth into future projections. The bias-corrected downscaled simulations effectively capture the distinct interdependencies between TAS and RH on WBGT across different regions, similar to the observed patterns during the historical period. While the absolute contribution of TAS to WBGT is larger than RH regardless of warming levels and regions, the relative increase in RH becomes more pronounced with warming. Under RCP8.5 scenario, unprecedentedly extreme WBGT is projected to emerge in the far future (2080–2099). In contrast, the effect of urbanization appears to be more dominant in the near future (2030–2049) as urban density under SSP5 scenario is projected to peak around the 2040s and gradually decrease afterwards. The reduction of RH is found in the intensely urbanized areas locally, but it does not significantly lower WBGT because the positive contribution of increased TAS is more dominant. As a result, highly urbanized regions still exhibit higher WBGT compared to other areas. In addition, urban heat island effect is more pronounced for compact areas with high urban density (i.e., PRD) and at night. Despite the smaller temperature increase from urban heat island effect compared to global warming, it can play a critical role in exacerbating heat stress, adding to the already dangerous humid and hot conditions.