Phase change materials (PCMs) have attracted considerable attention for their energy storage and thermal regulation properties. However, the solid–liquid leakage, low thermal conductivity, and single functionality of PCM composites hindered their applications, especially. In this paper, the phase change composite fibrous membranes with highly efficient thermal management ability are prepared by coaxial electrospinning technology with polyethylene glycol (PEG) as the PCM in the core layer and polyamide 6 (PA6) as the supporting materials. Convenient impregnation, a method with cesium-tungsten bronze (Cs0.32WO3, CWO) dispersion, was applied to integrate the infrared shielding agents into the shell. The prepared PEG/CWO/PA6 membranes exhibited a high thermal enthalpy of 92.04 J/g with a loading rate of 63.8%. PEG was well encapsulated in the fibers and maintained stability after 50 thermal cycles, with only 5.6% loss of heat latent and no obvious leakage. The synergistic effect of PCMs and infrared shielding agents ensured the application in thermal management, with an obvious decrease in temperature of about 2.8°C, especially under infrared irradiation. The phase change composite fibrous membranes synergistically fabricated with infrared shielding materials are controllable and anticipated to be highly potential materials for thermal control and temperature modulation in electronic and intelligent devices.