Semi-flexible composite mixture (SFCM) is a kind of pavement material formed by pouring cement-based grout material into a porous asphalt mixture with air voids from 20% to 30%. SFCM is widely used for its outstanding anti-rutting performance. Its mechanical performance is complicated due to its heterogeneity and interlocking structure. According to the present study, asphalt deforms at different temperatures, whereas cement-based grout has no similar characteristics. Rare research focuses on the temperature-based performance of SFCM. Therefore, the study was on the thermal performance of SFCM by seven open-graded asphalt mixture skeletons with different porosities and two types of grouts with early strength (ES) and high strength (HS). The test temperatures ranged from −10 °C to 60 °C. The mechanical investigation was performed using the semi-circular-bending (SCB) and beam bending tests. The strain sensor was used for analyzing the thermal performance of SFCM. The results show that the temperature significantly affected the SFCM's performance. The porosity was selected for three sections based on the trend of fracture energy (Gf) curves at 25 °C. The turning points were the porosity values of 20% and 26%. The initiation slope during elastic deformation increases with the porosity increase. This trend was more evident at intermediate temperature. The shrink strain of SFCM was lower than that of the usual asphalt mixture (AC). The thermal stress of the SFCM filled with HS (HS-SFCM) was higher than that of the SFCM filled with ES (ES-SFCM) at −10 °C. Moreover, the thermal failure characteristics of SFCM were influenced by porosity.