The low power density and high frequency loss of the core limits the miniaturization of high-frequency transformers (HFT) with an increase of the switching frequency. In this paper, a composite core with laminated nanocrystalline films and ferrite sheets is proposed based on the high saturation magnetic flux density of nanocrystalline and the low loss of ferrite. The two materials are laminated at a certain thickness ratio to form a composite unit. Then multiple layers of units are stacked to form the composite core. A homogenization model is established to calculate the equivalent permeability. Then the magnetic field strength of the composite core can be obtained, which can be used to calculate the magnetic flux density in different materials. An optimization model is built with the objective of optimizing the core loss and power density by adjusting the thickness ratio. Based on the non-dominated sorting genetic algorithm II (NSGA-II), it obtains the optimal thickness ratio. Simulation results show that the composite core increases the magnetic flux density from 0.3 T to 0.55 T over a ferrite core. A 100 V/200 V, 1 kW, 20 kHz composite core HFT prototype is developed. The power density is increased by 23.5% when compared to a ferrite HFT. The core loss is reduced by 37% when compared to nanocrystalline HFT, and the efficiency is increased from 94% to 96.5%.