The microstructures and mechanical properties of ferrite-based lightweight steel with different compositions were investigated by tensile test, scanning electron microscopy (SEM), transmission electron microscopy (TEM), X-ray diffraction (XRD) and thermodynamic calculation (TC). It was shown that the ferrite-based lightweight steels with 5 wt. % or 8 wt. % A1 were basically composed of ferrite, austenite and κ-carbide. As the annealing temperature increased, the content of the austenite in the steel gradually increased, while the κ-carbide gradually decomposed and finally disappeared. The mechanical properties of the steel with 5 wt.% A1 and 2 wt. % Cr, composed of ferrite and Cr7C3 carbide at different annealing temperatures, were significantly inferior to those of others. The steel containing 5 wt. % Al, annealed at 820 °C for 50 s then rapidly cooled to 400 °C and held for 180 s, can obtain the best product of strength and elongation (PSE) of 31242 MPa · %. The austenite stability of the steel is better, and its PSE is higher. In addition, the steel with higher PSE has a more stable instantaneous strain hardening exponent (n value), which is mainly caused by the effect of transformation induced plasticity (TRIP). When the κ-carbide or Cr7C3 carbide existed in the microstructure of the steel, there was an obvious yield plateau in the tensile curve, while its PSE decreased significantly.