Magnetic materials, as the microwave absorber, have huge advantages due to high microwave permeability and large microwave magnetic loss. To achieve a broadband microwave absorption, magnetic CoxNiy alloy particles were prepared in this paper by a simple co-precipitation method of H2C2O4 and a quick thermal decomposition process in vacuum, then deformed into micro-flakes by planetary ball milling method. Based on the CoxNiy alloy particles before and after ball milling, a three-layer absorber was designed and fabricated, with an effective absorption band of 5.4–18 GHz at a total thickness of 3.3 mm. It is found that the CoxNiy alloy particles show high coercive force and large microwave magnetic loss after ball milling, capable of greatly attenuating electro-magnetic (EM) waves, as a wave absorbing layer of the three-layer absorber. However, they are soft magnetic with relatively low coercive force and high microwave permeability before ball milling, allowing the incident EM waves to enter the absorbing layer when employed as an impedance matching layer. Through this study, the strategy combining the magnetic alloys before and after ball milling as well as the multilayer structure design, is feasible to achieve a broadband microwave absorption at a thin thickness, providing a simple and effective method for the broadband microwave absorption research and application.