The portable, non-destructive Leeb hardness tester is widely used to predict rock material properties. However, the lack of standardized procedures for its application in rock engineering can lead to inaccurate design parameters and increased risks. Therefore, this paper systematically explores effective methods for conducting the Leeb hardness test for rocks. 90 samples of six rock types from different regions were collected, totaling over 1800 test results. Furthermore, this study analyzes how factors such as the number of tests, testing practices, moisture conditions, surface roughness, and sample size influence Leeb hardness testing. Results show that each sample should be tested at least 20 times, with test points evenly distributed in the center area at least 10 mm from the edges of the cylindrical sample’s top and bottom surfaces. The Leeb hardness values in different moisture states relate to water absorption rates and elastic modulus. Thus, rocks with high absorption rates should be measured after drying. A 3D optical profilometer measured surface roughness, and the resulting Joint Roughness Coefficient (JRC) indicated that field polishing with a grinding machine meets test requirements. Considering size effects, cylindrical samples should have a minimum height of 25 mm and an optimal height-to-diameter ratio exceeding 2. A modified empirical formula is proposed. Rocks are catehorized into four hardness classes, and Leeb hardness is suggested as a supplementary parameter for comprehensive rock mass classification and engineering design.