The sulfur (S) mass fraction of carbonate minerals can be used to reconstruct the environmental conditions and S sources at the time of precipitation. As S is present in a wide range of host materials, there is an urgent need to develop a method to extract S from a single mineral phase. We have developed a method to extract structurally substituted sulfate, termed carbonate-associated sulfate (CAS), from geological and biogenic carbonate minerals using weakly acidic cation exchange resins. Two types of weakly acidic cation exchange resin (methacrylic acid and acrylic acid types) were tested to minimise the blank S contents and decompose carbonate. After repeated cleaning of the resins with high-purity water or HCl, the blank S contents were reduced to < 0.1 μg, which is < 0.1% of the CAS in the samples. The cleaned resin was used to dissolve 10 and 25 mg of the JCp-1 carbonate certified reference material (CRM; Japanese National Institute of Advanced Industrial Science and Technology, AIST). Samples and resin were added to 8 ml of high-purity water at resin/sample ratios of 2, 5, 10 and 20, set on a shaking table, and reacted. The supernatant solutions were sampled sequentially from 0.5 h to 87 h after the start of experiments. The results show that the optimal conditions for decomposing 10 mg of carbonate is a resin/sample ratio of ≥ 10 with a reaction time of ≥ 40 h. Carbonate-associated S mass fractions were measured for six geological and biogenic carbonate CRMs. The coefficient of variation in carbonate-associated S mass fractions was ≤ 7%, regardless of the type of resins used. The mass fractions determined with this method recover 74–94% of the total S mass fractions reported in previous studies, suggesting that this method dissolved carbonate, and did not leach other S-bearing fractions that are not resistant to weak acids. Another benefit of this method is that the decomposed solution can be introduced directly into the ion chromatograph, allowing for more sensitive analyses. We emphasise that this method can also be used for S isotopic measurements, as S contamination from other S-bearing mineral phases is low.