ANDROLOGY LAB CORNER*: Validation of Sperm Counting Methods Using Limits of Agreement

A variety of methods exist for counting sperm. Since the introduction of semen analysis, one of these methods, the hemocytometer, has been regarded as the gold standard by andrology laboratories and the World Health Organization (WHO, 1999). The flexible features of this approach, involving fixation and immobilization of sperm, dilution of highly concentrated samples, and the counting of sperm in a single plane, contribute to the accuracy of the Improved Neubauer hemocytometer and its relative ease of use. This method is strongly accepted within andrology clinics and has been clinically validated by a number of studies (Dunphy et al, 1989; Tomlinson et al, 1996, 1999; Guzick et al, 2001).

As technology and techniques improve, manufacturers are continually trying to develop newer, simpler, quicker and more accurate methods for determining sperm concentration. Busy assisted reproduction technology (ART) laboratories in particular would find a quicker yet comparatively accurate method highly desirable, since sperm counting is an essential part of the semen preparation process. Although modern methods may be faster, since unfixed, undiluted semen is used, some labs find these analyses more difficult to use and believe that the counting of motile sperm may produce erroneous results. Furthermore, the WHO states that the newly introduced methods “are convenient in that they can be used without dilution of the specimen, but that they may lack the accuracy of the hemocytometer technique especially for highly viscous and/or heterogeneous specimens. If such chambers are to be used, their adequate accuracy and precision must be established by comparison with hemocytometers” (WHO, 1999).

In addition, it is now a requirement of laboratory accreditation systems that laboratories provide clinical validation for all methods used (ie, demonstrate that they are “fit for purpose”). Currently, the only sperm counting method with a considerable body of evidence to support and clincally justify its use is the hemocytometer (Mortimer, 1994; WHO, 1999).

When using the hemocytometer, the sperm number is calculated using a fixed volume of semen under the coverslip and counting the sperm in a single plane. A significant association between pregnancy and the sperm concentration measured has consistently been shown for this method (Dunphy et al, 1989; Tomlinson et al, 1996, 1999; Guzick et al, 2001). Thus, 64% of laboratories involved in the analysis of semen use this method routinely (Keel et al, 2000).

However, despite its validity as a method, the use of hemocytometry is thought by many to be inconvenient, in that the hemocytometer must be cleaned and assembled prior to each counting event and it involves the use of dilution techniques that can introduce errors, either due to poor technique or the viscous nature of the semen itself. Mathematical mistakes can occur when applying the correction factor to determine the eventual counts, and the recommended dilution method uses fixatives, such as formal saline, which are often a reason for rejection of the method by embryologists in IVF laboratories.

Comparisons of other counting chambers with the hemocytometer, particularly those marketed as easy-to-use 1-step methods have generally not been favorable. Makler sperm counts have been shown to be generally higher than the corresponding counts obtained with the hemocytometer (Coetzee and Menkveld, 2001; Sukcharoen et al, 1994). Indeed, Ginsburg and Armant (1990) have found the Makler chamber counts to be 62% higher than those obtained with the hemocytometer when using latex beads. Other methods, such as the Leja slide (Gynotec Malden, Nieuw-Vennep, The Netherlands) or the Microcell (Conception Technologies, San Diego, Calif) have been shown to produce significantly lower average sperm counts as compared to the hemocytometer. In particular, marked differences were seen at high concentrations (Tomlinson et al, 2001). A number of potential sources of error have been suggested to cause these discrepancies when using 1-step methods. First, the recommendation that motile sperm are counted, which may mean a single spermatozoon is counted more than once or not counted at all. Second and perhaps more significantly is the phenomenum that affects capillary-loaded chambers, such as the Leja and Microcell slides, which is known as the Segre-Silberberg (SS) effect (Segre and Silberberg, 1961). The SS effect results in high-gradient fluid flow in thin capillary-loaded slides, which results in the sperm suspension being forced transversely towards the walls, causing uneven cell dispersion throughout the chamber. New measures have recently been introduced to compensate for this phenomenon. A correction factor is used, which makes allowances for changes in sample viscosity and appears to improve the performance of there slides in terms of agreement with the hemocytometer (Douglas-Hamilton et al, 2005). Third, many of the studies that have compared the performances of various methods with the gold standard of the hemocytometer are problematic in terms of the choice of statistics. Unfortunately, many of these studies have focused on analysing differences across a range of sperm counts, whereas a proper and more detailed analysis is one that compares each and every individual count and measures the agreement between these 2 parameters. Therefore, in the present study, we employ limits of agreement (LoA) to analyse comparative data. This approach, which is based on graphical techniques and simple calculations, allows comparisons between new measurement techniques (ie, the Leja and Makler chambers) and an established method (ie, the hemocytometer). Thus, we can evaluate whether the methods agree sufficiently for the new method to replace the old method and we can decide if the differences between the 2 methods are sufficiently small for the methods to be used interchangeably (Bland and Altman, 1986).

The objectives of the present study were: 1) to determine whether the sperm counts obtained using the Leja slide and Makler chamber compare favorably with the counts obtained using the hemocytometer; 2) to determine whether the accuracy and reliability of sperm counts obtained using the Leja slide and the Makler chamber are improved by prior fixation and dilution of the specimen; 3) to determine whether the sperm counts obtained using the Leja slide compared more favorably with the counts obtained using the hemocytometer when correction is made for the SS effect; and 4) to determine by the use of LOA whether these methods can be considered to be interchangeable, thereby providing validation for their routine clinical application.

The aim of the present investigation was to determine whether three different methods used for analyzing sperm counts in a fertility laboratory setting could be considered to be interchangable. All 3 methods are currently in use in laboratories throughout the United Kingdom and worlwide.

The ICC, CV, and CI values show that all the counting methods have a particularly high degree of repeatability/reliability. The hemocytometer was the most reliable method, with the highest ICC, lowest CV, and smallest 95% CI. This finding partially supports the hemocytometer as the gold standard method, although its precision is clearly not high. The Leja was the next most reliable method, although it was more reliable without the fixation/dilution step, which suggests that this step may introduce a degree of error. The analyses showed the Makler chamber to have significantly poorer reliability than the hemocytometer or Leja. The ICC of the Makler chamber was much lower for both undiluted and diluted samples, and the CV and CI showed great variability between the repeated counts.

The LOA analysis showed that the Leja slide could be compared favourably with the hemocytometer, particularly for the 1:1 dilutions. However, despite showing that the Leja 1:1 and hemocytometer have a mean difference of zero, the standard deviation was calculated as 0.9. Thus, the LOA lie between −2.04 and 1.7, which means that in a worst case scenario, specimens that contain 5 × 10⁶/mL in the hemocytometer could have counts between zero and 16 × 10⁶ sperm/mL. Similarly, the Leja method could determine the concentration of a sample with 20 × 10⁶ sperm/mL to be between 6 × 10⁶ and 38 × 10⁶ sperm/mL. Although the latter form of analysis may at first glance appear to lack precision, the same principle can be applied to repeat counts using the same chamber. In other words, in most cases, the difference between the chambers is no worse than repeated measures using the same chamber. There will always be occasional outliers and perhaps more outliers when semen is the test fluid in question and this could be due to any one of a number of factors (eg, hyperviscosity, sampling error, mathematical error, diluting error or chamber flaws). When assessing the agreement and interchangeability of a method, it has to be put in the context of the level of expectation. As there is innate variability in hemocytometer counts using either repeated measures or between individuals, we will undoubtedly detect differences when comparing this with other chambers. In this context, the Leja slide could be viewed as being interchangeable with the hemocytometer, particularly if the sample is first fixed and diluted.

All of the results show very clearly that applying the SS factor increases the level of agreement between the Leja counts and hemocytometer counts, as expected. This improvement was more pronounced for the 1:1 diluted samples, for which the mean difference with the hemocytometer became 0 for the modified Leja counts. This is reassuring, since the Leja Instruction Manual states that the correction factors were initially calculated by calibrating the Leja with the Improved Neubauer chamber.

The Makler chamber showed extremely poor agreement with all the other methods and had a tendency to overestimate sperm counts. In the worst case, a sample of 20 × 10⁶/mL could be found have between 9.2 × 10⁶ and 58.9 × 10⁶ sperm/mL, which could have serious consequences for the diagnostic or treatment laboratory. This finding supports previous studies that have found that the Makler produces high sperm counts (Ginsburg and Armant, 1990; Sukcharoen et al, 1994; Seaman et al, 1996; Coetzee and Menkveld, 2001; Lu et al, 2004). Laboratories provided with a latex bead solution with a known, fixed concentration of 35 × 10⁶ sperm/mL determined the concentration on the Makler to be 53.5 × 10⁶/mL on average. A similar study conducted by Seaman et al (1996) found that the Makler chamber overestimated the known bead concentrations by as much as 50%.

The counts determined using the Leja slide were significantly different from the counts obtained using the hemocytometer (P < .05), unless they specimens were first fixed and diluted. The Leja median was significantly lower than that of the hemocytometer, and this finding was supported by the LOA analysis, which showed that the Leja on average underestimated the sperm count when undiluted semen was used. This underestimation was consistent throughout the data distribution and was not related to the sperm concentrations of the samples. It was expected that the Leja would underestimate the sperm count when compared to the hemocytometer, since a study by Tomlinson et al (2001) has found Leja counts to be significantly lower (P < .0001) than the corresponding hemocytometer readings, although these counts were performed using a previous version of the Leja chamber and without knowledge of the SS factor.

Fixing and diluting the samples for analysis by the Makler method reduced the mean difference and increased the level of agreement with the hemocytometer counts. However the counts produced in the Makler chamber were significantly higher than those in the hemocytometer, regardless of whether they were performed on undiluted or diluted specimens.

In some instances, laboratories are clearly willing to compromise on the accuracy of sperm parameter measurements in order to maximise speed and convenience. The preference for methods of sperm counting and the selection of counting chambers may in many cases reflect the type of clinical service and the level of expertise provided by a particular laboratory. It is known that many IVF labs favor the 1-step methods owing to their convenience, even though this could indirectly compromise practice, in that future treatment decisions based on sperm thresholds could be derived from erroneous data.

It has been suggested previously that using different methodologies to analyze sperm concentrations is the major cause of variation in sperm counts between laboratories (Auger et al, 2000). Owing to disagreements between laboratories, a patient could be classified as normal by one and as infertile by another (Neuwinger et al, 1990). This has been known for some time and is supported by the findings of the present study. Sperm concentration discrepancies between the Leja and hemocytometer affected the diagnosis of 21 patients in the present study. Such erroneous diagnoses could have serious consequences for patients, some of whom may be misidentified as infertile (according to the WHO criteria), which could in turn have negative psychosocial implications and mean that some individuals would not receive the appropriate assistance in achieving pregnancy.

We know from previous studies that many of the current methods in use carry a degree of error, which is high in comparison with many other types of diagnostic testing, many of which are fully automated. Simple factors, such as the inherent viscosity and heterogeneity of semen, incomplete mixing, errors in pipetting, mathematics, and transcribing, are all contributory. It is likely that the general lack of consensus and lack of strong evidence linking sperm concentration with either natural or assisted conception are due to either poor practice or the intrinsic inaccuracies of many of the methods used.

Another source of error, and possibly the most significant, is sampling error. By its very nature, 1 small microlitre aliquot of semen may have very different qualities when compared to another. Therefore, only by better mixing, increasing the volume of seminal fluid, and increasing the number of sperm counted per analysis can we hope to improve the current degrees of accuracy and precision. However, this has to balanced against whether there is a need for such a degree of accuracy and also whether it is a good use of resources.

The relatively high risk of error during the analysis, be it due to the method used, sampling error, transcription or a case of mistaken identity, raises the question as to whether a diagnosis should ever be based upon the findings of a single semen sample. Adding in other variables, such as whether the sample is complete, length of sexual abstinence, illness, stress, and medications, all of which are known to affect sperm count, ensures that the relationship between male fertility and sperm count is weaker than perhaps it should be. The unfortunate consequence of this is that semen analysis continues to demonstrate poor predictive value in terms of either natural or assisted conception (eg, IUI, IVF, and ICSI).

The hemocytometer method has long been the accepted standard for the assessment of sperm concentration. Indeed, the WHO recommends in its guide to semen analysis (1999) that a hemocytometer, such as the Improved Neubauer, should be used for sperm concentration measurements. Despite this, an estimated 64% of laboratories used the hemocytometer, while 26% used the Makler chamber, and the remainder used alternative disposable methods, such as the Leja slide (Keel et al, 2000). One-step disposable methods, such as the Leja slide, and re-usable glass chambers, such as the Makler, have been put forward as convenient alternatives. The present study suggests that there is limited agreement between all of these methods, with the widest discrepancy between hemocytometry and the Makler method.

If the standard Leja method was modified by fixing and diluting the sample, the overall agreement with hemocytometery was very good. Therefore, if it can be shown that the 2 methods are likely to give the same counts, the 2 methods could be said to be interchangble, providing a degree of validation for the Leja method. The hemocytometer is marginally better in terms of precison but, like all of the methods, has intrinsic weaknesses in terms of reliability and repeatability, which may be related to the test fluid in question.

In conclusion, there is little to choose between the hemocytometer and diluted Leja method in terms of average sperm count, which suggests that the Leja slide may be suitable for use in clinical pactice. In contrast, the Makler method showed very poor agreement throughout the entire data range and cannot be recommended. The limited agreement seen between all 3 methods and their apparent lack of precision is a cause for concern. If the WHO continues to recommend the hemocytometer as the gold standard method, then there should at least be a reappraisal of the method. Testing inaccuracies are often blamed on poor technique or technical training, yet it is clear that all methods for sperm counting have inherent weaknesses. Laboratories must be better equipped in order to provide users of their service (eg, Ob/Gyn clinicians). with proper information about the test, including its limitations and the expected level of accuracy and precision, in order to allow efficient management of their male patients.