The free comment (FC) method enables the collection of insights on products based on consumers' natural language. The primary drawback is the need for extensive data pre-processing. This study compared the results of three data pre-processing techniques applied to FC data related to the perception of six madeleines by two panels of 100 consumers: manual pre-processing by four human experts, automated pre-processing by an expert system, and automated pre-processing by the large language model ChatGPT. Two modes of data collection were used: responses only with words or short expressions (“FC words”), or responses based on complete sentences (“FC sentences”). Various indicators (number of words extracted, number of concepts retained, pre-processing time, level of repeatability/discrimination/stability of findings) were computed and compared between data collection modes and pre-processing techniques. It was shown that the automated systems performed correctly with FC words; however, they were less effective at extracting relevant words from FC sentences. The findings from statistical analyses following automated pre-processing were less repeatable and discriminative compared to those from the most proficient human operators. It was also demonstrated that, beyond the overall differences between products, the pre-processing of FC data can be a major source of non-reproducibility in findings, depending on the operators and the level of detail they consider when extracting information. Finally, the advantages and disadvantages of each pre-processing technique were summarized, along with several recommendations for pre-processing and analysing FC data at the appropriate level of granularity to draw robust conclusions.