LLMs in e-commerce: A comparative analysis of GPT and LLaMA models in product review evaluation

Abstract

E-commerce has witnessed remarkable growth, especially following the easing of COVID-19 restrictions. Many people, who were initially hesitant about online shopping, have now embraced it, while existing online shoppers increasingly prefer the convenience of e-commerce. This surge in e-commerce has prompted the implementation of automated customer service processes, incorporating innovations such as chatbots and AI-driven sales. Despite this growth, customer satisfaction remains vital for E-commerce sustainability. Data scientists have made progress in utilizing machine learning to assess satisfaction levels but struggled to understand emotions within product reviews’ context. The recent AI revolution, marked by the release of powerful Large Language Models (LLMs) to the public, has brought us closer than ever before to understanding customer sentiment. This study aims to illustrate the effectiveness of LLMs by conducting a comparative analysis of two cutting-edge LLMs, GPT-3.5 and LLaMA-2, along with two additional Natural Language Process (NLP) models, BERT and RoBERTa. We evaluate the performance of these models before and after fine-tuning them specifically for product review sentiment analysis. The primary objective of this research is to determine if these specific LLMs, could contribute to understanding customer satisfaction within the context of an e-commerce environment. By comparing the effectiveness of these models, we aim to uncover insights into the potential impact of LLMs on customer satisfaction analysis and enhance our understanding of their capabilities in this particular context.