Protagoras: A Service for Tagging E-Commerce Products at Scale

Abstract

Despite widespread adoption of machine learning to solve real world problems, the implementation of ML solutions in production environment is more complicated than it seems. It is quite straightforward to write machine learning codes these days but they are not designed to be deployed in production scale where millions of requests per day is a norm. In this paper, we describe our implementation of a ML service for large scale product tagging in e-commerce called Protagoras. The problem of tagging products can be seen as multi-label classification where the labels are product tags. By performing the classification within each product category, the precision can be increased and the inference can be performed faster. Protagoras combined the scalability and speed of microservice implementation in Golang and robust machine learning implementation in Python. We present the architecture of the system with all its components including API endpoints, job queue, database, and monitoring. The benchmark shows that, even with 1000 classifiers in one category, the average latency for online inference is below 300 millisecond. The throughput can be further maximized by replicating the service into multiple servers.