Kubebench: A Benchmarking Platform for ML Workloads

Abstract

Machine Learning (ML) workloads are becoming mainstream in the enterprise but the plethora of choices around ML toolkits and multi-cloud infrastructure make it difficult to compare their performance and costs. In this paper, we motivate the need for benchmarking ML systems in a consistent way, discuss the requirements of an ML benchmarking platform, and propose a design that satisfies the requirements. We present Kubebench, an example open-source implementation of an ML benchmarking platform based on Kubeflow, itself an open-source project for managing any ML stack on Kubernetes, a widely used container management platform.