Bringing Serverless Computing to the HPC Community

Abstract

The next wave of cloud computing - the serverless computing model - is enjoying adoption at scale by different cloud computing vendors. The serverless computing model is already rapidly accelerating the development and deployment of enterprise applications. Unfortunately, the HPC community appears to be left behind in the revolution. The widespread assumption or opinion is that the serverless computing model is not beneficial for the HPC community because the serverless computing model was not primarily designed for scientific computing and users. In this talk, I promise to do three things. First, I'll show why existing popular opinion about HPC and serverless is pre-mature and inaccurate. Second, I'll demonstrate how HPC users and programmers can leverage the serverless computing model for mitigating some of the long-standing challenges and causes of frustration in HPC resource management. In the spirit of transparency, I admit that the path to effectively leveraging serverless for traditional HPC is challenging and has long stretches of darkness. I will identify and discuss some lamp posts on this not-so-well-lit path. Finally, I'll share my perspective on what new lampposts we should build as a community and where we should put them.