Optimized Execution of a Numerical Weather Forecast Model in a Cloud Cluster

Abstract

This study proposes strategies to reduce the financial cost of using cloud clusters, through Amazon Web Services (AWS) ParallelCluster, to run the weather forecast model Brazilian developments on the Regional Atmospheric Modeling System (BRAMS). We developed an instance selection algorithm that obtains and compares the costs of various instance types in different regions and markets, recommending those with the lowest costs. If the suggested instance is a Spot instance and is revoked by the cloud provider, the proposed strategy resumes the application execution from a pre-recorded checkpoint by rescheduling it on On-Demand instances. This study also presents a detailed analysis of BRAMS execution across various instance architectures and proposes a novel three-queue architecture for managing BRAMS execution on On-Demand and Spot instances within AWS ParallelCluster. The results obtained from small and large spatial domains executed in AWS ParallelCluster using the proposed strategies show that adopting a cloud cluster is a promising alternative for this type of High-Performance Computing application, compared with execution on a supercomputer.