Efficient Microservice Autoscaling through muOpt

Microservices have become the architecture of choice for cloud applications requiring high performance and scalability. Autoscaling, which dynamically adjusts resource allocation based on workload fluctuations, is key to optimizing performance and controlling costs. This paper presents μOpt, a computationally efficient, model-based autoscaler specifically designed for microservices. μOpt leverages a nonlinear optimization problem tied to a fluid approximation of a layered queuing network (LQN) model to determine optimal configurations that maximize key performance metrics—such as throughput, CPU usage, and response time—while minimizing operational costs. On a well-known benchmark application, our numerical experiments show that μOpt achieves fast solution times, enabling responsiveness to dynamic workloads. Compared to a state-of-the-art LQN-based autoscaler employing genetic algorithms, μOpt delivers improved application performance using fewer resources. To demonstrate the robustness and generalizability of our underlying model, we validate its prediction accuracy across ten randomly generated applications with diverse architectures, showing that its performance is a reliable foundation for autoscaling. Finally, it also outperforms Horizontal Pod Autoscaler, a production-ready solution for Kubernetes deployments in Google Cloud Platform, consistently reducing resource usage while more accurately tracking CPU utilization targets across both synthetic and real-world workloads.