Аdaptive load balancing algorithms for microservices architectures under stochastic burst traffic

Abstract

Abstract. This paper examines the problem of ensuring fault tolerance and minimizing latency in microservices architectures exposed to stochastic burst traffic. It is demonstrated that traditional deterministic load balancing algorithms (Round Robin, Least Connections) and reactive orchestration mechanisms exhibit critical inertia, resulting in the “thundering herd” problem and cascading failures under non-linear loads. To address this problem, Serhii Klymenko developed the proactive Predictive-Adaptive Load Balancer (PALB) algorithm. The method is founded upon a hybrid architecture combining short-term time series forecasting (Holt`s double exponential smoothing method) and probabilistic routing based on reinforcement learning (Contextual Multi-Armed Bandits, LinUCB). For the first time, the author introduces the “Ghost Queue Estimation” metric, which compensates for feedback latency in distributed systems. Experimental simulation results of a 10-fold load impulse burst indicated that PALB ensures a reduction in tail latency (\mathrm{P}_{\mathrm{99}}) from 4.5 s to 650 ms and maintains service availability at 99.98% compared to 96.5% for static counterparts. It is proven that the author`s approach enables a 40% increase in Goodput and obviates the need for economically inefficient resource over-provisioning.