Nginx load balancing decisions

Thu, 18 May 2017 00:00:00 +0000

Nginx as a reverse proxy and load balancer is well-documented. The configuration syntax is not the hard part. The decisions are.

Algorithm selection

Three algorithms cover most workloads.

Round-robin (the default). Requests cycle through backends sequentially. Weights let you bias toward higher-capacity servers. Simple, predictable, works well when request processing times are uniform.

upstream api {
 server api-01:8080 weight=3;
 server api-02:8080 weight=2;
 server api-03:8080 weight=1 backup;
 keepalive 32;
}

The backup directive keeps a server in reserve – it only receives traffic when all non-backup servers are down. Useful for a smaller instance that can keep the service alive during a partial outage but shouldn’t take production load normally.

Switching to Go

Wed, 07 Dec 2016 00:00:00 +0000

I’d been building production services with Node.js since 2013. Custom blog engines, API services, real-time backends. The ecosystem was rich, iteration was fast, and for moderate-load services, it worked well.

The limits showed up at scale. A latency-sensitive payment processing API exposed three problems that were manageable at lower traffic but compounded at higher volumes.

GC pauses. Node’s garbage collector introduced latency spikes that were unpredictable and hard to tune. For payment processing, where consistent response times matter more than peak throughput, this was the biggest concern. A p99 spike during a GC pause means a user staring at a spinner while their payment hangs.

Performance on vnykmshr

Nginx load balancing decisions

Algorithm selection

Switching to Go