2.3 Load balancing and autoscaling

The same thing, three names

One idea, sold under a different brand by each provider. Learn it once and you can read any job advertisement.

On AWS

ELB/ALB + EC2 Auto Scaling

On Azure

Load Balancer / Application Gateway + VM Scale Sets

On Google Cloud

Cloud Load Balancing + Managed Instance Groups

Work through these

• Layer 4 vs layer 7 load balancing; health checks

  A load balancer working at the transport layer only sees connections; one working at the application layer can read paths and headers and route on them. Health checks are what stop either from sending users to a broken machine.

• Horizontal vs vertical scaling and their limits

  Scaling out adds machines, scaling up makes one bigger, and each runs into a different wall. Knowing which wall you are approaching decides which lever is available.

• Scaling policies: target tracking, scheduled, step

  Scaling policies differ in what triggers them: holding a metric at a target, a timetable you set in advance, or fixed steps as load crosses thresholds. Choosing the wrong one produces either a slow response or a flapping fleet.

• Why scaling out fixes some outages and hides others

  Adding machines fixes shortages of capacity and hides shortages of correctness, because a failing dependency stays failing at any scale. Telling those two situations apart is the point of this item.