Load Balancer — quick learn

Load balancer is a scaling up solution that hides services group behind a black box, so that they pretend to be one service to clients.

[DNS round robin]

Simplest algorithm of load balancer to choose a backend service is DNS round robin. Load balancer side keeps a list of service IPs and rotate upon each request. The service whose IP is at top gets the request.

[L3/L4 load balancing]

With load balancing, client will send request to a virtual IP (VIP), this VIP is then re-written to real IP (RIP)of the service along with the request. Similarly, when data is sent back, RIP is rewritten to VIP in the data packet.

[L7 load balancing]

While in L3/L4 load balancing, load balancer re-writes packet in transport layer, in L7 load balancing, load balancer can read user cookie (identify user) and sends the request from same client to the same service. E.g. HAProxy

Pro of L7 compared to L3/L4: it establishes a 1:1 mapping between client and service, so only need to build a TCP connection (3 way handshake) once.

Con of L7 compared to L3/L4: it need to read cookies, increases the latency.

So L3/L4 is good for one visit per visitor case, L7 is good for multiple visits per visit case.

[Backup load balancer]

To prevent single point failure at load balancer side, we often need some backups. The back up load balancer keeps communicating with primary load balancer by sending health check requests (some simple API), if response indicates that the primary is down, backup load balancer will take over the primary position.

[Caching]

If visitor is likely to access same piece of data multiple times, universal cache is needed for L3/L4 or DNS round robin. Memcached would be a good choice

However if L7 load balancing is being used, we can take advantage of it to cache things in in-memory cache.

[Reverse/forward proxy]

Reverse proxy is usually the one does load balancing. To clarify, forward proxy is on behalf of the clients (server thought it’s talking with client while it’s actually talking to forward proxy), while reverse proxy is on behalf of the services (client thought it’s talking with servers while it’s actually talking with reverse proxy).