Building a Toy HTTP Load Balancer from Scratch in Go

I wanted to build a load balancer from scratch, mostly for fun, but also because load balancing is one of those infrastructure ideas that sounds simple until you actually try to make it work.

At the surface, a load balancer does one job:

Take incoming traffic and send it to one of several backend servers.

That is the friendly version. The real version has more texture. Which backend should get the request? What if one backend is down? What if the backend accepts the connection but never responds? How do you know what the balancer is doing? And how do you make the behavior visible enough to learn from?

This project answers those questions with a small Layer 7 HTTP load balancer written in Go. It is not meant to replace NGINX, HAProxy, Envoy, or a cloud load balancer. It is a learning project: compact, runnable, and honest about the core moving parts.

The finished project includes:

• an HTTP reverse proxy

• round-robin backend selection

• active health checks

• request timeouts

• limited retries

• per-backend runtime stats

• graceful shutdown

• demo backend servers

• an animated frontend that shows request flow live

Why HTTP First?

Load balancers usually sit at either Layer 4 or Layer 7.

Layer 4 load balancers work at the TCP level. They forward connections without understanding the application protocol inside those connections. That makes them powerful and general, but harder to explain visually when you are learning.

Layer 7 load balancers understand HTTP. They can inspect paths, headers, status codes, and methods. For a first build, HTTP is more approachable because you can test it with a browser, curl, and plain JSON endpoints.

So this project starts with an HTTP load balancer.

The client sends a request to the balancer:

client -> load balancer -> backend

The client does not need to know which backend actually handled the request. It only knows that it asked the load balancer for something and received a response.

That is the core idea of a reverse proxy.

The Shape of the Project

The repository is split into a few focused pieces:

cmd/loadbalancer      the main load balancer process

cmd/demo-backend     tiny backend servers for local testing

configs/demo.json    local demo configuration

internal/balancer    proxy, pool, health checks, stats, and UI

internal/config      JSON config loading and validation

The demo config tells the balancer where to listen and which backends to use:

{

  "listen_addr": ":8080",

  "backends": [

    "http://localhost:9001",

    "http://localhost:9002"

  ],

  "health_path": "/health",

  "health_interval": "2s",

  "request_timeout": "5s",

  "shutdown_timeout": "5s",

  "max_retries": 1,

  "max_replay_body_mib": 1

}

With that config, the load balancer listens on port 8080 and spreads traffic across two local demo servers on ports 9001 and 9002.

Backend Pool: The Heart of the Balancer

A load balancer needs a model of its upstream servers. In this project, each backend tracks:

• its URL

• whether it is healthy

• total request count

• failure count

• health check failure count

• in-flight request count

• last error

• last health check time

The backend pool owns a list of these backends and decides which one should receive the next request.

The first algorithm is round robin. That means requests cycle through healthy backends in order:

request 1 -> backend 1

request 2 -> backend 2

request 3 -> backend 1

request 4 -> backend 2

The important detail is that unhealthy backends are skipped. If backend 2 is down, traffic should continue to backend 1 instead of failing every other request.

That gives us the first real load balancing behavior:

healthy:   backend 1, backend 2

traffic:   1, 2, 1, 2

  

backend 2 fails

  

healthy:   backend 1

traffic:   1, 1, 1, 1

Simple, visible, useful.

Reverse Proxying with the Go Standard Library

Go makes the proxy part surprisingly approachable because the standard library includes net/http/httputil.

The balancer uses httputil.NewSingleHostReverseProxy to forward a request to the selected backend. Around that, the project adds the behavior a learning load balancer should expose:

• select a healthy backend from the pool

• increment request and in-flight counters

• set forwarding headers

• add response headers showing which backend handled the request

• record failures

• mark a backend unhealthy when proxying fails

The response includes headers like:

X-Backend: http://localhost:9001

X-Load-Balanced-By: scratch-go

Those headers are especially useful for demos because they let the frontend and curl output show which backend handled each request.

Health Checks: Keeping Bad Backends Out of Rotation

A load balancer should not blindly send traffic to a backend that is unavailable.

This project runs an active health checker. Every few seconds, it calls each backend's /health endpoint. If the backend returns a successful status code, it stays healthy. If the request fails or returns an error status, that backend is marked unhealthy.

That creates a feedback loop:

health checker -> backend /health

backend OK     -> keep in rotation

backend fails  -> remove from rotation

backend recovers -> add back to rotation

The demo backend includes a /toggle route, which makes this easy to test without killing processes:

curl http://localhost:9001/toggle

When a backend becomes unhealthy, the balancer's /stats endpoint reflects that state, and the animated UI turns that backend red.

Timeouts and Retries

Without timeouts, a proxy can get stuck waiting on a slow backend. That is bad for clients and bad for the balancer.

The project configures request timeouts through Go's HTTP transport:

• dial timeout

• TLS handshake timeout

• response header timeout

• idle connection timeout

It also supports limited retries. If a proxy attempt fails before a response is written, the balancer can try another healthy backend.

Retries are intentionally conservative. They are helpful for simple GET-like demos, but in real systems you have to think carefully about whether retrying a request is safe. Retrying a payment request, for example, is very different from retrying a request for a static page.

For this project, retries are there to show the concept without pretending the problem is finished.

Observability: The /stats Endpoint

One of the most useful parts of the project is the /stats endpoint.

It returns JSON like:

{

  "algorithm": "round_robin",

  "total": 2,

  "healthy": 2,

  "backends": [

    {

      "url": "http://localhost:9001",

      "healthy": true,

      "requests": 12,

      "failures": 0,

      "health_failures": 0,

      "in_flight": 0

    }

  ]

}

This matters because infrastructure is much easier to understand when it can explain itself.

The stats endpoint powers the frontend, but it is also useful directly:

curl http://localhost:8080/stats

You can watch request counts rise, backends leave rotation, and recovered backends rejoin the pool.

The Animated Frontend

After the backend worked, I added a small frontend served by the load balancer itself at:

http://localhost:8080/ui/

The frontend is not just decorative. It sends real requests through the load balancer and reads real backend state from /stats.

It shows:

• a client node

• a load balancer node

• backend cards

• animated request packets

• animated health check packets

• healthy and unhealthy backend states

• live request counters

• the active balancing algorithm

• the last routed backend

This turns the invisible behavior of a load balancer into something you can watch.

Clicking "Send" fires one request. Clicking "Burst" sends several. Leaving "Auto" on keeps traffic flowing so round robin becomes obvious.

When both backends are healthy, packets alternate between them. When one backend goes unhealthy, the animation skips it. That is the moment where load balancing stops being abstract.

Running It Locally

Start two demo backends:

go run ./cmd/demo-backend -addr :9001 -name backend-1

go run ./cmd/demo-backend -addr :9002 -name backend-2

Start the load balancer:

go run ./cmd/loadbalancer -config configs/demo.json

Then open:

http://localhost:8080/ui/

Or test with curl:

curl http://localhost:8080/hello

curl http://localhost:8080/hello

curl http://localhost:8080/stats

What This Project Teaches

The fun part of building a load balancer is that every feature points to a real systems idea.

Round robin teaches scheduling.

Health checks teach failure detection.

Timeouts teach resource protection.

Retries teach fault tolerance and the danger of repeated side effects.

Stats teach observability.

Graceful shutdown teaches lifecycle management.

The animated frontend teaches that tooling matters. If you can see a system move, you can understand it faster.

What I Would Add Next

This version is intentionally small, but it creates a good base for deeper experiments.

Good next steps would be:

• weighted round robin

• least-connections routing

• passive health checks with failure thresholds

• sticky sessions

• TLS termination

• config reloads

• richer metrics

• raw TCP Layer 4 forwarding

Each one opens a different door into how production load balancers work.

Final Thought

The best way to understand infrastructure is often to build a tiny version of it yourself.

This load balancer is small enough to read, but real enough to behave like the thing it is teaching. It accepts requests, chooses healthy backends, proxies traffic, watches for failures, exposes stats, and shows the whole flow in an animated UI.

That is a pretty satisfying result for a "just for fun" project.