Shrinking Docker Images Without Losing Your Mind

Every codebase has one area everyone is a little afraid to touch. On our team, it was everything related to autoscaling.

Most of the complexity was accidental. Over the years we'd layered three different approaches to autoscaling on top of each other, and each one made sense at the time.

How we got here

The first symptom was subtle. Nothing failed outright, but autoscaling kept drifting in the wrong direction, and every fix seemed to make the next one harder.

The setup was ordinary: Docker at the edges, Terraform doing most of the heavy lifting, and a handful of scripts nobody wanted to own. It worked well until usage roughly doubled over a single quarter.

The fix

The key insight was to treat autoscaling as part of the interface rather than an implementation detail. Once we did that, most of the other decisions followed naturally.

The core of it looks like this:

FROM golang:1.23 AS build
WORKDIR /src
COPY . .
RUN CGO_ENABLED=0 go build -o /app ./cmd/server

FROM gcr.io/distroless/static
COPY --from=build /app /app
ENTRYPOINT ["/app"]

This alone let us delete a surprising amount of code elsewhere in the project.

A few things that helped along the way:

Make the default the safe choice, not just the documented one.
Keep the old path around until the new one has survived a real incident.
Measure before and after, even if it's just a rough number in a spreadsheet.
Pair on the first few changes so the knowledge doesn't live in one head.

Caveats

We still haven't solved incident response. That's a post for another day, and honestly, I'm not sure we'll get it right the first time either.

There are cases where this falls apart. If your team is very small or your traffic is very spiky, the extra moving parts may not pay for themselves.

Closing thoughts

That's it for now. Next time I'll dig into infrastructure as code, which turned out to be closely related.