The CI bill was not a capacity problem

Sixteen pull requests went up in a burst, mostly generated. Every one came back red.

The annotation said the jobs had not started: account payments, spending limit. The obvious conclusion was that we had outgrown the free tier and needed to pay for more minutes.

That conclusion was wrong, and expensive to act on.

Read the failure before believing the label

The jobs had run for roughly zero minutes. That is the detail that matters. This was not a lint failure or a broken test. Nothing executed at all.

A test that fails and a job that never starts look similar in a pull request list — red is red. They have nothing in common as problems. One is about your code and one is about your account, and only the second one has a billing answer.

So the question changed from “how much capacity do we need” to “what consumed what we had”.

Three workflows, twice each

Three of the workflows ran the full heavy suite on both the push trigger and the pull request trigger. Every push to a feature branch with an open PR ran everything twice.

That is a 2x multiplier on the most expensive jobs in the repo, invisible unless you count runs rather than reading the config.

The burst made it visible. Sixteen PRs, roughly five workflows each, several doubled, plus a collection loop that re-triggered CI every time it resynced a branch. The volume did not create the problem. It just moved it above the threshold where somebody noticed.

The fix was small: cancel-in-progress concurrency on all four PR gates, and kill the double trigger so push and pull_request stop overlapping.

The advice I turned down

The standard recommendation at this point is to move heavy jobs onto a self-hosted runner. Your own box, no metered minutes, done.

I moved most of them. I refused for one, and the reason is worth stating.

One of those workflows is an isolation gate — it checks that the sellable engine contains no facts belonging to any specific tenant. It is the thing that stops a private detail leaking into a public repository.

A gate that verifies a box must not run on that box. The moment it does, a compromise or a misconfiguration of the environment takes the guard down with it, and you cannot tell the difference between “the check passed” and “the check could not fail”. Independence is the entire value of that job. Free minutes are not worth it.

Everything else moved. That one stayed where it could not be influenced by what it was guarding.

Then the better idea: stop using CI for most of it

Deduplicating runs was the small win. The bigger one was asking why the checks needed a remote runner at all.

Most of them do not. A test suite, a linter and a grep-based gate all run perfectly well on the machine that just wrote the code, and they run sooner — before the push, not after.

So gating moved local. The server suite went into a pre-push hook. Lint, isolation and review workflows went dormant, still invocable on demand. The health check dropped to a daily scheduled backstop.

Actions usage fell to near zero. No box acting as a runner. And it stays that way for anyone who forks the repository, which a self-hosted runner does not.

I verified the hook actually blocks rather than just runs: made a deliberate violation, confirmed the push was refused, then confirmed the dormant workflows triggered nothing.

The caveat, which bit later

A pre-push hook is bypassable. --no-verify and the gate is gone.

I knew that when I made the change and accepted it, because the alternative was paying for runs that mostly duplicated each other.

It cost me later. A suite of several hundred tests sat with no enforcement in CI at all for weeks, because local-first plus a bypass flag means the only thing standing between a broken commit and main is whether the person pushing was in a hurry. Production froze twice on a related class of problem before that got fixed properly.

Both things are true: the amplification diagnosis was right, and local-first was the right call for the cost. It was also a weaker guarantee than what it replaced, and I did not write that down anywhere at the time. If I had, the gap would have been obvious months earlier.

What generalises

Count runs, not minutes. The bill tells you how much you used. The run history tells you why, and the answer is usually duplication rather than need.

A job that never started is not a test failure. Read what actually happened before accepting the label on the tin.

Ask what a gate is guarding before you move it. Most jobs are fine on your own hardware. The ones verifying an environment’s integrity are not, and that distinction is easy to lose when you are optimising uniformly.

Write down what a trade-off costs you at the time you make it. Not to be diligent. Because the cost arrives months later, to somebody who was not in the room, and by then the reasoning is gone.

Related

• Claude Code orchestration - verification as part of the work rather than after it.
• Approval gates - why a control in the system beats a control in an instruction.