A missing check looks exactly like a passing one

Zero checks failed. Zero checks ran. Your dashboard shows the same tick for both.

You have this somewhere. I would bet on it.

Go and look at the thing you trust most. The pipeline that says the build is fine. The monitor that says the site is up. The nightly job that says the backup completed. Now ask it a slightly different question from the one it answers. Not “did anything fail?” but “did anything actually happen?”

Almost none of them answer the second one. They report on the checks that existed. If no check existed, there is nothing to report, and nothing to report gets rendered as nothing wrong.

An empty result set is the most agreeable thing in computing. It agrees with every question you ask it.

What is a green tick actually telling you?

It is telling you that everything which ran, passed. That is a genuinely useful fact and it is not the fact you think you are reading.

The gap between those two sentences is where automation quietly goes wrong. A human reading a dashboard fills the gap in without noticing, because a human has a mental model of what should be there. Software has no such model. It has a list, and the list is empty, and empty lists contain no failures.

This is not a bug in any particular tool. It is a property of asking “any problems?” instead of “any proof?”. Absence is the one answer that satisfies both a healthy system and a system that never woke up.

Here is where I walked into it on 27/08/2026.

I run an automated sweep that merges pull requests once they are clean. Its safety gate was one command: gh pr checks. Green means merge.

A pull request came up clean. Eighty-two files, four thousand seven hundred and sixty-five added lines, five green ticks, mergeable. The sweep was going to squash-merge it.

Every one of those five ticks was an external service. Four deployment previews, one security scanner. Not one of them runs a test. The actual test suite, the thing whose entire job is to say whether the change works, had never run against that branch at all. total_count: 0.

The gate was working perfectly. Five out of five checks green. It is just that the checks that mattered were not among the five, and the gate had no way to notice, because it was counting rather than expecting.

Why did the tests never run?

This is the part I find genuinely funny, in the way a thing is funny when it happens to someone else.

The workflow file said this:

on:
  pull_request:
    branches: [main]

Read that as a sentence. It reads as “gate what reaches main”. Sensible. Obviously correct. That is what everyone thinks it says, including me, for months.

It does not say that. It says: run this only when the pull request’s base branch is main. Any pull request stacked on top of another branch is silently skipped. Not failed. Not warned about. Skipped, with no artefact anywhere to say so.

So the branch had never been tested, and no amount of pushing to it would have changed that. The failure had a shape I had ruled out without checking, which is the only way to rule anything out badly.

Config that reads like a statement of intent, but is actually a narrow technical filter, is worth hunting for in your own setup. It is a whole genre. A cron entry pointing at a path that was renamed. A monitor watching a hostname that stopped resolving. A test file excluded by a glob that someone tightened in 2024. All of them go quiet, and quiet is indistinguishable from fine.

How do you check whether the check ran?

You add a second assertion, and you keep it stupid.

The script I wrote is about forty lines and answers exactly one question: does a run exist for this commit? Not whether it passed. Passing is already covered, and a check that judges the same thing twice will eventually disagree with itself, at which point you have two answers and no truth.

The part worth stealing is the exit codes:

• 0 - a run exists
• 1 - zero runs, this was never tested
• 2 - I could not find out

Three states, not two. That third one is the whole point.

“This is broken” is a finding about the thing you are checking. “I could not tell” is a finding about the checker. They demand completely different responses, and if you collapse them into one non-zero exit, you get an outage report every time your network hiccups, or worse, you fold uncertainty into success and you are back where you started.

Give not-knowing its own answer. Systems that cannot say “I don’t know” will always say something else instead, and it will usually be the reassuring thing.

Then I tested the tester and found the test was wrong, because of course it was. My “what if the tool is missing?” case built a fake environment with a stripped path, and the tool turned out to live somewhere my strip did not cover. The test found it, called the real service, and passed for entirely the wrong reason. On my machine. It failed on the server, where the tool sits elsewhere.

A test that passes for the wrong reason is the same defect as the merge gate, one level up. It counted a green result instead of expecting a specific one. I got to make the mistake twice in one afternoon, in two different fonts.

The one thing to take away

Every automated system you own has a question it answers and a question you think it answers. Write both down for one of them today. The gap is where your surprises live.

Then, for the check you rely on most, add the boring second assertion. Not a smarter check. A separate one, that only ever answers whether the first one fired.

For what it is worth, the branch did eventually get its first test run. It went red immediately, on a real bug, one nobody had seen because nothing had ever looked.

That is the argument for all of this in a single example. The test existed the whole time. It had simply never been asked.