A check that fails is useful. A check that passes when it should have failed is worse than no check at all, because you now have a reason to stop looking.
I found three of those in one fortnight, and they had nothing technically in common. What they shared was the shape: something answered yes to a question slightly different from the one I thought I was asking.
One: the build that only worked because of a cache
cargo tauri build had been fine for two releases. Then, from a clean checkout, it died:
mis-aligned LINKEDIT string poolon libsqlx_macros, at the moment sqlx does pub extern crate sqlx_macros.
The cause is a single line in Cargo.toml. strip = true under [profile.release] reads like an instruction about the binary you ship. It is not: it applies to everything built in that profile, including build scripts and procedural macros. Those are not shipped — they are dynamic libraries that the compiler itself loads while compiling your code — and a stripped proc-macro dylib cannot be loaded on macOS at all.
The fix is build-override, which turns stripping off for compiler plugins while leaving the app binary stripped. That last part I checked rather than assumed: zero debug symbols in both the executable and its library, after the change.
The interesting question is not what broke. It is why it broke then, on a setting that had been in the file for two successful releases.
The dylib was cached. Both of those releases linked against a copy produced before stripping ever reached it. The setting was wrong the whole time and invisible the whole time, and what finally exposed it was unrelated: enabling an image feature for the clipboard fix changed the dependency graph, which invalidated the cache, which caused sqlx's macro crate to be genuinely rebuilt for the first time since the flag went in.
I want to be precise about my own part in this, because it is the useful bit. When it first failed I argued the release profile could not be the cause, on the grounds that the profile had not changed. That was true and irrelevant. The profile was never the variable — the cache was. A green build is not evidence that a build setting is correct. Only a build from cold is, and I had not done one in months.
Two: the CI that ran nothing
I had two GitHub Actions workflows and a reasonable feeling of being covered.
Both fired only on a version tag or a manual dispatch. Neither ran a single test. A commit could land on the main branch having been checked by nothing whatsoever, and the first thing that would notice was the release script — which is late, and only fires on days you happen to be releasing.
That is a hard thing to see from the inside. The repository had workflows. The Actions tab had runs in it. The runs were mostly green. Nothing about the surface says "this fires four times a month and never on your code".
There is now a checks workflow that runs on every push and every pull request to main, and it runs one thing: the repository's own check script. Type checking, linting, the test suite, and the migration guard that the test suite starts with.
Calling the repo's own script rather than listing the steps in the workflow is the load-bearing choice. Any check spelled out in CI is a second copy of the truth, and the two drift — usually in the direction of CI running a subset of what you run locally, discovered on the day it matters. One script, two callers.
It deliberately builds nothing. A full desktop build needs a Rust toolchain and takes minutes; it has its own workflow and its own moment. What is wanted from a push is the answer to did that break anything inside a couple of minutes, which is a different product entirely.
Two and a half: the run that was always red
The other half of the same commit is the one I think about more.
The Windows build job ran on every version tag, and failed on every version tag — by design, because the code-signing secrets it needs are not set, because Windows is not on sale yet. Every release I had ever pushed produced a red run for a build nobody could buy. Both pushes of 1.0.0 included.
A CI failure that is always red is worse than having no CI, and not by a small margin. Red means nothing once it always means nothing, and the cost is not that one job — it is that you have trained yourself to close the tab. The next failure, the real one, arrives into an inbox where failure is background noise.
So the Windows job is manual-only until it can be green. The job itself is untouched and still correct; when Windows ships, the tag trigger goes back and the secrets get set. Turning a permanently-failing check off is not lowering the bar. Leaving it on is.
Three: the verification that read the wrong copy
The last one is my favourite, because it is a check whose failure mode is aborting a perfectly good release.
Publishing an update uploads the payload to object storage, verifies it, and then writes the manifest that points at it. The ordering matters: a manifest pointing at a payload that is not fully there is the one state that would hand a user a broken download. Verify sits between the two on purpose.
Re-publishing a version over itself failed with a signature mismatch. The upload was fine. The read-back was not: the storage API kept returning the previous release's bytes — through a delete-and-recreate as well — while the worker in front of the same bucket was already serving the new ones.
So the check aborted the publish in precisely the half-written state that its own ordering exists to prevent. A verification step that can fail spuriously is not a neutral cost. It has a blast radius, and here the radius was the thing it was protecting.
It now reads through the worker rather than the storage API, with a query string nothing has asked for before so no edge cache can answer it from a copy. That is also, on reflection, the stronger check. "What does the storage API say is stored" was never the question. "What would a user's updater actually receive" is, and it was a different question all along.
The common shape
None of these are exotic. Each is one component answering honestly, and me reading the answer as though it addressed the thing I cared about.
- The compiler said the build succeeded. It did — against a file produced under different settings.
- CI said green. It was — it had run nothing.
- The verification said the bytes matched. They did — the old ones.
The habit I have taken from it is to ask, of any check I am relying on, one question: what would have to be true for this to pass while the thing it protects is broken? Every one of these three had an obvious answer to that question, and I had never asked it, because a green tick is an answer-shaped object and it is genuinely hard to keep interrogating one.
The related discipline is to make a check fail on purpose once, in the state it is meant to catch. It costs ten minutes. Two of these three would not have survived it.