On Tueday I wrote about a Doritos bag mistaken for a gun, and a stolen-vehicle alert that led to the wrong family being held at gunpoint.
Both different. Both the same failure.
A machine produced a signal. That signal became an action. Somewhere in between, the part that should have decided whether the signal was enough was missing.
Today, I want to show you where that missing part sits.
Every AI system that touches a real-world decision has three layers. And most organisations build only two.
Layer 1: Data/Grounding Layer
This is the evidence.
In Aurora: a licence-plate number, a registration state, a vehicle description. In a claims workflow: photographs, weather data, repair estimates. In a control room: sensor readings, forecasts, asset condition.
These are facts, signals, and observations. They are the decision’s raw materials.
Layer 2: AI Layer
AI examines the evidence and returns an assessment:
“This appears to be a match.” “This claim may be fraudulent.” “This circuit may be at elevated risk.” “Here is my confidence level.”
That can be extremely useful.
But an assessment is not a decision.
Nobody would allow a detective to handcuff someone purely because they had a hunch. The detective gathers evidence and forms a view. Another process decides what that view is allowed to authorise. And in too many AI deployments, that process does not exist.
The model’s conclusion quietly becomes the action because nobody designed a place for it to go in between.
There is a missing layer, Layer 3 or the decision layer.
The courtroom
I like to fall back on analogies to help colour the picture. You can think of the decision layer as the courtroom process between an AI assessment and a real-world action.
That layer has three parts:
1. The spine: the law
Think of this as the law. Defined (designed) in advance. In the case of the decision process it defines:
- What evidence is sufficient for a particular action.
- Which discrepancies must stop or slow the process.
- Which decisions can be automated.
- Which require verification.
- Which always require accountable human judgment, regardless of a model’s confidence.
- Who has standing to override, escalate, or halt the process.
In Aurora, the spine would have specified that a plate match alone could never authorise a high-consequence stop — any mismatch in vehicle type or state required a person to verify it directly, and no action could proceed until they had.
The law is not written while the case is unfolding. It is written beforehand. In the case of building the decision spine, this is done by people who understand the workflow, its failure modes, and the consequence of being wrong.
2. The guardrail: the judge
Once the spine exists, a guardrail or the judge using our court room analogy, enforces it.
It checks a proposed action against the boundary already defined: allow, block, route for more evidence, or escalate to an authorised person.
The judge does not decide whether the law is wise. The judge holds it, consistently, case after case.
This is where much of the excellent engineering in AI safety, policy enforcement, and workflow orchestration belongs. But a judge can only enforce a law someone has actually written.
3. The automation: the bailiff
Only then comes automation.
The bailiff carries out an action that has already been authorised: send the customer notification, route the claim, log the event, dispatch the work order, or trigger an approved response.
It is safe to move quickly precisely because the judgment about when it may act was made upstream.
Where systems break
Aurora and Baltimore did not lack data. Nor did they lack technology capable of detecting a plate match or flagging a possible weapon.
They lacked a decision spine.
No one had clearly defined what a single match, a conflicting signal, or an uncertain classification was allowed to authorise. The result was predictable: a model output travelled directly from detection to action, with a person nominally present but no meaningful decision process for them to apply.
A courtroom with evidence and enforcement but no law is not a courtroom.
The work before AI goes live
This is not primarily a model problem.
The model can improve. The data can improve. The guardrail can become more sophisticated.
But none of those answers the first question:
Which decisions should never be closed by an AI’s confidence alone, however strong the case appears?
That question has to be designed into the operating process before the system goes live—not discovered in front of a customer, an operator, a regulator, or a jury.
Follow-Up
If you currently have an AI workflow in production or pilot, ask your team one question: “What is the system explicitly authorised to do below a certain confidence score?”
If you can’t find that answer in writing within 10 minutes, your boundary isn’t set.
If you’d like to stress-test that workflow before an edge case does it for you, let’s run a 30-minute boundary review: mattsheehan@spatialnext.io