Why Your Default Branch and Your LLM Are the Same Architecture

On 18 November 2025, Cloudflare's bot detection lost its input, and the two generations of proxy running side by side did different wrong things with it.

That incident is about a bot classifier, not a language model. I am using it anyway, because it is the cleanest illustration I know of a question people keep asking about LLMs: when do you replace deterministic code with a model, and what happens to the inputs neither one can decide?

A permissions change in a ClickHouse cluster at 11:05 UTC caused a Bot Management feature file to grow past a limit the proxy enforced at runtime. The feature count went from about 60 to more than 200, and the limit was 200. The file propagated.

The newer Rust proxy, FL2, hit the limit and panicked: called Result::unwrap() on an Err value. The thread died and the request returned a 5xx. Part of the internet went down for 5 hours and 46 minutes.

The older proxy did something else. In Cloudflare's own words, bot scores "were not generated correctly, resulting in all traffic receiving a bot score of zero." Every request got a number. And because a bot score of zero means "definitely automated," customers who had rules set to block bots "would have seen large numbers of false positives."

Same root cause, same missing input, two failures that look nothing alike. One crashed. The other computed a value it had no basis for and handed it downstream, where rules that could not tell the difference acted on it.

Almost every engineer reading that will file the crash under "bug" and the score of zero under "at least it stayed up." I want to argue that the second one is worse, and then argue something less obvious: neither proxy did the right thing, and the right thing is a third behaviour that was not built.

That third behaviour is to say "I cannot score this request" in a way the next component can read. Keep the last known good file, or emit a sentinel that downstream rules can branch on. FL2 did not refuse - it died, which is a different thing, and it is exactly what the parser further down this article is written to avoid. What both proxies lacked was a way to report "no answer available" as a value rather than as a crash or a zero.

That missing third behaviour is the subject here, and I am going to argue it marks the bottom of the architecture ladder better than the absence of a model does.

Why a path that always answers is not deterministic code

Part 1 of this series placed Level 0, deterministic code, at the bottom of an eight-rung ladder and gave the rule for climbing it: start at the floor and move up only when the current level fails. It defined Level 0 the way the field defines it. No model. Just code.

That definition is not load-bearing. Here is the one I want to defend:

Level 0 is not defined by the absence of a model. It is defined by the presence of an explicit refusal path, or by a domain you have closed and can show your working for - which is option 4 below. Closure is demonstrated the same way everything else here is: run with the refusal path in place until the refusal rate is zero by reason. Option 4 is option 3 that graduated, not option 3 skipped. A default branch and a model call in that same fallback position are the same architecture: both invent a value where no rule applied, and both hand the caller something it cannot distinguish from a decided answer.

Two consequences, and the first is uncomfortable. A rules-only router with a catch-all is not on the floor, whatever your diagram says. And a system with a model in it can be on the floor for the slice of traffic it fully specifies, provided it declares which slice that is.

A wider version of this claim is false, and I do not want to be read as making it. Part 1 named LLM-washing - a model on a path that was already fully specified. That is a real and separate defect: you pay tokens and latency for an answer ordinary code could compute. But a model on a fully specified path never meets an input where no rule applies, so it is not the failure this article is about. My claim concerns the fallback position specifically.

What matters here is coverage: the share of inputs the path chose to answer at all, as distinct from accuracy on the ones it answered. A system that always answers has coverage of 1.0 by construction. You could read that number off the source before a single request arrives, which is what makes it useless.

When to use an LLM instead of deterministic code: the missing criterion

The advice to build the simplest thing is not missing from the field. It is the field's most repeated advice, and it comes from the top.

Anthropic's Building Effective AI Agents, published in December 2024, is the canonical statement. It recommends "finding the simplest solution possible, and only increasing complexity when needed," and it goes further than most readers remember: "This might mean not building agentic systems at all." It also observes that "for many applications, optimizing single LLM calls with retrieval and in-context examples is usually enough."

I went back through that document looking for the criterion. It is not there. There is no accuracy threshold, no latency target, no cost ratio, and no coverage number that tells you the simple thing has stopped working. The guidance is to add complexity "only when simpler solutions fall short" and "only when it demonstrably improves outcomes." Both are correct. Neither one can be put on a dashboard.

That is the gap I want to close. The industry has excellent advice about direction and no instrument for position. The Anthropic guidance tells you which direction to move. It does not give you a number that says where you are standing.

The closest thing to a procedure I found is Cameron Palmer's "Should you even use an LLM?", published in July 2026, whose first question is whether the workflow can be completely specified in advance. That is the right question, and he asked it before I did. My disagreement is narrow, and it changes what you build. Palmer asks specifiability as a judgement the engineer makes at design time. I am arguing it has to be a measurement, produced at runtime by the deterministic implementation itself, on real traffic. Design-time judgement about specifiability is exactly the judgement Cloudflare's proxy authors made about a feature file that had never come close to its limit.

The wrong way: a rules router with a default branch

Here is the shape of router that turns up whenever a team correctly decides a problem does not need a model. It maps incident reports to an on-call rota. Substring match on the service name, a keyword scan for urgency, and a fallback for everything else.

ROTAS = {    "checkout-api": "payments-oncall",    "payment-gateway": "payments-oncall",    "search-api": "search-oncall",}URGENT_WORDS = ("5xx", "down", "outage", "fire")def route_total(report: str) -> tuple[str, str]:    """Map an incident report to (rota, severity). Always succeeds."""    lowered = report.lower()    severity = "SEV1" if any(word in lowered for word in URGENT_WORDS) else "SEV3"    for service, rota in ROTAS.items():        if service in report:            return rota, severity    return "platform-oncall", severity  # the default branch

Run it against six incident reports and compare each answer to what the incident turned out to be. I built these six to exercise the failure shapes this article is about, so read them as a worked example and not as a sample of anything:

report                                 routed to        sev   verdict------------------------------------------------------------------------------alert checkout-api http_5xx_rate=0.40  payments-oncall  SEV1  okalert search-api   http_5xx_rate=0.05  search-oncall    SEV1  WRONG (SEV3)alert cdn-edge     http_5xx_rate=0.42  platform-oncall  SEV1  WRONG (unowned)prose "everything is on fire..."       platform-oncall  SEV1  WRONG (payments)prose "card charged... no alert fired" platform-oncall  SEV1  WRONG (payments SEV2)prose "chekout-api returning errors"   platform-oncall  SEV3  WRONG (payments SEV1)answered 6/6    correct 1/6    flagged 0

One correct out of six. Flagged zero means the function never signalled low confidence on anything, so every wrong result is indistinguishable from the right one at the call site.

A reader should object here, and the objection is fatal to the comparison I was about to draw. This function has ordinary bugs, not just structural ones. The metric name http_5xx_rate contains the substring 5xx, which is in the urgency list, so a routine reindex at 0.05 becomes a SEV1 page at four in the morning. The word fired contains fire. if service in report is case-sensitive while the severity scan runs on lowered. And chekout-api is a typo the rules cannot reach, which under-pages a real SEV1 as a SEV3.

Those are specification bugs. Fixing them has nothing to do with refusal. If I compared this function against the honest version below, I would be crediting the refusal path with an improvement that came from writing better rules. That is a confound, and removing it is what the next section is for.

The ablation: what the refusal path actually buys

Here is the same router with the specification bugs fixed. It parses the alert grammar properly, looks ownership up in a table, and uses a per-metric severity line. It keeps the default branch, unchanged.

def route_total_fair(report: str) -> tuple[str, str]:    """Same tables as the honest router below. Default branch intact."""    alert = parse_alert(report)          # both defined in the next section    if alert is not None:        service = SERVICES.get(alert.service)        if service is not None:            return service.rota, severity_for(alert, service)    return "platform-oncall", "SEV3"     # the default branch, unchanged

report                                 routed to        sev   verdict------------------------------------------------------------------------------alert checkout-api http_5xx_rate=0.40  payments-oncall  SEV1  okalert search-api   http_5xx_rate=0.05  search-oncall    SEV3  okalert cdn-edge     http_5xx_rate=0.42  platform-oncall  SEV3  WRONG (unowned)prose "everything is on fire..."       platform-oncall  SEV3  WRONG (payments)prose "card charged... no alert fired" platform-oncall  SEV3  WRONG (payments SEV2)prose "chekout-api returning errors"   platform-oncall  SEV3  WRONG (payments SEV1)answered 6/6    correct 2/6    flagged 0

One report flips, and the per-metric severity line is what flipped it. The ownership lookup changed nothing on this set: route_total's substring match already resolved both service names correctly. It removes a class of bug these six reports happen not to trigger. Refusal touched neither.

Hold on to that, because the honest router in the next section answers the same two reports at the same two severities. That is not six data points. On every input route answers, route_total_fair returns the identical pair - same tables, same severity rule, same code path - and route refuses a superset of what route_total_fair sends to the default branch. The refusal path cannot buy accuracy. It buys the labels on everything it refuses. That is the claim of this article stated as an ablation rather than a slogan, and it is narrower and more defensible than "the honest version is more accurate," which is not true.

Using an LLM as your fallback branch changes nothing

Now replace the fallback with a model. This is the version that ships in 2026, and it is usually presented as the fix for everything above.

def classify_with_llm(report: str) -> tuple[str, str]:    """Ask a model for (rota, severity). Always returns a pair, never refuses."""    ...def route_with_model(report: str) -> tuple[str, str]:    """Same contract, with the fallback delegated to a model."""    alert = parse_alert(report)    if alert is not None:        service = SERVICES.get(alert.service)        if service is not None:            return service.rota, severity_for(alert, service)    return classify_with_llm(report)     # the default branch, wearing a coat

This genuinely fixes the four the tables could not reach. The model reads "Everything is on fire, checkout is down" and routes it to payments. It reads the prose about the charged card and gets there too. It handles chekout-api without noticing the typo. I want to be careful with that last one, because the obvious claim about it is false: difflib.get_close_matches resolves chekout-api to checkout-api in one line of standard library, deterministically, on the floor. What fuzzy matching costs you is a threshold, and a threshold is a new source of confident wrong matches that has to be tuned against a corpus you do not have. Three of these four inputs really do need language understanding. The typo is not one of them, and I will come back to why that matters.

It also changes nothing about the property that matters here, because coverage is still 1.0. Ask the function which answers came from the table and which came from the model. It has no idea, and neither does its caller.

And it added things. There is now a probabilistic remote dependency in the routing path, and a dict lookup on the tail of your traffic is now a billed network round trip - on exactly the slice you cannot forecast, which is why you cannot price it without the refusal rate. There is a failure mode the rules version did not have: when the model is degraded, slow, or rate limited, classify_with_llm either raises into a caller that probably catches broadly, or returns a plausible rota that is wrong for a new reason.

The security picture breaks the "same architecture" framing, and not in the direction I would have chosen. Incident reports are frequently attacker-influenceable - customer tickets, third-party webhooks, alert bodies carrying user-supplied strings. An attacker who wants to suppress a page sends prose, which deterministically forces the fallback branch. Under route_total_fair that gets them platform-oncall at SEV3, which for a real payments SEV1 is a missed page - so the deterministic version is exploitable too, just to a fixed destination. Under route_with_model the attacker additionally chooses where the page goes and how loud it is. Two things follow, and only one of them helps my argument. This axis favours rules over a model, not refusal over invention. But it is also the one place counting the fallback pays a security dividend, and the refusal path is what makes that counter structural rather than the first thing to go missing: a burst of injected prose shows up as a spike instead of as nothing at all. A counter catches a burst. It does not catch the single crafted report that suppresses one page.

Answer Obligation: the requirement nobody wrote down

Answer Obligation is the unstated design requirement that a decision path must return an answer for every input it receives. Almost nobody chooses it. You inherit it from the interface: the signature says the function returns a rota, the caller has no branch for anything else, so the implementation invents something when its rules run out. After that, the only remaining question is what does the inventing - a default branch or a model - and that question is a detail.

Name Answer Obligation for one reason: it is the part of this you can actually delete. The rules table will never cover unbounded human prose, and no model gets reliable enough that you stop watching it. The return type, though, is yours.

The strong version of the mechanism is false. I would rather kill it here than have a reader kill it for me. Answer Obligation does not make the refusal rate impossible to measure. One line in the default branch measures it:

    metrics.increment("route.fallback", reason="no rota matched")    return "platform-oncall", "SEV3"

That works. The default branch survives, the return type never changes, and you can answer the dashboard question. The model version gets a free partial count, because every fallback is a billed request. That gives you a numerator and nothing else: no denominator, no split by reason, no attribution to a traffic slice. It tells you what you spent, not what share of your inputs you failed to specify.

So the claim is not impossibility. It is this: nothing stops you counting the fallback, and nothing makes you. The caller cannot tell the difference, so no downstream code ever fails for want of the counter, so the counter is the first thing to go missing and the last thing anybody notices. Changing the return type is the version that survives the engineer who added the metric leaving the team. That is weaker than impossibility and it is the claim that holds.

The tell is a metric you cannot produce. Go and try it now: what share of incidents hit the fallback last month, broken down by reason? If your dashboard cannot answer that, the rest of this article is theory to you. Find out why it cannot, then come back.

Total and partial functions, applied to architecture selection

There is a version of this argument from type theory, and it is a useful frame provided I do not overclaim what it derives.

Routing an arbitrary incident report to a rota is a partial function. Its domain is smaller than its type suggests: there exist inputs for which no correct output can be specified from the rules alone. No amount of engineering removes that.

Functional programming has a standard set of moves for a partial function when the type demands a total one; Alexis King's write-up on partiality and bottoms covers most of them. Architecture selection is choosing among these, and that is the whole trick:

1. Supply a value where the function was undefined. Your default branch.
2. Let something else supply it. Congratulations, you have bought a model.
3. Return a value that means "undefined here." Option, Maybe, or Result. In architecture terms, Level 0 built honestly.
4. Or shrink the domain until the function is total on it, which means scoping the deterministic path to the traffic it specifies and declaring the rest out of scope.

The frame is doing less work than it looks. The canonical taxonomy's second move is "leave the function undefined on the new values," not "ask a model" - substituting a model there is mine, not the literature's. And at the type level, options 1 and 2 are not merely similar, they are indistinguishable: the type system cannot see where a value came from. That is the point, not a hole in it. But it means the taxonomy names the equivalence. It does not prove it.

Push it and it breaks. By the totality argument alone, a default branch and routing to a human operator are the same move, since both make the function total. But routing to a human is option 3, and option 3 is what I am recommending. So totality is not the distinction that matters. Provenance is. What separates 3 and 4 from 1 and 2 is not whether the function returns, it is whether the caller is told which move happened - and provenance is orthogonal to the return type, as the decided_by field below shows.

Options 3 and 4 also look like the same move and are not, and the difference lands on the rota rather than in the code. Option 3 sends the leftovers to a human queue, which somebody has to staff. Option 4 refuses them at the edge, which means somebody has to go and tell a stakeholder that a class of input is now out of scope. Option 4 gives you the better system and the harder conversation, so these are not four equal choices and I would not present them to a team as if they were.

flowchart TD
    IN["An input whose correct output<br/>the rules cannot specify"] --> Q{"What does the path do?"}

    Q -->|"1 - supply a value"| A["default: return PLATFORM"]
    Q -->|"2 - let a model supply one"| B["default: ask the model"]
    Q -->|"3 - return 'undefined here'"| C["return Unroutable(reason)"]
    Q -->|"4 - shrink the domain"| D["reject at the edge,<br/>declare the rest out of scope"]

    A --> X["Caller cannot tell.<br/>Counter optional."]
    B --> X
    C --> Y["Caller must handle it.<br/>Counter by reason,<br/>and the checker asks for it."]
    D --> Y

    style IN fill:#95A5A6,color:#2C2C2A
    style Q fill:#FFD93D,color:#2C2C2A
    style A fill:#E74C3C,color:#FFFFFF
    style B fill:#E74C3C,color:#FFFFFF
    style C fill:#6BCF7F,color:#2C2C2A
    style D fill:#6BCF7F,color:#2C2C2A
    style X fill:#E74C3C,color:#FFFFFF
    style Y fill:#4A90E2,color:#FFFFFF

Notice which axis gets the argument. Almost every debate in this field is about what supplies the invented value. Almost none is about what the caller is told.

The right way: make the refusal a return value

Same tables as route_total_fair. The difference is the return type, plus one exit route can take that route_total_fair cannot: an alert naming a metric with no severity line. That exit is a deliberate partial refusal - I know who owns the service and not how loud to page - and I would rather surface that the alert grammar has outrun the severity table than page at the service floor and hide it. No report in this set reaches it.

import refrom dataclasses import dataclassfrom typing import assert_never@dataclass(frozen=True)class Alert:    service: str    metric: str    value: float@dataclass(frozen=True)class Service:    rota: str    floor: str  # the severity floor for anything paging on this service@dataclass(frozen=True)class Routed:    rota: str    severity: str    decided_by: str@dataclass(frozen=True)class Unroutable:    reason: strDecision = Routed | UnroutableSERVICES: dict[str, Service] = {    "checkout-api": Service(rota="payments-oncall", floor="SEV1"),    "payment-gateway": Service(rota="payments-oncall", floor="SEV2"),    "search-api": Service(rota="search-oncall", floor="SEV3"),}# The value at which a metric is a SEV1 on its own, whatever the service floor.SEV1_LINE: dict[str, float] = {    "http_5xx_rate": 0.25,    "p99_latency_ms": 3000.0,}# fullmatch, not match: the whole string is an alert line or none of it is._ALERT = re.compile(    r"ALERT\s+service=(?P<service>[a-z0-9-]+)"    r"\s+metric=(?P<metric>[a-z0-9_]+)"    r"\s+value=(?P<value>\d+(?:\.\d+)?)"    r"\s+for=\d+m")def parse_alert(report: str) -> Alert | None:    """Parse a machine alert line. Return None for anything that is not one.    Returning None rather than raising is deliberate. A human sentence is not    a malformed alert. It is a different kind of input, and `route` has to be    able to say which one arrived.    """    match = _ALERT.fullmatch(report.strip())    if match is None:        return None    return Alert(        service=match["service"],        metric=match["metric"],        value=float(match["value"]),    )def severity_for(alert: Alert, service: Service) -> str:    """A metric past its own SEV1 line escalates. A quiet metric never de-escalates."""    line = SEV1_LINE.get(alert.metric)    if line is not None and alert.value >= line:        return "SEV1"    return service.floordef route(report: str) -> Decision:    """Route one incident report, or say why no rule reached it.    Four exits, all labelled. Nothing leaves this function carrying a value    that no rule produced.    """    alert = parse_alert(report)    if alert is None:        return Unroutable("report is prose, not an alert line")    service = SERVICES.get(alert.service)    if service is None:        return Unroutable(f"no rota owns service {alert.service!r}")    if alert.metric not in SEV1_LINE:        return Unroutable(f"no severity line defined for metric {alert.metric!r}")    return Routed(        rota=service.rota,        severity=severity_for(alert, service),        decided_by=(            f"{alert.metric}={alert.value} on {alert.service} (floor {service.floor})"        ),    )

Three things in that function are doing argumentative work. The logic is the boring part.

There is no else. The function exits in one of four places and every one is labelled: three refusals carrying distinct reasons, and one decision carrying the rule that produced it. Each exit can carry its own counter.

Decision is a union, so the caller has to decide what a refusal means. That is weaker than it sounds in Python, and I want to be exact about how weak. Python does not compile, and a type checker only forces exhaustiveness if the caller writes the match and asks for it. A caller that logs str(decision) or widens to Any type-checks fine and drops the incident silently. So the guarantee is conditional: provided the caller is type-checked and does not widen, the refusal reaches a human because the checker will not let it not.

fullmatch is doing the work people usually assign to anchors. An earlier draft used re.match with ^ and $, and the ^ was decorative, because re.match anchors at position 0 already. Only the tail anchor was load-bearing. Switch to re.search and the pattern finds the alert grammar anywhere it appears, including inside a sentence a human typed, so a report quoting an alert while asking whether it is stale would parse cleanly, route on the fragment, and discard the actual question. That is not a hypothetical I invented for the paragraph. In January 2026 an authorisation filter controlling which GitHub users could trigger AWS CodeBuild builds was missing its anchors, so any user identifier containing a trusted identifier as a substring passed the check. Because the identifiers are sequential, researchers obtained administrator access on affected AWS-managed open source repositories.

Where the refusals go

A refusal is not a safety improvement until something receives it. This is the part most versions of this argument leave out, and leaving it out is dangerous, because on an alerting path an unhandled refusal is a dropped incident.

def dispatch(report: str) -> None:    match route(report):        case Routed(rota=rota, severity=severity, decided_by=why):            page(rota, severity, context=why)        case Unroutable(reason=reason):            metrics.increment("route.unroutable", reason=reason)            page("platform-oncall", "SEV3", context=f"unroutable: {reason}")        case unreachable:            assert_never(unreachable)

page and metrics are your own; the block is the shape, not something to paste.

Note what did not change. platform-oncall still gets woken up. In most organisations the platform rota already is the unroutable queue, and "delete the default branch" is bad advice if it is read as "drop the destination." Keep the destination, delete the invention. The page now arrives carrying the reason no rule applied, and the counter increments with that reason attached.

Run the honest router against the same six reports:

ALERT service=checkout-api metric=http_5xx_rate value=0.40 for=8m  -> page payments-oncall at SEV1     decided by: http_5xx_rate=0.4 on checkout-api (floor SEV1)ALERT service=search-api metric=http_5xx_rate value=0.05 for=30m  -> page search-oncall at SEV3     decided by: http_5xx_rate=0.05 on search-api (floor SEV3)ALERT service=cdn-edge metric=http_5xx_rate value=0.42 for=3m  -> UNROUTABLE: no rota owns service 'cdn-edge'Everything is on fire, checkout is down  -> UNROUTABLE: report is prose, not an alert lineCard charged but the order page spun forever. No alert fired.  -> UNROUTABLE: report is prose, not an alert linechekout-api returning errors  -> UNROUTABLE: report is prose, not an alert linerouted 2/6 (2 correct)    refused 4/6    wrong 0wrong 0 because nothing was answered that could be wrong. All fourrefusals still page platform-oncall - now carrying the reason.

Compare that against route_total_fair at matched coverage, because comparing 100 percent coverage against 40 percent coverage is the exact error the next section is about. On the two reports both functions answer, both are right, twice. The refusal path changed no answer. What it changed is the other four: four confident wrong pages became four labelled refusals split across two reasons, one of which - no rota owns service 'cdn-edge' - is a data problem that got filed as an architecture problem, and is now visible as one.

The Zero Row: what a floor that can refuse actually measures

The Zero Row is what you get when you measure the deterministic implementation of a task before any model exists, and record both halves: what it costs, and how often it cannot answer. Every later claim that a rung improved something is a comparison against that row. Without it you are quoting a percentage with nothing under the line.

It is a term I use in the book this series accompanies, and it has nothing to do with ZeroR, the majority-class baseline classifier from classical machine learning, beyond the similarity of the names. ZeroR is a model you train. The Zero Row is a row in a cost table you measure.

For the reference system in that book - the incident triage assistant its companion repository ships - the row reads 2.5 microseconds at the median and 13.4 at the 99th. I measured it over 1,200 routes replayed against the fixture corpus, with zero tokens and no network call. Those are in-process Python timings, so the 99th percentile is telling you more about garbage collection than about routing, and I would not quote it as a production latency. On its own the figure is noise. Print the next rung's row beside it and the ratio is what language understanding costs on that path.

My own instrument is rigged, and it is rigged in my favour. It prices the two rungs in different currencies. The Zero Row as stated charges the model rung for tokens and gives the floor its costs for free, and the floor has real ones: table maintenance, and - once you adopt the refusal path - human triage of every refusal. At a refusal rate in the tens of percent on an alerting path, that is a headcount line rather than a rounding error. A Zero Row that omits it will flatter the floor in every ratio computed against it. Add a third column for human cost per refusal with its denominator, and option 4 starts looking as attractive as I already claimed it was.

I have not seen a cost table with a refusal-rate column in it, including the first version of my own. The half that governs the climbing decision is the refusal rate.

This has a formal shape worth borrowing. In selective classification a system is allowed to abstain, and coverage - the fraction of inputs on which it chose to predict - is a first-class measured quantity traded against risk. The reject option goes back to Chow in 1970. The modern deep learning version is Geifman and El-Yaniv's 2017 result, and their headline number is the point I am making: 2 percent top-5 error on ImageNet guaranteed with probability 99.9 percent, at roughly 60 percent coverage. The guarantee exists only because the system was permitted to refuse on the other 40 percent.

The same thing happens one level up. A system with no reject option has coverage pinned at 1.0, so you cannot plot a risk-coverage curve for it at all. One point, no curve. That is also why the comparison in the last section had to be done at matched coverage. A system that answers everything and a system that answers 40 percent are not comparable on accuracy, and treating them as if they were is the error the whole selective-prediction literature exists to prevent.

Why a deterministic refusal is better evidence than a model's abstention

Models can abstain too. Ask for a confidence score, threshold it, route the low-confidence tail to a human. That is a real pattern, it ships, and it is better than a bare default branch.

It is also a weaker instrument than a rules-based refusal, and in 2026 somebody measured why. Ling and colleagues found something I did not expect, and named it Abstention Inflation. Models decline questions they can demonstrably answer, and the trigger is the presence of an extra option rather than uncertainty. The control that convinced me: swap "Unknown" for an unrelated random word and you get the same effect. It emerges through instruction tuning and not from sampling, so it is a stable property of the deployed model and not noise you can average away.

The obvious version of this comparison overclaims, so here is the real one. A deterministic refusal is just as much an artifact of how you built it. Loosen the regex and the refusal rate drops with no change in task difficulty, which is structurally the same phenomenon Ling and colleagues found when they changed a label. Both numbers are construction-dependent.

The difference is legibility. You can read the reason for a deterministic refusal off the source, attribute it to a specific line, and change it deliberately. The model's abstention rate moves for reasons you cannot see and cannot attribute. So the floor reports on itself more honestly than the model does - not because it is uncontaminated, but because its contamination is inspectable.

These are also not the same kind of error. Abstention Inflation is a false-refusal phenomenon: the model declining what it could have answered. The deterministic analogue would be a regex refusing inputs the rules could actually handle. My own router does that once, on purpose: the unknown-metric exit refuses an alert whose owner is known. Everywhere else it refuses inputs the rules cannot specify, which is the rules working. The comparison here is between instruments, not between error rates.

The honest counter-argument: what the floor cannot do

The strongest objection to all of this is that the floor is simply bad, and that a refusal rate this high is an outage rather than a measurement.

There is a number attached to that objection, and I would rather quote it than argue around it. Trooskens and colleagues ran plain regular-expression extraction against direct model calls on 5,680 invoices from the DocILE dataset. The regex version was 4,915 times faster. It also scored 20.3 percent on the Key Information Localization and Extraction (KILE) metric, against 80.0 percent for a hybrid that used deterministic orchestration with bounded model calls for the semantic subtasks. The deterministic version collapsed where you would predict: 10.1 percent on customer names, 9.2 percent on currency amounts.

A gap of roughly 60 points is not a rounding error, and nobody should read this article as a claim that rules usually win. On that task, on those fields, they do not.

Here is where I have to concede something that cuts against my own prescription. That paper's regex arm had no refusal path a caller could act on, and the authors still produced the complete escalation map: the size of the gap, the field-level breakdown, the exact places the floor fails. I do not know from the paper whether its low score came from wrong values or from missing ones, and the distinction is this article's whole subject, so I will not guess at it. What produced that map was an offline evaluation against a labelled ground truth, which is an eval practice and not an architecture choice.

So the disjunction is real and I should state it plainly. An offline eval against ground truth gets you the same map without touching your return type. It costs a labelled corpus and a scoring harness, and it goes stale the moment your traffic distribution shifts. The difference is smaller than I would like and it is real. The eval needs a corpus that stays representative of your traffic; the refusal counter needs a hundred labels once and then reports continuously, on live traffic, without anyone keeping a fixture set current. Point-in-time against continuous, maintained against one-off. If you have the corpus and the harness, use them. Most teams on an alerting path have neither.

There is a second cost, and the typo shows it. My six-report router refuses chekout-api with the same label it gives to genuinely unstructured human prose - and as noted above, fuzzy matching would have caught that one without a model at all. That is the point: the typo belongs in the bucket a better alert template or an edit-distance pass would fix, not in the bucket that argues for a rung. So the prose bucket does not split itself. The no rota owns this service bucket splits perfectly well - that one is a data problem and reads as one. But report is prose, not an alert line lumps together "needs language understanding" and "a better alert template would have fixed this," and those call for opposite responses. Only the first argues for a rung.

Closing that gap takes an afternoon and no tooling. Pull a sample of refusals and sort them into the two buckets yourself. Do not hand this part to a classifier.

Why the alerting path is the one that must stay on the floor

Everything so far applies to any path. There is one class of path where the argument gets stronger, and it is the one that gets LLM-washed most often, because incident reports are unstructured and unstructured reads as "needs a model."

Start with the availability arithmetic, which is standard even if its independence assumption is not. A three-nines dependency on a four-nines path costs you about a nine, and for most features you would take that trade.

The problem is correlation. Incidents are not sampled uniformly across the year. They cluster around load spikes, deploys, and provider degradation, which are the conditions under which a shared third-party API is also having a bad day. So the window in which the dependency is most likely to be struggling overlaps the window in which the path carries the most weight. No staging environment reproduces that. Staging is not having an incident at the same moment production is.

The status of that claim needs stating. The availability arithmetic is standard and citable. The correlation argument is reasoning from shared-fate failure, not a measurement: I could not find published data on correlated failure of third-party model APIs during load events, and I am not going to present my inference as a finding.

One thing has not been shown, and I am not going to paper over it. I could not find a single public post-mortem in which a language model on an incident-routing or alerting path caused or extended an outage. Not one. So this section argues a predicted failure mode from structure, with Cloudflare as a structurally similar precedent on a request path rather than an alerting path. If you know of a published incident, I would genuinely like to read it.

The best evidence I found came from somewhere else entirely, and from someone not writing about architecture selection at all. Wei Wu spent eight weeks cataloguing silent failures in his own agent runtime. One of them was a nine-hour gateway outage in which three separate alarms each failed to fire, one because a quiet-hours filter suppressed both notification channels including the one that existed for emergencies. The rule Wu derives is the sharpest version of this principle I have found stated anywhere: an alert path must not depend on the failing subject. Gateway-down alerts must travel a channel the gateway cannot take down.

Read it, and read the caveat with it. This is one operator studying his own system over eight weeks, with 22 documented incidents. It is a strong catalogue of mechanisms and it is not an industry statistic. Its most useful contribution here is smaller: a fail-open guard caught a NameError in eight separate validation checks and dutifully skipped all eight, reporting success. Eight checks that never ran, reported as eight checks that passed. That is Answer Obligation inside a guard clause, one level below architecture, doing exactly what a default branch does to a router.

Wu's rule also cuts against me, and I would rather raise that than have a reviewer raise it. If an alert path must not depend on the failing subject, then a third-party model API is a channel your failing subject cannot take down - while your own rules engine, same region, same cluster, behind the same load balancer as the system that is on fire, is far more shared-fate with the incident than the external API is.

Both are true, because they are different axes. Wu's rule is about coupling to the failing subject. Mine is about coupling to a probabilistic dependency whose degradation modes are silent: slow, malformed, rate-limited, subtly reshaped. The configuration that satisfies both is a deterministic path that is also off-box, and if you are building this from scratch that is the thing to build.

Separate the two jobs instead of banning the model. Deciding who to wake up and deciding what to tell them have different tolerances for being wrong and for being slow. Route on rules in the pager service, and let a summariser enrich the ticket afterwards, out of band, where a 30-second model timeout costs you a worse ticket rather than a missed page.

AWS shipped this exact branch, with no AI anywhere near it. Route 53 health checks expose a setting called InsufficientDataHealthStatus, and the documented behaviour is that a new health check counts as healthy until there is enough data to say otherwise. A health-checking system - an alerting path - converting "I do not have the data" into "everything is fine."

But look at what AWS actually did, because it is closer to my recommendation than to the pathology. They did not bury the branch. They named it, documented it, and made it an operator-visible choice with three settings: healthy, unhealthy, or last known status. Defaulting a brand new check to healthy is the right call, because the alternative trips failover on every check you create. The point is not that AWS got it wrong. The point is that you can tell which behaviour you have, and change it, because somebody made the insufficient-data branch a first-class named thing instead of an invisible else. That is not option 3 - the resolver is still handed a definite status, and only the operator is told which move happened. It is option 1 with the branch named, documented and selectable, which is the part of option 3 that survives when you cannot change the return type.

How to run the Floor Test on code you already own

Part 1's diagnostic question - the book calls it the Floor Test - is whether you can write a unit test that fully specifies the correct output for any given input. If yes, no model.

Running it against a system that already exists takes one extra step, and that step is where teams go wrong. You are not testing whether the implementation is deterministic. That is easy, and it is beside the point. You are testing whether you can state the correct output for each input the path receives.

Code that queries a database, calls an internal API, or drains a live queue is still on the floor. What moves you off it is a single input for which no test can state the right answer.

1. Sample a hundred real inputs and write down the correct output for each, by hand. This is the step everybody skips, and skipping it is why the rest of the argument never gets settled. Not a model-assisted labelling run, not a sample of the inputs you remember, and not a hundred rows from the happy path. A hundred real ones, once, by a person who knows what the right answer was. If you can specify all hundred, any model in the fallback position is decoration. If you cannot specify eleven of them, that is roughly five to nineteen percent, exact binomial at 95 percent - imprecise, and still decisive, because nobody argues about the order of magnitude and the order of magnitude is what picks the rung. Budget an afternoon.

2. Delete the invention, not the destination. Change the return type so callers cannot ignore a refusal, and keep paging the same fallback rota with the reason attached.

3. Run it in shadow mode for a week without changing what gets paged. I do not know what share of traffic the default branch was absorbing. Neither do you, which is the point of running the week.

4. Sort thirty of those refusals by hand before anybody proposes a rung. Thirty is enough to see whether one bucket dominates and nowhere near enough to estimate a proportion, so treat it as a direction and not a number. You are splitting "needs language understanding" from "a better alert template would have fixed this," and only the first argues for a model.

5. Write the Zero Row down, both halves: latency with its method, and the refusal rate with its denominator, split by reason. "No rule matched" is not one of the reasons. "Report is prose, not an alert line" and "no rota owns this service" are different findings with different fixes. A refusal rate without a denominator is the same error Part 5 catalogued in the other direction, where a counter of zero was read as a verdict.

6. Before you climb, write down what the floor cannot do, not what it does badly. "This path receives inputs whose meaning decides the answer, and meaning is not a property of the string" points at exactly one rung. "Our rules are not accurate enough" argues equally well for any rung on the ladder, which means it argues for none of them.

Step 6 is the discipline the whole procedure exists to serve. Rule churn is the most common argument for climbing off the floor, and it is not evidence. A table that needs an edit every week is telling you the input space you assumed was closed is open. It is not an argument for a model. A model has never heard of the service you added last Tuesday either, and it will guess about that service with considerably more confidence than your table did. The evidence is the Floor Test failing on a real input, and churn is only what sent you to look for it.

The floor is a measurement, not a rung

The other seven rungs are read against Level 0. That makes it an instrument, and an instrument that cannot report its own failure is not measuring anything.

So let me state the claim in the form that survived the argument, rather than the form I opened with. On the one axis this article is about - whether the caller can tell that no rule applied - a default branch and a model in the fallback position are the same architecture. On cost, on security, and on billing they are not, and I have said where each one diverges. The axis nobody argues about is the one that decides whether you can ever price the climb.

Two teams, two architecture diagrams, the same defect. One writes a rules engine with a catch-all and believes it is on the floor. The other puts a model in the fallback and believes it has climbed. Both shipped a function that invents a value where no rule applied. The rules team wrote coverage of 1.0 into a dict literal. The model team wrote it into a fallback call and paid per token for the privilege. Neither can now price what the model bought, or what handing it back would cost.

The first team is an awkward case for my own definition. If a rules-only router with a catch-all is not on the floor, where is it? Not Level 1, which requires a model. It is below the floor: Level 0 with the instrument removed. That is not a rung and should not be drawn as one. The Zero Row is measured per traffic slice rather than per system, which is also how a system with a model in it can sit on the floor for the part of its traffic it fully specifies.

Cloudflare's two proxies are still the cleanest illustration I know, provided it is read correctly. One crashed where everybody could see it. The other emitted a confident bot score of zero for every request on Earth, and the signal that something was wrong on that path came from customers watching legitimate traffic get blocked, not from the score itself. Neither had the third behaviour. Neither could say "no score available" as a value.

I would rather operate the system that failed where somebody could see it. Not because crashing is safe - it took down part of the internet - but because a system that fails visibly is one you can still make decisions about, and a system that invents a value is one you cannot.

Delete the invention and keep the destination. Change the return type so the caller has to look. Count the refusals by reason, print the denominator beside them, and sort thirty of them by hand before anybody proposes a model. That is the whole of Level 0, and it is the only rung in this series that is cheaper to measure than to argue about.

References

• Cloudflare. (2025, November 18). Cloudflare outage on November 18, 2025. https://blog.cloudflare.com/18-november-2025-outage/
• ThousandEyes. (2025). Cloudflare Outage Analysis: November 18, 2025. https://www.thousandeyes.com/blog/cloudflare-outage-analysis-november-18-2025
• Schluntz, E. and Zhang, B. (2024, December 19). Building Effective AI Agents. Anthropic. https://www.anthropic.com/engineering/building-effective-agents
• Palmer, C. (2026, July). Should you even use an LLM? https://cameronmpalmer.medium.com/should-you-even-use-an-llm-b4f3b7914f4d
• Chow, C.K. (1970). "On Optimum Recognition Error and Reject Tradeoff." IEEE Transactions on Information Theory 16(1):41-46.
• Geifman, Y. and El-Yaniv, R. (2017). "Selective Classification for Deep Neural Networks." NeurIPS 2017. https://papers.neurips.cc/paper/7073-selective-classification-for-deep-neural-networks
• Geifman, Y. and El-Yaniv, R. (2019). "SelectiveNet: A Deep Neural Network with an Integrated Reject Option." ICML 2019. https://proceedings.mlr.press/v97/geifman19a.html
• Franc, V., Prusa, D. and Voracek, V. (2023). "Optimal Strategies for Reject Option Classifiers." Journal of Machine Learning Research 24. https://jmlr.org/papers/volume24/21-0048/21-0048.pdf
• Larson, S. et al. (2019). "An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction." EMNLP-IJCNLP 2019, 1311-1316. https://aclanthology.org/D19-1131/
• Ling, Z. et al. (2025, revised 2026). "LLM Abstention Can Be a Prompt Artifact, in Addition to Genuine Uncertainty." arXiv:2507.16199. https://arxiv.org/abs/2507.16199
• Wu, W. (2026). "When Errors Become Narratives: A Longitudinal Taxonomy of Silent Failures in a Production LLM Agent Runtime." arXiv:2606.14589. https://arxiv.org/abs/2606.14589
• Trooskens, G. et al. (2026). "Compiled AI: Deterministic Code Generation for LLM-Based Workflow Automation." arXiv:2604.05150. https://arxiv.org/abs/2604.05150
• Amazon Web Services. How Amazon Route 53 Determines Whether a Health Check Is Healthy. https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/dns-failover-determining-health-of-endpoints.html
• Amazon Web Services. Availability and Beyond: Understanding and Improving the Resilience of Distributed Systems on AWS. https://docs.aws.amazon.com/pdfs/whitepapers/latest/availability-and-beyond-improving-resilience/availability-and-beyond-improving-resilience.pdf
• InfoQ. (2026, January). Two Missing Characters: How a Regex Flaw Exposed AWS GitHub Repos to Supply-Chain Risk. https://www.infoq.com/news/2026/01/aws-github-vulnerability/
• King, A. Partial Functions and Nontermination (Hackett guide). https://lexi-lambda.github.io/hackett/guide-bottoms.html
• Kumar, R. (2026). The 7 GenAI Architectures: A Field Guide to Choosing the Right AI System - Before You Overbuild It. Chapter 3, "Level 0: The Floor." https://7genai.ranjankumar.in/

Related Articles

AI Engineering

• Building a Production MCP Server: Architecture, Pitfalls, and Best Practices
• Unified Observability Across Agent Fleets: Building the Control Plane Metric Layer
• Why Single Agents Fail at Scale: The Five-Mode Failure Taxonomy

System Design

• Descending: Why You Cannot Delete an Idle AI Agent Layer