FastAPI + LangGraph: What a Client Disconnect Commits

I wired a four-node LangGraph agent to a real uvicorn server, opened a client, and closed the connection after the second chunk, about six hundred milliseconds in. The charge node had already run. notify and finish had not. The thread's stored state stopped at next=['notify'].

Then I sent one more message on the same thread_id, the way a user does when they come back. The graph ran again from the start, so charge fired a second time. The stored transcript ended up as [plan, charge, plan, charge, notify, finish], with the duplicate charge sitting in plain sight and nothing anywhere flagging it.

That is the visible version. Change one setting and the duplicate stops being visible at all, which is where this gets worse.

Nothing crashed, no worker died, no region failed over, and the infrastructure stayed healthy for every millisecond of that sequence. Someone closed a connection.

Measured 2026-08-04 on langgraph 1.2.9, langgraph-checkpoint-sqlite 3.1.0, langgraph-checkpoint-postgres 3.1.1, psycopg 3.3.4, fastapi 0.118.3, starlette 0.48.0, uvicorn 0.38.0, postgres:16. Full methodology and limits at the end.

Why your FastAPI handler decides LangGraph's interruption behaviour

The usual framing for FastAPI and LangGraph is that a stateless request layer meets a stateful runtime, and you fix the gap by bolting persistence underneath: add a checkpointer, add async, add streaming. I published a version of that argument here in February, and it was answering the wrong question, because persistence is not what breaks at this seam.

It was worse than incomplete, actually. That article told readers to make HTTP handlers thin wrappers that start execution and return immediately, which is the detached shape below, the one row in the table that leaves no evidence when it goes wrong. I recommended it without knowing what it commits. This piece replaces it, which is why its URL now brings you here.

The claim this article owns: in a self-hosted FastAPI service, the structural shape of your request handler determines whether an interrupted agent run stops, completes, or half-completes. Not your checkpointer. Not your durability setting. Three handler shapes that pass the same review and serve the same response produce three different durable records, in two outcome classes, from one disconnect.

I call this Handler-Shaped Durability.

A note on the name, since "durability" already means something specific in the Temporal and DBOS literature I cite below. There it means surviving crashes. Here the subject is interruption and cancellation semantics, which is a narrower thing wearing the same word. I am bending the term deliberately, because the setting that governs it is literally called durability.

The disconnect path is not rare. It is a closed laptop, a mobile handoff, a 60-second proxy idle timeout, a user who lost interest. In a consumer-facing agent it runs at volume, and it drives your system down the code path you built for crash recovery and have probably never tested.

What DBOS, Diagrid, and open LangGraph issues already say about this

Most of this territory is occupied, so let me mark the boundary before claiming anything.

The consequence here, a side effect that executed with no durable record of it, is thoroughly documented. DBOS states the mechanism plainly: a workflow can be interrupted after completing a step but before recording its checkpoint, so on recovery it has no record the step ran and executes it again. Two arXiv papers work the same ground from opposite ends. Atomix (2602.14849) treats tool use as a transactional problem; Temporary Authority, Permanent Effects (2607.10487) takes the authorization version. Crab (2604.28138) gave it the best name, the "agent-OS semantic gap", where agent frameworks see tool calls but never their operating-system effects. Diagrid argues LangGraph's checkpoints are not durable execution at all, citing no failure detection, no automatic resumption, and no duplicate-execution prevention.

None of that is mine, and I am not going to re-argue any of it.

The concurrency result later in this article also rests on a classical foundation: concurrent read-modify-write without optimistic concurrency control forks or loses writes. That is a textbook result from the 1980s, not a discovery.

Two things are missing from all of those sources.

The trigger. Every one of them frames this as crash recovery, where a worker dies or a region fails or a sandbox is restored. Diagrid's analysis does not address HTTP request boundaries, client disconnects, or request cancellation. The double-execution write-ups assume you deliberately called interrupt(). None of them treat ordinary web traffic on healthy infrastructure as the thing that puts you on the recovery path.

The control surface. All of them reason at the runtime layer. None look at the handler, which is where the decision gets made.

Ably's "transport gap" piece is the nearest adjacent framing: HTTP streaming binds the connection, the device, and the server to one request, and a drop on either side ends the session with no way to resume. That is a claim about the stream. Nothing in it says what the interruption committed on the way out.

Three FastAPI handler patterns for LangGraph streaming, compared

Here is the graph, unchanged across every measurement below.

import asyncio, json, os, timefrom contextlib import asynccontextmanagerfrom typing import Annotated, TypedDictfrom fastapi import FastAPIfrom fastapi.responses import StreamingResponsefrom langgraph.graph import StateGraph, START, ENDfrom langgraph.checkpoint.sqlite.aio import AsyncSqliteSaverclass S(TypedDict):    steps: Annotated[list, lambda a, b: a + b]def record(node: str, thread: str) -> None:    """The measuring instrument: an append-only log of what really executed."""    with open("side_effects.log", "a", encoding="utf-8") as fh:        fh.write(f"{time.time():.3f}\t{thread}\t{node}\t{os.getpid()}\n")def mk(name: str, delay: float):    async def node(state: S, config):        await asyncio.sleep(delay)        record(name, config["configurable"]["thread_id"])        return {"steps": [name]}    return nodebuilder = StateGraph(S)for n, d in [("plan", 0.3), ("charge", 0.3), ("notify", 1.5), ("finish", 0.5)]:    builder.add_node(n, mk(n, d))builder.add_edge(START, "plan")builder.add_edge("plan", "charge")builder.add_edge("charge", "notify")builder.add_edge("notify", "finish")builder.add_edge("finish", END)G = None@asynccontextmanagerasync def lifespan(app: FastAPI):    global G    async with AsyncSqliteSaver.from_conn_string("seam.sqlite") as saver:        G = builder.compile(checkpointer=saver)        yieldapp = FastAPI(lifespan=lifespan)

Two things about that code. record() writes a line to a local file, which is the friendliest possible stand-in for a real side effect. I come back to why that matters in the limits section. And the delays exist to make the interruption window observable: notify is deliberately long, so a client that leaves after the second chunk, about 0.6s in, stops between charge and notify with 1.5s of node work still ahead.

The graph decides where a run can stop. The handler decides whether it stops. Those are different variables, and the three handlers below change only the second one.

A thread sitting at next=['notify'] looks like it should resume at notify. It would, if you passed None: G.astream(None, cfg) picks up the pending task and runs only what is left. Every handler below passes an input dict, {"steps": []}, because a chat endpoint carries the user's new message. Supplying input re-enters the graph from START. Completed nodes run again.

That is not a bug, and it is not a setting anyone got wrong. Resuming and starting a new turn are genuinely different operations, and the runtime tells them apart by whether you passed input. The catch is that on a chat endpoint, the ordinary path is always the re-entry path, and the thread it re-enters may be one an interrupted run left in the middle.

Shape 1: stream the graph inline

@app.post("/inline")async def inline(thread: str):   # demo only - derive thread from the session    async def gen():        cfg = {"configurable": {"thread_id": thread}}        async for c in G.astream({"steps": []}, cfg, stream_mode="updates"):            yield f"data: {json.dumps(list(c.keys()))}\n\n"    return StreamingResponse(gen(), media_type="text/event-stream")

Shape 2: decouple with a background task

This is the pattern a FastAPI collaborator recommends in discussion #13349 for continuing work after a client disconnect, since Starlette offers no way to suppress the cancellation. It is copied from there a lot.

_BG: set[asyncio.Task] = set()async def _drive(thread: str, q: asyncio.Queue) -> None:    cfg = {"configurable": {"thread_id": thread}}    outcome = ("done", None)    try:        async for chunk in G.astream({"steps": []}, cfg, stream_mode="updates"):            # NEVER `await q.put(...)` here. The premise of this shape is that the            # consumer leaves; a bounded queue would then block the producer forever            # and stall the run mid-graph, holding a checkpointer connection.            try:                q.put_nowait(("chunk", list(chunk.keys())))            except asyncio.QueueFull:                pass                       # client gone or slow; the run is the point    except asyncio.CancelledError:        # CancelledError is a BaseException, so it skips the arm below. Without        # this, a cancelled run closes the client's stream exactly like a        # successful one.        outcome = ("error", "cancelled")        raise    except Exception as exc:        outcome = ("error", repr(exc))    finally:        # The outcome must never be the thing that gets dropped. A slow-but-present        # client can fill the queue; if the sentinel is dropped then gen() waits on        # q.get() forever and the connection is held for good. Evict a chunk instead.        while True:            try:                q.put_nowait(outcome)                break            except asyncio.QueueFull:                try:                    q.get_nowait()          # drop the oldest chunk, never the outcome                except asyncio.QueueEmpty:                    break@app.post("/background")async def background(thread: str):  # demo only - derive thread from the session    q: asyncio.Queue = asyncio.Queue(maxsize=100)    t = asyncio.create_task(_drive(thread, q))    _BG.add(t)                     # keep a strong ref or the task can be GC'd    t.add_done_callback(_BG.discard)    async def gen():        while True:            kind, payload = await q.get()            if kind == "chunk":                yield f"data: {json.dumps(payload)}\n\n"                continue            if kind == "error":        # do not let a crash look like a clean close                yield f"event: error\ndata: {json.dumps(payload)}\n\n"            break    return StreamingResponse(gen(), media_type="text/event-stream")

Three details, each of which I got wrong first time round.

The strong reference matters: asyncio.create_task returns a task the loop holds only weakly, and CPython's docs warn it can be garbage-collected mid-execution. The copies I have read discard the return value.

The queue bound matters more, and in the opposite direction to the obvious one. Unbounded leaks memory for the life of an abandoned run; bounded with a blocking put is worse, because the consumer is expected to leave and the producer then blocks forever. put_nowait with a dropped chunk keeps the run moving, and so does evicting the oldest, which is what the sentinel path needs. Choosing what to drop is part of choosing this shape.

And the sentinel carries the outcome. A bare None on both paths means a crashed run terminates the client's stream exactly like a successful one, which is this article's own complaint reproduced inside its recommended code.

Shape 3: shield the run from cancellation

@app.post("/shielded")async def shielded(thread: str):  # demo only - derive thread from the session    async def gen():        cfg = {"configurable": {"thread_id": thread}}        agen = G.astream({"steps": []}, cfg, stream_mode="updates")        while True:            try:                c = await asyncio.shield(agen.__anext__())            except StopAsyncIteration:                break            yield f"data: {json.dumps(list(c.keys()))}\n\n"    return StreamingResponse(gen(), media_type="text/event-stream")

What each one actually does

The client reads two chunks and closes the connection. Measured on a real uvicorn server over a real socket, sampled at 0.3s and again at 4.3s after the disconnect:

Read the middle row again. The decoupling pattern finishes the run, writes a complete terminal transcript, and marks the thread done, for a session the user left before the second chunk. The notify node fired its side effect for a reader who was already gone, and nothing in the record distinguishes that from a delivered notification.

Honestly, those are two outcome classes, not three. shielded is a degenerate inline that buys exactly one superstep, because asyncio.shield guards a single await while a LangGraph run is a sequence of them. When the request task is cancelled, gen() is closed, which tears down the underlying async generator; shielding each step in a loop does not rescue it. Practitioners hit this directly, and there is an open LangChain forum thread on anyio.CancelScope(shield=True) failing inside a LangGraph node.

The real variable is binary: does the object driving the graph outlive the request's scope? Note that this is about ownership, not parentage. asyncio.shield wraps its argument in ensure_future, so the shielded await is already a top-level task rather than a child of the request, and it still dies with the request because the async generator it is pulling from belongs to gen().

The shape I did not measure, which is probably the one you should use

All three shapes above run the graph inside the API worker. Plenty of production teams do something else entirely: the handler enqueues the run to a durable queue or a separate worker service, and the HTTP handler only subscribes to a channel for streaming. Under that shape a disconnect has no effect on the run, the run survives a worker crash too, and a reconnecting client can resume the stream. It is better than all three of mine on every axis measured here, and it is broadly what LangGraph Platform does.

I did not measure it, because the scope of this article is the single-service FastAPI deployment that most teams actually start with, and the three shapes above are what that codebase contains.

That paragraph reads like an escape hatch. It is not one. Going out-of-process moves the trigger and leaves the hazard. A redelivered queue message re-enters the graph the same way a returning user does, so everything below about idempotency keys still applies, and now the trigger is your broker's at-least-once delivery instead of a closed laptop. Stream resumption also needs a replayable channel with offsets; plain pub/sub gives a reconnecting client a gap, not a resume.

One more variant, though it is not a drop-in for any handler above. A non-streaming endpoint defined with def and calling G.invoke(...) runs in Starlette's threadpool, which the event loop cannot cancel, so it completes on disconnect with no create_task anywhere. You cannot get there by changing one keyword on the handlers above, since their bodies contain await and async for. It is not a handler-level change at all: every node has to be sync too, because the sync runner cannot invoke a coroutine node. In practice that means a second graph, not a second endpoint. You also give up streaming entirely and occupy one of anyio's 40 worker threads for the length of the run.

LangGraph durability modes compared: sync vs async vs exit

The obvious next move is to blame the checkpointer configuration, so I measured all three modes.

LangGraph 1.x exposes three. From the shipped source in langgraph/pregel/main.py, the default resolves to "async":

durability = config.get(CONF, {}).get(CONFIG_KEY_DURABILITY, "async")

The legacy checkpoint_during=False flag maps onto "exit", so older code can be sitting on that setting without anyone having chosen it recently.

Same disconnect, same graph, inline handler throughout, all three modes:

The side-effect column never moves. In all three modes plan and charge executed, and charge appended its line to side_effects.log. In your service, that line is a POST to a payment provider.

Only the record changes. At durability="exit" the run executed both nodes and left zero checkpoints, so the runtime holds no evidence the thread ever ran. The user's next message then completes cleanly and produces a thread whose final state describes exactly one well-formed booking. Note that the exit row's empty next means "no checkpoint exists", not "reached a terminal state", and nothing downstream can tell those apart.

So durability controls bookkeeping. It sets how much evidence an interrupted run leaves behind and has no say over what that run already did. If you reached for durability="sync" because it sounded safest, you bought a better audit trail and nothing else, unless you also restructure the graph so the effect sits alone in a node, which is fix 5 below.

One caveat on my own table. The disconnect lands at the start of notify, about 0.6s in, milliseconds after charge's superstep dispatched its checkpoint write. That is a real window for an in-flight async write to be caught by the cancellation, and the write won in every trial anyway. So this harness did not rule out LangGraph's open, unanswered issue #5672, which reports a run cancelled while an async write is still pending. It just never hit it. A node fast enough to close that gap should make async and sync diverge, and I did not construct that case.

Does this change with AsyncPostgresSaver and multiple uvicorn workers?

Everything so far runs on SQLite in one process, which is not production. So I re-ran it against postgres:16 with AsyncPostgresSaver, under uvicorn --workers 4, four real OS processes each with its own event loop and pool.

import osfrom contextlib import asynccontextmanagerfrom langgraph.checkpoint.postgres.aio import AsyncPostgresSaverfrom psycopg_pool import AsyncConnectionPoolDSN = os.environ["AGENTS_DSN"]     # never hardcode; never commit sslmode=disablePOOL = None@asynccontextmanagerasync def lifespan(app: FastAPI):    global G, POOL    # from_conn_string() opens one AsyncConnection. Every checkpoint operation in    # this worker would then serialise on it. Use a pool.    async with AsyncConnectionPool(        DSN, min_size=2, max_size=10, open=False,        # from_conn_string() sets these for you; constructing a pool yourself does not.        # prepare_threshold=0 in particular is required behind pgbouncer.        kwargs={"autocommit": True, "prepare_threshold": 0},    ) as pool:        POOL = pool        G = builder.compile(checkpointer=AsyncPostgresSaver(pool))        yieldapp = FastAPI(lifespan=lifespan)

Run AsyncPostgresSaver(pool).setup() once as a deployment migration rather than at startup. With four workers it races across processes on first boot, and it has no business in the request path.

Keeping the connection open for the app's lifetime is the part people get wrong. Closing it at the end of startup is a failure I have hit and seen reported repeatedly, and it shows up later under load rather than at boot.

The inline and background results reproduce exactly, down to the next values. This is not a SQLite artifact and not a single-process artifact.

One disclosure about that sentence: I ran the measurements with AsyncPostgresSaver.from_conn_string(DSN), which opens a single connection. The pool form above is what you should deploy, because a single connection serialises every checkpoint operation in the worker, but the numbers below were produced by the from_conn_string variant.

Multi-worker adds one failure mode a single process cannot show you. A disconnect leaves the thread mid-run. Then the user retries, or double-clicks, or the client reconnects on its own. Two requests now arrive for the same thread_id, and with four workers they land in different processes that know nothing about each other.

Diagrid raised this in the abstract, noting LangGraph has no built-in coordination if two processes resume the same thread at once. I could not find a published measurement, so here is one.

The protocol, since the counts below depend on it: one request to the inline handler, disconnected after the second chunk, leaving the thread at next=['notify'] with one charge already executed. Then two POSTs to that same thread_id, issued concurrently against uvicorn --workers 4. Counts come from parsing side_effects.log. Ten trials:

My first instinct was that a write had been lost. That was wrong, and checking it changed the finding.

On a three-charge trial I dumped the raw rows. There were 16 checkpoints, and one parent had two children: a fork, not a lost write. Two workers each read the same parent and each wrote a child, and both branches persisted intact. aget_state() returns the head of one of them.

But the third charge is not recoverable from the other branch either. Walking every checkpoint in the thread, the highest charge count in any single checkpoint's state was 2, while 3 executed. Each branch inherits the abandoned run's charge from the shared parent and then adds its own, so every branch tops out at two.

The checkpoint tree branches, and effects accumulate along every path. What executed is the sum over every edge in the tree, one execution per checkpoint transition, siblings included. What any single checkpoint records is one root-to-leaf path, so the head of a branch under-reports by exactly the effects that ran on its siblings.

Worth being clear that no write was lost, and that this is structural rather than lucky. The Postgres saver keys checkpoints on (thread_id, checkpoint_ns, checkpoint_id) with a fresh id per checkpoint, so two concurrent writers cannot collide. Forking is the only outcome the schema permits. Last-writer-wins is not on the menu.

What I cannot give you is a tidy rule for when it is two and when it is three, and the reason is worth more than the rule would have been.

The obvious model says it should always be three: the abandoned run charged once, each of the two requests re-enters from START, so each charges once. That model is testable, so I instrumented the server to record per-request outcomes, since a StreamingResponse hides a mid-stream failure from the client. Both requests completed, every trial. The count still came out two in five trials and three in five.

Then I counted every node rather than just charge:

Every path through this graph runs plan before charge. So for any collection of runs that each start at START, plan executions must be greater than or equal to charge executions. Two trials violate that. At least one execution began in the middle of the graph, which means it resumed from a checkpoint the other in-flight request had just written rather than starting its own turn.

That is as far as the measurement takes me. Every node's execution count is independently nondeterministic, and the counts do not correspond to any whole number of complete runs. I am not going to offer a mechanism for the exact interleaving, because I have counts rather than a trace of it. What the counts do establish is narrower and enough: two requests on one thread_id do not resolve into "both ran" or "one won". They interleave into a state no single-request run can produce.

That is why the idempotency key below is not optional. It is the only mechanism here that survives two workers forking, because it moves deduplication to the one component that sees both attempts. Nothing upstream will warn you: StreamingResponse writes 200 before the generator produces a byte.

Handler-Shaped Durability

The definition, tight enough to quote in a review comment:

Handler-Shaped Durability - in a service that runs a durable runtime in-process behind HTTP, the structural shape of the request handler determines the runtime's interruption semantics. Cancellation authority is delegated to the transport layer by default, and the handler's shape decides whether that delegation is accepted, which is why an interrupted run stops, completes, or half-completes for reasons no checkpointer setting can see.

That is scoped to self-hosted graphs inside your own ASGI app, and the scope matters, because LangGraph Platform proves the point from the other side. Its runs API exposes on_disconnect, taking cancel or continue. LangChain had to lift this decision out of the handler and make it a first-class run parameter, which is the same decision this article says your handler shape is making silently. If you self-host, you do not have that parameter, and the decision reverts to whoever wrote the handler.

Which gives you a review question that mostly does not get asked. When you read a FastAPI handler driving an agent, you are reading a durability policy. asyncio.create_task is not a concurrency detail there; it decides that abandoned runs will complete and be recorded as successes. Nobody writes that decision down, because it does not look like one. You can configure a checkpointer perfectly and still have the question answered for you, three files away, by someone fixing a streaming bug.

Where request lifetime and thread lifetime diverge

flowchart TD
    A["Client sends message"] --> B["FastAPI handler starts"]
    B --> C["graph.astream() begins"]
    C --> D["plan executed"]
    D --> E["charge executed<br/>(real-world effect)"]
    E --> F{"Client disconnects"}

    F -->|"inline<br/>CancelledError propagates"| G["Run STOPS<br/>next=['notify']<br/>abandoned, visible in checkpoints"]
    F -->|"background<br/>task detached"| H["Run COMPLETES<br/>next=[]<br/>looks like success"]
    F -->|"shielded<br/>one await protected"| I["Run PARTIAL<br/>next=['finish']<br/>abandoned, one superstep later"]

    H --> H2["No local evidence of abandonment<br/>reconciliation has nothing to find"]
    G --> J["User's next message<br/>runs graph from START"]
    I --> J
    J --> K["charge runs a SECOND time"]

    style A fill:#4A90E2,color:#FFFFFF
    style B fill:#4A90E2,color:#FFFFFF
    style C fill:#4A90E2,color:#FFFFFF
    style D fill:#98D8C8,color:#2C2C2A
    style E fill:#FFD93D,color:#2C2C2A
    style F fill:#7B68EE,color:#FFFFFF
    style G fill:#FFA07A,color:#2C2C2A
    style H fill:#E74C3C,color:#FFFFFF
    style H2 fill:#E74C3C,color:#FFFFFF
    style I fill:#FFA07A,color:#2C2C2A
    style J fill:#95A5A6,color:#FFFFFF
    style K fill:#C2185B,color:#FFFFFF

Node fills reflect default durability. At durability="exit" nothing in the plan and charge rows is committed at all, though both still execute.

How to handle client disconnects in a FastAPI + LangGraph service

The fix is not a cleverer in-process handler shape. All three are defensible, and they need to be chosen rather than inherited.

1. Put an idempotency key on every effect, scoped to the step

This is the only change that touches the effect. Everything else touches the record.

class S(TypedDict):    steps: Annotated[list, lambda a, b: a + b]    booking_id: str          # supplied by the caller, NOT minted inside the graph    amount: int              # same: arrives with the request    charge_id: strasync def charge(state: S, config) -> dict:    thread = config["configurable"]["thread_id"]    # Stable across replays because booking_id came in with the request.    key = f"{thread}:{state['booking_id']}:charge"    # Call unconditionally. Do not read-then-write: two forked workers both read    # "nothing found" and both charge.    try:        result = await payments.charge(amount=state["amount"], idempotency_key=key)    except payments.IdempotencyKeyInUse:        # A completed prior attempt replays the original response. A concurrent        # one - the interleaving measured above - gets 409 idempotency_key_in_use        # instead (Stripe's semantics; check your provider's). Retry the identical        # call with the same key until the original settles and its response is        # replayed. You cannot look a charge up by idempotency key, so this is a        # retry, not a lookup. Never mint a new key; that is how you charge twice.        result = await _retry_same_call(key, state["amount"])    return {"steps": ["charge"], "charge_id": result.id}

The handlers at the top of this article send {"steps": []}. This fix supersedes that: the call becomes G.astream({"steps": [], "booking_id": body.booking_id, "amount": body.amount}, cfg, ...), with both values arriving in the request body.

Two details decide whether this works. Every key input must survive a replay, so booking_id arrives with the user's request and is never minted by an upstream node. If plan generated it, the next message would mint a fresh one, the key would differ, and the card would be charged twice. And the state schema has to declare charge_id. An undeclared key is filtered out of the write rather than rejected: I ran it, and a node returning {"steps": [...], "charge_id": "ch_123"} against a schema without charge_id completes with no exception and no charge_id in the final state. The provider's charge id disappears between the node returning it and the checkpoint recording it, silently. That is this article's own failure mode in miniature.

2. Serialise concurrent requests on one thread

The measured fork happens because nothing stops two workers entering the same thread. A per-thread lock is the only remedy that prevents it rather than cleaning up after it.

# Borrowing from the checkpointer's pool deadlocks at max_size: a lock is held for# the whole run, so N runs hold N saver connections and each still needs one more# to write its next checkpoint. It is the bounded-queue mistake, one layer out.LOCK_POOL = AsyncConnectionPool(DSN, min_size=1, max_size=20, open=False)@asynccontextmanagerasync def thread_lock(pool, thread_id: str):    """Transaction-scoped advisory lock, held for the life of the run."""    async with pool.connection() as conn, conn.transaction():        await conn.execute("SET LOCAL lock_timeout = '5s'")   # else it waits forever        # xact, not session. Postgres releases this on commit OR rollback, so it        # survives the request task being cancelled. A session-scoped        # pg_advisory_lock() would need an explicit unlock in a `finally` - and an        # explicit unlock in a `finally`. Teardown awaits are not guaranteed under        # cancellation. A second cancel while the unlock is in flight leaves the lock        # held on a connection that goes straight back to the pool, and the next        # borrower owns a lock on someone else's thread. An xact lock does not depend        # on your cleanup running at all: Postgres releases it when the transaction        # ends, including when the backend's connection simply drops.        await conn.execute(            "SELECT pg_advisory_xact_lock(hashtextextended(%s, 0))", (thread_id,)        )        yield

hashtextextended rather than hashtext: the latter returns 32 bits, so collision probability is already around 40% at 65,000 distinct threads and passes 50% by roughly 77,000. Two unrelated conversations then serialise against each other for no visible reason. Postgres also makes no cross-version guarantee about hash stability, so a rolling major upgrade can leave two backends hashing one thread_id to different keys, which is an argument for a lease table over an advisory lock if you upgrade in place.

Wrap the whole run with it: async with thread_lock(LOCK_POOL, thread): async for chunk in G.astream(...). That means one lock connection per in-flight run, for the length of the run, which is your real sizing constraint.

The lock_timeout makes contention observable: a second request on a busy thread raises LockNotAvailable in five seconds instead of hanging, and that is what you return a 409 on.

It costs you two things. A transaction held open for the length of an LLM-driven run is an idle in transaction connection, which blocks vacuum and gets killed outright by any sane idle_in_transaction_session_timeout. And a worker that dies holding the lock only releases it when its connection actually drops. For long runs the better shape is a short-lived lease row (thread_id, owner, expires_at) refreshed by heartbeat, which is also exactly the active set that the reconciliation sweep below already needs.

3. Decide the disconnect policy explicitly, in one place

from enum import Enumclass OnDisconnect(str, Enum):    ABANDON = "abandon"      # let cancellation propagate; reconcile later    COMPLETE = "complete"    # detach and finish; the user will not see itDISCONNECT_POLICY = OnDisconnect.ABANDON@app.post("/policy")async def policy(thread: str):    # demo only - derive thread from the session    q: asyncio.Queue = asyncio.Queue(maxsize=100)    t = asyncio.create_task(_drive(thread, q))    _BG.add(t)    t.add_done_callback(_BG.discard)    async def gen():        try:            while True:                kind, payload = await q.get()                if kind == "chunk":                    yield f"data: {json.dumps(payload)}\n\n"                    continue                if kind == "error":                    yield f"event: error\ndata: {json.dumps(payload)}\n\n"                break        finally:            # The generator closing is the disconnect signal. This one line is            # the entire policy.            if DISCONNECT_POLICY is OnDisconnect.ABANDON:                t.cancel()    return StreamingResponse(gen(), media_type="text/event-stream")

The cancellation has to live in gen()'s teardown. The tempting version does not work. You cannot write await run(thread, q) in a streaming handler and rely on await to propagate cancellation: the handler has to return a StreamingResponse before the run finishes, so any helper it awaits would block the response from ever starting. The only way to wire that helper up is create_task, at which point it is top-level, nothing cancels it, and ABANDON silently behaves exactly like COMPLETE. You would ship a policy constant that reads correctly in review and does nothing at runtime.

So the durability decision is one line inside a finally that nobody reads. Which value you pick depends on whether a half-finished run costs you more than an undelivered one. The point is that today the value is implied by whether somebody reached for create_task.

4. Reconcile abandoned threads instead of resuming them blindly

A non-empty next means the thread is not at a terminal state. That covers four different situations: abandoned, paused at an interrupt(), paused at a static interrupt_before/interrupt_after breakpoint, or executing on another worker right now. Only the first is yours to clean up. The static-breakpoint case is the awkward one, because it carries no task.interrupts to filter on, so the check has to come from your own graph configuration rather than from the snapshot. Treating all four as abandoned is how a reconciliation job starts issuing refunds against live bookings.

from datetime import datetime, timezone# Must come from your own compiled graph's config; it is not on the snapshot.STATIC_BREAKPOINTS = frozenset(INTERRUPT_BEFORE) | frozenset(INTERRUPT_AFTER)async def find_abandoned(graph, thread_ids, older_than_s: float, active: set[str]):    now = datetime.now(timezone.utc)    stale = []    for tid in thread_ids:        snap = await graph.aget_state({"configurable": {"thread_id": tid}})        if not snap.next:            continue                                  # terminal        if any(t.interrupts for t in snap.tasks):            continue                                  # waiting on a human        if STATIC_BREAKPOINTS.intersection(snap.next):            continue                                  # static breakpoint: no                                                      # task.interrupts to see        if tid in active:            continue                                  # a live lease says it is running        if not snap.created_at:            continue                                  # Optional[str]; defensive, not reachable above        ts = datetime.fromisoformat(snap.created_at.replace("Z", "+00:00"))        ts = ts if ts.tzinfo else ts.replace(tzinfo=timezone.utc)        if (now - ts).total_seconds() > older_than_s:            stale.append((tid, list(snap.next)))    return stale

active has to come from a real lease or heartbeat. Age alone cannot separate abandoned from in-flight, because a slow tool call looks exactly like a dead run.

Then reconcile against the provider rather than the checkpoint, since the fork result above showed the checkpoint tree cannot represent what actually executed. And note the sweep finds nothing at durability="exit", where no rows exist to find.

5. Separate the effect from the turn, at durability="sync"

Isolating charge in its own node bounds the window between effect and record to one awaited checkpoint write, but only at durability="sync". From the shipped source:

# wait for checkpointif durability_ == "sync":    loop._put_checkpoint_fut.result()

At the default async the write is dispatched and not awaited, so the run proceeds while it is still in flight and restructuring the graph buys nothing. This is the one case where sync earns its cost. It is also the same conclusion the interrupt() double-execution work reaches from a different direction.

Production checklist for FastAPI + LangGraph services

• Every node with an external side effect takes an idempotency key derived from replay-stable state, never from a value minted inside the graph
• Effects call the provider unconditionally and rely on its replay semantics, rather than a local read-then-write that two forked workers both pass
• Concurrent requests on one thread_id are serialised by a lock or lease you own, because the runtime does not coordinate across workers
• The disconnect policy is a named constant that a handler actually reads
• You have opened a client, disconnected it mid-run, and looked at your checkpoint table. Until you do, you do not know which row above you are
• durability is set deliberately, and you know it controls the record rather than the effect
• Nothing depends on asyncio.shield to protect a multi-step run
• Background tasks keep a strong reference and use try/finally to release their consumer
• The abandoned-thread sweep excludes interrupt()-paused and lease-active threads
• Reconciliation treats the provider as authoritative, since the checkpoint tree under-reported in half my trials
• Legacy checkpoint_during=False has been audited, since it maps to durability="exit"
• thread_id is derived from the authenticated session and never accepted from the client
• Load tests include client disconnects, not only completed requests

What idempotency keys still do not fix

Keys make duplicate effects safe. They do not make the first effect correct when it belonged to a plan that never completed, and they cannot roll it back. If your agent charged a card as step two of a five-step booking that died, a key stops the second charge and refunds nobody. That is a compensation problem needing a compensating transaction.

There is also no way to make durability="exit" safe at this seam. It is a reasonable throughput choice for runs with no external effects. Behind an HTTP handler with a node that writes to the world, it removes the only local evidence you would have used to find the damage.

The seam is a decision you are already making

Both layers work exactly as designed. Starlette cancels on disconnect because that is correct for HTTP. LangGraph preserves what completed because that is correct for a durable runtime. The gap is that cancellation authority is delegated to the transport by default, so the shape of a handler that mentions no agent and no checkpointer is what settles your interruption semantics.

Handler-Shaped Durability is already in your codebase. Somebody chose it when they wrote the handler. The only open question is whether they knew that at the time.

How this was measured, and what I did not measure

Stack as listed at the top, on 2026-08-04. Single-process results held identical across three consecutive runs. Handler-shape and durability-mode results were then reproduced on Postgres under uvicorn --workers 4.

The concurrency result is reported as a distribution over n=10 rather than a single number, because it is a race and the outcome varies. The first time I ran it I got the three-charge case, and stopping there would have turned a coin flip into a certainty. With n=10, "5 of 10" carries a 95% confidence interval of roughly 19% to 81%, so read it as "this happens often", not as a rate.

Six limits I have not closed:

The side effect is too easy. record() is a synchronous local file write: atomic, fast, and containing no cancellation point. A real charge is await payments.charge(...), an await that can be cancelled after the request leaves the socket and before the response is read. That produces a fourth outcome this harness cannot show, where the effect's status is unknown rather than merely unrecorded. That case is strictly harder than anything above, and it is the one that most justifies idempotency keys.

The instrument is a shared file. Four processes append to one log, and the counts behind the n=10 table come from parsing it. Each line is written in append mode and sits well under the 8 KiB buffer, so each record reaches the file in one write(). (PIPE_BUF is the usual citation here and it is the wrong one: it governs pipes and FIFOs, not regular files. The guarantee for a regular file is the atomic offset update that O_APPEND gives you.) A per-run identifier would still be better than a timestamp.

No proxy, no load balancer. Everything runs over loopback, and this one cuts against the thesis, so it deserves more than a mention. With proxy_buffering on in nginx, the origin may not learn about a disconnect until it writes past the buffer, which means the inline shape can behave like background purely because of deployment topology. The honest statement is narrower than "the handler decides": the handler decides what happens once cancellation arrives, and the proxy decides whether and when it arrives. Both are policy that nobody wrote down, which is the same complaint one layer up.

Starlette's behaviour is a moving target. Cancel-on-disconnect for StreamingResponse has changed materially across versions. Every table here depends on 0.48.0; check your own pin before trusting the rows.

Out-of-process execution is untested, as is request.is_disconnected() and StreamingResponse(background=BackgroundTask(...)).

Background completion is bounded by the worker's lifetime. Nothing drains _BG at shutdown, so the middle table row holds only until the next deploy, restart, or scale-in.

References

• LangGraph. Run Cancellation Causes Loss of Streamed State Not Yet Persisted as a Checkpoint. Issue #5672, langchain-ai/langgraph. Open, no maintainer response as of 2026-08-04. https://github.com/langchain-ai/langgraph/issues/5672
• LangGraph. durability resolution and the sync wait, langgraph/pregel/main.py at v1.2.9. https://github.com/langchain-ai/langgraph/blob/1.2.9/libs/langgraph/langgraph/pregel/main.py
• LangGraph Platform. How to cancel a run (on_disconnect: cancel / continue, default continue). https://docs.langchain.com/langsmith/cancel-run
• FastAPI. Continue a StreamingResponse function after client disconnect. Discussion #13349. https://github.com/fastapi/fastapi/discussions/13349
• FastAPI. How can I cancel the request handler gracefully when a client disconnects? Discussion #8805. https://github.com/fastapi/fastapi/discussions/8805
• LangChain Forum. anyio.CancelScope(shield=True) not working inside langgraph node. https://forum.langchain.com/t/anyio-cancelscope-shield-true-not-working-inside-langgraph-node/3307
• Diagrid. Why Checkpoints Aren't Durable Execution. https://www.diagrid.io/blog/checkpoints-are-not-durable-execution-why-langgraph-crewai-google-adk-and-others-fall-short-for-production-agent-workflows
• DBOS. The Case for Co-Locating Workflow State with Your Data. https://www.dbos.dev/blog/co-locating-workflow-state-with-your-data
• Ably. Stateful agents, stateless infrastructure: the transport gap AI teams are patching by hand. https://ably.com/blog/stateful-agents-stateless-infrastructure-ai-transport-gap
• Atomix: Timely, Transactional Tool Use for Reliable Agentic Workflows. arXiv:2602.14849. https://arxiv.org/html/2602.14849
• Crab: A Semantics-Aware Checkpoint/Restore Runtime for Agent Sandboxes. arXiv:2604.28138. https://arxiv.org/html/2604.28138v1
• Temporary Authority, Permanent Effects: Commit-Time Authorization for LLM Agents. arXiv:2607.10487. https://arxiv.org/html/2607.10487
• Python. asyncio.create_task - note on holding a strong reference to the returned task. https://docs.python.org/3/library/asyncio-task.html#asyncio.create_task
• Official Docs: LangGraph Durable Execution
• Official Docs: FastAPI Lifespan Events

Related Articles

Agentic AI

• LangGraph Checkpoints Restore Your Limits, Not Just Your State
• State Architecture for Agent Networks: The Resume Is the Dangerous Part
• LangGraph Evals Test the Answer, Not the Thread
• LangGraph Compaction Deletes the View, Not the Record