ARCHETYPE // Design DecisionArchitecture Opinion• 2026-07-14 • 9 min read

An LLM with fifteen tools and a while-loop is not an agent. It's a liability.

Why I replaced autonomous agent loops with typed deterministic state machines—and the specific production failures that forced me to.

Every AI product failure I've encountered in production has the same root cause: engineers gave an LLM unrestricted tool access and an open-ended execution loop, then were surprised when it routed itself into recursive self-correction spirals that burned $0.40 per request and returned incoherent results 40% of the time. The correct architecture is not a better prompt. It is a typed finite state machine.

During the first production deployment of Gitloom's code analysis pipeline, 40% of repository indexing requests entered recursive tool-calling loops when the LLM encountered ambiguous TypeScript compiler errors. The failure mode was not a bug in the LLM—it was a category error in the architecture: we had given a probabilistic reasoning engine the responsibility of making deterministic control flow decisions.

System Topology Specification — ASCII Flow

Unconstrained Agent Loop vs. Typed LangGraph State Machine

The state machine does not constrain the LLM's reasoning. It constrains the LLM's ability to make routing decisions that the application should own.

UNCONSTRAINED LOOP (What failed in production):
[User Request]
    |
    v
[LLM: "What tools do I need?"] <-------+
    |                                   |
    v (Tool selected by LLM)            |
[Execute Tool] --> [Ambiguous Output] --+  <-- LOOP (no exit condition)
    |
    v (If LLM decides it's done)
[Response]  <-- (40% coherence rate on ambiguous inputs)

TYPED STATE MACHINE (What replaced it — Gitloom v2):
[User Request]
    |
    v
[Schema Validator] -- (Invalid schema) --> [Structured Error: 400]
    |
    v (Valid)
[Planner Node: determine task type]
    |
    +---> [code_analysis] ---> [Retriever: pgvector similarity top-k=8]
    |                                |
    |                         (confidence >= 0.75)
    |                                v
    |                         [Synthesis Node] --> [Structured JSON Response]
    |
    +---> [summary] ---------> [Direct LLM call, no retrieval needed]
    |
    +---> (unknown) ----------> [Human Review Gate: return 202 + callback URL]

The central promise of agent-based AI systems is seductive: give an LLM the tools, give it a goal, and let it figure out the path. In demos, this works because demos use curated, unambiguous inputs. In production, ambiguous inputs are the norm.

When Gitloom v1 launched, our code analysis pipeline was built exactly this way: a single system prompt with fifteen tool definitions (file retrieval, AST parsing, type checking, documentation lookup) and an autonomous `while (!done)` execution loop. Engineers building the demo loved it. It felt magical watching the agent self-direct through a repository.

The first time a user submitted a TypeScript monorepo with circular type references, the agent entered a loop. The LLM called the type-checker tool, received ambiguous output, called it again with different parameters, received more ambiguity, and continued for 11 iterations before the 30-second timeout killed the request. The user got a 504. We got a $0.43 API bill for a failed query.

The failure was not the LLM's fault. LLMs are extraordinarily good at reasoning about language. They are not designed to make deterministic control flow decisions under computational uncertainty. We had given the LLM a job it was not architected to perform.

The correct insight: routing decisions belong to the application, not the model. An LLM should answer 'what does this code do?' — not 'should I call this tool again or is my current context sufficient?'

We rebuilt the pipeline as a typed LangGraph state machine with five nodes. The Planner Node classifies the incoming request into one of three task types (`code_analysis`, `summary`, `unknown`) using a single structured LLM call with a strict JSON schema output. The LLM never again makes a routing decision after this point.

The `code_analysis` path runs a pgvector similarity search against the indexed repository, retrieves the top-8 semantically relevant chunks, and passes them to a Synthesis Node that generates the answer. The LLM at the Synthesis Node has exactly the context it needs — no more retrieval decisions to make.

The `unknown` path routes immediately to a human review gate that returns a 202 Accepted with a callback URL. The user is told their request required human triage. We stopped trying to force the model to answer questions it could not reliably answer. This is a better product decision, not a compromise.

After the rebuild: p99 latency dropped from 14.2 seconds to 3.1 seconds. Token cost per successful query dropped 68%. The incoherence rate on ambiguous inputs dropped to zero — because ambiguous inputs no longer reach the synthesis node.

The opinion I'll defend: any AI system where an LLM is making control flow decisions is not well-designed. Control flow belongs in typed application code. Reasoning belongs in the model. The moment you blur that boundary, you have traded observability, debuggability, and cost predictability for the appearance of autonomy.

All Engineering Notes Explore Case Studies

Initializing System StateSUB-100MS EDGE SLA // INGRESS ROUTING

UNCONSTRAINED LOOP (What failed in production): [User Request] | v [LLM: "What tools do I need?"] <-------+ | | v (Tool selected by LLM) | [Execute Tool] --> [Ambiguous Output] --+ <-- LOOP (no exit condition) | v (If LLM decides it's done) [Response] <-- (40% coherence rate on ambiguous inputs) TYPED STATE MACHINE (What replaced it — Gitloom v2): [User Request] | v [Schema Validator] -- (Invalid schema) --> [Structured Error: 400] | v (Valid) [Planner Node: determine task type] | +---> [code_analysis] ---> [Retriever: pgvector similarity top-k=8] | | | (confidence >= 0.75) | v | [Synthesis Node] --> [Structured JSON Response] | +---> [summary] ---------> [Direct LLM call, no retrieval needed] | +---> (unknown) ----------> [Human Review Gate: return 202 + callback URL]