Introduction
Building production-ready AI agents that reliably complete multi-step tasks is harder than it looks. A single poorly‑formed tool call or an off‑topic response can derail an entire workflow. In this post, I share how I designed a robust agentic system using LangGraph and a set of quality gates that catch failures early and enforce retries.
We'll walk through the architecture, the code, and the lessons learned from deploying this pattern in a real‑world research assistant.
The Challenge of Agentic Workflows
When agents are given autonomy to call tools, chain thoughts, and generate final outputs, they inevitably make mistakes:
- Hallucinated facts that are not grounded in retrieved context.
- Tool calls that fail due to malformed parameters or missing dependencies.
- Loops where the agent repeats itself without making progress.
- Outputs that do not meet required formatting or citation standards.
Traditional linear pipelines (prompt → LLM → output) offer no built‑in recovery. Once a mistake is made, the system fails silently or produces low‑quality results. What we need is a stateful, self‑correcting workflow that can detect errors, branch to fallback handlers, and retry with modified inputs.
Why LangGraph?
LangGraph is a library for building stateful, multi‑agent applications with cyclic graphs. It provides exactly the primitives we need:
- State management – a shared state object that persists across nodes.
- Conditional edges – we can route to different nodes based on the state (e.g., pass quality gate → continue, fail → retry).
- Cycles – allowing retry loops and iterative refinement.
- Human‑in‑the‑loop – breaks can be inserted for manual intervention.
I paired LangGraph with CrewAI for task delegation and tool binding, creating a hybrid that gives me graph‑level control while keeping agent definitions clean.
Quality Gates: The Safety Net
A quality gate is a node in the graph that evaluates the output of a previous node against a set of criteria. If the output passes, the workflow continues; if it fails, the graph routes to a correction or retry node.
In my system, I implemented four gates:
- Readability Gate – uses the Flesch Reading Ease score; if below 30, the writer is asked to simplify the report.
- Citation Gate – ensures at least three proper citations are present; if missing, a citation generator is triggered.
- Fact‑Check Gate – compares claims against retrieved documents; if a claim lacks support, the researcher is re‑assigned.
- Completeness Gate – checks that all sections defined in the plan are present; if not, the writer revises.
Each gate returns a score and a diagnostic message, which becomes part of the state and informs the next step.
Architecture Overview
The graph is composed of four main agent nodes (Planner, Researcher, Writer, Translator) and four quality‑gate nodes interleaved. Below is a simplified representation of the flow:
Fact‑check & citationsPass → Writer | Fail → Researcher
After the first draft, a Readability Gate and Completeness Gate run in parallel. If both pass, the report moves to the final step (optional translation). If either fails, the graph loops back to the Writer with a revision prompt. The Writer has a maximum of three retries before the graph raises a flag for human review.
Implementing the Workflow
Here's a condensed version of the graph definition using LangGraph (Python):
from langgraph.graph import StateGraph, END
from langgraph.checkpoint import MemorySaver
class AgentState(TypedDict):
topic: str
plan: str
research: List[str]
draft: str
quality_score: float
attempts: int
final_report: str
builder = StateGraph(AgentState)
builder.add_node("planner", plan_node)
builder.add_node("researcher", research_node)
builder.add_node("writer", write_node)
builder.add_node("quality_gate", quality_gate_node)
builder.add_node("translator", translate_node)
builder.set_entry_point("planner")
builder.add_edge("planner", "researcher")
builder.add_edge("researcher", "quality_gate")
builder.add_conditional_edges(
"quality_gate",
lambda state: "writer" if state["quality_score"] >= 1.8 else "researcher",
{
"writer": "writer",
"researcher": "researcher"
}
)
def should_continue(state):
if state["attempts"] >= 3:
return "translator"
return "writer" if state["quality_score"] < 1.8 else "translator"
builder.add_conditional_edges("writer", should_continue)
builder.add_edge("translator", END)
memory = MemorySaver()
graph = builder.compile(checkpointer=memory)The quality_gate_node calls a separate evaluator that checks citations, readability, and fact‑consistency. The state holds the score and the number of retry attempts, allowing the graph to either loop or exit.
For tool failures, each tool call is wrapped in a retry decorator with exponential backoff, and the graph is designed to continue with partial data if a non‑critical tool fails.
Testing and Evaluation
I built a test harness that runs the graph on a suite of 20 topics, recording:
- Success rate – final report passes all gates after ≤3 attempts.
- Average attempts – number of writer iterations needed.
- Readability score – final Flesch score.
- Citation count – average number of unique sources.
After tuning the gate thresholds, the success rate improved from 62% to 89%, and the average attempts dropped from 2.1 to 1.4. The most common failure mode was the fact‑check gate, which caught hallucinated facts in 23% of initial drafts, but after the retry logic, 94% of those were corrected.
Results and Lessons Learned
The quality‑gate pattern dramatically increased the reliability of the agent system. Key takeaways:
- Explicit gates force clarity. Defining what "good" means upfront (e.g., "must have 3 citations") avoids vague LLM instructions.
- Retries with context work. When the writer receives the gate's diagnostic message (e.g., "too complex, simplify"), the revision is almost always better.
- Graph state is your friend. Storing attempt counts and partial outputs allows the system to learn from failures and avoid repeating the same mistakes.
- Human‑in‑the‑loop is essential. Even with retries, about 5% of cases still needed a manual override—always keep an escape hatch.
I've open‑sourced the core workflow on GitHub for others to experiment with.
Conclusion
Agentic AI workflows are powerful but brittle. By combining LangGraph's stateful graph with a suite of quality gates, you can build systems that self‑correct and produce consistently high‑quality outputs. The pattern scales from simple research assistants to complex multi‑step reasoning engines. If you're building with agents, don't just prompt and pray—gate it, test it, and iterate.
I'd love to hear how you're implementing quality checks in your own agent systems—reach out on Twitter or GitHub.