9.6 Agent frameworks

What this is and why it exists

Frameworks package the loop, the state and the orchestration, and they are genuinely useful — provided you know what they are doing for you. The reason this topic is framed as "know what it is hiding" rather than "learn this one" is that abstractions here have a shelf life measured in months, while the concepts underneath do not. Somebody who learned the loop can pick up any framework in an afternoon; somebody who learned only a framework has to start again when it changes.

The vocabulary

• Orchestration — deciding what runs when, and what state it sees.
• Graph — steps as nodes with explicit edges deciding what follows what.
• State — the data carried between steps.
• Checkpoint — persisted state a run can be resumed from.
• Role-based orchestration — agents given personas and responsibilities.
• Vendor toolkit — a provider's own library for building on their service.
• Lock-in — how much of your code assumes one provider's shape.
• Escape hatch — a supported way to do something the abstraction did not anticipate.

The mental model

Everything a framework provides falls into four categories, and naming them is what lets you evaluate any framework, including ones that do not exist yet.

The loop: think, act, observe, terminate. You can write this, and the previous topic had you do so.

The state: what carries between steps, how it is updated, how much is kept. This is the memory topic, and it is genuine work.

The orchestration: which step runs next, what happens in parallel, how a failure is handled, how a run resumes. This is where frameworks earn most of their value, because it is real distributed-systems engineering.

The integrations: adapters for providers, tools and stores. Convenient, shallow, and the part most likely to be out of date.

Ask of any framework: which of the four am I buying, and could I write that part myself in a day? If the answer is the loop, you are paying an abstraction for forty lines. If it is orchestration with persistence and resumability, that is worth real money.

The graph-based approach makes the structure explicit, and that explicitness is its main advantage. Steps are nodes, edges say what may follow what, and a state object is threaded through and updated by each node. Two things follow. The control flow is visible and reviewable — you can see every path, which is exactly what an autonomous loop denies you, and it makes conditional routing, parallel steps and cycles-with-limits ordinary rather than clever.

And because state is explicit, it can be checkpointed: persisted after each node so a run survives a crash, a restart or a deployment, and can be resumed from where it stopped. That is what makes long-running and human-in-the-loop agents practical — an agent that pauses for approval and continues tomorrow needs its state to exist somewhere durable, and building that yourself is a real project. Checkpointed graph state is the distinctive idea here, and it is the strongest single reason to use a framework in this space.

Role-based orchestration organises around several agents with assigned personas — a researcher, a writer, a critic — passing work between them. It reads naturally, it demonstrates well, and the abstraction fits some problems and obscures others.

The honest assessment is two-sided. It genuinely helps where the work decomposes into distinct skills with different tools and different context, and where the handovers are few and well-defined. It hurts where the roles are theatre: the same model with three different personas, producing longer transcripts, higher cost and behaviour that is hard to attribute when it goes wrong. Debugging is the real cost — when the output is poor, the question is which agent, at which handover, on which turn, and the abstraction is designed to keep you from having to think about that. The multi-agent topic argues that a single agent with good tools frequently outperforms the committee, and that argument applies here.

Vendor toolkits are convenient and they tie your code to one provider's shape. Not merely the interface — the message format, the tool-calling convention, the streaming protocol, the error semantics. Some of that tie is unavoidable and fine. The question is where it sits: a thin layer of your own between your logic and the provider means a change is one file, while provider types spread through your codebase means a change is a rewrite. Write that layer early, when it is twenty lines, and note that the same argument applies to a framework — a framework is also a shape your code assumes.

When to drop the framework is the point of the topic, and there are recognisable signs.

You are fighting the abstraction: spending more time on how to express something in its terms than on the thing itself. You cannot debug beneath it — a run failed and the framework's logging tells you which node, not which request, and you cannot see the actual prompt sent. You need control it does not offer, over context assembly, retries or termination. Its version churn is costing you more than it saves. Or you have realised, as the first topic suggested, that your problem was a chain and the framework was orchestrating a sequence you could have written.

Dropping it is usually less work than it sounds: the loop is short, the state is yours, and orchestration is only large when you need persistence and resumability — which is the case worth keeping a framework for.

The recommendation this topic ends on. Build the loop yourself first, once, on a real task, so you know what is underneath. Then adopt a framework for the parts that are real engineering — persistent state, resumable runs, parallelism — and keep your own thin layer between your logic and both the framework and the provider. Learn the concepts, use the tools, and expect the tools to change, because in this area they will.

What you should now be able to explain or do

Name the four categories a framework provides and evaluate one against them. Say what explicit graphs make visible and why that matters for an autonomous loop. Explain checkpointing and why it is the strongest reason to adopt a framework here. Give both sides of role-based orchestration and name its real cost. Identify where provider lock-in sits and write a thin layer against it. Recognise the five signs that a framework should be dropped. Say why building the loop once first is the prerequisite for all of it.

Check yourself