All chapters
  1. Start hereOne agent asks. Another answers. The rest is the contract between them
  2. Foundations · 1One model, one tool, two agents: who is who, and who is responsible
  3. Foundations · 2Message passing: sender, recipient, payload, address and correlation
  4. Foundations · 3Topologies: central orchestrator, peer-to-peer and hierarchy
  5. Mechanics · 4Request and reply, or a task: synchronous versus asynchronous exchange
  6. Mechanics · 5Contracts: messages, artifacts, typed schemas, errors and versions
  7. Mechanics · 6The task lifecycle: submitted, working, input-required, completed, failed, canceled
  8. Mechanics · 7Discovery: Agent Cards, skills and how an agent says what it can do
  9. Protocol · 8A2A: the Agent2Agent protocol, part by part
  10. Protocol · 9A2A versus MCP: delegating to an agent versus calling a tool
  11. Protocol · 10Handoff, delegation or shared state: three ways to work together
  12. Production · 11How updates arrive: streaming, push notifications, callbacks, polling and queues
  13. Production · 12Security: identity, delegated scopes, trust across domains, authorization
  14. Production · 13Reliability: retries, idempotency, order, timeouts and compensation
  15. Production · 14Observability: one trace across three agents
  16. Reference · 15Frameworks: A2A SDKs, LangGraph, AutoGen, CrewAI and plain code
  17. Reference · 16Advanced: consensus claims, coordination failure, cost and isolation
  18. ReferenceGlossary
  19. AboutAbout this site and its sources
Foundations · 3

Topologies: central orchestrator, peer-to-peer and hierarchy

The same workflow routed three ways. Who knows what, how many messages it costs, and what breaks when one agent stops answering.

What you will understand
Three routings
A central orchestrator, a peer chain and a hierarchy, on the same case.
What each costs
Messages sent, who sees which payload, and what stops when one agent is down.
An old debate
Orchestration versus choreography, as microservice teams knew it, applied to agents.

The intake team wants to know, for every quote request, where it is right now: waiting for evidence, waiting for rules, or done. The evidence team wants something else: it wants to send its artifact straight to the underwriting agent, without a stop in the middle, because every extra hop adds a message, a place to fail and another component holding customer data.

Both wishes are reasonable, and they pull in opposite directions. The first asks for one agent in the middle that sees everything. The second asks for agents that pass the work along directly. The team cannot settle it with an opinion about which is "more agentic". It needs to know what each arrangement does to the same case: how many messages, who learns what, and what happens on the day the assessment agent stops answering.

Who may send to whom#

A topology is the answer to one question: who may send messages to whom. For example, "the intake agent may send only to the coordinator, and the coordinator may send to everyone" is a topology. It is a property of the wiring, decided before the first message; each run then follows it.

Three topologies cover most systems:

A central orchestration topology has one agent or program in the middle, the orchestrator, that sends every request and receives every reply. The other agents never talk to each other. In the example the orchestrator is a helper called the coordinator.

A peer-to-peer topology has no agent in the middle. Each agent sends its requests directly to the agent that does the next part of the work. In the example the intake agent asks the assessment agent, and the assessment agent sends its artifact on to the underwriting agent itself.

A hierarchical topology is a tree: a top agent delegates to agents below it, which may delegate further. In the example a supervisor delegates the evidence work to an assessment-lead, which splits it into two requests to the assessment agent and merges the two results.

Plain English. An orchestrator is a project manager who forwards every email. Peer-to-peer is colleagues who pass the folder to the next desk. A hierarchy is a manager who hands the job to a team lead, who splits it among the team.

Diagram: One workflow, three routings · UW-403

Each lane is one topology, read left to right along the work. The orchestrator lane passes every request and reply through the coordinator. The peer lane passes the work from agent to agent. The hierarchy lane delegates through a supervisor and a team lead that splits the evidence work in two.

  1. Central orchestrator
    1. Intake agent: Sends one workflow request and waits for the final artifact.
    2. Coordinator: Asks the assessment agent, then the underwriting agent, and reports back. Sees every payload.
    3. Assessment, underwriting: Each answers the coordinator only.
  2. Peer-to-peer
    1. Intake agent: Delegates the evidence directly. Learns that the evidence is complete; never sees the recommendation.
    2. Assessment agent: Returns the artifact to the intake agent and sends it on to the underwriting agent.
    3. Underwriting agent: Writes the recommendation and stores the row for the underwriter.
  3. Hierarchy
    1. Supervisor: Asks a team lead for the evidence, then the underwriting agent for the recommendation.
    2. Assessment lead: Sends two evidence requests to the assessment agent in parallel and merges the two artifacts.
    3. Assessment agent: Works on task-a1 and task-a2 at once.

The smallest primitive: the same exchange, routed three ways#

Nothing new is needed to build any of these. Each is a composition of the exchange from Messages: one request, replies that carry its correlation id, a task id from the callee. What changes is who holds which request open, and therefore who knows what.

The example builds all three from the same agents. This is the wiring, from run_case.py:build in examples/uw-agents/. Only three lines differ between topologies: whether there is a coordinator, a supervisor with an assessment-lead, or an assessment agent that forwards its work.

if topology == "orchestrator":
    agents["coordinator"] = Relay("coordinator", world, tracer, evidence_to="assessment-agent", **kw)
if topology == "hierarchy":
    # the supervisor's deadline covers the lead's own two attempts of 30 s each
    agents["supervisor"] = Relay("supervisor", world, tracer, evidence_to="assessment-lead",
                                 deadlines={"evidence.request/v1": 90_000}, **kw)
    agents["assessment-lead"] = AssessmentLead("assessment-lead", world, tracer, **kw)
agents["assessment-agent"] = AssessmentAgent("assessment-agent", world, tracer, cards["assessment-agent"],
                                             forward_to="underwriting-agent" if topology == "peer" else None, **kw)

The three helpers (coordinator, supervisor, assessment-lead) are part of the wiring, not agents anyone discovers, so they have no Agent Card.

The insurance example: UW-403 three times#

Case UW-403 is the conflicting roof age: the application says 9 years, the roof-data provider says 22. The expected result is escalate, citing R-ROOF-03, routed to a senior underwriter. Here it is in the peer topology. The agents here are scripted stand-ins, so these results show how the mechanism works, not how good any model is.

/opt/homebrew/bin/python3 run_case.py --case UW-403 --topology peer --dry-run
UW-403 · topology peer · run run-7f3a · agents: scripted stand-ins (no model called)
  time ms  from                 to                 type          state           ids
       50  intake-agent       → assessment-agent   task.request  -               msg-0001 corr msg-0001 task -        try 1
       70  assessment-agent   → intake-agent       task.status   working         msg-0002 corr msg-0001 task task-a1  try 1
      640  assessment-agent   → intake-agent       task.artifact completed       msg-0003 corr msg-0001 task task-a1  try 1
      640  assessment-agent   → underwriting-agent task.request  -               msg-0004 corr msg-0004 task -        try 1
      660  underwriting-agent → assessment-agent   task.status   working         msg-0005 corr msg-0004 task task-u1  try 1
      706  underwriting-agent → assessment-agent   task.artifact completed       msg-0006 corr msg-0004 task task-u1  try 1
messages: 6 sent · 6 delivered · 0 duplicates dropped · 0 stale updates ignored · 0 deadlines fired
tasks (caller's view): evidence completed · recommendation completed
recommendation: escalate · citations R-ROOF-03 · missing - · evidence complete · routed to the senior underwriter · stated_confidence 0.8
tool calls: get_roof_assessment 1 · search_guidance 1 · find_similar_decisions 1 · check_eligibility 1 · record_recommendation 1
rows stored: 1 · human_decision: none (no person has decided yet)
trace: 1 trace_id · 22 spans · last message at 726 ms · run ends at 726 ms (simulated)
expected vs actual: PASS (10 of 10 checks)

Six messages, and the same recommendation the orchestrator topology produces with nine and the hierarchy with fifteen. Read the to column: the recommendation goes back to the assessment agent, which asked for it. The intake agent never sees it. It learns that the evidence is complete, and that is all. The recommendation reaches the underwriter through the stored row in the underwriter cockpit, not through the intake agent.

The three runs side by side (from the example's --all output and summary lines; simulated times):

OrchestratorPeerHierarchy
Messages for UW-4039615
Last message (simulated)786 ms726 ms646 ms
Spans in the trace292243
Who holds the whole picturethe coordinatornobodythe supervisor
Who receives the recommendationcoordinator, then intake agentassessment agentsupervisor, then intake agent
Who receives the submissioncoordinator, assessment, underwritingassessment, underwritingsupervisor, lead, assessment, underwriting

The hierarchy is the fastest here despite the most messages, because the lead runs the two evidence sub-requests in parallel. That is a property of this example's timings, not a law. The general point holds: message count, latency and visibility move independently.

The routing lab#

Pick a topology and a case, then take an agent down. A silent agent receives messages and never answers; the callers' deadlines decide what happens next. The numbers come from real runs of the example.

TopologyMessages, UW-401Sees the recommendationAssessment agent silentUnderwriting agent silentMiddle agent silent
Orchestrator9coordinator, intake agentdeadline, retry, deadline; escalate marked evidence unavailable; 1 rowno recommendation, 0 rows1 message; the workflow stays submitted; nothing times out
Peer6assessment agentdeadline, retry, deadline; escalate marked evidence unavailable; 1 rowno recommendation, 0 rows(no middle agent)
Hierarchy15supervisor, intake agent3 deadlines, 2 retries; escalate marked evidence unavailable; 1 rowno recommendation, 0 rows1 message; the workflow stays submitted; nothing times out

A failure mode: the silent middle#

The last column of that table is the most important result on this page. When the coordinator or the supervisor stops answering, the run sends one message and then nothing happens at all. No deadline fires, because the intake agent's request to the middle agent has no deadline in this example: the intake agent was written to trust the coordinator to report back. The quote request sits in state submitted forever, and no alarm says so.

That is what single point of failure means: one component whose failure stops the whole system. An orchestrator is one by construction. The peer topology has no such point, but it pays elsewhere: nobody can answer "where is this request now?" without collecting the traces of every agent, and a change to the order of the steps means changing the agents themselves.

The lesson is not "avoid orchestrators". It is that every hop needs a deadline, including the hop to the component you trust most. The example sets a 30,000 ms deadline on evidence requests and 10,000 ms on recommendation requests, but none on the intake agent's request to the middle, and the lab shows the consequence.

Production pitfall. An orchestrator that also holds the only copy of the workflow's state turns a crash into lost work. If the middle agent must exist, its state belongs in durable storage, and its callers still need deadlines. Saved execution state is the subject of memory.sammuthu.com.

Orchestration and choreography#

Backend engineers had this debate before agents. In a microservice saga, orchestration means one service tells the others what to do next; choreography means each service reacts to events from the others, with no conductor. A peer chain of agents is choreography: each agent knows only its next step.

Before AI. An orchestrated saga in a Java shop is often a workflow engine calling services in order and running compensations on failure. A choreographed one is services listening on Kafka topics and emitting their own events. The trade-offs carry over unchanged: orchestration gives one place to read and change the flow and one place to fail; choreography removes the central point and spreads the flow across every participant's code.

Two things are different with agents. The steps are less predictable, so the central view an orchestrator gives is worth more when debugging. And an orchestrating agent, if it is itself driven by a model, may make routing decisions that are hard to test. Many systems end up with plain code in the middle and agents at the edges. That choice, what runs next and under which condition, is the subject of orchestrate.sammuthu.com.

What this is not. This page compares who may send to whom. It does not teach how to design the control flow itself: graphs, conditions, scheduling and workflow state belong to orchestrate.sammuthu.com.

How frameworks express a topology#

Libraries make some topologies easier than others. Some build a graph of steps with explicit edges, which suits a central plan. Some give agents a way to transfer control to another agent, which suits a peer chain. Some offer a manager agent that delegates to workers, which is a hierarchy. The A2A protocol, a standard for messages between agents, takes no position: it defines one exchange between a client and a remote agent, and any topology is built by composing those exchanges. Which library makes which topology natural is on Frameworks, from each one's documentation.

Checklist: choosing a topology#

The same idea for a coding agent. A planning agent can orchestrate: it asks the test agent, waits, then asks the review agent with the test results. Or the agents can form a peer chain: the test agent sends its report straight to the review agent. In the first, the planning agent can always say where the pull request stands. In the second, the review starts sooner, and nobody holds the whole picture unless the traces are joined.

Counting messagesWhy the orchestrator costs 9 and the peer chain 6

Each delegation in the example costs three messages: the request, a working status, and the artifact. The peer chain has two delegations (intake to assessment, assessment to underwriting) and the assessment agent's artifact goes back to the intake agent on the same correlation, so six. The orchestrator has three delegations (intake to coordinator, coordinator to assessment, coordinator to underwriting), so nine. The hierarchy has five (intake to supervisor, supervisor to lead, lead to assessment twice, supervisor to underwriting), so fifteen.

So the count is about 3 × (number of delegations) here. In general, a middle agent adds one delegation per participant it fronts. Status messages are the part to watch: a callee that sends progress every few seconds multiplies the count by the length of the task, and at scale the status traffic, not the work, dominates. Updates compares ways to keep that traffic in check, and Advanced puts a cost on every extra message.

Visibility is a data-protection questionWho receives the submission is decided by the topology

In the orchestrator topology the coordinator receives every payload: the submission, the evidence, the recommendation. That is what makes it useful, and it is also a component that must be allowed to hold all of it, protected as such, and audited. The hierarchy adds two such components. The peer chain spreads the data across fewer holders; the intake agent never holds the evidence.

When agents belong to different organizations, this becomes the deciding factor. A central orchestrator run by one party sees everything the others send; a peer chain lets each party see only what its neighbor sends. Security adds the other half: each hop should also carry only the scopes the next agent needs.

Hybrids, and how to tell which you haveMost real systems mix the three

Real systems are rarely pure. A common shape is a plain-code orchestrator for the main path, with a peer exchange inside one step (an agent that consults another agent directly) and a hierarchy where one step fans out. That is fine, as long as each exchange is still a complete one: correlation, task id, deadline, trace context.

A practical way to see the real topology of a running system is to draw it from the traces: every receive span names a sender and a recipient, and the set of pairs that occurs is the topology in practice. A pair that appears in the traces but not in the design document is a hidden dependency. Tracing shows the span tree this relies on.

In one breath#

A topology says who may send to whom. A central orchestrator sees everything and is a single point of failure; a peer chain has no middle and no single view; a hierarchy delegates through layers and multiplies messages. In the example the same case costs 9, 6 or 15 messages and ends with the same recommendation. Whatever the topology, every hop needs a deadline, including the hop to the agent you trust most.

Next: every hop so far was answered in under a second. Real evidence can take minutes and may need a person. Sync and async asks whether the caller should wait on the line or take a ticket.