The paper, presented by Sydney from Lingchain, discusses criteria for choosing multi-agent architectures, though it initially cautions that a multi-agent pattern might not always be necessary, with complex tasks often being better handled by a single agent with well-designed tools. Multi-agent systems are recommended for increasingly complex problems.

The evaluation framework consists of four core criteria:

1. Distributed Development: Assesses the ability of different teams to independently maintain components or agents based on their specialties.
2. Parallelization: Measures whether multiple agents can execute concurrently.
3. Multihop Conversational Support: Determines if the architecture facilitates sequential calls to multiple sub-agents, carrying context from previous interactions.
4. Direct User Interaction: Evaluates if sub-agents can converse directly with the user.

The paper then analyzes four distinct multi-agent architectures:

5. Sub-agents (Supervisor Pattern):
   • Mechanism: A central supervisor agent orchestrates and coordinates sub-agents, treating them as tools. All message routing is mediated by the main agent, which dictates the invocation sequence and timing.
   • Distributed Development: Scores $\mathbf{5/5}$, as it excels in scenarios where different teams manage distinct sub-agents.
   • Parallelization: Scores $\mathbf{5/5}$, as agents inherently support parallel tool calling, enabling concurrent invocation of sub-agents.
   • Multihop Conversational Support: Scores $\mathbf{5/5}$, being easily organized through iterative cycles of model and tool calling loops.
   • Direct User Interaction: Scores low, as direct user interaction with sub-agents is not natively supported or easily achievable, despite technical possibilities via interrupts.

6. Handoffs Pattern:
   • Mechanism: Agents possess the capability to transfer control to other agents using tool-calling mechanisms. An initial user request goes to an entry-point agent, and subsequent agents can pass control among themselves before generating a final response.
   • Distributed Development: Scores low, as it is challenging to develop agents independently when they require specific capabilities to hand off to one another.
   • Parallelization: Scores low, as it is not a primary strength of this architecture.
   • Multihop Conversational Support: Scores $\mathbf{5/5}$, being exceptionally well-suited for enabling rich, multi-turn conversations with context transfer.
   • Direct User Interaction: Scores $\mathbf{5/5}$, noted as potentially the best architecture for allowing direct user interaction with various agents in the chain.

7. Skills (Quasi Multi-agent):
   • Mechanism: This is presented as a quasi-multi-agent architecture where a single core agent remains in control but dynamically loads specialized prompts and knowledge (skills) as needed. This approach is referred to as progressive disclosure for context management.
   • Distributed Development: Scores $\mathbf{5/5}$, as different teams can effectively manage distinct skills based on their expertise.
   • Parallelization: Scores $\mathbf{3/5}$. While multiple skills can be loaded and called in parallel, the process is described as two-step, preventing a perfect score.
   • Multihop Conversational Support: Scores $\mathbf{5/5}$, as the single core agent can readily make multiple sequential calls to skills.
   • Direct User Interaction: Scores $\mathbf{5/5}$, due to the user interacting directly with a single, consistent core agent.

8. Router Architecture:
   • Mechanism: Involves a routing step that classifies input and directs it to one or more specialized agents. The results from these agents are then synthesized into a combined response. Both the router and synthesizer components can be agentic or deterministic.
   • Distributed Development: Scores $\mathbf{3/5}$, as the lack of a standardized protocol (unlike tools or skills) makes distributed development more challenging, though still feasible.
   • Parallelization: Scores $\mathbf{5/5}$, as the router can invoke multiple sub-agents concurrently or one at a time.
   • Multihop Conversational Support: Scores $\mathbf{0/5}$. This architecture is not designed for multihop conversations as sequential invocations of an agent with state are difficult to manage, suggesting wrapping a stateful router in a tool if needed.
   • Direct User Interaction: Scores $\mathbf{3/5}$. The presence of routing and synthesizing layers around agent invocation makes user interaction less direct compared to other architectures.

The paper concludes by emphasizing the importance of starting with a simpler single-agent approach and scaling up to multi-agent patterns only as problem complexity dictates. A summary table consolidating the scores on a five-star scale is provided.