Typed Outputs, Untyped Routing: The Design Split in Codex Custom Agents

The Two Contracts

Simon Willison noted the addition of subagent and custom agent support to OpenAI’s Codex CLI. The OpenAI Agents SDK powers this: wrap an Agent object as a tool via .as_tool(), give it a name and description, and an orchestrating agent can invoke it when appropriate.

Two contracts govern how custom agents behave in this system. They operate at different levels of formality, and the design difference between them is intentional.

The first contract is the routing contract: how the orchestrating model decides which custom agent to call. This lives entirely in natural language. The tool_description string you pass to .as_tool() is the only signal the orchestrator has when making routing decisions. There is no type system, no schema validation, no static analysis of whether the description accurately reflects what the agent does.

The second contract is the output contract: what the subagent returns to its caller. This can be made precise and enforceable via Pydantic. The Agents SDK accepts an output_type parameter on Agent definitions:

from pydantic import BaseModel
from agents import Agent

class SecurityFinding(BaseModel):
    severity: str  # "critical" | "high" | "medium" | "low"
    vulnerability_type: str
    file_path: str
    line_number: int | None
    description: str

class SecurityReviewOutput(BaseModel):
    findings: list[SecurityFinding]
    files_reviewed: list[str]
    clean: bool

security_reviewer = Agent(
    name="security-reviewer",
    instructions="Review code for OWASP Top 10 vulnerabilities. "
                 "Focus on injection, broken authentication, and access control.",
    model="o4",
    tools=[read_file],
    output_type=SecurityReviewOutput
)

When this agent completes its work, the SDK uses structured output mode to constrain the model’s response to match the schema. The orchestrator receives a Python object. It can access result.findings[0].severity directly, without parsing prose.

The asymmetry is clear: routing is natural language and unverifiable; output structure is typed and enforced.

Why Routing Resists Typing

The routing decision is a semantic match: does the current task description fit what this agent does? A type system cannot answer that question, because type systems operate on syntax, not meaning. Two tool descriptions can be syntactically different and semantically equivalent, or syntactically similar and semantically divergent. Formalizing routing as types would require either enumerating every possible routing case in advance, or defining an ontology that captures the meaning of each agent’s scope in machine-readable terms.

Academic multi-agent systems tried this approach. JADE (Java Agent Development Framework), the dominant framework for multi-agent systems research in the 2000s, used FIPA’s Agent Communication Language for inter-agent messaging. Agents registered capabilities using formal ontology classes. The middleware matched messages to capabilities based on semantic relationships in a predefined ontology. The DFService (Directory Facilitator) maintained a registry of agent capabilities, and agents queried it to discover which agents could handle a given task type.

The formalism worked within closed, carefully scoped research environments. In open domains, it collapsed. Every new capability required updating a shared ontology, which required coordination among all agents that used it. Adding a new agent type could break existing routing if its capability ontology terms overlapped with existing entries. The cost of maintaining the formal routing system grew with the number of agent types, making it impractical for any system expected to evolve over time.

Codex’s approach offloads routing to the language model, which handles semantic matching without requiring explicit ontology definitions. “Analyze for SQL injection vulnerabilities” and “check for injection risks in the provided code” route to the same agent, because the model understands they refer to the same thing. The routing system gains flexibility at the cost of static verifiability. You cannot confirm at definition time whether two custom agents’ descriptions overlap or conflict; that only surfaces when you observe the orchestrator’s actual routing decisions in practice.

Why Outputs Benefit from Types

The output direction has the opposite structure. Once a subagent completes work and returns a result, the orchestrator needs to use that result. If the return value is prose, the orchestrator must parse it. Parse errors compound: the subagent interprets its task in natural language, formulates a response in natural language, and the orchestrator interprets that response in natural language. Three inference steps, three places where semantic drift can occur.

Structured outputs collapse this. The model is constrained to produce output that fits the Pydantic schema, and the orchestrator receives a typed object. The schema validation happens at the model level: if the model would produce output that does not fit the schema, structured output mode forces correction before the response is returned, rather than passing malformed prose that the caller then misparses.

The practical improvement shows up most clearly in partial failure cases. When an untyped subagent fails partway through its work, it may return something like “I found some issues but could not complete the full analysis.” The orchestrator has to detect that this is an incomplete result, extract whatever was found, and decide how to proceed. When a typed subagent encounters the same situation, the partial result is still a valid SecurityReviewOutput instance: findings contains what was actually found, and clean is False. The error handling is structural rather than semantic.

The REST API Parallel

This tension between typed and untyped contracts appeared in REST API design a decade ago, and the resolution there is instructive. When REST APIs began replacing RPC-style interfaces, request routing was documented loosely: URL patterns and query parameters described in prose documentation, README files, and wiki pages. Response bodies were the immediate target for formalization, through JSON Schema and eventually the OpenAPI specification. Client libraries needed to parse responses; parsing required schemas; schemas required formal definitions.

OpenAPI eventually covered request parameters as well, but response schema formalization drove adoption. Developers needed contract testing for API responses because their code depended on the shape of what came back. Request routing was flexible and recoverable; response parsing errors were not. A client that sent a malformed request got a 400 and could adjust. A client that received an unexpected response shape failed in unpredictable ways.

Custom agent output schemas occupy the same structural position. A client’s orchestration logic depends on the shape of what a subagent returns. The routing description is more forgiving, because if the orchestrator calls the wrong agent or passes malformed input, it can observe the unexpected result and reason about what happened. If the subagent returns prose when structured data was expected, no amount of orchestrator reasoning reliably extracts the right information.

Implications for Building Custom Agents

This split clarifies where to spend engineering effort when defining custom agents.

The output schema deserves careful upfront design. Once downstream orchestration logic depends on SecurityReviewOutput.findings, changing the field structure is a breaking change. Other agents may check result.clean, branch on finding.severity, or aggregate files_reviewed across multiple subagent calls. Get the schema right initially, because the output type is load-bearing in a way that the routing description is not. Use Pydantic’s validators where the schema needs invariants enforced: if severity should only accept specific values, model that as a Literal type or an Enum, not a bare str.

The routing description functions as living documentation. Observe how the orchestrator invokes the agent in practice, use the Agents SDK trace API to inspect the model’s reasoning when it decides to call or skip an agent, and revise the description accordingly. The description should include triggering conditions alongside capability summaries, because the orchestrator needs to know when to call the tool, not just what it does. But this text can change without breaking any structural dependencies.

Tool access restrictions occupy a third category. They are not a type contract or a routing signal; they are an access control list that determines what resources a subagent can touch. Like the output schema, they should be defined carefully upfront, because they determine the security properties of the whole system. A subagent with broad file-write access is a different attack surface than one restricted to a specific directory, regardless of what its description claims it should do. The InjecAgent benchmark found attack success rates compounding across multi-agent hierarchies, making per-subagent tool restriction the primary structural defense against prompt injection propagation.

What Is Still Missing

The Agents SDK’s current design covers half of what a full contract system would provide. Output contracts are typed and enforceable via output_type. Input contracts are not. The orchestrator calls a custom agent with whatever input it decides to pass, constrained only by the description’s guidance in prose. There is no input schema the SDK enforces at call time, no validation that the orchestrator passed a file path when the subagent expected a file path rather than a raw code block.

Input schema enforcement, analogous to request body validation in OpenAPI, is the natural next layer. Some teams currently define input contracts in the description text and rely on the orchestrator to follow them, which works inconsistently. The tool description post on this blog covers how vague input descriptions produce mis-routed calls; typed input schemas would make those errors fail fast at the SDK level rather than silently produce wrong behavior at runtime.

Codex’s subagent feature, as Simon Willison covered, frames delegation as a mechanism for scaling what a coding agent can tackle. The output/routing asymmetry is not visible at that framing level, but it is where the reliability engineering lives once you are past the tutorial case. An agent system with well-designed output schemas and loosely iterated routing descriptions will behave more consistently than one where both are prose, because the typed half is the half that orchestrators, parsers, and downstream logic depend on structurally. The natural language half is what made the system practical to build; the typed half is what makes it practical to maintain.