What happened
OpenAI published a technical breakdown of how it optimized the Codex agent loop using WebSockets and connection-scoped caching within its Responses API. The post explains the specific engineering decisions made to reduce API overhead and improve model response latency for agentic workflows — situations where an AI model repeatedly calls tools, receives results, and issues follow-up requests in a loop.
The core change involves replacing traditional HTTP request-response cycles with persistent WebSocket connections. In a standard HTTP setup, each turn of an agent loop requires a new connection to be established, headers to be transmitted, and context to be re-sent or re-fetched. With WebSockets, the connection stays open across multiple turns, eliminating that per-request overhead.
Alongside the transport change, OpenAI introduced connection-scoped caching. Rather than re-processing shared context — such as system prompts or tool definitions — on every request within a session, the API can cache that information at the connection level and reuse it across turns. According to OpenAI’s post, this combination meaningfully reduced the latency developers would observe when running multi-step agentic tasks through the Responses API.
Why it matters
Agentic workflows are structurally different from single-turn completions. A coding agent like Codex might run dozens of tool calls in a single task — reading files, executing code, checking outputs, and revising its approach. Each round-trip to the model adds latency, and that latency compounds across a long agent loop.
The HTTP overhead that is acceptable for a one-shot query becomes a measurable drag when multiplied across 20 or 30 sequential API calls. WebSockets address this at the transport layer, while connection-scoped caching addresses it at the context layer. Together, they target two distinct sources of per-turn cost.
This also signals how OpenAI is thinking about the Responses API as infrastructure for agents specifically, not just a general-purpose completion endpoint. The engineering choices described — persistent connections, scoped caching — are optimizations that only matter if you expect long-running, multi-turn sessions to be a primary use case.
Who should care
Developers building AI agents on top of OpenAI’s Responses API will find this most directly relevant. If your application involves repeated model calls within a session — whether for coding assistance, automated research, or multi-step task execution — the WebSocket transport and caching behavior described here can reduce the latency your users experience.
Engineering teams evaluating the Responses API against other approaches should also take note. The connection-scoped caching model changes the cost and performance profile of long sessions compared to stateless HTTP calls, which affects how you would architect an agent runtime.
Developers using the older Completions API or building on third-party wrappers that abstract away the transport layer may not see these benefits automatically — the gains are tied to how the Responses API connection is managed.
Practical takeaway
If you are building with the OpenAI Responses API and running multi-turn agent loops, review whether your current integration uses persistent WebSocket connections or re-establishes HTTP connections on each turn. Switching to the WebSocket transport, where supported, is the most direct way to capture the latency improvements OpenAI describes.
For connection-scoped caching to be effective, structure your sessions so that shared context — system prompts, tool schemas, static instructions — is consistent across turns within a connection rather than being modified or re-sent each time. This allows the API to cache and reuse that material rather than reprocessing it.
OpenAI’s Codex agent loop serves as the reference implementation here. Reviewing how that loop is structured, as described in the post, provides a concrete model for how to organize your own agent’s turn management and context handling.