GPT-Live Gets the Interface Right. The Hard Part Is Still State.

OpenAI's GPT-Live is a meaningful change in voice AI architecture. It can listen and speak at the same time, and it can hand deeper work to another model without freezing the conversation. That makes the interface feel less like a walkie-talkie. It does not remove the harder production problem: keeping the system's state correct while audio, tools, and background reasoning all move at once.

That distinction matters. Natural speech is easy to hear in a demo. State errors appear later, usually after the user interrupts, changes the request, or asks a follow-up while the first tool call is still running.

What GPT-Live changes

OpenAI describes GPT-Live as a full duplex model. Instead of waiting for a clean end of turn, it continuously processes input while generating output. The model can decide whether to speak, keep listening, pause, interrupt, or call a tool many times during the exchange.

The company also separated the live conversation from deeper work. GPT-Live handles the immediate audio loop. When a request needs search or more reasoning, it delegates that task to GPT-5.5 and can keep the conversation moving while the background work runs. OpenAI explains the architecture in its GPT-Live announcement.

This split is sensible. A live voice model needs speed and timing. A reasoning model needs room to think. Asking one model invocation to do both jobs forces a tradeoff on every turn.

Older cascaded systems exposed the cost of that tradeoff. Speech moved through transcription, a language model, and text-to-speech in sequence. Turn-based audio models removed some of that delay, but they still treated silence as a boundary. A short pause could look like the end of a thought. Background noise could trigger a response. An interruption could arrive after the model had already committed to the next turn.

Full duplex audio gives the system more information and more chances to react. That is real progress. It also creates more concurrent events for the application to reconcile.

An interruption is a state change

Suppose a user asks a voice agent to move an appointment to Tuesday. The agent starts a calendar lookup, then says it is checking availability. Before the tool returns, the user interrupts: "Actually, make that Thursday afternoon."

The audio layer heard the correction. The rest of the system now has work to do.

The Tuesday lookup may need to be cancelled. If cancellation is not possible, its result must be marked stale. The user's intent record must change to Thursday afternoon. Any speech queued from the first request must be dropped. A second tool call may begin. If the Tuesday result arrives after the Thursday request, the orchestrator must know which event wins.

A model can sound completely natural while getting this sequence wrong. It can acknowledge Thursday, receive the older Tuesday result, and confirm Tuesday with perfect timing and a pleasant voice. The failure is not speech quality. The failure is event ordering.

This is why interruption handling cannot live only inside the audio model. The application needs explicit rules for cancellation, stale results, intent revisions, and queued speech. Those rules need tests.

Delegation creates a shared-state problem

GPT-Live's delegation design is the right direction for difficult voice tasks. It also means two systems may work on the same conversation at once.

The live model manages timing and keeps the user engaged. The background model searches, reasons, or completes a longer task. Both need access to conversation state. Both may produce information that changes what the system should say next.

A shared transcript alone is not enough. The application needs structured state that records the active request, pending tasks, completed tasks, cancelled tasks, tool results, and the latest user correction. Each update needs an identity and an order. Otherwise, the system cannot tell whether a result belongs to the current request or to a request the user already changed.

An append-only event log works well here. Store the user's words, interruption events, model decisions, tool starts, tool results, cancellations, and speech output as separate events. Build the current state from that history. When something goes wrong, the team can replay the sequence instead of guessing from a final transcript.

That replay matters because many voice failures are timing failures. The same words may work nine times and fail once because two events arrived in a different order. A transcript hides that. An event trace shows it.

The production stack around full duplex

A production full duplex voice agent needs at least six systems working together:

1. Continuous audio input that separates the user's voice from noise and from the agent's own output.
2. An interruption policy that decides when to pause, stop, acknowledge, or keep listening.
3. Task delegation that sends slow work away from the live audio loop.
4. Shared conversation state with clear ownership and versioning.
5. Cancellation and recovery paths for stale calls, failed tools, and reconnects.
6. Replayable traces that preserve timing and event order.

None of these replaces the model. They make the model usable under real conditions.

The temptation is to treat a full duplex API as the finished conversation layer. It is closer to a better runtime primitive. The application still decides what an interruption means, which task is current, when speech should stop, and whether a late result is safe to use.

How to evaluate the system

A voice evaluation should test sequences, not polished single turns. Start with ordinary requests, then change the timing and order of events.

Interrupt the agent while it is speaking. Correct the request while a tool is running. Let the older tool return after the correction. Delay a background reasoning result. Drop the audio connection and reconnect. Send a partial tool response. Repeat the same scenario with background noise.

For each run, score more than the final answer. Check whether the agent stopped speaking at the right time, preserved the latest intent, cancelled stale work, ignored late results, and explained recovery without confusing the user.

The event trace should make every failure reproducible. If a team cannot replay the order that caused a bad confirmation, it cannot know whether a fix worked. It can only hope the timing changes next time.

Model evaluations still matter. GPT-Live may improve turn-taking and conversational flow, and OpenAI reports that its evaluators preferred it over Advanced Voice Mode in matched conversations. Production evaluation has a different job. It measures whether the whole application stays correct when the conversation stops following the happy path.

Natural speech is the beginning

GPT-Live moves voice AI toward continuous interaction. That is a better foundation than a chain of rigid turns, especially for interruptions and background work.

The competitive work now moves up a layer. Teams need clear state models, cancellation rules, event ordering, and traces that show what happened. Those systems are less visible than a natural demo, but they decide whether the agent can be trusted after the user changes their mind.

Full duplex voice gets the interaction model closer to human conversation. Production engineering keeps the conversation correct.