Realtime API | OpenAI API

Generating ephemeral API keys

In the beta interface, there were multiple endpoints for generating ephemeral keys for either Realtime sessions or transcription sessions. In the GA interface, there is only one REST API endpoint used to generate keys - POST /v1/realtime/client_secrets.

To create a session and receive a client secret you can use to initialize a WebRTC or WebSocket connection on a client, you can request one like this using the appropriate session configuration:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
const sessionConfig = JSON.stringify({
    session: {
        type: "realtime",
        model: "gpt-realtime",
        audio: {
            output: { voice: "marin" },
        },
    },
});

const response = await fetch("https://api.openai.com/v1/realtime/client_secrets", {
    method: "POST",
    headers: {
        Authorization: `Bearer ${apiKey}`,
        "Content-Type": "application/json",
    },
    body: sessionConfig,
});

const data = await response.json();
console.log(data.value); // e.g. ek_68af296e8e408191a1120ab6383263c2

These tokens can safely be used in client environments like browsers and mobile applications.

New URL for WebRTC SDP data

When initializing a WebRTC session in the browser, the URL for obtaining remote session information via SDP is now /v1/realtime/calls:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
const baseUrl = "https://api.openai.com/v1/realtime/calls";
const model = "gpt-realtime";
const sdpResponse = await fetch(baseUrl, {
    method: "POST",
    body: offer.sdp,
    headers: {
        Authorization: `Bearer YOUR_EPHEMERAL_KEY_HERE`,
        "Content-Type": "application/sdp",
    },
});

const sdp = await sdpResponse.text();
const answer = { type: "answer", sdp };
await pc.setRemoteDescription(answer);

New event names and shapes

When creating or updating a Realtime session in the GA interface, you must now specify a session type, since now the same client event is used to create both speech-to-speech and transcription sessions. The options for the session type are:

• realtime for speech-to-speech
• transcription for realtime audio transcription

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
import WebSocket from "ws";

const url = "wss://api.openai.com/v1/realtime?model=gpt-realtime";
const ws = new WebSocket(url, {
    headers: {
        Authorization: "Bearer " + process.env.OPENAI_API_KEY,
    },
});

ws.on("open", function open() {
    console.log("Connected to server.");

    // Send client events over the WebSocket once connected
    ws.send(
        JSON.stringify({
            type: "session.update",
            session: {
                type: "realtime",
                instructions: "Be extra nice today!",
            },
        })
    );
});

Configuration for input modalities and other properties have moved as well, notably output audio configuration like model voice. Check the API reference for the latest event shapes.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
ws.on("open", function open() {
    ws.send(
        JSON.stringify({
            type: "session.update",
            session: {
                type: "realtime",
                model: "gpt-realtime",
                audio: {
                    output: { voice: "marin" },
                },
            },
        })
    );
});

Finally, some event names have changed to reflect their new position in the event data model:

• response.text.delta → response.output_text.delta
• response.audio.delta → response.output_audio.delta
• response.audio_transcript.delta → response.output_audio_transcript.delta

New conversation item events

For response.output_item, the API has always had both .added and .done events, but for conversation level items the API previously only had .created, which by convention is emitted at the start when the item added.

We have added a .added and .done event to allow better ergonomics for developers when receiving events that need some loading time (such as MCP tool listing or input audio transcriptions if these were to be modeled as items in the future).

Current event shape for conversation items added:

1
2
3
4
5
6
{
    "event_id": "event_1920",
    "type": "conversation.item.created",
    "previous_item_id": "msg_002",
    "item": Item
}

New events to replace the above:

1
2
3
4
5
6
{
    "event_id": "event_1920",
    "type": "conversation.item.added",
    "previous_item_id": "msg_002",
    "item": Item
}

1
2
3
4
5
6
{
    "event_id": "event_1920",
    "type": "conversation.item.done",
    "previous_item_id": "msg_002",
    "item": Item
}

Input and output item changes

All Items

Realtime API sets an object=realtime.item param on all items in the GA interface.

Function Call Output

status : Realtime now accepts a no-op status field for the function call output item param. This aligns with the Responses API implementation.

Message

Assistant Message Content

The type properties of output assistant messages now align with the Responses API:

• type=text → type=output_text (no change to text field name)
• type=audio → type=output_audio (no change to audio field name)