TRAE-AI编程
OpenAI WebSocket 连接
2026-11-06 12:40 更新
Go使用1.26或更高版本，保存main.go，初始化live-demo模块并安装github.com/openai/openai-go/v3@v3.85.1；Java使用17或更高版本、com.openai:openai-java:4.95.0和LiveConnectionWebsocketExample；Ruby使用3.3或更高版本，安装openai 0.110.0与async-websocket并保存为client.rb。接入音源与播放器后，可运行node client.mjs、python client.py、go run .、ruby client.rb或Java程序。 选择所用 API，查看对应连接步骤和事件。
将服务器连接到 GPT-Live
GPT-Live 音频连接推荐 WebRTC，服务端到服务端集成也包括在内。
同时支持 WebSocket。主 WebSocket 双向传音频和 JSON 事件，项目 API 密钥保存在可信服务器。
本指南介绍向 GPT-Live 流式发送音频。监控或控制现有会话见服务端控制；将推理和工具后端接到 Responses API 见 Responses WebSocket 模式。
认证并启动会话
- 连接不带查询参数的
wss://api.openai.com/v1/live/sessions。使用Authorization: Bearer $OPENAI_API_KEY和示例连接头认证。 - 第一条消息发送
session.start，把模型、对话指令、音频格式、声音和委派配置放在session中。 - 等
session.started后再发送音频或命令；它包含解析后的会话配置和 ID。
示例使用 Marin、24 kHz PCM16，以及带网页搜索的 Responses 后端。保持对话指令简短。后端指令、工具与权限见委派与工具。
使用 SDK 流式发送音频
Node.js 执行 npm install openai ws，保存为 client.mjs。macOS 或 Linux 的 Python 安装 openai[realtime]，保存为 client.py。在服务器设置 OPENAI_API_KEY，使用支持 Live 的 SDK 版本。示例从标准输入读取 24 kHz 单声道原始 PCM16，将相同格式的返回音频写入标准输出，需接入应用采集与播放。日志和转录走标准错误，避免污染音频流。
来源示例 1（javascript）：
程序文字译释241：保持简洁，需要最新信息的请求委派给能搜索网页的后端。
程序文字译释247：最终会话用量。
程序文字译释257：会话已就绪。
程序文字译释258：收尾未完成：没有收到session.closed。
程序文字译释259：连接关闭，未取得最终用量。
// [中文] 示例 1：使用 SDK 流式发送音频；以下保留本语言原始示例。
// [中文] 原代码明确设置的模型值：gpt-live-1、gpt-5.6-luna；各值的作用按其所在配置对象区分。
// [中文] 本段构造的事件类型：session.start、session.input_audio.append、session.close；不要将构造载荷等同于收到成功确认。
// [中文] 沿原示例执行这些方法：ws.socket.on、ws.send、ws.on、ws.close、ws.socket.platformSocket.terminate；异步调用及事件接收顺序保留。
// [中文] 将 Base64 载荷解码成原始字节；字节后续写文件、缓冲或播放的位置按本例处理。
// [中文] 需要发送的原始字节先编码为 Base64；编码不会替代重采样或改变编解码格式。
// [中文] 区分发出 session.close 与收到 session.closed；最终用量以接收事件的处理分支为准。
// [中文] 标准输入是原始 PCM，标准输出是返回音频，日志与转录写标准错误，避免破坏音频流。
// [中文] PCM 保持完整双字节采样，输入结束不会自动结束会话；示例通过 SIGINT 请求正常关闭。
import OpenAI from "openai";
import { LiveWS } from "openai/resources/live/ws";
// stdin and stdout carry raw mono PCM16 audio at 24 kHz, not WAV files.
// Supply microphone bytes continuously and play stdout in the same format.
process.stdin.pause();
const ws = new LiveWS(new OpenAI());
let started = false;
let closing = false;
let finalized = false;
let pendingByte = Buffer.alloc(0);
let closeTimeout;
ws.socket.on("open", () => {
ws.send({
type: "session.start",
event_id: "event_start",
session: {
model: "gpt-live-1",
instructions:
"Be concise. Delegate requests needing current information to the backend, which can search the web.",
audio: {
format: { type: "audio/pcm", rate: 24000 },
output: { voice: "marin" },
},
delegation: {
type: "responses",
responses: {
model: "gpt-5.6-luna",
tools: [{ type: "web_search" }],
tool_choice: "auto",
},
},
},
});
});
process.stdin.on("data", (chunk) => {
if (!started || closing || ws.socket.readyState !== 1) return;
const bytes = Buffer.concat([pendingByte, chunk]);
const completeLength = bytes.length - (bytes.length % 2);
pendingByte = bytes.subarray(completeLength);
if (completeLength) {
ws.send({
type: "session.input_audio.append",
audio: bytes.subarray(0, completeLength).toString("base64"),
});
}
});
// Register the final-event handler before any close command can be sent.
ws.on("event", (event) => {
if (event.type === "session.started") {
started = true;
console.error("Session ready", event.session.id);
process.stdin.resume();
} else if (event.type === "session.output_audio.delta") {
process.stdout.write(Buffer.from(event.delta, "base64"));
} else if (event.type === "session.closed") {
finalized = true;
clearTimeout(closeTimeout);
process.stdin.pause();
console.error("Final session usage", event.usage);
ws.close();
} else {
// Includes transcript deltas and nested response.event usage.
console.error(JSON.stringify(event));
}
});
process.on("SIGINT", () => {
if (closing) return;
if (!started || ws.socket.readyState !== 1) {
ws.socket.platformSocket.terminate();
return;
}
closing = true;
process.stdin.pause();
ws.send({ type: "session.close" });
closeTimeout = setTimeout(() => {
console.error("Incomplete finalization: session.closed was not received");
process.exitCode = 1;
ws.socket.platformSocket.terminate();
}, 15_000);
});
ws.on("error", (error) => {
console.error(error.message);
process.exitCode = 1;
});
ws.socket.on("close", () => {
clearTimeout(closeTimeout);
process.stdin.pause();
if (!finalized) {
console.error("Connection closed without final session usage");
process.exitCode = 1;
}
});来源示例 2（python）：
沿用程序文字译释241。
沿用程序文字译释247。
沿用程序文字译释257。
沿用程序文字译释259。
程序文字译释260：收尾未完成：缺少session.closed事件。
## [中文] 把标准输入的24 kHz单声道PCM16音频持续送入Live连接，把返回音频写到标准输出；日志与转录走标准错误，关闭时等待会话结束。
import asyncio
import base64
import os
import signal
import sys
from openai import AsyncOpenAI
from openai.types.live.session_config_param import SessionConfigParam
async def main() -> None:
# stdin/stdout carry raw mono PCM16 at 24 kHz, not WAV files.
session: SessionConfigParam = {
"model": "gpt-live-1",
"instructions": "Be concise. Delegate requests needing current information to the backend, which can search the web.",
"audio": {
"format": {"type": "audio/pcm", "rate": 24000},
"output": {"voice": "marin"},
},
"delegation": {
"type": "responses",
"responses": {
"model": "gpt-5.6-luna",
"tools": [{"type": "web_search"}],
"tool_choice": "auto",
},
},
}
loop = asyncio.get_running_loop()
chunks: asyncio.Queue[bytes] = asyncio.Queue()
finalized = asyncio.Event()
closing = False
pending = b""
close_task: asyncio.Task[None] | None = None
def read_audio() -> None:
chunk = os.read(sys.stdin.fileno(), 4800)
if chunk:
chunks.put_nowait(chunk)
else:
# EOF does not end a Live session. Use SIGINT to finalize it.
loop.remove_reader(sys.stdin.fileno())
async with AsyncOpenAI() as client:
async with client.live.connect() as connection:
async def send_audio() -> None:
nonlocal pending
while True:
chunk = pending + await chunks.get()
complete = len(chunk) - len(chunk) % 2
pending = chunk[complete:]
if complete and not closing:
await connection.session.input_audio.append(
audio=base64.b64encode(chunk[:complete]).decode("ascii")
)
async def close_session() -> None:
nonlocal closing
closing = True
loop.remove_reader(sys.stdin.fileno())
await connection.session.close()
try:
await asyncio.wait_for(finalized.wait(), timeout=15)
except asyncio.TimeoutError:
print(
"Incomplete finalization: no session.closed event",
file=sys.stderr,
)
await connection.close()
def request_close() -> None:
nonlocal close_task
if close_task is None:
close_task = asyncio.create_task(close_session())
# Start receiving before a close command can be requested.
await connection.session.start(session=session, event_id="event_start")
sender = asyncio.create_task(send_audio())
try:
async for event in connection:
if event.type == "session.started":
print("Session ready", event.session.id, file=sys.stderr)
loop.add_reader(sys.stdin.fileno(), read_audio)
loop.add_signal_handler(signal.SIGINT, request_close)
elif event.type == "session.output_audio.delta":
sys.stdout.buffer.write(base64.b64decode(event.delta))
sys.stdout.buffer.flush()
elif event.type == "session.closed":
print("Final session usage", event.usage, file=sys.stderr)
finalized.set()
break
else:
print(event.model_dump_json(), file=sys.stderr)
finally:
loop.remove_reader(sys.stdin.fileno())
loop.remove_signal_handler(signal.SIGINT)
sender.cancel()
await asyncio.gather(sender, return_exceptions=True)
if close_task is not None:
await close_task
if not finalized.is_set():
raise RuntimeError("Connection closed without final session usage")
if __name__ == "__main__":
asyncio.run(main())
连接音源与播放器后运行 node client.mjs 或 python client.py。出现 Session ready 后，以录音采样率持续提供麦克风流。一次管入整份文件不能模拟实时麦克风。音源 EOF 不会结束对话，向进程发送 SIGINT 才请求正常关闭。
示例只连接音频流；采集、缓冲、播放和必要重采样由应用负责。评估模型前，先用实际设备和网络测试这些部分。
选择音频格式
启动时设置 session.audio.format，输入输出共用且会话内不可修改：
{"type":"audio/pcm","rate":24000}：24 kHz、单声道、有符号 16 位小端 PCM，默认格式。{"type":"audio/pcm","rate":16000}：16 kHz，同样单声道、有符号 16 位小端 PCM。{"type":"audio/pcmu","rate":8000}：8 kHz G.711 μ-law，每采样一个字节。{"type":"audio/pcma","rate":8000}：8 kHz G.711 A-law，每采样一个字节。
原始字节以 Base64 编码，不含 WAV 等容器头。PCM 块必须包含完整 16 位采样，字节数须为偶数。示例把尾部单字节留到下一输入块。除此之外块边界可任意，但流必须连续有序。
实际采样率不同于配置时应重采样，仅修改设置不会转换字节。G.711 应直接转发编解码字节，移除 PCM 两字节对齐逻辑，并让播放器使用同一编解码器。匹配的 G.711 流可不转 PCM 直通。电话接入见电话集成。
收发事件
每个事件作为 JSON 文本消息发送，音频以 Base64 嵌入：
- 发送音频：
session.input_audio.append的audio为 Base64 原始字节，不返回追加确认。 - 接收音频：解码
session.output_audio.delta的delta，按配置格式有序入播放队列。 - 接收转录：把
session.input_transcript.delta和session.output_transcript.delta的delta追加到对应一方。 - 接收后端事件：Responses 委派处理各
response.event封装的内部event。 - 处理错误：从
error读取拒绝命令和会话错误；有error.client_event_id时用于匹配命令。
播放由应用音频队列追踪。GPT-Live 主 WebSocket 输出音频不带时间字段，也没有输出音频完成事件。字幕用转录时间戳组织，任务由后端事件追踪。
GPT-Live 随连续音频自行决定何时聆听和说话。response.create 用于启动或续接被委派后端工作。见委派与工具。
配置运行中的会话
Live 模型、初始对话指令、音频格式、声音与委派模式在启动时固定。session.update 仅更新当前模式支持的设置，省略项保留原值，成功返回含完整解析配置的 session.updated。
用 session.instructions.append 增补指令，用 session.input_audio.mute、session.input_audio.unmute 控制输入。输入静音不取消后端工作，也不停止生成语音。上下文、转录、输入控制和用量见管理会话。
关闭会话
结束时发送 session.close，先注册 session.closed 监听，持续接收直到该事件再释放连接。示例最多等 15 秒，未收到终结事件则报告收尾未完成。
保留最终语音用量和已收到后端用量。时长更新是累计快照，不要相加。最终事件前传输失败或超时，最终用量未确认。完整生命周期见管理会话。
Realtime API：WebSockets 广泛支持实时数据传输，适合服务端到服务端连接 Realtime。浏览器与移动端推荐 WebRTC。
后端直接经 WebSocket 连接 API，可用标准 API 密钥，因为令牌只在安全后端可用。
通过 WebSocket 连接
以下给出多种连接示例。除 WebSocket URL，还需带 OpenAI 密钥的认证头。应用分配安全标识符时，在 OpenAI-Safety-Identifier 发送稳定、保护隐私的用户标识。
浏览器可用 WebRTC 指南中的临时令牌建立 WebSocket，但多数浏览器或移动场景用 WebRTC 更稳健。
Node.js 的 ws 模块：
来源示例 3（javascript）：
程序文字译释224：已连接服务器。
// [中文] 把标准输入的24 kHz单声道PCM16音频持续送入Live连接，把返回音频写到标准输出；日志与转录走标准错误，关闭时等待会话结束。
package main
import (
"context"
"encoding/base64"
"fmt"
"github.com/openai/openai-go/v3"
"github.com/openai/openai-go/v3/live"
"io"
"os"
"os/signal"
"time"
)
func run() error {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
client := openai.NewClient()
connection, err := client.Live.Connect(ctx, live.LiveConnectionOptions{})
if err != nil {
return err
}
defer connection.Abort()
session := live.SessionConfigParam{
Model: "gpt-live-1",
Instructions: openai.String("Be concise. Delegate requests needing current information to the backend, which can search the web."),
Audio: live.SessionConfigAudioParam{Format: live.AudioFormatParamOfAudioPCM(24000), Output: live.SessionConfigAudioOutputParam{Voice: live.SessionConfigAudioOutputVoiceUnionParam{OfString: openai.String("marin")}}},
Delegation: live.SessionConfigDelegationUnionParam{OfResponses: &live.SessionConfigDelegationResponsesParam{Responses: live.ResponsesDelegationConfigParam{
Model: "gpt-5.6-luna", Tools: []live.ResponsesDelegationConfigToolUnionParam{{OfWebSearch: &live.ResponsesDelegationConfigToolWebSearchParam{}}},
ToolChoice: live.ResponsesDelegationConfigToolChoiceUnionParam{OfLiveToolChoiceEnum: openai.String("auto")},
}}},
}
if err := connection.Send(ctx, live.ClientEventUnionParam{OfSessionStart: &live.SessionStartEventParam{Session: session, EventID: openai.String("event_start")}}); err != nil {
return err
}
events := make(chan live.ServerEventUnion)
failures := make(chan error, 2)
go func() {
for {
event, err := connection.Recv(ctx)
if err != nil {
select {
case failures <- err:
case <-ctx.Done():
}
return
}
select {
case events <- event:
case <-ctx.Done():
return
}
}
}()
audio := make(chan []byte)
interrupts := make(chan os.Signal, 1)
signal.Notify(interrupts, os.Interrupt)
defer signal.Stop(interrupts)
timer := time.NewTimer(30 * time.Second)
defer timer.Stop()
started, closing := false, false
for {
select {
case <-timer.C:
return fmt.Errorf("session startup or finalization timed out")
case err := <-failures:
return err
case <-interrupts:
if !started {
return fmt.Errorf("interrupted before session.started")
}
if closing {
continue
}
closing = true
timer.Reset(15 * time.Second)
if err := connection.Send(ctx, live.ClientEventUnionParam{OfSessionClose: &live.SessionCloseEventParam{}}); err != nil {
return err
}
case chunk := <-audio:
if closing {
continue
}
if err := connection.Send(ctx, live.ClientEventUnionParam{OfSessionInputAudioAppend: &live.InputAudioAppendEventParam{Audio: base64.StdEncoding.EncodeToString(chunk)}}); err != nil {
return err
}
case event := <-events:
switch event.Type {
case "error":
return fmt.Errorf("Live error: %s", event.RawJSON())
case "session.started":
if started {
continue
}
started = true
timer.Stop()
fmt.Fprintln(os.Stderr, "Session ready", event.AsSessionStartedEvent().Session.ID)
// stdin/stdout carry raw mono PCM16 at 24 kHz, not WAV files.
go func() {
pending := []byte{}
for {
buffer := make([]byte, 4800)
n, err := os.Stdin.Read(buffer)
pending = append(pending, buffer[:n]...)
complete := len(pending) - len(pending)%2
if complete > 0 {
chunk := append([]byte(nil), pending[:complete]...)
pending = pending[complete:]
select {
case audio <- chunk:
case <-ctx.Done():
return
}
}
if err != nil {
if err != io.EOF {
select {
case failures <- err:
case <-ctx.Done():
}
}
// EOF does not close the session. Press Ctrl+C to finalize it.
return
}
}
}()
case "session.output_audio.delta":
chunk, err := base64.StdEncoding.DecodeString(event.AsOutputAudioDeltaEvent().Delta)
if err != nil {
return err
}
if _, err = os.Stdout.Write(chunk); err != nil {
return err
}
case "session.closed":
if !started {
return fmt.Errorf("session closed before startup")
}
fmt.Fprintln(os.Stderr, "Final session usage", event.AsSessionClosedEvent().Usage)
return nil
default:
fmt.Fprintln(os.Stderr, event.RawJSON())
}
}
}
}
func main() {
if err := run(); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
Python 的 websocket-client：
来源示例 4（python）：
沿用程序文字译释224。
程序文字译释261：收到事件。
// [中文] 把标准输入的24 kHz单声道PCM16音频持续送入Live连接，把返回音频写到标准输出；日志与转录走标准错误，关闭时等待会话结束。
import com.openai.client.okhttp.OkHttpClient;
import com.openai.core.ClientOptions;
import com.openai.helpers.LiveConnection;
import com.openai.models.live.*;
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicBoolean;
public final class LiveConnectionWebsocketExample {
public static void main(String[] args) throws Exception {
var session =
SessionConfig.builder()
.model("gpt-live-1")
.instructions(
"Be concise. Delegate requests needing current information to the backend, which can search the web.")
.audio(
SessionConfig.Audio.builder()
.audioPcmFormat(AudioFormat.AudioPcm.Rate.of(24000L))
.output(SessionConfig.Audio.Output.builder().voice("marin").build())
.build())
.responsesDelegation(
ResponsesDelegationConfig.builder()
.model("gpt-5.6-luna")
.addToolWebSearch()
.toolChoice(ResponsesDelegationConfig.ToolChoice.LiveToolChoiceEnum.AUTO)
.build())
.build();
var closing = new AtomicBoolean();
var finished = new CountDownLatch(1);
var audio = new ArrayBlockingQueue<byte[]>(8);
var failures = new ArrayBlockingQueue<Exception>(1);
try (var http = OkHttpClient.builder().build();
var connection =
LiveConnection.connect(ClientOptions.builder().fromEnv().httpClient(http).build())) {
Thread shutdown =
new Thread(
() -> {
closing.set(true);
try {
if (!finished.await(15, TimeUnit.SECONDS)) {
System.err.println("Incomplete finalization: no session.closed event");
connection.close();
}
} catch (InterruptedException interrupted) {
Thread.currentThread().interrupt();
}
});
Runtime.getRuntime().addShutdownHook(shutdown);
try {
connection.send(
ClientEvent.ofSessionStart(
SessionStartEvent.builder().session(session).eventId("event_start").build()));
boolean started = false, closeSent = false;
long deadline = System.nanoTime() + TimeUnit.SECONDS.toNanos(30);
var incoming = connection.receiveAsync();
while (true) {
if (System.nanoTime() > deadline)
throw new IllegalStateException("Session startup or finalization timed out");
if (!failures.isEmpty()) throw failures.take();
if (closing.get() && started && !closeSent) {
closeSent = true;
deadline = System.nanoTime() + TimeUnit.SECONDS.toNanos(15);
connection.send(ClientEvent.ofSessionClose(SessionCloseEvent.builder().build()));
}
var chunk = audio.poll();
if (chunk != null && !closing.get())
connection.send(
ClientEvent.ofSessionInputAudioAppend(
InputAudioAppendEvent.builder()
.audio(Base64.getEncoder().encodeToString(chunk))
.build()));
ServerEvent event;
try {
event = incoming.get(20, TimeUnit.MILLISECONDS);
} catch (TimeoutException waiting) {
continue;
}
if (event.isErrorEvent()) throw new IllegalStateException(event.toString());
if (event.isSessionStarted() && !started) {
started = true;
deadline = Long.MAX_VALUE;
System.err.println("Session ready " + event.asSessionStarted().session().id());
// stdin/stdout carry raw mono PCM16 at 24 kHz, not WAV files.
Thread reader =
new Thread(
() -> {
try {
int pending = -1;
while (!closing.get()) {
byte[] buffer = new byte[4800];
int offset = pending < 0 ? 0 : 1;
if (offset == 1) buffer[0] = (byte) pending;
int count = System.in.read(buffer, offset, buffer.length - offset);
// EOF does not close the session. Press Ctrl+C to finalize it.
if (count < 0) return;
count += offset;
pending = count % 2 == 0 ? -1 : Byte.toUnsignedInt(buffer[count - 1]);
int complete = count - count % 2;
if (complete > 0) audio.put(Arrays.copyOf(buffer, complete));
}
} catch (Exception error) {
failures.offer(error);
}
});
reader.setDaemon(true);
reader.start();
} else if (event.isSessionOutputAudioDelta()) {
System.out.write(Base64.getDecoder().decode(event.asSessionOutputAudioDelta().delta()));
System.out.flush();
} else if (event.isSessionClosed()) {
if (!started) throw new IllegalStateException("Session closed before startup");
System.err.println("Final session usage " + event.asSessionClosed().usage());
return;
} else {
System.err.println(event);
}
incoming = connection.receiveAsync();
}
} finally {
closing.set(true);
finished.countDown();
try {
Runtime.getRuntime().removeShutdownHook(shutdown);
} catch (IllegalStateException shuttingDown) {
/* The hook is already draining. */
}
}
}
}
}
Ruby OpenAI SDK：先执行 gem install openai async-websocket。
来源示例 5（ruby）：
程序文字译释262：已连接Realtime API，后接主机名。
程序文字译释263：已收到事件，后接类型。
## [中文] 把标准输入的24 kHz单声道PCM16音频持续送入Live连接，把返回音频写到标准输出；日志与转录走标准错误，关闭时等待会话结束。
require "openai"
require "base64"
$stdin.binmode
$stdout.binmode
session = {
model: "gpt-live-1",
instructions: "Be concise. Delegate requests needing current information to the backend, which can search the web.",
audio: {
format: {
type: :"audio/pcm",
rate: 24_000
},
output: { voice: :marin }
},
delegation: {
type: :responses,
responses: {
model: "gpt-5.6-luna",
tools: [{ type: :web_search }],
tool_choice: :auto
}
}
}
client = OpenAI::Client.new
client.live.connect do |connection|
state = {
closing: false,
finalized: false,
started: false
}
threads = {}
mutex = Mutex.new
previous = trap("INT") do
threads[:closer] ||= Thread.new do
mutex.synchronize do
state[:closing] = true
if state[:started]
connection.send_event(type: :"session.close")
else
connection.close
end
end
sleep 15
unless state[:finalized]
warn "Incomplete finalization: no session.closed event"
connection.close
end
end
end
begin
threads[:startup] = Thread.new do
sleep 30
unless state[:started]
warn "Session startup timed out"
connection.close
end
end
connection.send_event(type: :"session.start", session: session, event_id: "event_start")
connection.each do |event|
case event
when OpenAI::Models::Live::ErrorEvent
raise "Live error: #{event.to_json}"
when OpenAI::Models::Live::SessionStartedEvent
next if state[:started]
state[:started] = true
warn "Session ready #{event.session.id}"
# stdin/stdout carry raw mono PCM16 at 24 kHz, not WAV files.
threads[:sender] = Thread.new do
pending = +"".b
loop do
pending << $stdin.readpartial(4800)
complete = pending.bytesize - pending.bytesize % 2
chunk = pending.slice!(0, complete).to_s
mutex.synchronize do
break if state[:closing]
connection.send_event(type: :"session.input_audio.append", audio: Base64.strict_encode64(chunk)) unless chunk.empty?
end
end
rescue EOFError
# EOF does not close the session. Press Ctrl+C to finalize it.
nil
rescue => error
warn "Audio input failed: #{error.class}"
connection.close
end
when OpenAI::Models::Live::OutputAudioDeltaEvent
$stdout.write(Base64.strict_decode64(event.delta))
$stdout.flush
when OpenAI::Models::Live::SessionClosedEvent
raise "Session closed before startup" unless state[:started]
warn "Final session usage #{event.usage}"
state[:finalized] = true
break
else
warn event.to_json
end
end
raise "Connection closed without final session usage" unless state[:finalized]
ensure
trap("INT", previous)
threads[:startup]&.kill
threads[:startup]&.join
threads[:sender]&.kill
threads[:sender]&.join
threads[:closer]&.kill
threads[:closer]&.join
end
end// [中文] 用本语言客户端连接指定WebSocket端点，区分Live与Realtime的凭证、URL和事件；应用继续负责音频收发与会话生命周期。
import WebSocket from "ws";
const url = "wss://api.openai.com/v1/realtime?model=gpt-realtime-2.1";
const ws = new WebSocket(url, {
headers: {
Authorization: "Bearer " + process.env.OPENAI_API_KEY,
"OpenAI-Safety-Identifier": "hashed-user-id",
},
});
let connected = false;
ws.on("message", function incoming(message) {
const event = JSON.parse(message.toString());
if (event.type === "error") throw new Error(event.error.message);
if (event.type === "session.created") {
connected = true;
console.log(`Connected to the Realtime API: ${event.session.id}`);
ws.close(1000);
}
});
ws.on("close", () => {
if (!connected)
throw new Error("Connection closed before the session was created");
});## [中文] 用本语言客户端连接指定WebSocket端点，区分Live与Realtime的凭证、URL和事件；应用继续负责音频收发与会话生命周期。
## example requires websocket-client library:
## pip install websocket-client
import os
import json
import websocket
OPENAI_API_KEY = os.environ["OPENAI_API_KEY"]
url = "wss://api.openai.com/v1/realtime?model=gpt-realtime-2.1"
headers = [
"Authorization: Bearer " + OPENAI_API_KEY,
"OpenAI-Safety-Identifier: hashed-user-id",
]
session_created = False
def on_message(ws, message):
global session_created
data = json.loads(message)
if data["type"] == "session.created":
session_created = True
print("Connected to the Realtime API:", data["session"]["id"])
ws.close()
elif data["type"] == "error":
ws.close()
ws = websocket.WebSocketApp(
url,
header=headers,
on_message=on_message,
)
ws.run_forever()
if not session_created:
raise RuntimeError("Connection closed before the Realtime session was created.")## [中文] 用本语言客户端连接指定WebSocket端点，区分Live与Realtime的凭证、URL和事件；应用继续负责音频收发与会话生命周期。
require "openai"
client = OpenAI::Client.new(
default_headers: { "OpenAI-Safety-Identifier" => "hashed-user-id" }
)
client.realtime.connect(model: "gpt-realtime-2.1") do |connection|
created = connection.find do |event|
raise event.error.message if event.is_a?(OpenAI::Realtime::RealtimeErrorEvent)
next unless event.is_a?(OpenAI::Realtime::SessionCreatedEvent)
puts("Connected to the Realtime API: #{event.session[:id]}")
true
end
raise "Connection closed before the session was created" unless created
end
Java OpenAI SDK：Java 17 或更高，依赖 com.openai:openai-java:4.75.1。
来源示例 6（java）：
沿用程序文字译释224。
// [中文] 示例 6：通过 WebSocket 连接；以下保留本语言原始示例。
// [中文] 原代码明确设置的模型值：gpt-realtime-2.1；各值的作用按其所在配置对象区分。
// [中文] 沿原示例执行这些方法：connection.receive；异步调用及事件接收顺序保留。
import com.openai.client.okhttp.OkHttpClient;
import com.openai.core.ClientOptions;
import com.openai.helpers.RealtimeConnection;
import com.openai.helpers.RealtimeWebSocketOptions;
import com.openai.models.realtime.*;
var http = OkHttpClient.builder().build();
var options =
ClientOptions.builder()
.fromEnv()
.httpClient(http)
.putHeader("OpenAI-Safety-Identifier", "hashed-user-id")
.build();
try (http;
var connection =
RealtimeConnection.connect(
options,
RealtimeWebSocketOptions.builder().model("gpt-realtime-2.1").build())) {
while (true) {
var event = connection.receive();
if (event.error().isPresent())
throw new IllegalStateException(event.error().orElseThrow().toString());
if (event.sessionCreated().isPresent()) {
System.out.println("Connected to server.");
System.out.println(event);
break;
}
}
}// [中文] 用本语言客户端连接指定WebSocket端点，区分Live与Realtime的凭证、URL和事件；应用继续负责音频收发与会话生命周期。
// [中文] 本段为配置或调用片段，需结合外层程序与已建立连接使用；不作为独立完整程序宣称编译通过。
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel()
client := openai.NewClient()
conn, err := client.Realtime.Connect(ctx, realtime.RealtimeConnectionOptions{}, option.WithQuery("model", "gpt-realtime-2.1"), option.WithHeader("OpenAI-Safety-Identifier", "hashed-user-id"))
if err != nil {
return err
}
defer conn.Close()
.NET OpenAI SDK：执行 dotnet add package OpenAI --version 2.14.0。Realtime 使用实验性 SDK 类型，示例包含必要警告指令。
来源示例 7（csharp）：
沿用程序文字译释224。
程序文字译释264：会话创建前连接已关闭。
// [中文] 示例 7：通过 WebSocket 连接；以下保留本语言原始示例。
// [中文] 沿原示例执行这些方法：client.StartConversationSessionAsync、session.ReceiveUpdatesAsync；异步调用及事件接收顺序保留。
using OpenAI.Realtime;
#pragma warning disable OPENAI002
string key = Environment.GetEnvironmentVariable("OPENAI_API_KEY")!;
RealtimeClient client = new(key);
RealtimeSessionClientOptions options = new();
options.Headers["OpenAI-Safety-Identifier"] = "hashed-user-id";
using CancellationTokenSource timeout = new(TimeSpan.FromSeconds(30));
using RealtimeSessionClient session = await client.StartConversationSessionAsync("gpt-realtime-2.1", options, cancellationToken: timeout.Token);
await foreach (RealtimeServerUpdate update in session.ReceiveUpdatesAsync(timeout.Token))
{
if (update is RealtimeServerUpdateError error)
throw new InvalidOperationException(error.Error.Message);
if (update is RealtimeServerUpdateSessionCreated)
{
Console.WriteLine("Connected to server.");
Console.WriteLine(System.ClientModel.Primitives.ModelReaderWriter.Write(update));
return;
}
}
throw new InvalidOperationException("Connection closed before session creation.");浏览器标准 WebSocket：
来源示例 8（javascript）：
沿用程序文字译释224。
// [中文] 用本语言客户端连接指定WebSocket端点，区分Live与Realtime的凭证、URL和事件；应用继续负责音频收发与会话生命周期。
/*
Note that in client-side environments like web browsers, we recommend
using WebRTC instead. It is possible, however, to use the standard
WebSocket interface in browser-like environments like Deno and
Cloudflare Workers.
*/
const ws = new WebSocket(
"wss://api.openai.com/v1/realtime?model=gpt-realtime-2.1",
[
"realtime",
// Use a short-lived token fetched from your application server.
"openai-insecure-api-key." + OPENAI_REALTIME_EPHEMERAL_KEY,
]
);
ws.addEventListener("open", function open() {
console.log("Connected to server.");
});
ws.addEventListener("message", function incoming(event) {
console.log(event.data);
});发送与接收事件
WebSocket 以文本字符串形式收发 JSON 序列化事件。以下 Node.js 原理同样适用于其他库：
来源示例 9（javascript）：
沿用程序文字译释224。
程序文字译释225：今天格外友善一些！
// [中文] 示例 9：发送与接收事件；以下保留本语言原始示例。
// [中文] 本段构造的事件类型：session.update；不要将构造载荷等同于收到成功确认。
// [中文] 沿原示例执行这些方法：ws.on、ws.send；异步调用及事件接收顺序保留。
import WebSocket from "ws";
const url = "wss://api.openai.com/v1/realtime?model=gpt-realtime-2.1";
const ws = new WebSocket(url, {
headers: {
Authorization: "Bearer " + process.env.OPENAI_API_KEY,
"OpenAI-Safety-Identifier": "hashed-user-id",
},
});
ws.on("open", function open() {
console.log("Connected to server.");
// Send client events over the WebSocket once connected
ws.send(
JSON.stringify({
type: "session.update",
session: {
type: "realtime",
instructions: "Be extra nice today!",
},
})
);
});
// Listen for and parse server events
ws.on("message", function incoming(message) {
console.log(JSON.parse(message.toString()));
});WebSocket 可能是实时模型最底层的交互接口，音频 Base64 编码块的发送与处理均由应用负责。收发音频见 Realtime 对话指南。
以上内容是否对您有帮助：
更多建议：