TRAE-AI编程
OpenAI WebRTC 连接
2026-11-06 12:40 更新
Go使用1.26或更高版本：保存为main.go，执行go mod init live-demo，再安装github.com/openai/openai-go/v3@v3.85.1，使用go run .运行。Java使用17或更高版本与com.openai:openai-java:4.95.0，运行LiveConnectionWebrtcExample。Ruby使用3.3或更高版本，安装openai 0.110.0与webrick，保存为server.rb并运行ruby server.rb。各语言服务器变体只运行一个。 选择所用 API，查看对应连接步骤和事件。
将浏览器连接到 GPT-Live
GPT-Live 在浏览器、移动端及服务端的音频连接推荐使用 WebRTC。本指南展示浏览器示例：麦克风与生成语音经协商媒体轨道传输，数据通道承载转录、会话更新和委派工作的 JSON 事件。
浏览器创建 SDP offer，应用服务器用项目 API 密钥调用 POST /v1/live/sessions 换取 answer。密钥与会话配置保存在可信服务器。
开始之前
需要：
- 有 GPT-Live 访问权限的项目 API 密钥。
- 所选 SDK 示例的服务器运行环境。Node.js 示例要求 22.6 或更高。
- 拥有麦克风权限、运行在 HTTPS 或 localhost 上的浏览器。
示例使用 gpt-5.6-terra Responses 委派与托管网页搜索。后端指令和应用工具见委派与工具，语音与后端用量见成本优化。
理解连接顺序
- 通过用户操作请求麦克风权限，把音轨加到对等连接。
- 创建数据通道，在 SDP offer 之前注册监听器。
- 设置本地描述，等待 ICE 候选收集，再将 offer 发给服务器。
- 服务器向 OpenAI 提交包含
session和transport: { type: "webrtc", sdp: ... }的 JSON。 - 将返回 answer 设为远端描述。等数据通道收到
session.started再发送应用命令。
HTTP 请求已启动会话，不要在数据通道发送 session.start。oai-events 是通道标签。
POST /v1/live/sessions 初始化时先计收 15 秒语音，运行后抵扣时长费用，不是在会话时长之外再加 15 秒。见 WebRTC 初始化费用。
创建应用服务器
在新目录保存示例，并设置环境变量 OPENAI_API_KEY。Node.js 使用 server.mjs，执行 npm install openai express；Python 安装 openai。示例绑定 127.0.0.1，接受 http://localhost:3000 的会话请求，并从运行目录提供 index.html。
选择一种服务器语言：两个版本都在 3000 端口提供 index.html 和同一个 /api/session。SDK 版本必须支持 Live，一次只运行一个版本。
来源示例 1（javascript）：
程序文字译释237：示例提示：保持简洁，需要最新信息的请求委派给可搜索网页的后端；后端在需要最新事实时使用网页搜索，为语音对话返回简明、有依据的结果。错误文字：请求来源异常；需要SDP offer；请在服务器设置OPENAI_API_KEY。Origin检查并不替代应用用户认证与授权。
程序文字译释238：创建Live会话失败。
// [中文] 示例 1：创建应用服务器；以下保留本语言原始示例。
// [中文] 原代码明确设置的模型值：gpt-live-1、gpt-5.6-terra；各值的作用按其所在配置对象区分。
// [中文] 沿原示例执行这些方法：client.live.create；异步调用及事件接收顺序保留。
import express from "express";
import OpenAI from "openai";
import { readFile } from "node:fs/promises";
import { resolve } from "node:path";
const app = express();
const client = new OpenAI({ maxRetries: 0 });
const port = 3000;
const origin = `http://localhost:${port}`;
const indexPath = resolve("index.html");
app.use(express.json({ limit: "64kb" }));
app.get("/", async (_request, response) => {
response.type("html").send(await readFile(indexPath, "utf8"));
});
// Local-only demo. Add your application's authentication and authorization
// before exposing session creation to other users.
app.post("/api/session", async (request, response) => {
if (request.headers.origin !== origin) {
response.status(403).json({ error: "Unexpected request origin" });
return;
}
if (typeof request.body?.sdp !== "string" || !request.body.sdp.trim()) {
response.status(400).json({ error: "An SDP offer is required" });
return;
}
if (!process.env.OPENAI_API_KEY) {
response.status(503).json({ error: "Set OPENAI_API_KEY on the server" });
return;
}
try {
const result = await client.live.create({
session: {
model: "gpt-live-1",
instructions:
"Be concise. Delegate requests needing current information to the backend, which can search the web.",
delegation: {
type: "responses",
responses: {
model: "gpt-5.6-terra",
instructions:
"Use web search when current facts are needed. Return concise, grounded results for a spoken conversation.",
tools: [{ type: "web_search" }],
tool_choice: "auto",
},
},
},
transport: {
type: "webrtc",
sdp: request.body.sdp,
},
});
// Preserve the SDK's typed session ID and SDP answer.
response.status(201).json(result);
} catch (error) {
if (!(error instanceof OpenAI.APIError)) throw error;
console.error("Live session creation failed", error.status);
response
.status(error.status ?? 502)
.json({ error: "Live session creation failed" });
}
});
app.listen(port, "127.0.0.1", () => console.log(`Open ${origin}`));来源示例 2（python）：
程序文字译释239：请求来源异常。
程序文字译释240：必须提供SDP offer。
程序文字译释241：保持简洁，需要最新信息的请求委派给能搜索网页的后端。
程序文字译释242：需要最新事实时用网页搜索，为语音对话返回简明、有依据的结果。
程序文字译释243：创建Live会话失败。
## [中文] 应用服务器持有API密钥，提供页面和/api/session端点，把浏览器SDP交给Live创建会话，再将协商结果返回浏览器。
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from openai import APIError, OpenAI
from openai.types.live.media_session_config_param import MediaSessionConfigParam
from pydantic import BaseModel, Field, ValidationError
client = OpenAI(max_retries=0)
origin = "http://localhost:3000"
class SDPOffer(BaseModel):
sdp: str = Field(min_length=1)
class SessionHandler(BaseHTTPRequestHandler):
def reply(
self, status: int, body: bytes, content_type: str = "application/json"
) -> None:
self.send_response(status)
self.send_header("Content-Type", content_type)
self.end_headers()
self.wfile.write(body)
def do_GET(self) -> None:
if self.path != "/":
self.reply(404, b"{}")
return
self.reply(200, Path("index.html").read_bytes(), "text/html")
def do_POST(self) -> None:
# Local-only demo: add application authentication before exposing it.
if self.path != "/api/session":
self.reply(404, b"{}")
return
if self.headers.get("Origin") != origin:
self.reply(403, b'{"error":"Unexpected request origin"}')
return
length = int(self.headers.get("Content-Length", "0"))
if length > 65536:
self.reply(413, b'{"error":"SDP offer is too large"}')
return
if length <= 0:
self.reply(400, b'{"error":"An SDP offer is required"}')
return
try:
offer = SDPOffer.model_validate_json(self.rfile.read(length))
if not offer.sdp.strip():
self.reply(400, b'{"error":"An SDP offer is required"}')
return
except ValidationError:
self.reply(400, b'{"error":"An SDP offer is required"}')
return
session: MediaSessionConfigParam = {
"model": "gpt-live-1",
"instructions": "Be concise. Delegate requests needing current information to the backend, which can search the web.",
"delegation": {
"type": "responses",
"responses": {
"model": "gpt-5.6-terra",
"instructions": "Use web search when current facts are needed. Return concise, grounded results for a spoken conversation.",
"tools": [{"type": "web_search"}],
"tool_choice": "auto",
},
},
}
try:
result = client.live.create(
session=session, transport={"type": "webrtc", "sdp": offer.sdp}
)
except APIError as error:
self.reply(
getattr(error, "status_code", 502),
b'{"error":"Live session creation failed"}',
)
return
# Return the SDK's typed session ID and SDP answer unchanged.
self.reply(201, result.model_dump_json().encode())
if __name__ == "__main__":
print(f"Open {origin}")
ThreadingHTTPServer(("127.0.0.1", 3000), SessionHandler).serve_forever()// [中文] 应用服务器持有API密钥，提供页面和/api/session端点，把浏览器SDP交给Live创建会话，再将协商结果返回浏览器。
package main
import (
"encoding/json"
"fmt"
"github.com/openai/openai-go/v3"
"github.com/openai/openai-go/v3/live"
"github.com/openai/openai-go/v3/option"
"io"
"log"
"net/http"
"os"
"strings"
"time"
)
func main() {
client := openai.NewClient(option.WithMaxRetries(0))
mux := http.NewServeMux()
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/" || r.Method != http.MethodGet {
http.NotFound(w, r)
return
}
http.ServeFile(w, r, "index.html")
})
mux.HandleFunc("/api/session", func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
w.WriteHeader(http.StatusMethodNotAllowed)
return
}
if r.Header.Get("Origin") != "http://localhost:3000" {
http.Error(w, "Forbidden", 403)
return
}
if os.Getenv("OPENAI_API_KEY") == "" {
http.Error(w, "Server API key is not configured", 503)
return
}
body, err := io.ReadAll(http.MaxBytesReader(w, r.Body, 64*1024))
if err != nil {
http.Error(w, "Invalid SDP offer", 413)
return
}
var offer struct {
SDP string `json:"sdp"`
}
if err := json.Unmarshal(body, &offer); err != nil || strings.TrimSpace(offer.SDP) == "" {
http.Error(w, "SDP offer is required", 400)
return
}
result, err := client.Live.New(r.Context(), live.LiveNewParams{
Session: live.MediaSessionConfigParam{
Model: "gpt-live-1",
Instructions: openai.String("Be concise. Delegate requests needing current information to the backend, which can search the web."),
Delegation: live.MediaSessionConfigDelegationUnionParam{OfResponses: &live.MediaSessionConfigDelegationResponsesParam{Responses: live.ResponsesDelegationConfigParam{
Model: "gpt-5.6-terra",
Instructions: openai.String("Use web search when current facts are needed. Return concise, grounded results for a spoken conversation."),
Tools: []live.ResponsesDelegationConfigToolUnionParam{{OfWebSearch: &live.ResponsesDelegationConfigToolWebSearchParam{}}},
ToolChoice: live.ResponsesDelegationConfigToolChoiceUnionParam{OfLiveToolChoiceEnum: openai.String("auto")},
}}},
},
Transport: live.LiveNewParamsTransport{Sdp: offer.SDP},
})
if err != nil {
log.Printf("Live session creation failed: %T", err)
http.Error(w, "Session creation failed", 502)
return
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusCreated)
if err := json.NewEncoder(w).Encode(result); err != nil {
log.Print("Could not write session response")
}
})
server := &http.Server{Addr: "127.0.0.1:3000", Handler: mux, ReadHeaderTimeout: 5 * time.Second}
fmt.Println("Open http://localhost:3000")
log.Fatal(server.ListenAndServe())
}// [中文] 应用服务器持有API密钥，提供页面和/api/session端点，把浏览器SDP交给Live创建会话，再将协商结果返回浏览器。
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.models.live.*;
import com.sun.net.httpserver.HttpServer;
import java.net.InetSocketAddress;
import java.nio.charset.StandardCharsets;
import java.nio.file.*;
public final class LiveConnectionWebrtcExample {
public static void main(String[] args) throws Exception {
var client = OpenAIOkHttpClient.builder().fromEnv().maxRetries(0).build();
var server = HttpServer.create(new InetSocketAddress("127.0.0.1", 3000), 0);
server.createContext(
"/",
exchange -> {
try (exchange) {
var path = exchange.getRequestURI().getPath();
int status;
byte[] body;
String contentType = "text/plain; charset=utf-8";
if (path.equals("/") && exchange.getRequestMethod().equals("GET")) {
status = 200;
body = Files.readAllBytes(Path.of("index.html"));
contentType = "text/html; charset=utf-8";
} else if (!path.equals("/api/session")) {
status = 404;
body = "Not found".getBytes(StandardCharsets.UTF_8);
} else if (!exchange.getRequestMethod().equals("POST")) {
status = 405;
body = "Method not allowed".getBytes(StandardCharsets.UTF_8);
} else if (!"http://localhost:3000"
.equals(exchange.getRequestHeaders().getFirst("Origin"))) {
status = 403;
body = "Forbidden".getBytes(StandardCharsets.UTF_8);
} else if (System.getenv("OPENAI_API_KEY") == null
|| System.getenv("OPENAI_API_KEY").isBlank()) {
status = 503;
body = "Server API key is not configured".getBytes(StandardCharsets.UTF_8);
} else {
var offer = exchange.getRequestBody().readNBytes(64 * 1024 + 1);
String sdp = "";
if (offer.length <= 64 * 1024) {
try {
var request = com.openai.core.ObjectMappers.jsonMapper().readTree(offer);
if (request != null && request.path("sdp").isTextual()) {
sdp = request.path("sdp").asText();
}
} catch (com.fasterxml.jackson.core.JsonProcessingException error) {
// Invalid browser JSON is rejected below without creating a session.
}
}
if (offer.length > 64 * 1024 || sdp.isBlank()) {
status = offer.length > 64 * 1024 ? 413 : 400;
body = "Invalid SDP offer".getBytes(StandardCharsets.UTF_8);
} else {
try {
var result =
client
.live()
.create(
LiveCreateParams.builder()
.session(
MediaSessionConfig.builder()
.model("gpt-live-1")
.instructions(
"Be concise. Delegate requests needing current information to the backend, which can search the web.")
.responsesDelegation(
ResponsesDelegationConfig.builder()
.model("gpt-5.6-terra")
.instructions(
"Use web search when current facts are needed. Return concise, grounded results for a spoken conversation.")
.addToolWebSearch()
.toolChoice(
ResponsesDelegationConfig.ToolChoice
.LiveToolChoiceEnum.AUTO)
.build())
.build())
.transport(LiveCreateParams.Transport.builder().sdp(sdp).build())
.build());
body = com.openai.core.ObjectMappers.jsonMapper().writeValueAsBytes(result);
status = 201;
contentType = "application/json";
} catch (RuntimeException error) {
System.err.println(
"Live session creation failed: " + error.getClass().getSimpleName());
status = 502;
body = "Session creation failed".getBytes(StandardCharsets.UTF_8);
}
}
}
exchange.getResponseHeaders().set("Content-Type", contentType);
exchange.sendResponseHeaders(status, body.length);
exchange.getResponseBody().write(body);
}
});
Runtime.getRuntime()
.addShutdownHook(
new Thread(
() -> {
server.stop(0);
client.close();
}));
server.start();
System.out.println("Open http://localhost:3000");
}
}## [中文] 应用服务器持有API密钥，提供页面和/api/session端点，把浏览器SDP交给Live创建会话，再将协商结果返回浏览器。
require "openai"
require "webrick"
require "json"
client = OpenAI::Client.new(api_key: ENV["OPENAI_API_KEY"] || "", max_retries: 0)
server = WEBrick::HTTPServer.new(BindAddress: "127.0.0.1", Port: 3000)
server.mount_proc "/" do |request, response|
response["Content-Type"] = "text/plain; charset=utf-8"
if request.path == "/" && request.request_method == "GET"
response["Content-Type"] = "text/html; charset=utf-8"
response.body = File.read("index.html")
elsif request.path != "/api/session"
response.status = 404
response.body = "Not found"
elsif request.request_method != "POST"
response.status = 405
response.body = "Method not allowed"
elsif request["origin"] != "http://localhost:3000"
response.status = 403
response.body = "Forbidden"
elsif ENV["OPENAI_API_KEY"].to_s.empty?
response.status = 503
response.body = "Server API key is not configured"
else
raise WEBrick::HTTPStatus::LengthRequired if request.content_length.nil?
raise WEBrick::HTTPStatus::BadRequest if request["transfer-encoding"]
raise WEBrick::HTTPStatus::RequestEntityTooLarge if request.content_length.to_i > 64 * 1024
offer = request.body.to_s
raise WEBrick::HTTPStatus::RequestEntityTooLarge if offer.bytesize > 64 * 1024
begin
payload = JSON.parse(offer)
rescue JSON::ParserError
raise WEBrick::HTTPStatus::BadRequest
end
sdp = payload.is_a?(Hash) ? payload["sdp"] : nil
raise WEBrick::HTTPStatus::BadRequest unless sdp.is_a?(String) && !sdp.strip.empty?
begin
result = client.live.create(
session: {
model: "gpt-live-1",
instructions: "Be concise. Delegate requests needing current information to the backend, which can search the web.",
delegation: {
type: :responses,
responses: {
model: "gpt-5.6-terra",
instructions: "Use web search when current facts are needed. Return concise, grounded results for a spoken conversation.",
tools: [{ type: :web_search }],
tool_choice: :auto
}
}
},
transport: {
type: :webrtc,
sdp: sdp
}
)
response.status = 201
response["Content-Type"] = "application/json"
response.body = result.to_json
rescue OpenAI::Errors::Error => error
warn "Live session creation failed: #{error.class}"
response.status = 502
response.body = "Session creation failed"
end
end
end
trap("INT") { server.shutdown }
puts "Open http://localhost:3000"
server.start
向其他用户开放前，应为 /api/session 添加应用认证、授权、请求限制与 HTTPS。此本地示例的 Origin 检查不等于用户认证。
创建浏览器客户端
在服务器运行目录创建 index.html：
来源示例 3（html）：
<!-- [中文] 示例 3：创建浏览器客户端；以下保留本语言原始示例。 -->
<!-- [中文] 此 HTML 只建立页面控件与模块脚本位置；后面的 JavaScript 片段应放入该模块脚本。 -->
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>GPT-Live connection</title>
</head>
<body>
<script type="module">
// Paste the browser code below here.
</script>
</body>
</html>
把以下代码放入模块脚本。它添加开始、结束控件，连接麦克风与输出音频，并处理事件。/api/session 是应用服务器路由。
来源示例 4（javascript）：
程序文字译释244：开始对话。
程序文字译释245：结束对话。
程序文字译释246：请点击音频控件中的播放以收听助手。
程序文字译释247：最终会话用量。
程序文字译释248：对话已结束。
程序文字译释249：断开时没有收到最终会话用量。
程序文字译释250：收集ICE候选超时。
程序文字译释251：缺少本地SDP offer。
程序文字译释252：已创建会话。
程序文字译释253：正在结束对话……
程序文字译释254：收尾未完成：未收到session.closed。
// [中文] 请求正常关闭并等待session.closed或对应终结状态；发送关闭事件不等同于已收到最终用量或释放所有资源。
const start = document.createElement("button");
start.textContent = "Start conversation";
const stop = document.createElement("button");
stop.textContent = "End conversation";
stop.disabled = true;
const status = document.createElement("p");
const audio = new Audio();
audio.autoplay = true;
audio.controls = true;
document.body.append(start, stop, status, audio);
let peer;
let events;
let microphone;
let closeTimeout;
let ready = false;
let finalized = false;
function cleanup() {
clearTimeout(closeTimeout);
microphone?.getTracks().forEach((track) => track.stop());
const closingEvents = events;
events = undefined;
closingEvents?.close();
peer?.close();
audio.srcObject = null;
ready = false;
start.disabled = false;
stop.disabled = true;
}
start.addEventListener("click", async () => {
start.disabled = true;
finalized = false;
status.textContent = "Connecting…";
try {
const connection = new RTCPeerConnection();
peer = connection;
connection.addEventListener("track", (event) => {
audio.srcObject = new MediaStream([event.track]);
audio.play().catch(() => {
status.textContent =
"Select play on the audio controls to hear the assistant.";
});
});
microphone = await navigator.mediaDevices.getUserMedia({ audio: true });
for (const track of microphone.getAudioTracks()) {
connection.addTrack(track, microphone);
}
// Create the event channel before creating the SDP offer.
events = connection.createDataChannel("oai-events");
events.addEventListener("message", ({ data }) => {
const event = JSON.parse(data);
if (event.type === "session.started") {
ready = true;
stop.disabled = false;
status.textContent = "Connected: " + event.session.id;
} else if (event.type === "session.closed") {
finalized = true;
console.log("Final session usage", event.usage);
status.textContent = "Conversation ended.";
cleanup();
} else {
// Save transcript and nested Responses events as needed by your app.
console.log(event);
}
});
events.addEventListener("close", (event) => {
if (event.target !== events) return;
if (!finalized) {
status.textContent = "Disconnected without final session usage.";
cleanup();
}
});
const offer = await connection.createOffer();
await connection.setLocalDescription(offer);
if (connection.iceGatheringState !== "complete") {
await new Promise((resolve, reject) => {
const timeout = setTimeout(() => {
connection.removeEventListener("icegatheringstatechange", onState);
reject(new Error("Timed out while gathering ICE candidates"));
}, 10_000);
function onState() {
if (connection.iceGatheringState !== "complete") return;
clearTimeout(timeout);
connection.removeEventListener("icegatheringstatechange", onState);
resolve(undefined);
}
connection.addEventListener("icegatheringstatechange", onState);
onState();
});
}
const sdp = connection.localDescription?.sdp;
if (!sdp) throw new Error("Missing local SDP offer");
const response = await fetch("/api/session", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ sdp }),
});
if (!response.ok) throw new Error(await response.text());
const result = await response.json();
console.log("Created session", result.session.id);
await connection.setRemoteDescription({
type: "answer",
sdp: result.transport.sdp,
});
// The HTTP request started this session. Do not send session.start here.
} catch (error) {
status.textContent =
error instanceof Error ? error.message : String(error);
cleanup();
}
});
stop.addEventListener("click", () => {
if (!ready || !events || events.readyState !== "open") return;
stop.disabled = true;
status.textContent = "Finishing the conversation…";
// The session.closed handler is already registered. Keep media and events
// alive while pending work drains; only clean up after the final event.
events.send(JSON.stringify({ type: "session.close" }));
closeTimeout = setTimeout(() => {
status.textContent = "Incomplete finalization: no session.closed event.";
cleanup();
}, 15_000);
});
运行 node server.mjs 或 python server.py，打开 http://localhost:3000，选择 Start conversation（开始对话）。状态变为 Connected（已连接）后，提需要最新信息的问题以测试托管搜索。浏览器阻止自动播放时，使用音频控件。
读取会话响应
请求成功返回 HTTP 201，JSON 包含会话 ID 和 SDP answer：
来源示例 5（json）：
程序文字译释255：这里替换为真实SDP answer。
{
"session": { "id": "live_123" },
"transport": { "type": "webrtc", "sdp": "<SDP answer>" }
}
读取 result.session.id，将 result.transport.sdp 交给 setRemoteDescription。会话 ID 是不透明标识，连同前缀原样保存。
降低启动延迟见 WebRTC 与 WARP。
处理媒体与事件
麦克风与生成语音走媒体轨道，SDP 协商音频格式，因此省略 audio.format。不要在数据通道发送 session.input_audio.append，也不要期待收到 session.output_audio.delta。
数据通道用于转录增量、会话命令和嵌套 response.event。生命周期见管理会话，服务器需要独立事件连接时见服务端控制。
结束时发送 session.close，持续接收直到 session.closed 后，才关闭对等连接和麦克风轨道。示例在发送前注册最终事件监听。若先断连或超时，最终用量未确认。见用量与正常关闭。
Realtime API：WebRTC 是构建实时应用的一组标准接口。OpenAI Realtime API 支持通过 WebRTC 对等连接访问实时模型。
浏览器语音到语音应用推荐先看语音智能体，了解 Agents SDK 管理 Realtime 的较高层辅助接口。WebRTC 灵活强大，但比 Agents SDK 更底层。
浏览器或移动端连接实时模型时，推荐 WebRTC 而非 WebSocket，以获得更稳定表现。界面构建可参阅 MDN WebRTC 文档。
概览
Realtime 支持浏览器通过统一接口或临时 API 密钥连接。统一接口配置更简单、连接更快，但应用服务器处于初始化关键路径。
通过统一接口连接
浏览器流程如下：
- 浏览器用对等连接的 SDP 数据向开发者控制的服务器请求。
- 服务器把 SDP 和会话配置组合成 multipart 表单，用标准 API 密钥向 Realtime API 发送。
通过统一接口创建会话
构建小型服务端应用，或集成到现有应用，向 /v1/realtime/calls 请求。用后端标准 API 密钥认证。
以下 Node.js Express 示例创建 Realtime 会话：
来源示例 6（javascript）：
程序文字译释115：生成令牌时出错。
程序文字译释116：生成令牌失败。
// [中文] 保留原例中客户端、输入和返回值的处理顺序；本段需结合所在小节的连接前提使用，未通过真实API调用验证。
import express from "express";
const app = express();
// Parse raw SDP payloads posted from the browser
app.use(express.text({ type: ["application/sdp", "text/plain"] }));
const sessionConfig = JSON.stringify({
type: "realtime",
model: "gpt-realtime-2.1",
audio: { output: { voice: "marin" } },
});
// An endpoint which creates a Realtime API session.
app.post("/session", async (req, res) => {
const fd = new FormData();
fd.set("sdp", req.body);
fd.set("session", sessionConfig);
try {
const r = await fetch("https://api.openai.com/v1/realtime/calls", {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
"OpenAI-Safety-Identifier": "hashed-user-id",
},
body: fd,
});
// Send back the SDP we received from the OpenAI REST API
const sdp = await r.text();
res.set(
"Content-Type",
r.ok ? "application/sdp" : (r.headers.get("Content-Type") ?? "text/plain")
);
const location = r.headers.get("Location");
if (location) res.set("Location", location);
res.status(r.status).send(sdp);
} catch (error) {
console.error("Session creation error:", error);
res.status(500).json({ error: "Failed to create session" });
}
});
app.listen(3000);// [中文] 保留原例中客户端、输入和返回值的处理顺序；本段需结合所在小节的连接前提使用，未通过真实API调用验证。
// [中文] 本段为配置或调用片段，需结合外层程序与已建立连接使用；不作为独立完整程序宣称编译通过。
func handler(client openai.Client) http.Handler {
mux := http.NewServeMux()
mux.HandleFunc("POST /session", func(w http.ResponseWriter, r *http.Request) {
offer, err := io.ReadAll(http.MaxBytesReader(w, r.Body, 64*1024))
if err != nil || strings.TrimSpace(string(offer)) == "" {
http.Error(w, "A valid SDP offer is required", http.StatusBadRequest)
return
}
opts := []option.RequestOption{
option.WithHeader("OpenAI-Safety-Identifier", "hashed-user-id"),
}
response, err := client.Realtime.Calls.New(r.Context(), realtime.CallNewParams{
Sdp: string(offer),
Session: realtime.RealtimeSessionCreateRequestParam{
Model: "gpt-realtime-2.1",
Audio: realtime.RealtimeAudioConfigParam{
Output: realtime.RealtimeAudioConfigOutputParam{
Voice: realtime.RealtimeAudioConfigOutputVoiceUnionParam{
OfString: openai.String("marin"),
},
},
},
},
}, opts...)
if err != nil {
http.Error(w, "Could not create Realtime call", http.StatusBadGateway)
return
}
defer response.Body.Close()
w.Header().Set("Content-Type", "application/sdp")
w.Header().Set("Location", response.Header.Get("Location"))
w.WriteHeader(response.StatusCode)
if _, err := io.Copy(w, response.Body); err != nil {
log.Print("Could not write SDP answer")
}
})
return mux
}
若应用为终端用户分配安全标识符，在服务端请求的 OpenAI-Safety-Identifier 头传入，例如哈希内部用户 ID 等稳定、保护隐私的值。由可信后端设置，不由浏览器设置。
连接服务器
浏览器用标准 WebRTC API 经应用服务器连接，直接向服务器 POST SDP：
来源示例 7（javascript）：
// [中文] 示例 7：连接服务器；以下保留本语言原始示例。
// [中文] 沿原示例执行这些方法：pc.addTrack、pc.createDataChannel、pc.createOffer、pc.setLocalDescription、pc.setRemoteDescription；异步调用及事件接收顺序保留。
// Create a peer connection
const pc = new RTCPeerConnection();
// Set up to play remote audio from the model
audioElement.current = document.createElement("audio");
audioElement.current.autoplay = true;
pc.ontrack = (e) => (audioElement.current.srcObject = e.streams[0]);
// Add local audio track for microphone input in the browser
const ms = await navigator.mediaDevices.getUserMedia({
audio: true,
});
pc.addTrack(ms.getTracks()[0]);
// Set up data channel for sending and receiving events
const dc = pc.createDataChannel("oai-events");
// Start the session using the Session Description Protocol (SDP)
const offer = await pc.createOffer();
await pc.setLocalDescription(offer);
const sdpResponse = await fetch("/session", {
method: "POST",
body: offer.sdp,
headers: {
"Content-Type": "application/sdp",
},
});
const answer = {
type: "answer",
sdp: await sdpResponse.text(),
};
await pc.setRemoteDescription(answer);使用临时令牌连接
临时密钥方式的浏览器流程：
- 浏览器向开发者服务器请求临时密钥。
- 服务器用标准密钥向 OpenAI REST API申请，并返回新临时密钥。
- 浏览器使用临时密钥认证，直接与 Realtime API 建立 WebRTC 对等连接。
创建临时令牌
需要服务端应用向 REST API 请求临时密钥，使用标准密钥认证。以下 Express 示例负责签发：
来源示例 8（javascript）：
沿用程序文字译释115。
沿用程序文字译释116。
// [中文] 在可信应用服务器创建临时凭证，再供客户端建立实时连接；标准API密钥保持在服务器，不写入浏览器。
import express from "express";
const app = express();
const sessionConfig = JSON.stringify({
session: {
type: "realtime",
model: "gpt-realtime-2.1",
audio: {
output: {
voice: "marin",
},
},
},
});
// An endpoint which would work with the client code above - it returns
// the contents of a REST API request to this protected endpoint
app.get("/token", async (req, res) => {
res.set("Cache-Control", "no-store");
try {
const response = await fetch(
"https://api.openai.com/v1/realtime/client_secrets",
{
method: "POST",
headers: {
Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
"Content-Type": "application/json",
"OpenAI-Safety-Identifier": "hashed-user-id",
},
body: sessionConfig,
}
);
const data = await response.json();
res.status(response.status).json(data);
} catch (error) {
console.error("Token generation error:", error);
res.status(500).json({ error: "Failed to generate token" });
}
});
app.listen(3000);// [中文] 在可信应用服务器创建临时凭证，再供客户端建立实时连接；标准API密钥保持在服务器，不写入浏览器。
// [中文] 本段为配置或调用片段，需结合外层程序与已建立连接使用；不作为独立完整程序宣称编译通过。
func handler(client openai.Client) http.Handler {
mux := http.NewServeMux()
mux.HandleFunc("GET /token", func(w http.ResponseWriter, r *http.Request) {
secret, err := client.Realtime.ClientSecrets.New(r.Context(), realtime.ClientSecretNewParams{
Session: realtime.ClientSecretNewParamsSessionUnion{
OfRealtime: &realtime.RealtimeSessionCreateRequestParam{
Model: "gpt-realtime-2.1",
Audio: realtime.RealtimeAudioConfigParam{
Output: realtime.RealtimeAudioConfigOutputParam{
Voice: realtime.RealtimeAudioConfigOutputVoiceUnionParam{
OfString: openai.String("marin"),
},
},
},
},
},
}, option.WithHeader("OpenAI-Safety-Identifier", "hashed-user-id"))
if err != nil {
http.Error(w, "Could not create client credential", http.StatusBadGateway)
return
}
w.Header().Set("Content-Type", "application/json")
w.Header().Set("Cache-Control", "no-store")
if _, err := io.WriteString(w, secret.RawJSON()); err != nil {
log.Print("Could not write client credential")
}
})
return mux
}任何能收发 HTTP 的平台都可实现类似端点。标准 OpenAI API 密钥只能在服务器使用，不能放进浏览器。
临时令牌方式中，在服务器创建客户端密钥的请求设置 OpenAI-Safety-Identifier。API 将标识符绑定到令牌，浏览器以后用令牌连接时无需再发送安全标识符。
连接服务器
浏览器先从服务器取令牌，再用临时令牌向 Realtime API POST SDP：
来源示例 9（javascript）：
// [中文] 示例 9：连接服务器；以下保留本语言原始示例。
// [中文] 本段涉及的 HTTP 端点：/v1/realtime/calls；认证和请求体保持原示例形式。
// [中文] 沿原示例执行这些方法：pc.addTrack、pc.createDataChannel、pc.createOffer、pc.setLocalDescription、pc.setRemoteDescription；异步调用及事件接收顺序保留。
// Get a session token for OpenAI Realtime API
const tokenResponse = await fetch("/token");
const data = await tokenResponse.json();
const EPHEMERAL_KEY = data.value;
// Create a peer connection
const pc = new RTCPeerConnection();
// Set up to play remote audio from the model
audioElement.current = document.createElement("audio");
audioElement.current.autoplay = true;
pc.ontrack = (e) => (audioElement.current.srcObject = e.streams[0]);
// Add local audio track for microphone input in the browser
const ms = await navigator.mediaDevices.getUserMedia({
audio: true,
});
pc.addTrack(ms.getTracks()[0]);
// Set up data channel for sending and receiving events
const dc = pc.createDataChannel("oai-events");
// Start the session using the Session Description Protocol (SDP)
const offer = await pc.createOffer();
await pc.setLocalDescription(offer);
const sdpResponse = await fetch("https://api.openai.com/v1/realtime/calls", {
method: "POST",
body: offer.sdp,
headers: {
Authorization: `Bearer ${EPHEMERAL_KEY}`,
"Content-Type": "application/sdp",
},
});
const answer = {
type: "answer",
sdp: await sdpResponse.text(),
};
await pc.setRemoteDescription(answer);通过 WARP 降低连接延迟
WARP 可缩短 Realtime 语音启动时间，优化可分别启用或组合成完整握手。
原生客户端设置、浏览器支持、Origin Trial 和统一连接流程见 WebRTC 与 WARP。
发送与接收事件
会话通过开发者发送的客户端事件，以及 API 产生的服务端事件共同管理。
WebRTC 不必像 WebSocket 那样逐个处理音频事件，按上述配置的对等连接会代为完成。
其他事件通过 WebRTC 数据通道收发：
来源示例 10（javascript）：
程序文字译释256：你好！
// [中文] 保留原例中客户端、输入和返回值的处理顺序；本段需结合所在小节的连接前提使用，未通过真实API调用验证。
// Reuse dc from the preceding browser connection example.
// Listen for server events
dc.addEventListener("message", (e) => {
const event = JSON.parse(e.data);
console.log(event);
});
// Send client events
const event = {
type: "conversation.item.create",
item: {
type: "message",
role: "user",
content: [
{
type: "input_text",
text: "hello there!",
},
],
},
};
const sendGreeting = () => dc.send(JSON.stringify(event));
if (dc.readyState === "open") {
sendGreeting();
} else if (dc.readyState === "connecting") {
dc.addEventListener("open", sendGreeting, { once: true });
} else {
throw new Error("The data channel has closed before greeting was sent.");
}对话管理详见 Realtime 对话。Realtime Console 提供轻量 WebRTC 示例应用。
以上内容是否对您有帮助：
更多建议：