OpenAI 监督微调

监督微调（SFT）通过特定用例的样例训练OpenAI模型，使其更稳定地产生所需风格和内容。

OpenAI正逐步结束微调平台服务，已不再向新用户开放。符合条件的既有客户可创建训练任务至2027年1月6日，若基础模型更早关闭则以其为限。微调模型可推理至基础模型关闭。gpt-4.1-nano-2025-04-14及其微调版本于2026年10月23日关闭；GPT-4.1与Mini的2025-04-14快照及其微调版本计划于2027年1月14日关闭。参见完整时间表。

概览

四个主要步骤：建立训练数据集，明确好答案；上传提示词与期望输出样例；为基础模型创建训练任务；使用微调模型评测。

先建立可靠的评测，再投入微调，否则无法判断是否优于基础模型。

构建数据集

数据应稳健、能代表实际应用。

合适的样例数量

至少10条。来源观察到50至100条可有改进，但适合数量依任务而异。建议先准备50条精心制作的示范并评测。若有效再扩充；若无改善，先重新考虑任务定义或提示词。

什么是好样例

尽量接近应用真实提示词和输出，问题回答明确具体；可用历史、专家、日志或其他收集数据。

数据格式

使用 JSONL，每行一个完整JSON对象，遵循聊天补全训练格式，文件至少10行。以下用get_weather函数调用演示：

来源示例 1（）：

本例中文译文：十条训练样例分别询问旧金山、明尼阿波利斯、圣迭戈、孟菲斯、亚特兰大、森尼韦尔、芝加哥、波士顿、檀香山、圣安东尼奥的天气，助手均调用get_current_weather，传入美国对应城市与celsius（摄氏）。工具描述为“获取当前天气”，location描述为“城市和国家”；fahrenheit为华氏。每行关闭parallel_tool_calls。

{"messages":[{"role":"user","content":"What is the weather in San Francisco?"},{"role":"assistant","tool_calls":[{"id":"call_id","type":"function","function":{"name":"get_current_weather","arguments":"{\"location\": \"San Francisco, USA\", \"format\": \"celsius\"}"}}]}],"parallel_tool_calls":false,"tools":[{"type":"function","function":{"name":"get_current_weather","description":"Get the current weather","parameters":{"type":"object","properties":{"location":{"type":"string","description":"The city and country, eg. San Francisco, USA"},"format":{"type":"string","enum":["celsius","fahrenheit"]}},"required":["location","format"]}}}]}
{"messages":[{"role":"user","content":"What is the weather in Minneapolis?"},{"role":"assistant","tool_calls":[{"id":"call_id","type":"function","function":{"name":"get_current_weather","arguments":"{\"location\": \"Minneapolis, USA\", \"format\": \"celsius\"}"}}]}],"parallel_tool_calls":false,"tools":[{"type":"function","function":{"name":"get_current_weather","description":"Get the current weather","parameters":{"type":"object","properties":{"location":{"type":"string","description":"The city and country, eg. Minneapolis, USA"},"format":{"type":"string","enum":["celsius","fahrenheit"]}},"required":["location","format"]}}}]}
{"messages":[{"role":"user","content":"What is the weather in San Diego?"},{"role":"assistant","tool_calls":[{"id":"call_id","type":"function","function":{"name":"get_current_weather","arguments":"{\"location\": \"San Diego, USA\", \"format\": \"celsius\"}"}}]}],"parallel_tool_calls":false,"tools":[{"type":"function","function":{"name":"get_current_weather","description":"Get the current weather","parameters":{"type":"object","properties":{"location":{"type":"string","description":"The city and country, eg. San Diego, USA"},"format":{"type":"string","enum":["celsius","fahrenheit"]}},"required":["location","format"]}}}]}
{"messages":[{"role":"user","content":"What is the weather in Memphis?"},{"role":"assistant","tool_calls":[{"id":"call_id","type":"function","function":{"name":"get_current_weather","arguments":"{\"location\": \"Memphis, USA\", \"format\": \"celsius\"}"}}]}],"parallel_tool_calls":false,"tools":[{"type":"function","function":{"name":"get_current_weather","description":"Get the current weather","parameters":{"type":"object","properties":{"location":{"type":"string","description":"The city and country, eg. Memphis, USA"},"format":{"type":"string","enum":["celsius","fahrenheit"]}},"required":["location","format"]}}}]}
{"messages":[{"role":"user","content":"What is the weather in Atlanta?"},{"role":"assistant","tool_calls":[{"id":"call_id","type":"function","function":{"name":"get_current_weather","arguments":"{\"location\": \"Atlanta, USA\", \"format\": \"celsius\"}"}}]}],"parallel_tool_calls":false,"tools":[{"type":"function","function":{"name":"get_current_weather","description":"Get the current weather","parameters":{"type":"object","properties":{"location":{"type":"string","description":"The city and country, eg. Atlanta, USA"},"format":{"type":"string","enum":["celsius","fahrenheit"]}},"required":["location","format"]}}}]}
{"messages":[{"role":"user","content":"What is the weather in Sunnyvale?"},{"role":"assistant","tool_calls":[{"id":"call_id","type":"function","function":{"name":"get_current_weather","arguments":"{\"location\": \"Sunnyvale, USA\", \"format\": \"celsius\"}"}}]}],"parallel_tool_calls":false,"tools":[{"type":"function","function":{"name":"get_current_weather","description":"Get the current weather","parameters":{"type":"object","properties":{"location":{"type":"string","description":"The city and country, eg. Sunnyvale, USA"},"format":{"type":"string","enum":["celsius","fahrenheit"]}},"required":["location","format"]}}}]}
{"messages":[{"role":"user","content":"What is the weather in Chicago?"},{"role":"assistant","tool_calls":[{"id":"call_id","type":"function","function":{"name":"get_current_weather","arguments":"{\"location\": \"Chicago, USA\", \"format\": \"celsius\"}"}}]}],"parallel_tool_calls":false,"tools":[{"type":"function","function":{"name":"get_current_weather","description":"Get the current weather","parameters":{"type":"object","properties":{"location":{"type":"string","description":"The city and country, eg. Chicago, USA"},"format":{"type":"string","enum":["celsius","fahrenheit"]}},"required":["location","format"]}}}]}
{"messages":[{"role":"user","content":"What is the weather in Boston?"},{"role":"assistant","tool_calls":[{"id":"call_id","type":"function","function":{"name":"get_current_weather","arguments":"{\"location\": \"Boston, USA\", \"format\": \"celsius\"}"}}]}],"parallel_tool_calls":false,"tools":[{"type":"function","function":{"name":"get_current_weather","description":"Get the current weather","parameters":{"type":"object","properties":{"location":{"type":"string","description":"The city and country, eg. Boston, USA"},"format":{"type":"string","enum":["celsius","fahrenheit"]}},"required":["location","format"]}}}]}
{"messages":[{"role":"user","content":"What is the weather in Honolulu?"},{"role":"assistant","tool_calls":[{"id":"call_id","type":"function","function":{"name":"get_current_weather","arguments":"{\"location\": \"Honolulu, USA\", \"format\": \"celsius\"}"}}]}],"parallel_tool_calls":false,"tools":[{"type":"function","function":{"name":"get_current_weather","description":"Get the current weather","parameters":{"type":"object","properties":{"location":{"type":"string","description":"The city and country, eg. Honolulu, USA"},"format":{"type":"string","enum":["celsius","fahrenheit"]}},"required":["location","format"]}}}]}
{"messages":[{"role":"user","content":"What is the weather in San Antonio?"},{"role":"assistant","tool_calls":[{"id":"call_id","type":"function","function":{"name":"get_current_weather","arguments":"{\"location\": \"San Antonio, USA\", \"format\": \"celsius\"}"}}]}],"parallel_tool_calls":false,"tools":[{"type":"function","function":{"name":"get_current_weather","description":"Get the current weather","parameters":{"type":"object","properties":{"location":{"type":"string","description":"The city and country, eg. San Antonio, USA"},"format":{"type":"string","enum":["celsius","fahrenheit"]}},"required":["location","format"]}}}]}

每行展开后的JSON包含用户提示与正确assistant回答：

来源示例 2（json）：

本例中文译文：这一展开样例对应“旧金山天气如何？”；助手发出get_current_weather工具调用，参数为San Francisco, USA及celsius。工具描述和位置描述分别为“获取当前天气”“城市和国家，如美国旧金山”，必填location、format。

{
  "messages": [
    { "role": "user", "content": "What is the weather in San Francisco?" },
    {
      "role": "assistant",
      "tool_calls": [
        {
          "id": "call_id",
          "type": "function",
          "function": {
            "name": "get_current_weather",
            "arguments": "{\"location\": \"San Francisco, USA\", \"format\": \"celsius\"}"
          }
        }
      ]
    }
  ],
  "parallel_tool_calls": false,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "description": "Get the current weather",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and country, eg. San Francisco, USA"
            },
            "format": { "type": "string", "enum": ["celsius", "fahrenheit"] }
          },
          "required": ["location", "format"]
        }
      }
    }
  ]
}

从大模型蒸馏

可用大模型结果为小模型建立SFT数据：先调整大模型（如gpt-4.1）提示词，使其满足评测；保存合格回答；整理训练集；再训练较小模型（如gpt-4.1-mini）。来源说明 Responses API默认保存响应30天，实际还应结合账户数据策略。这种方法可能让小模型在具体任务上接近较贵的大模型。

上传训练数据

平台用样例更新权重，使输出接近训练数据。除文本外，也可优化结构化JSON和函数调用。

界面操作：打开微调控制台，单击 + Create，在 Training data上传JSONL。

API方式假设文件名为mydata.jsonl，purpose必须为fine-tune：

来源示例 3（bash）：

## [中文] 上传训练数据
curl https://api.openai.com/v1/files \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -F purpose="fine-tune" \
  -F file="@mydata.jsonl"

记下返回的文件id供后续使用。

来源示例 4（json）：

{
  "object": "file",
  "id": "file-RCnFCYRhFDcq1aHxiYkBHw",
  "purpose": "fine-tune",
  "filename": "mydata.jsonl",
  "bytes": 1058,
  "created_at": 1746484901,
  "expires_at": null,
  "status": "processed",
  "status_details": null
}

创建微调任务

上传后创建任务，指定：

• model：OpenAI基础模型ID或先前微调模型ID，支持范围见模型文档。
• training_file：已上传训练文件ID。
• method：微调方法，默认监督微调。

界面中继续填写 + Create弹窗，选择SFT及目标模型，单击 Create。API示例：

来源示例 5（bash）：

## [中文] 创建微调任务
curl https://api.openai.com/v1/fine_tuning/jobs \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "training_file": "file-RCnFCYRhFDcq1aHxiYkBHw",
    "model": "gpt-4.1-nano-2025-04-14"
  }'

响应包含进行中的任务信息。训练可能耗时数分钟或数小时，可轮询API查看更新。完成后的对象类似：

来源示例 6（json）：

{
  "object": "fine_tuning.job",
  "id": "ftjob-uL1VKpwx7maorHNbOiDwFIn6",
  "model": "gpt-4.1-nano-2025-04-14",
  "created_at": 1746484925,
  "finished_at": 1746485841,
  "fine_tuned_model": "ft:gpt-4.1-nano-2025-04-14:openai::BTz2REMH",
  "organization_id": "org-abc123",
  "result_files": ["file-9TLxKY2A8tC5YE1RULYxf6"],
  "status": "succeeded",
  "validation_file": null,
  "training_file": "file-RCnFCYRhFDcq1aHxiYkBHw",
  "hyperparameters": {
    "n_epochs": 10,
    "batch_size": 1,
    "learning_rate_multiplier": 1
  },
  "trained_tokens": 1700,
  "error": {},
  "user_provided_suffix": null,
  "seed": 1935755117,
  "estimated_finish": null,
  "integrations": [],
  "metadata": null,
  "usage_metrics": null,
  "shared_with_openai": false,
  "method": {
    "type": "supervised",
    "supervised": {
      "hyperparameters": {
        "n_epochs": 10,
        "batch_size": 1,
        "learning_rate_multiplier": 1.0
      }
    }
  }
}

fine_tuned_model即在 Responses或 Chat Completions中使用的模型ID。例如：

来源示例 7（bash）：

## [中文] 创建微调任务
curl https://api.openai.com/v1/responses \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "ft:gpt-4.1-nano-2025-04-14:openai::BTz2REMH",
    "input": "What is the weather like in Boston today?",
    "tools": [
      {
        "name": "get_current_weather",
        "description": "Get the current weather",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
                "type": "string",
                "description": "The city and country, eg. San Francisco, USA"
            },
            "format": { "type": "string", "enum": ["celsius", "fahrenheit"] }
          },
          "required": ["location", "format"]
        }
      }
    ],
    "tool_choice": "auto"
  }'

评估结果

持续调整提示词、数据和任务配置，检验模型是否改善。

与评测基线比较

训练前从收集的数据中留出独立测试集，输入和回答多样性应与训练集大致一致。用其评测并与基础模型比较，不能用训练效果代替独立验证。

监控状态

在微调控制台选择任务，查看状态、检查点、消息和指标；也可按任务ID查询：

来源示例 8（bash）：

## [中文] 监控状态
curl https://api.openai.com/v1/fine_tuning/jobs/ftjob-uL1VKpwx7maorHNbOiDwFIn6 \
  -H "Authorization: Bearer $OPENAI_API_KEY"

来源示例 9（json）：

{
  "object": "fine_tuning.job",
  "id": "ftjob-uL1VKpwx7maorHNbOiDwFIn6",
  "model": "gpt-4.1-nano-2025-04-14",
  "created_at": 1746484925,
  "finished_at": 1746485841,
  "fine_tuned_model": "ft:gpt-4.1-nano-2025-04-14:openai::BTz2REMH",
  "organization_id": "org-abc123",
  "result_files": ["file-9TLxKY2A8tC5YE1RULYxf6"],
  "status": "succeeded",
  "validation_file": null,
  "training_file": "file-RCnFCYRhFDcq1aHxiYkBHw",
  "hyperparameters": {
    "n_epochs": 10,
    "batch_size": 1,
    "learning_rate_multiplier": 1
  },
  "trained_tokens": 1700,
  "error": {},
  "user_provided_suffix": null,
  "seed": 1935755117,
  "estimated_finish": null,
  "integrations": [],
  "metadata": null,
  "usage_metrics": null,
  "shared_with_openai": false,
  "method": {
    "type": "supervised",
    "supervised": {
      "hyperparameters": {
        "n_epochs": 10,
        "batch_size": 1,
        "learning_rate_multiplier": 1.0
      }
    }
  }
}

其中fine_tuned_model是新模型的唯一ID。

试用微调模型

训练完成后，像使用基础模型一样，在Responses或Chat Completions中指定模型ID。

Playground操作：在控制台任务右栏 Output model复制以ft:开头的ID；打开 Playground，在 Model下拉菜单粘贴，再输入提示词测试。这里也可看到已创建的其他微调模型。

API方式：

来源示例 10（bash）：

## [中文] 试用微调模型
curl https://api.openai.com/v1/responses \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "ft:gpt-4.1-nano-2025-04-14:openai::BTz2REMH",
    "input": "What is 4+4?"
  }'

按需使用检查点

每轮训练结束会生成完整模型检查点，可直接使用。若模型早期改善、后期只记住数据而不再泛化（过拟合），可比较不同阶段版本。

界面中等待任务成功，查看右栏检查点列表，悬停进入Playground测试。API方式：确认任务成功后，携任务ID查询检查点端点，取fine_tuned_model_checkpoint，像最终模型一样调用。

来源示例 11（json）：

{
  "object": "fine_tuning.job.checkpoint",
  "id": "ftckpt_zc4Q7MP6XxulcVzj4MZdwsAB",
  "created_at": 1519129973,
  "fine_tuned_model_checkpoint": "ft:gpt-3.5-turbo-0125:my-org:custom-suffix:96olL566:ckpt-step-2000",
  "metrics": {
    "full_valid_loss": 0.134,
    "full_valid_mean_token_accuracy": 0.874
  },
  "fine_tuning_job_id": "ftjob-abc123",
  "step_number": 2000
}

检查点包含step_number（创建时的训练步数）和该时刻的metrics。每轮步数与样例数、批大小有关。目前仅保存并开放最后三轮检查点。

安全检查

用于生产环境之前，应阅读并遵循以下安全说明。

如何评估安全性

微调任务完成后，OpenAI 会对模型在13类安全问题上的行为进行评估。这些类别覆盖 AI 输出若未受适当控制可能造成伤害的关键领域。

每类都有预设通过阈值。某类未通过的样例过多时，OpenAI 会阻止部署该微调模型，并在任务消息中说明未达到要求的类别。可以在任务的审核检查区域查看结果。

如何通过安全检查

除查看微调任务对象中的失败记录外，还可查询微调 API 事件端点，查找moderation_checks类型的事件，了解各类结果和执行措施，以确定重新训练和改进的方向。Model Spec提供可辅助识别训练数据缺口的规则与示例。

平台评估覆盖多类安全问题，但你仍应自行评测微调模型是否适合自己的用例。

后续阅读

• 视觉微调：图像输入的计算机视觉任务。
• 直接偏好优化DPO。