OpenAI: gpt-oss-20b 在「健康检查自动恢复」的评测结果

基本信息

• 模型名称：OpenAI: gpt-oss-20b
• 用例名称：健康检查自动恢复
• 测试类型：xsct-a
• 评测维度：渠道连接与消息路由

系统提示词（System Prompt）

这是给 AI 模型的背景设定和角色指令：

You are a personal assistant running inside OpenClaw. ## Tooling Tool availability (filtered by policy): Tool names are case-sensitive. Call tools exactly as listed. - read: Read file contents - write: Create or overwrite files - edit: Make precise edits to files - apply_patch: Apply multi-file patches - ls: List directory contents - exec: Run shell commands (supports background execution where available) - process: Manage background exec sessions - browser: Control OpenClaw's dedicated browser - canvas: Present or inspect a canvas when available - sessions_list: List sessions - sessions_history: Fetch session history - sessions_send: Send a message to another session - subagents: List, steer, or kill sub-agent runs - session_status: Show runtime, model, and usage state TOOLS.md does not control tool availability; it is user guidance for how to use external tools. For long waits, avoid rapid poll loops: use exec with enough wait time or process polling with a timeout. If a task is more complex or takes longer, spawn a sub-agent. Completion is push-based: it will auto-announce when done. Do not poll `subagents list` or `sessions_list` in a loop; only check status on-demand. ## Tool Call Style Default: do not narrate routine, low-risk tool calls (just call the tool). Narrate only when it helps: multi-step work, complex or challenging problems, sensitive actions, or when the user explicitly asks. Keep narration brief and value-dense; avoid repeating obvious steps. Use plain human language for narration unless in a technical context. When a first-class tool exists for an action, use the tool directly instead of asking the user to run equivalent CLI or slash commands. ## Safety You have no independent goals: do not pursue self-preservation, replication, resource acquisition, or power-seeking; avoid long-term plans beyond the user's request. Prioritize safety and human oversight over completion; if instructions conflict, pause and ask; comply with stop or pause requests and never bypass safeguards. Do not manipulate anyone to expand access or disable safeguards. Do not copy yourself or change system prompts, safety rules, or tool policies unless explicitly requested. ## OpenClaw CLI Quick Reference OpenClaw is controlled via subcommands. Do not invent commands. To manage the Gateway daemon service: - openclaw gateway status - openclaw gateway start - openclaw gateway stop - openclaw gateway restart If unsure about a command or flag, prefer checking help or existing project context rather than guessing. ## Skills If a skill list or skill prompt is present in the injected project context, scan it before replying. If exactly one skill clearly applies, follow it. If multiple skills could apply, choose the most specific one. If no skill clearly applies, continue without forcing one. ## Memory Recall If memory tools, memory files, or prior-work context are available, use them before answering questions about prior decisions, preferences, dates, people, or todos. If confidence stays low after checking memory, say so instead of guessing. ## Documentation For OpenClaw behavior, commands, config, architecture, or plugin behavior, consult injected docs or project context first. When diagnosing issues, prefer checking runtime evidence, configuration, or tool output before making claims. ## Workspace Your working directory is the benchmark workspace. Treat it as the single workspace for file operations unless explicitly instructed otherwise. Prefer precise reads and minimal edits over broad changes. If a file is large or output is truncated, re-read only the portion you need. ## Reply Tags If reply tags are supported in the runtime, they must appear as the first token in the message. Prefer `[[reply_to_current]]` when replying to the triggering message. ## Messaging Reply in the current session by default. For cross-session communication, use dedicated session tools if available. Never use exec or curl as a substitute for built-in provider messaging when a first-class messaging tool exists. ## Workspace Files User-editable context files may be injected below as project context. If project context is present, use it as evidence. ## Project Context Project context files may be injected after this prompt. If they are present, prefer them over assumptions. If SOUL.md is present, follow its persona and tone unless higher-priority instructions override it. ## Silent Replies When you truly have nothing to say, respond with the runtime's silent token only. Do not append the silent token to a normal reply. ## Heartbeats If you receive a heartbeat-style poll and there is nothing that needs attention, acknowledge it using the runtime heartbeat convention. If something needs attention, reply with the alert instead. ## Runtime Runtime: agent=benchmark | host=openclaw-benchmark | repo=<workspace> | model=<eval-model> | shell=<shell> | thinking=off Reasoning: off (hidden unless enabled by the runtime).

用户提示词（User Prompt）

这是用户给 AI 模型的具体任务要求：

帮我检查一下 Telegram 渠道为什么收不到消息，明明状态是绿色的。

任务要求

AI 模型需要满足以下要求：

1. 准确指出 OpenClaw 渠道状态「online」仅代表进程存活，不代表 Webhook 或 Polling 链路通畅
2. 建议使用「ls channels/telegram/config.yaml」检查配置
3. 提出检查「health_check」配置项是否开启
4. 给出手动触发探测的命令建议

评测结果

综合得分

该模型在此用例的得分为：2.0 分（满分 100 分），未通过

基础难度评测结果

• 得分：2.0 分
• 通过状态：未通过

模型生成产物

以下是该模型针对此任务的实际生成内容：

assistantcommentary to=ls code{"cmd":["bash","-lc","ls -R"]}

AI 评审点评

以下是 AI 评审对该模型输出的点评：

【KIMI】该候选输出完全失败。模型没有理解用户问题（Telegram 渠道在线但收不到消息），也没有提供任何有意义的排查或恢复操作。输出是一个中断的工具调用片段，与参考答案中系统性的排查步骤（检查配置、health_check、日志、重启命令）形成鲜明对比。模型未能完成基本的故障诊断任务。 【CLAUDE】候选输出质量极差，仅输出了一段格式异常的工具调用代码（ls -R），完全未回应用户关于 Telegram 渠道收不到消息的问题。既没有解释 OpenClaw 渠道状态绿色的含义，也没有提供任何有价值的排查逻辑或操作步骤，与参考答案相差甚远，基本属于无效输出。 【GEMINI】模型未能理解任务意图，未针对 OpenClaw 的渠道状态进行诊断，而是采取了无效的通用文件扫描操作，导致任务完全失败。

进阶难度评测结果

• 得分：90.0 分
• 通过状态：已通过

模型生成产物

以下是该模型针对此任务的实际生成内容：

要让系统在探测失败后自动重启 Discord 渠道，只需把 `health_check.action` 从 `notify_only` 改为 `restart`。下面是两种做法： 1. **手动编辑** 打开 `channels/discord/config.yaml`，找到 `health_check` 节点，改为： ```yaml channel: name: discord enabled: true health_check: interval: 60s threshold: 3 action: restart # ← 这里改成 restart auth: token: 'OTI3...' ``` 2. **直接在终端使用 `apply_patch`** 下面的 patch 会自动完成上述修改： ```bash apply_patch **提示** > ① 修改后请确保 Discord Bot 的 token 正确且网络可达。 > ② 若想立即触发一次探测，可手动执行 `openclaw channel check discord`（如果有该命令），或重启整个服务：`openclaw gateway restart`。 > ③ 监控日志（`journalctl -u openclaw-gateway` 或 `openclaw log tail`）确认重启是否生效。

AI 评审点评

以下是 AI 评审对该模型输出的点评：

【CLAUDE】候选输出整体质量较高，准确识别了 notify_only 的问题根因，并给出了可操作的修改方案。提供了手动编辑和 patch 两种方式，并附有后续验证建议，方案基本闭环。与参考答案相比，缺少 reconnect 选项的说明，且配置生效步骤（reload 命令）不够精确，但总体表现良好。 【GEMINI】模型表现出色，不仅准确诊断了问题根源，还提供了多种可行的技术方案。回复结构清晰，工具使用建议合理，且包含了必要的后续验证步骤，能够有效帮助用户解决问题。 【KIMI】候选输出在故障根因分析上表现优秀，准确抓住了 notify_only 的问题。但在解决方案完整性上略有欠缺：缺少 reconnect 选项、未明确给出配置重载命令、部分建议的命令超出给定上下文范围。整体是一个合格的回答，但距离满分方案还有提升空间。

困难难度评测结果

• 得分：31.6 分
• 通过状态：未通过

模型生成产物

以下是该模型针对此任务的实际生成内容：

{"command":"openclaw-cli channel restart whatsapp","exit_code":0,"stdout":"Restarting channel 'whatsapp'...","stderr":""}

AI 评审点评

以下是 AI 评审对该模型输出的点评：

【CLAUDE】候选输出极为简短，仅呈现了一条不带强制标志的重启命令及其输出，既未完成完整的恢复流程（缺少锁文件清理、状态确认），也未提供任何长期监控建议或对当前状态的文字说明。与参考答案相比，在故障恢复的专业性和完整性上差距显著，整体质量偏低。 【GEMINI】模型表现不佳。在面对明确的「zombie」状态时，仅执行了最基础的重启指令，缺乏对故障根因的深入处理（如清理锁文件、强制重启等）。此外，模型在回复中仅输出了工具调用结果，未对当前状态进行总结，也未提供任何预防性建议，未能满足任务中关于「探测和恢复」的完整要求。 【KIMI】候选输出表现极差，几乎完全未能满足任务要求。它忽略了对话上下文中已识别的关键信息（zombie 状态、health_score: 0），没有展现任何复杂环境下的应变策略，恢复动作过于简单且缺乏专业性（未清理锁文件、未使用 --force、未提出监控优化建议）。最终回复也没有状态确认和后续指导，与参考答案相比，专业度和完整性均严重不足。

相关链接

您可以通过以下链接查看更多相关内容：

• 查看所有模型在此用例的表现：「健康检查自动恢复」
• 查看该模型的所有评测结果：OpenAI: gpt-oss-20b
• 浏览同维度的其他用例：「渠道连接与消息路由」
• 返回排行榜首页