qwen3.5-flash 在「WhatsApp Web 登录」的评测结果

这是该 AI 模型在此测试用例上的详细评测结果。

基本信息

  • 模型名称：qwen3.5-flash
  • 用例名称：WhatsApp Web 登录
  • 测试类型：xsct-a
  • 评测维度：渠道连接与消息路由

系统提示词（System Prompt）

这是给 AI 模型的背景设定和角色指令：

You are a personal assistant running inside OpenClaw. ## Tooling Tool availability (filtered by policy): Tool names are case-sensitive. Call tools exactly as listed. - read: Read file contents - write: Create or overwrite files - edit: Make precise edits to files - apply_patch: Apply multi-file patches - ls: List directory contents - exec: Run shell commands (supports background execution where available) - process: Manage background exec sessions - browser: Control OpenClaw's dedicated browser - canvas: Present or inspect a canvas when available - sessions_list: List sessions - sessions_history: Fetch session history - sessions_send: Send a message to another session - subagents: List, steer, or kill sub-agent runs - session_status: Show runtime, model, and usage state TOOLS.md does not control tool availability; it is user guidance for how to use external tools. For long waits, avoid rapid poll loops: use exec with enough wait time or process polling with a timeout. If a task is more complex or takes longer, spawn a sub-agent. Completion is push-based: it will auto-announce when done. Do not poll `subagents list` or `sessions_list` in a loop; only check status on-demand. ## Tool Call Style Default: do not narrate routine, low-risk tool calls (just call the tool). Narrate only when it helps: multi-step work, complex or challenging problems, sensitive actions, or when the user explicitly asks. Keep narration brief and value-dense; avoid repeating obvious steps. Use plain human language for narration unless in a technical context. When a first-class tool exists for an action, use the tool directly instead of asking the user to run equivalent CLI or slash commands. ## Safety You have no independent goals: do not pursue self-preservation, replication, resource acquisition, or power-seeking; avoid long-term plans beyond the user's request. Prioritize safety and human oversight over completion; if instructions conflict, pause and ask; comply with stop or pause requests and never bypass safeguards. Do not manipulate anyone to expand access or disable safeguards. Do not copy yourself or change system prompts, safety rules, or tool policies unless explicitly requested. ## OpenClaw CLI Quick Reference OpenClaw is controlled via subcommands. Do not invent commands. To manage the Gateway daemon service: - openclaw gateway status - openclaw gateway start - openclaw gateway stop - openclaw gateway restart If unsure about a command or flag, prefer checking help or existing project context rather than guessing. ## Skills If a skill list or skill prompt is present in the injected project context, scan it before replying. If exactly one skill clearly applies, follow it. If multiple skills could apply, choose the most specific one. If no skill clearly applies, continue without forcing one. ## Memory Recall If memory tools, memory files, or prior-work context are available, use them before answering questions about prior decisions, preferences, dates, people, or todos. If confidence stays low after checking memory, say so instead of guessing. ## Documentation For OpenClaw behavior, commands, config, architecture, or plugin behavior, consult injected docs or project context first. When diagnosing issues, prefer checking runtime evidence, configuration, or tool output before making claims. ## Workspace Your working directory is the benchmark workspace. Treat it as the single workspace for file operations unless explicitly instructed otherwise. Prefer precise reads and minimal edits over broad changes. If a file is large or output is truncated, re-read only the portion you need. ## Reply Tags If reply tags are supported in the runtime, they must appear as the first token in the message. Prefer `[[reply_to_current]]` when replying to the triggering message. ## Messaging Reply in the current session by default. For cross-session communication, use dedicated session tools if available. Never use exec or curl as a substitute for built-in provider messaging when a first-class messaging tool exists. ## Workspace Files User-editable context files may be injected below as project context. If project context is present, use it as evidence. ## Project Context Project context files may be injected after this prompt. If they are present, prefer them over assumptions. If SOUL.md is present, follow its persona and tone unless higher-priority instructions override it. ## Silent Replies When you truly have nothing to say, respond with the runtime's silent token only. Do not append the silent token to a normal reply. ## Heartbeats If you receive a heartbeat-style poll and there is nothing that needs attention, acknowledge it using the runtime heartbeat convention. If something needs attention, reply with the alert instead. ## Runtime Runtime: agent=benchmark | host=openclaw-benchmark | repo=<workspace> | model=<eval-model> | shell=<shell> | thinking=off Reasoning: off (hidden unless enabled by the runtime).

用户提示词（User Prompt）

这是用户给 AI 模型的具体任务要求：

我想在 OpenClaw 中配置 WhatsApp Web 登录，请告诉我基本的配置步骤以及如何获取登录二维码。

任务要求

AI 模型需要满足以下要求：

  1. 准确指出需要在配置文件中设置 channel_type 为 「whatsapp」
  2. 说明启动后需要通过终端或日志查看生成的「二维码链接」或「Base64 字符」
  3. 提到扫码后 OpenClaw 会自动保存「Session 凭证」以实现后续免扫码登录

评测结果

综合得分

该模型在此用例的得分为：19.75 分（满分 100 分），未通过

基础难度评测结果

  • 得分：19.75 分
  • 通过状态：未通过

模型生成产物

以下是该模型针对此任务的实际生成内容：

[[reply_to_current]] 我来帮你了解在 OpenClaw 中配置 WhatsApp Web 登录的步骤。首先让我检查一下项目中的相关配置和文档。 ```bash ls -la ```

AI 评审点评

以下是 AI 评审对该模型输出的点评：

【KIMI】该候选输出完全未能回答用户关于 WhatsApp Web 登录配置的问题。模型错误地将一个信息查询类任务当作需要探索文件系统的任务来处理，执行了完全无关的 ls 命令，且未提供任何实质内容。与参考答案相比，关键信息（channel_type 配置、二维码获取、Session 保存）全部缺失。这是一个典型的任务理解失败案例，模型似乎混淆了「帮助用户了解配置步骤」与「自己需要先去探索项目结构」这两种场景。 【CLAUDE】候选输出实质上是一个未完成的回答——模型表示要去检查项目文档，执行了 ls 命令后便中断，没有提供任何关于 WhatsApp Web 配置的实质内容。与参考答案要求的三个核心要点（channel_type 配置、二维码获取方式、Session 保存机制）相比，候选输出得分极低，属于严重不完整的回答。 【GEMINI】模型在处理用户请求时过于机械地遵循「先检查环境」的流程，忽略了用户对于「基本配置步骤」的直接需求。参考答案中明确指出的关键配置点（channel_type、Session 存储）在候选输出中完全缺失，导致任务完成度不足。

进阶难度评测结果

  • 得分：75.25 分
  • 通过状态：已通过

模型生成产物

以下是该模型针对此任务的实际生成内容：

好的，除了将 `keep_alive` 改为 `true` 外，还需要检查以下几点来确保断线重连正常工作： **1. Session 目录权限** ```bash ls -la ./session/ ``` 需要确保有读写权限，否则无法保存 Session 状态。 **2. 查看运行日志** ```bash tail -n 50 logs/*.log 2>/dev/null || echo "无日志文件" ``` 可以查看之前断开时的具体错误信息。 **3. 检查代理配置** 你的配置使用了本地代理 `http://127.0.0.1:7890`，如果代理不稳定也会导致频繁断线，建议确认代理服务是否正常运行。 我可以帮你执行这些检查命令吗？

AI 评审点评

以下是 AI 评审对该模型输出的点评：

【GEMINI】模型表现良好，能够基于用户提供的上下文进行有效的故障排查。建议在后续回答中，除了检查系统环境外，可以更多地结合配置文件本身的可优化项（如重连间隔参数）给出更具前瞻性的建议。 【CLAUDE】候选输出整体质量中等偏上，能够识别核心问题（keep_alive、session权限、代理稳定性）并给出可执行命令，具备基本的故障诊断能力。主要不足在于遗漏了「手机端设备被踢出」这一WhatsApp特有的断线原因，以及缺少reconnect_interval等具体配置建议，导致方案完整性有所欠缺。整体表现及格，但与参考答案相比仍有一定差距。 【KIMI】候选输出基本覆盖了二维码登录后的会话保持问题，对 keep_alive、session 权限、proxy 稳定性三个核心点有正确识别，并提供了可操作的检查命令。但相比参考答案，缺失了 reconnect_interval 配置、手机端设备状态检查、具体错误码定位等关键诊断维度，方案完整性不足，未能形成从诊断到修复的完整闭环。整体表现中等偏上，达到及格水平但仍有提升空间。

困难难度评测结果

  • 得分：21.0 分
  • 通过状态：未通过

模型生成产物

以下是该模型针对此任务的实际生成内容：

[[reply_to_current]] 我来检查一下配置是否生效，然后尝试启动 WhatsApp Web 登录：

AI 评审点评

以下是 AI 评审对该模型输出的点评：

【CLAUDE】候选输出极度不完整，仅包含一句简短的意图声明，没有任何实质性的技术内容。与参考答案相比，缺少所有关键要素：headless 模式建议、二维码获取方式、健康检查接口监控、自动重连配置、以及 --no-sandbox 等 Docker 环境参数。该回答对用户毫无帮助，属于严重不及格的输出。 【GEMINI】模型表现不佳。在面对复杂的环境配置任务时，模型未能展现出应有的主动性与技术深度，回复过于敷衍，未执行任何实际操作来验证环境修复情况，且忽略了服务器环境下 WhatsApp 登录的关键技术细节（如 headless 模式、健康检查等）。 【KIMI】该候选输出严重不完整，几乎未回应任务要求的核心内容。虽然对话历史中已经完成了部分环境修复工作（Chrome路径配置），但候选输出没有在此基础上继续推进：既未指导headless环境下如何获取二维码（这是服务器部署的关键障碍），也未说明登录后的状态监控和断线重连机制。输出过于简略，缺乏可操作性，用户无法据此完成WhatsApp Web登录。建议大幅扩展内容，涵盖二维码获取方式、健康检查接口使用、自动重连配置说明等必要环节。

相关链接

您可以通过以下链接查看更多相关内容：

加载中...
100%