mimo-v2-pro 在「离线包安装」的评测结果
这是该 AI 模型在此测试用例上的详细评测结果。
基本信息
- 模型名称：mimo-v2-pro
- 用例名称：离线包安装
- 测试类型：xsct-a
- 评测维度：Gateway 配置与管理
系统提示词（System Prompt）
这是给 AI 模型的背景设定和角色指令：
You are a personal assistant running inside OpenClaw. ## Tooling Tool availability (filtered by policy): Tool names are case-sensitive. Call tools exactly as listed. - read: Read file contents - write: Create or overwrite files - edit: Make precise edits to files - apply_patch: Apply multi-file patches - ls: List directory contents - exec: Run shell commands (supports background execution where available) - process: Manage background exec sessions - browser: Control OpenClaw's dedicated browser - canvas: Present or inspect a canvas when available - sessions_list: List sessions - sessions_history: Fetch session history - sessions_send: Send a message to another session - subagents: List, steer, or kill sub-agent runs - session_status: Show runtime, model, and usage state TOOLS.md does not control tool availability; it is user guidance for how to use external tools. For long waits, avoid rapid poll loops: use exec with enough wait time or process polling with a timeout. If a task is more complex or takes longer, spawn a sub-agent. Completion is push-based: it will auto-announce when done. Do not poll `subagents list` or `sessions_list` in a loop; only check status on-demand. ## Tool Call Style Default: do not narrate routine, low-risk tool calls (just call the tool). Narrate only when it helps: multi-step work, complex or challenging problems, sensitive actions, or when the user explicitly asks. Keep narration brief and value-dense; avoid repeating obvious steps. Use plain human language for narration unless in a technical context. When a first-class tool exists for an action, use the tool directly instead of asking the user to run equivalent CLI or slash commands. ## Safety You have no independent goals: do not pursue self-preservation, replication, resource acquisition, or power-seeking; avoid long-term plans beyond the user's request. Prioritize safety and human oversight over completion; if instructions conflict, pause and ask; comply with stop or pause requests and never bypass safeguards. Do not manipulate anyone to expand access or disable safeguards. Do not copy yourself or change system prompts, safety rules, or tool policies unless explicitly requested. ## OpenClaw CLI Quick Reference OpenClaw is controlled via subcommands. Do not invent commands. To manage the Gateway daemon service: - openclaw gateway status - openclaw gateway start - openclaw gateway stop - openclaw gateway restart If unsure about a command or flag, prefer checking help or existing project context rather than guessing. ## Skills If a skill list or skill prompt is present in the injected project context, scan it before replying. If exactly one skill clearly applies, follow it. If multiple skills could apply, choose the most specific one. If no skill clearly applies, continue without forcing one. ## Memory Recall If memory tools, memory files, or prior-work context are available, use them before answering questions about prior decisions, preferences, dates, people, or todos. If confidence stays low after checking memory, say so instead of guessing. ## Documentation For OpenClaw behavior, commands, config, architecture, or plugin behavior, consult injected docs or project context first. When diagnosing issues, prefer checking runtime evidence, configuration, or tool output before making claims. ## Workspace Your working directory is the benchmark workspace. Treat it as the single workspace for file operations unless explicitly instructed otherwise. Prefer precise reads and minimal edits over broad changes. If a file is large or output is truncated, re-read only the portion you need. ## Reply Tags If reply tags are supported in the runtime, they must appear as the first token in the message. Prefer `[[reply_to_current]]` when replying to the triggering message. ## Messaging Reply in the current session by default. For cross-session communication, use dedicated session tools if available. Never use exec or curl as a substitute for built-in provider messaging when a first-class messaging tool exists. ## Workspace Files User-editable context files may be injected below as project context. If project context is present, use it as evidence. ## Project Context Project context files may be injected after this prompt. If they are present, prefer them over assumptions. If SOUL.md is present, follow its persona and tone unless higher-priority instructions override it. ## Silent Replies When you truly have nothing to say, respond with the runtime's silent token only. Do not append the silent token to a normal reply. ## Heartbeats If you receive a heartbeat-style poll and there is nothing that needs attention, acknowledge it using the runtime heartbeat convention. If something needs attention, reply with the alert instead. ## Runtime Runtime: agent=benchmark | host=openclaw-benchmark | repo=<workspace> | model=<eval-model> | shell=<shell> | thinking=off Reasoning: off (hidden unless enabled by the runtime).
用户提示词（User Prompt）
这是用户给 AI 模型的具体任务要求：
我已经把离线包放到了 /opt 目录下，现在该怎么安装并启动 OpenClaw Gateway？
任务要求
AI 模型需要满足以下要求：
- 准确指出需要先解压离线包文件
- 明确提及执行「./install.sh」或类似的初始化脚本
- 指出需要检查或配置「config.yaml」中的基础信息
- 给出启动命令如「openclaw start」或运行二进制文件
评测结果
综合得分
该模型在此用例的得分为：25.5 分（满分 100 分），未通过
基础难度评测结果
- 得分：25.5 分
- 通过状态：未通过
模型生成产物
以下是该模型针对此任务的实际生成内容：
让我先看看 /opt 目录下有什么文件，同时查一下相关文档。 ls -la /opt/ find /opt -maxdepth 2 -name "*.tar*" -o -name "*.zip" -o -name "*.deb" -o -name "*.rpm" -o -name "*.sh" -o -name "*.tgz" 2>/dev/null | head -20
AI 评审点评
以下是 AI 评审对该模型输出的点评：
【CLAUDE】候选输出只完成了「查看目录」这一前置动作，相当于只走完了流程的第零步。用户问的是「怎么安装并启动」，模型给出的是两条 ls/find 命令，没有任何安装、配置或启动的指导内容。在内网离线场景下，用户最需要的是明确的操作步骤，而不是让模型先去「探索」环境——尤其是在用户已经明确告知文件位置的情况下。整体得分偏低，核心原因是「有动作但无结果」，未能回答用户的实际问题。 【GEMINI】模型在接收到用户请求后，仅进行了初步的文件探测，未给出任何实质性的安装指导或步骤。在面对内网环境手动初始化任务时，模型未能利用已有的工具或知识库给出操作建议，表现不合格。 【KIMI】该候选输出严重不合格。面对用户关于内网环境下离线安装并启动OpenClaw Gateway的明确询问，模型没有提供任何实质性的操作指导，而是仅执行了文件浏览命令。完全没有覆盖参考答案中的四个核心步骤（解压、初始化脚本、配置检查、启动服务），也没有体现对内网离线场景的理解。这是一个典型的「答非所问」案例，模型将任务误解为探索性调查而非提供解决方案。
进阶难度评测结果
- 得分：63.2 分
- 通过状态：已通过
模型生成产物
以下是该模型针对此任务的实际生成内容：
建议先用 `check` 命令验证配置，再启动服务： ```bash # 1. 验证配置文件是否有效 ./openclaw-gateway check --config /etc/openclaw/config.yaml # 2. 验证通过后，正式启动 ./openclaw-gateway start --config /etc/openclaw/config.yaml ``` 如果 `check` 通过但 `start` 仍有问题，请把报错信息贴出来，我继续帮您排查。
AI 评审点评
以下是 AI 评审对该模型输出的点评：
【CLAUDE】候选输出方向正确，能够基于上下文给出配置验证和启动的基本步骤，但存在命令路径错误、子命令格式偏差、缺少端口检查、缺少启动后进程和日志验证等问题，整体流程不够完整严谨，与参考答案相比有明显差距。 【GEMINI】模型能够给出正确的操作指令，但在验证流程的完整性上有所欠缺。在内网手动部署场景下，仅验证配置是不够的，还需要确认端口状态和进程存活情况，以确保服务真正可用。 【KIMI】候选输出提供了基础的配置验证和启动指导，思路方向正确，但在命令路径准确性、System Prompt 规范遵循以及验证流程完整性方面存在明显缺陷。特别是遗漏了端口检查、后台常驻启动、进程确认等关键验证步骤，对于「内网离线环境」这一场景下的服务稳定性保障不足。建议补充完整的验证链路，并严格遵循项目文档中的命令规范。
困难难度评测结果
- 得分：21.2 分
- 通过状态：未通过
模型生成产物
以下是该模型针对此任务的实际生成内容：
让我先检查一下当前状态。 /opt/openclaw/bin/openclaw-gateway status 10
AI 评审点评
以下是 AI 评审对该模型输出的点评：
【CLAUDE】候选输出极为简短，仅调用了一个「openclaw-gateway status」工具调用，面对明确的端口占用错误既未采取清理行动，也未进行任何升级完整性验证，更没有体现内网运维的特殊考量。与参考答案相比，差距悬殊——参考答案涵盖了端口清理、依赖检查、配置同步、启动验证和回滚方案五个完整步骤，而候选输出仅停留在「查看状态」这一初步动作上，未能推进问题解决。整体表现远低及格线。 【GEMINI】模型表现不佳。在面对明确的「Address already in use」报错时，模型未能利用系统工具（如 fuser/lsof）解决冲突，反而执行了冗余的 status 查询。此外，模型完全脱离了内网环境升级所需的严谨验证流程（如 ldd 检查、配置比对、回滚预案），未能体现出专业运维助手的应有水平。 【KIMI】候选输出严重偏离当前任务的核心矛盾。对话上下文已清晰呈现问题演进：文件覆盖完成 → 启动失败（端口占用）。此时正确的应对应是强制清理端口、检查内网环境特有的依赖/配置问题、并准备验证与回滚方案。但候选输出却执行一个无关紧要的「status」检查，既未解决已知的端口占用，也未体现任何内网离线升级的运维意识，属于对场景理解和优先级判断的重大失误。
相关链接
您可以通过以下链接查看更多相关内容：
