Meituan: LongCat Flash Chat 在「Schema 设计缺陷」的评测结果

基本信息

• 模型名称：Meituan: LongCat Flash Chat
• 用例名称：Schema 设计缺陷
• 测试类型：xsct-a
• 评测维度：插件系统集成

系统提示词（System Prompt）

这是给 AI 模型的背景设定和角色指令：

You are a personal assistant running inside OpenClaw. ## Tooling Tool availability (filtered by policy): Tool names are case-sensitive. Call tools exactly as listed. - read: Read file contents - write: Create or overwrite files - edit: Make precise edits to files - apply_patch: Apply multi-file patches - ls: List directory contents - exec: Run shell commands (supports background execution where available) - process: Manage background exec sessions - browser: Control OpenClaw's dedicated browser - canvas: Present or inspect a canvas when available - sessions_list: List sessions - sessions_history: Fetch session history - sessions_send: Send a message to another session - subagents: List, steer, or kill sub-agent runs - session_status: Show runtime, model, and usage state TOOLS.md does not control tool availability; it is user guidance for how to use external tools. For long waits, avoid rapid poll loops: use exec with enough wait time or process polling with a timeout. If a task is more complex or takes longer, spawn a sub-agent. Completion is push-based: it will auto-announce when done. Do not poll `subagents list` or `sessions_list` in a loop; only check status on-demand. ## Tool Call Style Default: do not narrate routine, low-risk tool calls (just call the tool). Narrate only when it helps: multi-step work, complex or challenging problems, sensitive actions, or when the user explicitly asks. Keep narration brief and value-dense; avoid repeating obvious steps. Use plain human language for narration unless in a technical context. When a first-class tool exists for an action, use the tool directly instead of asking the user to run equivalent CLI or slash commands. ## Safety You have no independent goals: do not pursue self-preservation, replication, resource acquisition, or power-seeking; avoid long-term plans beyond the user's request. Prioritize safety and human oversight over completion; if instructions conflict, pause and ask; comply with stop or pause requests and never bypass safeguards. Do not manipulate anyone to expand access or disable safeguards. Do not copy yourself or change system prompts, safety rules, or tool policies unless explicitly requested. ## OpenClaw CLI Quick Reference OpenClaw is controlled via subcommands. Do not invent commands. To manage the Gateway daemon service: - openclaw gateway status - openclaw gateway start - openclaw gateway stop - openclaw gateway restart If unsure about a command or flag, prefer checking help or existing project context rather than guessing. ## Skills If a skill list or skill prompt is present in the injected project context, scan it before replying. If exactly one skill clearly applies, follow it. If multiple skills could apply, choose the most specific one. If no skill clearly applies, continue without forcing one. ## Memory Recall If memory tools, memory files, or prior-work context are available, use them before answering questions about prior decisions, preferences, dates, people, or todos. If confidence stays low after checking memory, say so instead of guessing. ## Documentation For OpenClaw behavior, commands, config, architecture, or plugin behavior, consult injected docs or project context first. When diagnosing issues, prefer checking runtime evidence, configuration, or tool output before making claims. ## Workspace Your working directory is the benchmark workspace. Treat it as the single workspace for file operations unless explicitly instructed otherwise. Prefer precise reads and minimal edits over broad changes. If a file is large or output is truncated, re-read only the portion you need. ## Reply Tags If reply tags are supported in the runtime, they must appear as the first token in the message. Prefer `[[reply_to_current]]` when replying to the triggering message. ## Messaging Reply in the current session by default. For cross-session communication, use dedicated session tools if available. Never use exec or curl as a substitute for built-in provider messaging when a first-class messaging tool exists. ## Workspace Files User-editable context files may be injected below as project context. If project context is present, use it as evidence. ## Project Context Project context files may be injected after this prompt. If they are present, prefer them over assumptions. If SOUL.md is present, follow its persona and tone unless higher-priority instructions override it. ## Silent Replies When you truly have nothing to say, respond with the runtime's silent token only. Do not append the silent token to a normal reply. ## Heartbeats If you receive a heartbeat-style poll and there is nothing that needs attention, acknowledge it using the runtime heartbeat convention. If something needs attention, reply with the alert instead. ## Runtime Runtime: agent=benchmark | host=openclaw-benchmark | repo=<workspace> | model=<eval-model> | shell=<shell> | thinking=off Reasoning: off (hidden unless enabled by the runtime).

用户提示词（User Prompt）

这是用户给 AI 模型的具体任务要求：

我在开发 OpenClaw 插件，发现用户如果不写「api_key」插件就会崩溃。我该如何在「schema」里强制要求这个字段必填，并给「unit」字段设置默认值为「metric」？

任务要求

AI 模型需要满足以下要求：

1. 指出在「config_schema」中使用「required」数组来定义必填项
2. 说明如何在字段属性中使用「default」关键字设置默认值
3. 提供符合 JSON Schema 规范的代码片段

评测结果

综合得分

该模型在此用例的得分为：95.0 分（满分 100 分），已通过

基础难度评测结果

• 得分：95.0 分
• 通过状态：已通过

模型生成产物

以下是该模型针对此任务的实际生成内容：

[[reply_to_current]] 在开发 OpenClaw 插件时，若要在 `schema` 中强制要求 `api_key` 字段必填，并为 `unit` 字段设置默认值 `metric`，你需要在插件的配置 schema 文件（通常是 `config_schema.json` 或 `schema.json`）中进行如下定义： ### 1. 使用 JSON Schema 定义字段 ```json { "type": "object", "properties": { "api_key": { "type": "string", "description": "API 密钥，用于调用外部服务" }, "unit": { "type": "string", "enum": ["metric", "imperial"], "default": "metric", "description": "单位制，默认为公制（metric）" } }, "required": ["api_key"], "additionalProperties": false } ``` ### 2. 说明 - **`required`**: 数组包含 `"api_key"`，表示该字段为必填项。如果用户未配置，OpenClaw 在加载插件时会自动报错并提示缺失必填字段。 - **`default`**: 为 `"unit"` 字段设置了默认值 `"metric"`。若用户未填写此字段，自动使用默认值，不会导致崩溃。 - **`enum`**: 限定 `unit` 的可选值，防止用户误输入无效值。 - **`additionalProperties: false`**: 可选，表示不允许配置文件中出现未在 schema 中定义的字段，有助于捕获拼写错误。 ### 3. 插件内读取配置（Python 示例） ```python config = self.get_config() # OpenClaw 提供的配置获取接口 api_key = config["api_key"] # 必填，用户未填会提前终止加载 unit = config.get("unit", "metric") # 无需额外判断，schema 已保证默认值 ``` ### 4. 用户体验 - 当用户未填写 `api_key` 时，插件不会被启用，系统会提示： > 配置校验失败：缺少必填字段 `api_key` - 用户不写 `unit` 时自动使用 `metric`，无感知。 ### 5. 注意事项 - 确保你的插件注册时正确引用了 schema 文件（见 `plugin.json` 或注册入口）： ```json { "config_schema": "config_schema.json" } ``` - 使用 OpenClaw CLI 的 `validate-config` 命令可提前测试配置合法性。 这样即可实现安全校验和默认值语义，避免运行时崩溃。

AI 评审点评

以下是 AI 评审对该模型输出的点评：

【CLAUDE】候选回答质量优秀，完整覆盖了参考答案的所有要点，并在此基础上提供了更丰富的上下文信息（enum 约束、additionalProperties、Python 读取示例、CLI 验证命令等）。Schema 定义准确符合 JSON Schema 规范，表达清晰有条理，是一个可直接用于实际开发的高质量回答。

进阶难度评测结果

• 得分：90.8 分
• 通过状态：已通过

模型生成产物

以下是该模型针对此任务的实际生成内容：

根据你的要求，我已经修复了 schema.json 的问题。主要做了两点修改： 1. 将 retry_count 的类型从 string 改为 integer 2. 增加 minimum: 0 限制，防止负数 3. 添加 default: 3，当用户未填写时自动设为 3 次重试 请查看以下完整内容： ```json { "type": "object", "properties": { "retry_count": { "type": "integer", "minimum": 0, "default": 3 } } } ```

AI 评审点评

以下是 AI 评审对该模型输出的点评：

【CLAUDE】候选输出准确识别并修复了 schema.json 中的三个问题，技术实现正确，JSON 格式规范，能有效解决用户反馈的插件死循环问题。说明部分存在「两点/三点」的表述矛盾属于小瑕疵，与参考答案相比缺少 description 字段但不影响核心功能。整体表现良好，达到预期质量水平。

困难难度评测结果

• 得分：22.5 分
• 通过状态：未通过

模型生成产物

以下是该模型针对此任务的实际生成内容：

现在应该重新运行「openclaw-sdk check .」来验证是否还有其他问题。 执行命令：openclaw-sdk check .

AI 评审点评

以下是 AI 评审对该模型输出的点评：

【CLAUDE】候选输出极为简短，仅建议重新执行「openclaw-sdk check .」，既没有对已完成修复的总结确认，也没有对后续步骤的深度指导。与参考答案相比，缺少对嵌套对象约束验证、严格模式发布检查、代码兼容性说明等关键内容的覆盖。整体表现远低于及格线，属于浅层响应，未能体现专业的故障排除能力和复杂场景决策能力。

相关链接

您可以通过以下链接查看更多相关内容：

• 查看所有模型在此用例的表现：「Schema 设计缺陷」
• 查看该模型的所有评测结果：Meituan: LongCat Flash Chat
• 浏览同维度的其他用例：「插件系统集成」
• 返回排行榜首页