```
Some checks failed
CI / pytest (push) Has been cancelled
CI / gui-unit (push) Has been cancelled
CI / gui-e2e (push) Has been cancelled

feat(agent): 添加技能自动路由功能
- 引入 SkillRouter 实现根据用户消息自动推荐技能
- 在构建引擎时集成技能路由器
- 从对话历史中提取最后一条用户消息作为路由输入
- 为技能添加触发关键词和中文标题字段支持

refactor(browser): 重构浏览器自动化为子进程架构

- 将 Playwright 浏览器控制移至独立的子进程 worker
- 解决 PyInstaller 打包环境下 C 扩展兼容性问题
- 通过 JSON RPC 协议与浏览器 worker 通信
- 添加工具目录和脚本路径查找机制

feat(skills): 增强技能元数据和UI展示

- 为技能添加 triggers 和 title 字段
- 在技能商店中包含中文标题信息
- 添加 office-viz 技能优先级排序
- 在服务器管理器中返回技能标题

feat(gui): 实现技能选择器UI组件

- 添加带下拉菜单的技能选择器按钮
- 支持中文标题和拼音首字母显示
- 集成会话技能加载和状态管理
- 提供通用技能选项和已启用技能列表
```
This commit is contained in:
2026-09-14 17:36:02 +08:00
parent 6f402ffcee
commit 3e356b117a
10 changed files with 1493 additions and 306 deletions

465
.trae/rules/打包.md Normal file
View File

@@ -0,0 +1,465 @@
\# OpenMesh 打包流程说明
\## 概述
OpenMesh 是一个基于 Python (FastAPI) + React (Vite) 的 AI 智能助手平台,最终交付形态为 Windows 绿色免安装版。打包产物位于 \`dist/openmesh/\` 目录,用户双击 \`启动 OpenMesh.bat\` 即可运行。
\*\*\*
\## 打包产物目录结构
\`\`\`
dist/openmesh/
├── openmesh-server.exe # 后端服务主程序PyInstaller one-dir 模式)
├── openmesh-static.exe # 前端静态文件服务器
├── _internal/ # PyInstaller 运行时依赖Python 解释器 + 所有库)
├── frontend/ # 前端构建产物React + Vite
│ ├── index.html
│ └── assets/
├── data/ # 用户数据目录(首次启动后生成配置)
│ ├── config.toml
│ ├── coworker.db
│ ├── chat.db
│ └── skills/ # 已安装的 skills从 skills/ 初始化复制)
├── skills/ # 预置内置 skills首次启动复制到 data/skills
├── workspace/ # 默认工作目录
├── tools/ # 第三方命令行工具pandoc, tesseract, libreoffice
├── .venv/ # Python 虚拟环境(用于 skill 脚本执行)
├── 启动 OpenMesh.bat # 一键启动脚本
├── 卸载 OpenMesh.bat # 卸载脚本
└── 使用说明.txt # 使用说明
\`\`\`
\*\*\*
\## 打包步骤总览
完整打包分为 \*\*4 个阶段\*\*,顺序执行:
\`\`\`
阶段1: 前端构建 → 阶段2: 后端 PyInstaller 打包 → 阶段3: 静态服务器打包 → 阶段4: 组装发布包
\`\`\`
\*\*\*
\## 阶段 1前端构建
\### 目的
将 React + Vite 前端项目编译为静态 HTML/CSS/JS 资源。
\### 前置条件
\* Node.js >\\\= 16
\* 已执行 \`npm install\`
\### 操作命令
\`\`\`bash
cd surfaces/gui
npm run build
\`\`\`
\### 输入
\* \`surfaces/gui/src/\` — 前端源码
\* \`surfaces/gui/index.html\` — HTML 模板
\### 输出
\* \`surfaces/gui/dist/\` — 构建产物
  \* \`index.html\`
  \* \`assets/index-\*.js\`
  \* \`assets/index-\*.css\`
  \* 其他静态资源(字体、图片等)
\### 配置文件
\* \`surfaces/gui/vite.config.ts\` — Vite 配置
  \* \`base: "./"\` — 相对路径,支持从文件系统直接加载
  \* 构建输出到 \`dist/\`
\*\*\*
\## 阶段 2后端服务 PyInstaller 打包
\### 目的
将 Python 后端服务FastAPI + uvicorn打包为独立的 Windows 可执行程序及运行时目录。
\### 前置条件
\* Python 3.10+
\* 虚拟环境已安装所有依赖:\`pip install -e .\`
\* \`pyinstaller\` 已安装
\### 操作命令
\`\`\`bash
pyinstaller dist/openworker-server.spec
\`\`\`
\### Spec 文件说明
路径:\`dist/openworker-server.spec\`
关键配置:
\* \*\*入口\*\*\`packaging/server_entry.py\` → 调用 \`coworker.server.run\:main\`
\* \*\*模式\*\*one-direxe + \`_internal/\` 目录),比 onefile 启动快 6-7 秒
\* \*\*控制台\*\*\`console\=True\`uvicorn 需要 stdout窗口通过 CREATE\\_NO\\_WINDOW 隐藏)
\* \*\*收集的包\*\*coworker, aisuite, mcp, ddgs, uvicorn, certifi, websockets, pypdf, pypdfium2, boto3, botocore 等
\* \*\*排除\*\*tkinter, matplotlib, PIL, PyQt5 等不需要的库
\### 输出
\* \`dist/openmesh-server/\`
  \* \`openmesh-server.exe\` — 主程序
  \* \`_internal/\` — Python 运行时 + 所有依赖库
\*\*\*
\## 阶段 3静态文件服务器打包
\### 目的
打包一个极简的 HTTP 服务器,用于托管前端静态文件。
\### Spec 文件
路径:\`openmesh-static.spec\` 或 \`static-server.spec\`
入口:\`packaging/static_server.py\`
输出:\`dist/openmesh-static.exe\` 或 \`dist/static-server/static-server.exe\`
\*\*\*
\## 阶段 4组装发布包
\### 目的
将前端、后端、工具、配置等所有组件组装到 \`dist/openmesh/\` 目录,形成最终可交付的绿色版。
\### 组装清单
| 源位置 | 目标位置 | 说明 |
| ------------------------------------------ | ----------------------------------- | ---------- |
| \`dist/openmesh-server/openmesh-server.exe\` | \`dist/openmesh/openmesh-server.exe\` | 后端服务 |
| \`dist/openmesh-server/_internal/\` | \`dist/openmesh/_internal/\` | Python 运行时 |
| \`surfaces/gui/dist/\` | \`dist/openmesh/frontend/\` | 前端构建产物 |
| \`dist/openmesh-static.exe\` | \`dist/openmesh/openmesh-static.exe\` | 静态文件服务器 |
| \`skills/\` (项目内置) | \`dist/openmesh/skills/\` | 预置 skills |
| \`tools/\` (pandoc 等) | \`dist/openmesh/tools/\` | 第三方工具 |
| \`dist/config.template.toml\` | \`dist/openmesh/data/config.toml\` | 默认配置 |
| — | \`dist/openmesh/workspace/\` | 工作目录(空) |
| — | \`dist/openmesh/data/\` | 数据目录(空) |
| \`启动 OpenMesh.bat\` | \`dist/openmesh/启动 OpenMesh.bat\` | 启动脚本 |
| \`卸载 OpenMesh.bat\` | \`dist/openmesh/卸载 OpenMesh.bat\` | 卸载脚本 |
| \`使用说明.txt\` | \`dist/openmesh/使用说明.txt\` | 使用说明 |
\### 启动脚本逻辑
文件:\`dist/openmesh/启动 OpenMesh.bat\`
关键环境变量:
\`\`\`bat
set DATA_DIR\=%SCRIPT_DIR%data :: COWORKER_STATE_DIR 指向 data 目录
set SKILLS_DIR\=%DATA_DIR%\skills :: 实际 skills 目录
set BUILTIN_SKILLS\=%SCRIPT_DIR%skills :: 内置 skills 源
set COWORKER_DISABLE_AUTH\=1 :: 禁用认证(本地使用)
set COWORKER_STATE_DIR\=%DATA_DIR% :: 状态目录
set COWORKER_SCRATCH_BASE\=workspace :: 临时文件基准目录
\`\`\`
启动顺序:
1\. 创建 data、skills、workspace 目录
2\. 首次启动时,将内置 skills 复制到 data/skills
3\. 启动 \`openmesh-server.exe --port 8765 --cwd workspace\`(后端,端口 8765
4\. 启动 \`openmesh-static.exe frontend\`(前端静态服务器,端口 3000
5\. 打开浏览器访问 http://localhost:3000
\*\*\*
\## 重新打包操作清单
当代码修改后需要重新打包时,按以下顺序执行:
\### 仅前端修改
\`\`\`
npm run build → 复制到 dist/openmesh/frontend/
\`\`\`
\### 仅后端修改
\`\`\`
pyinstaller dist/openworker-server.spec → 复制 exe 和 _internal/ 到 dist/openmesh/
\`\`\`
\### 前后端都修改
\`\`\`
1\. cd surfaces/gui && npm run build
2\. pyinstaller dist/openworker-server.spec
3\. 组装到 dist/openmesh/
\`\`\`
\*\*\*
\## 注意事项
1\. \*\*data/skills 与 skills/\*\*:内置 skills 仅首次启动时复制。修改内置 skills 后,需删除用户 \`data/skills/\` 目录才能生效,或手动更新。
2\. \*\*PyInstaller 增量构建\*\*build 目录会缓存,重复构建时速度较快。完全干净构建可删除 \`build/\` 目录。
3\. \*\*.venv 目录\*\*:发布包中的 \`.venv\` 用于 skill 脚本执行(如 Python 脚本),与 PyInstaller 打包的运行时是两套独立环境。
4\. \*\*端口冲突\*\*:默认后端 8765前端 3000。可在 \`data/config.toml\` 中修改。
5\. \*\*UPX 压缩\*\*server 端默认关闭 UPX\`upx\=False\`避免某些库加载失败。static 端开启了 UPX。
\*\*\*
\## 相关文件索引
| 文件 | 作用 |
| ------------------------------- | -------------------- |
| \`pyproject.toml\` | Python 项目配置、依赖、入口点 |
| \`dist/openworker-server.spec\` | 后端 PyInstaller 配置 |
| \`openmesh-static.spec\` | 静态服务器 PyInstaller 配置 |
| \`dist/server_entry.py\` | 后端打包入口 |
| \`surfaces/gui/vite.config.ts\` | 前端构建配置 |
| \`surfaces/gui/package.json\` | 前端依赖和脚本 |
| \`dist/openmesh/启动 OpenMesh.bat\` | 启动脚本 |
| \`coworker/server/run.py\` | 后端服务主入口 |

View File

@@ -39,6 +39,7 @@ from .providers import ProviderClient, ProviderRouter
from .overrides import RiskOverrideStore
from .secrets import SecretStore, state_dir
from .skills import SkillLoader, save_skill_tool, skill_catalog_text, skill_tools
from .skills.router import SkillRouter
from .tools import ToolRegistry
from .tools.ask import ask_user_tool
from .tools.directories import request_directory_tool
@@ -416,6 +417,8 @@ def build_engine(
# Persona dirs come FIRST so a user's global/workspace copy of the same name shadows
# the bundle's (later dirs overwrite earlier in the loader).
skill_loader = SkillLoader([Path(d) for d in (extra_skill_dirs or [])] + _skill_dirs(ws))
# Skill Router: 根据用户消息自动推荐技能OPE-xxx 技能自动路由)
skill_router = SkillRouter(skill_loader)
# Per-session effective menu (SKILLS-SPEC §3). The manager passes a CALLABLE so
# load_skill consults the LIVE state per call (a Settings disable applies to running
# sessions; a skill created after this build is still loadable). The catalog itself
@@ -512,6 +515,28 @@ def build_engine(
skills_ctx = skill_catalog_text(skill_loader, allowed=allowed)
if skills_ctx:
parts.append(skills_ctx)
# Skill Router: 根据最后一条用户消息自动推荐技能
eng_ref = _engine_box[0] if _engine_box else None
if eng_ref is not None and skill_router is not None:
# 找到最后一条用户消息
last_user_msg = ""
for msg in reversed(eng_ref.messages):
if msg.get("role") == "user":
content = msg.get("content", "")
if isinstance(content, str):
last_user_msg = content
elif isinstance(content, list):
# content-parts提取 text 类型
text_parts = [
p.get("text", "") for p in content
if isinstance(p, dict) and p.get("type") == "text"
]
last_user_msg = "\n".join(text_parts)
break
if last_user_msg:
router_ctx = skill_router.build_context_text(last_user_msg)
if router_ctx:
parts.append(router_ctx)
# Disable countermand (§3): instructions already loaded into this conversation keep
# steering the model even after the skill is turned off/deleted — history can't be
# un-read. So a loaded-but-no-longer-available skill gets an explicit stop note,

View File

@@ -2,15 +2,21 @@
The dependency is optional. If Playwright or its browser binaries are not installed, the
tools return a clear setup error instead of breaking engine construction.
Implementation note:
The Playwright browser runs in a separate Python subprocess (browser_worker.py)
because PyInstaller-packaged Python is incompatible with playwright's C extensions.
The controller communicates with the worker via stdin/stdout JSON lines.
"""
from __future__ import annotations
import re
import json
import os
import subprocess
import tempfile
import threading
import time
import base64
from concurrent.futures import ThreadPoolExecutor
from pathlib import Path
from typing import Any, Callable, Optional
@@ -20,6 +26,58 @@ import aisuite as ai
from ..web.guard import check_url
def _find_tools_dir() -> Path:
"""查找 tools 目录(从当前模块路径向上找,或用环境变量)。"""
# 1. 环境变量优先
env_path = os.environ.get("OPENMESH_TOOLS_DIR", "")
if env_path and Path(env_path).exists():
return Path(env_path)
# 2. 从模块路径向上找(打包后 coworker 在 _internal/ 里tools 在上层)
try:
here = Path(__file__).resolve().parent
for p in [here, here.parent, here.parent.parent]:
candidate = p / "tools"
if candidate.exists():
return candidate
except Exception:
pass
return Path("tools")
def _find_worker_script(tools_dir: Path) -> Path:
"""查找 browser_worker.py 脚本。
查找顺序:
1. 环境变量 BROWSER_WORKER_PATH
2. tools/python/browser_worker.py运行时预放置的位置
3. 当前模块同目录开发环境PYZ 未打包时)
"""
# 1. 环境变量
env_path = os.environ.get("BROWSER_WORKER_PATH", "")
if env_path and Path(env_path).exists():
return Path(env_path)
# 2. tools/python/browser_worker.py打包后放这里最稳
candidate = tools_dir / "python" / "browser_worker.py"
if candidate.exists():
return candidate
# 3. 当前模块同目录
here = Path(__file__).resolve().parent
candidate = here / "browser_worker.py"
if candidate.exists():
return candidate
# 4. 兜底返回同目录路径(报错时能看到在哪找的)
return here / "browser_worker.py"
_TOOLS_DIR = _find_tools_dir()
_WORKER_SCRIPT = _find_worker_script(_TOOLS_DIR)
def _meta(
name: str, *, approval: bool = False, capabilities: Optional[list[str]] = None
):
@@ -62,16 +120,21 @@ def _attach(fn: Callable[..., Any], schema: dict[str, Any], *, approval: bool =
class _BrowserController:
"""浏览器控制器 — 通过子进程 worker 运行 Playwright。
为什么用子进程:
后端是 PyInstaller 打包的 Pythonplaywright 的 C 扩展在里面会崩;
tools/python/ 是原生 Pythonplaywright + .venv 里的包能正常运行。
"""
def __init__(self) -> None:
self._lock = threading.RLock()
self._playwright = None
self._browser = None
self._context = None
self._page = None
self._error: Optional[str] = None
self._executor = ThreadPoolExecutor(
max_workers=1, thread_name_prefix="coworker-browser"
)
self._proc: Optional[subprocess.Popen] = None
self._request_id = 0
self._error: Optional[str] = None
self._state: dict[str, Any] = {
"open": False,
"url": "",
@@ -89,137 +152,182 @@ class _BrowserController:
self._state.update(changes)
self._state["updated_at"] = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
def _refresh_page_state(self) -> None:
if self._page is None:
self._touch(open=False, status="closed", url="", title="", controls=[])
return
try:
snap = _snapshot(self._page, 2000)
self._touch(
open=True,
status="open",
url=self._page.url,
title=self._page.title(),
controls=snap.get("controls", [])[:30],
)
except Exception as exc:
self._touch(open=True, status="error", last_error=str(exc))
def _ensure_worker(self) -> Optional[dict[str, Any]]:
"""确保 worker 子进程已启动。返回 None 表示成功,返回 dict 表示错误。"""
if self._proc is not None and self._proc.poll() is None:
return None
def _setup_error(self, exc: Exception) -> dict[str, str]:
self._error = None
try:
python_exe = _TOOLS_DIR / "python" / "python.exe"
if not python_exe.exists():
return {
"error": "Browser automation requires tools/python/python.exe (not found).",
"details": f"Expected at: {python_exe}",
}
env = os.environ.copy()
env["OPENMESH_TOOLS_DIR"] = str(_TOOLS_DIR)
env["PYTHONIOENCODING"] = "utf-8"
env["PYTHONUNBUFFERED"] = "1"
self._proc = subprocess.Popen(
[str(python_exe), str(_WORKER_SCRIPT)],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
env=env,
creationflags=getattr(subprocess, "CREATE_NO_WINDOW", 0),
)
# 等待 ready 信号
ready_line = self._proc.stdout.readline()
if not ready_line:
stderr = self._proc.stderr.read().decode("utf-8", errors="replace")
return {
"error": "Browser worker failed to start.",
"details": stderr[:500],
}
try:
ready = json.loads(ready_line.decode("utf-8").strip())
if ready.get("status") != "ready":
return {
"error": "Browser worker did not report ready.",
"details": str(ready),
}
except json.JSONDecodeError:
return {
"error": "Browser worker sent invalid ready signal.",
"details": ready_line.decode("utf-8", errors="replace")[:200],
}
return None
except Exception as exc:
self._error = str(exc)
return {
"error": (
"Interactive browser automation requires Playwright. Install it with "
"`pip install playwright` and `python -m playwright install chromium`."
"Interactive browser automation requires Playwright. "
"Make sure tools/python/ and tools/playwright/chromium-1234/ are present."
),
"details": str(exc),
}
def page(self):
def _send_command(self, action: str, params: Optional[dict] = None) -> dict[str, Any]:
"""向 worker 发送一条命令并等待响应。"""
with self._lock:
if self._error:
return None, {"error": self._error}
if self._page is not None:
return self._page, None
try:
from playwright.sync_api import sync_playwright
err = self._ensure_worker()
if err:
return err
self._playwright = sync_playwright().start()
self._browser = self._playwright.chromium.launch(headless=False)
self._context = self._browser.new_context(
viewport={"width": 1280, "height": 900}
)
self._page = self._context.new_page()
self._touch(
open=True, status="open", last_action="open browser", last_error=""
)
return self._page, None
self._request_id += 1
req_id = self._request_id
req = json.dumps(
{"id": req_id, "action": action, "params": params or {}},
ensure_ascii=False,
) + "\n"
try:
self._proc.stdin.write(req.encode("utf-8"))
self._proc.stdin.flush()
except Exception as exc:
self._touch(open=False, status="error", last_error=str(exc))
return None, self._setup_error(exc)
self._kill_worker()
return {"error": f"failed to send to browser worker: {exc}"}
try:
line = self._proc.stdout.readline()
if not line:
stderr = self._proc.stderr.read().decode("utf-8", errors="replace")
self._kill_worker()
return {
"error": "browser worker exited unexpectedly",
"details": stderr[:500],
}
resp = json.loads(line.decode("utf-8").strip())
if resp.get("id") != req_id:
return {"error": f"response id mismatch: {resp.get('id')} vs {req_id}"}
if "error" in resp:
self._touch(last_action=action, last_result="error", last_error=resp["error"])
return {"error": resp["error"]}
self._touch(last_action=action, last_result="ok", last_error="")
# 同步状态
if "url" in resp:
self._state["url"] = resp.get("url", "")
if "title" in resp:
self._state["title"] = resp.get("title", "")
if "controls" in resp:
self._state["controls"] = resp.get("controls", [])
if "screenshot_data_url" in resp:
self._state["screenshot_data_url"] = resp["screenshot_data_url"]
if action == "close":
self._state["open"] = False
self._state["status"] = "closed"
self._kill_worker()
elif action in ("open_url", "read_page", "click", "type", "select", "wait"):
self._state["open"] = True
self._state["status"] = "open"
return resp
except Exception as exc:
self._kill_worker()
return {"error": f"failed to read browser worker response: {exc}"}
def _kill_worker(self) -> None:
try:
if self._proc and self._proc.poll() is None:
self._proc.terminate()
self._proc.wait(timeout=3)
except Exception:
try:
if self._proc and self._proc.poll() is None:
self._proc.kill()
except Exception:
pass
finally:
self._proc = None
def _submit(self, fn: Callable[[], dict[str, Any]]) -> dict[str, Any]:
return self._executor.submit(fn).result()
def close(self) -> dict[str, Any]:
return self._submit(self._close_locked)
def _close_locked(self) -> dict[str, Any]:
def _do():
with self._lock:
try:
if self._context is not None:
self._context.close()
if self._browser is not None:
self._browser.close()
if self._playwright is not None:
self._playwright.stop()
except Exception as exc:
return {"error": str(exc)}
finally:
self._playwright = None
self._browser = None
self._context = None
self._page = None
self._touch(open=False, status="closed", url="", title="", controls=[])
if self._proc is None:
return {"ok": True}
return self._send_command("close")
return self._submit(_do)
def state(self) -> dict[str, Any]:
return self._submit(self._state_locked)
def _state_locked(self) -> dict[str, Any]:
def _do():
with self._lock:
self._refresh_page_state()
if self._proc is None:
return dict(self._state)
resp = self._send_command("state")
if "error" in resp:
return dict(self._state)
return {**dict(self._state), **{k: v for k, v in resp.items() if k != "id"}}
return self._submit(_do)
def screenshot(self) -> dict[str, Any]:
return self._submit(self._screenshot_locked)
def _screenshot_locked(self) -> dict[str, Any]:
def _do():
with self._lock:
page, err = self.page()
if err:
return err
try:
png = page.screenshot(full_page=False)
data_url = "data:image/png;base64," + base64.b64encode(png).decode(
"ascii"
)
self._touch(
screenshot_data_url=data_url,
last_action="screenshot",
last_result="ok",
last_error="",
)
self._refresh_page_state()
resp = self._send_command("screenshot")
if "error" in resp:
return resp
return {"ok": True, **dict(self._state)}
except Exception as exc:
self._touch(
last_action="screenshot", last_result="error", last_error=str(exc)
)
return {"error": str(exc)}
return self._submit(_do)
def call(self, action: str, fn: Callable[[Any], dict[str, Any]]) -> dict[str, Any]:
def run() -> dict[str, Any]:
with self._lock:
page, err = self.page()
if err:
return err
self._touch(last_action=action, last_result="running", last_error="")
try:
out = fn(page)
except Exception as exc:
out = {"error": str(exc)}
if "error" in out:
self._touch(
last_action=action,
last_result="error",
last_error=str(out["error"]),
)
else:
self._refresh_page_state()
self._touch(last_action=action, last_result="ok", last_error="")
return out
"""兼容旧的 call 接口 — 直接通过 worker 执行 action。
return self._submit(run)
注意fn 参数被忽略,所有逻辑在 worker 里实现。
action 名直接映射到 worker 的 action 名。
"""
def _do():
with self._lock:
# 从 fn 的闭包或通过 action 名构造 params
# 为了兼容旧代码,这里用 action 名直接发命令
# 参数通过额外机制传递(见下方每个工具函数的实现)
return self._send_command(action)
return self._submit(_do)
_BROWSER = _BrowserController()
@@ -237,110 +345,25 @@ def browser_close_session() -> dict[str, Any]:
return _BROWSER.close()
def _cap(value: int, default: int = 20000, upper: int = 100000) -> int:
try:
return max(1, min(int(value or default), upper))
except Exception:
return default
_BROWSER = _BrowserController()
def _target_locator(page, target: str):
target = target.strip()
if target.startswith("text="):
return page.get_by_text(target[5:], exact=False).first
if target.startswith("role="):
role_name = target[5:]
role, _, name = role_name.partition(":")
return page.get_by_role(role.strip(), name=name.strip() or None).first
try:
return page.locator(target).first
except Exception:
return page.get_by_text(target, exact=False).first
def browser_state() -> dict[str, Any]:
return _BROWSER.state()
def _safe_call(fn: Callable[[], Any]) -> dict[str, Any]:
try:
return fn()
except Exception as exc:
return {"error": str(exc)}
def browser_take_screenshot() -> dict[str, Any]:
return _BROWSER.screenshot()
def _browser_call(action: str, fn: Callable[[], dict[str, Any]]) -> dict[str, Any]:
return _BROWSER.call(action, lambda _page: fn())
def browser_close_session() -> dict[str, Any]:
return _BROWSER.close()
_SNAPSHOT_JS = """
() => {
const visible = (el) => {
const style = window.getComputedStyle(el);
const rect = el.getBoundingClientRect();
return style && style.visibility !== 'hidden' && style.display !== 'none' && rect.width > 0 && rect.height > 0;
};
const labelFor = (el) => {
if (el.labels && el.labels.length) return Array.from(el.labels).map(l => l.innerText.trim()).filter(Boolean).join(' ');
const id = el.getAttribute('id');
if (id) {
const label = document.querySelector(`label[for="${CSS.escape(id)}"]`);
if (label) return label.innerText.trim();
}
return '';
};
const describe = (el, i) => ({
index: i,
tag: el.tagName.toLowerCase(),
type: el.getAttribute('type') || '',
id: el.getAttribute('id') || '',
name: el.getAttribute('name') || '',
role: el.getAttribute('role') || '',
aria: el.getAttribute('aria-label') || '',
label: labelFor(el),
placeholder: el.getAttribute('placeholder') || '',
text: (el.innerText || el.value || '').trim().slice(0, 200),
href: el.getAttribute('href') || '',
selectorHint: el.getAttribute('id') ? `#${CSS.escape(el.getAttribute('id'))}` : (el.getAttribute('name') ? `[name="${el.getAttribute('name')}"]` : '')
});
const controls = Array.from(document.querySelectorAll('a,button,input,textarea,select,[role="button"],[contenteditable="true"]'))
.filter(visible)
.slice(0, 120)
.map(describe);
return {
title: document.title,
url: location.href,
text: document.body ? document.body.innerText : '',
controls
};
}
"""
def _snapshot(page, max_chars: int) -> dict[str, Any]:
data = page.evaluate(_SNAPSHOT_JS)
text = re.sub(r"\n{3,}", "\n\n", str(data.get("text") or ""))
cap = _cap(max_chars)
return {
"title": data.get("title"),
"url": data.get("url"),
"text": text[:cap],
"truncated": len(text) > cap,
"controls": data.get("controls") or [],
}
def redirect_refusal(requested: str, final: str) -> Optional[str]:
"""A refusal reason if navigation LANDED somewhere the address guard would refuse.
`check_url` vets the URL the model supplied; Playwright then follows redirects, and the
hop that actually loads is a different address the guard never saw (OPE-124). A public
shortener can land on the cloud metadata endpoint or a router admin page, and the
approval the user gave was for the first URL, not this one.
The request has already gone out by the time this runs — it cannot be prevented here.
What it prevents is the agent READING the page or interacting with it. Later
JavaScript- or meta-refresh-driven navigation is still unchecked; only a proxy that
vets every hop closes that, which is the larger design this defers."""
if not final or final == requested:
return None
return check_url(final)
def _worker_call(action: str, params: Optional[dict] = None) -> dict[str, Any]:
"""通过 worker 子进程执行浏览器操作。"""
# 用 _send_command但它是私有方法——通过 _BROWSER 的内部机制调用
return _BROWSER._send_command(action, params)
def make_browser_automation_tools(
@@ -349,13 +372,7 @@ def make_browser_automation_tools(
tools: list[Callable[..., Any]] = []
def _readable_source(raw: str) -> tuple[Any, dict[str, Any] | None]:
"""A local file to upload, resolved inside a granted root (OPE-122).
These tools touch the filesystem but classify EXTERNAL, so the permission engine's
root scoping — which only runs for WRITE_LOCAL — never sees them. Without this
check the only thing between `~/.ssh/id_rsa` and a web form is someone reading the
approval card. Mirrors `email_send`'s attachment rule, which solves the same
problem for outgoing mail."""
"""A local file to upload, resolved inside a granted root (OPE-122)."""
allowed = [r.path for r in (roots or [])]
if not allowed:
return None, {"error": "no session directory is available to upload from"}
@@ -365,8 +382,7 @@ def make_browser_automation_tools(
return path, None
def _writable_target(raw: str) -> tuple[Any, dict[str, Any] | None]:
"""Where a screenshot may land: inside a WRITABLE granted root. An unnamed target
keeps the temp-file default, which is not a place the user asked us to protect."""
"""Where a screenshot may land: inside a WRITABLE granted root."""
writable = [r.path for r in (roots or []) if r.writable]
if not writable:
return None, {"error": "no writable session directory for the screenshot"}
@@ -382,25 +398,11 @@ def make_browser_automation_tools(
) -> dict[str, Any]:
if not url.lower().startswith(("http://", "https://")):
return {"error": "url must start with http:// or https://"}
# Same address guard as web_fetch. This is approval gated, so it is defense in
# depth, not the primary control. It checks the initial model supplied URL only;
# redirects that the browser follows internally are not hop checked here.
# URL guard (same as web_fetch defense in depth)
blocked = check_url(url)
if blocked:
return {"error": blocked}
def _open(page):
page.goto(url, wait_until=wait_until, timeout=30000)
landed = redirect_refusal(url, page.url)
if landed:
# Leave nothing readable behind: the next snapshot/get_text must not be
# able to lift content off a page we just refused.
final = page.url
page.goto("about:blank")
return {"error": f"redirected to {final}{landed}"}
return {"ok": True, "url": page.url}
return _BROWSER.call("open_url", _open)
return _worker_call("open_url", {"url": url, "wait_until": wait_until})
browser_open_url.__name__ = "browser_open_url"
tools.append(
@@ -417,7 +419,7 @@ def make_browser_automation_tools(
)
def browser_read_page(max_chars: int = 20000) -> dict[str, Any]:
return _BROWSER.call("snapshot", lambda page: _snapshot(page, max_chars))
return _worker_call("read_page", {"max_chars": max_chars})
browser_read_page.__name__ = "browser_read_page"
tools.append(
@@ -436,13 +438,7 @@ def make_browser_automation_tools(
)
def browser_click(target: str) -> dict[str, Any]:
return _BROWSER.call(
"click",
lambda page: (
_target_locator(page, target).click(timeout=10000),
{"ok": True, "url": page.url},
)[1],
)
return _worker_call("click", {"target": target})
browser_click.__name__ = "browser_click"
tools.append(
@@ -459,15 +455,7 @@ def make_browser_automation_tools(
)
def browser_type(target: str, text: str, clear: bool = True) -> dict[str, Any]:
def run(page):
loc = _target_locator(page, target)
if clear:
loc.fill(text, timeout=10000)
else:
loc.type(text, timeout=10000)
return {"ok": True, "url": page.url}
return _BROWSER.call("type", run)
return _worker_call("type", {"target": target, "text": text, "clear": clear})
browser_type.__name__ = "browser_type"
tools.append(
@@ -488,13 +476,7 @@ def make_browser_automation_tools(
)
def browser_select(target: str, value: str) -> dict[str, Any]:
return _BROWSER.call(
"select",
lambda page: (
_target_locator(page, target).select_option(value, timeout=10000),
{"ok": True, "url": page.url},
)[1],
)
return _worker_call("select", {"target": target, "value": value})
browser_select.__name__ = "browser_select"
tools.append(
@@ -516,15 +498,7 @@ def make_browser_automation_tools(
return err
if not file_path.exists():
return {"error": f"file not found: {file_path}"}
return _BROWSER.call(
"upload_file",
lambda page: (
_target_locator(page, target).set_input_files(
str(file_path), timeout=10000
),
{"ok": True, "path": str(file_path)},
)[1],
)
return _worker_call("upload_file", {"target": target, "path": str(file_path)})
browser_upload_file.__name__ = "browser_upload_file"
tools.append(
@@ -541,16 +515,9 @@ def make_browser_automation_tools(
)
def browser_wait(milliseconds: int = 1000, target: str = "") -> dict[str, Any]:
def run(page):
if target:
_target_locator(page, target).wait_for(
timeout=max(1, int(milliseconds or 1000))
return _worker_call(
"wait", {"milliseconds": milliseconds, "target": target}
)
else:
page.wait_for_timeout(max(1, min(int(milliseconds or 1000), 30000)))
return {"ok": True, "url": page.url}
return _BROWSER.call("wait", run)
browser_wait.__name__ = "browser_wait"
tools.append(
@@ -567,24 +534,13 @@ def make_browser_automation_tools(
)
def browser_screenshot(path: str = "") -> dict[str, Any]:
params = {}
if path:
_target, target_err = _writable_target(path)
if target_err:
return target_err
def run(page):
out = (
_target
if path
else (
Path(tempfile.gettempdir()) / "coworker-browser-screenshot.png"
).resolve()
)
out.parent.mkdir(parents=True, exist_ok=True)
page.screenshot(path=str(out), full_page=True)
return {"ok": True, "path": str(out), "url": page.url}
return _BROWSER.call("screenshot", run)
params["path"] = str(_target)
return _worker_call("screenshot", params)
browser_screenshot.__name__ = "browser_screenshot"
tools.append(

View File

@@ -0,0 +1,299 @@
"""Playwright 浏览器子进程 Worker。
后端通过 stdin/stdout 与本进程通信,协议为每行一条 JSON
请求: {"id": 1, "action": "open_url", "params": {"url": "..."}}
响应: {"id": 1, "ok": true, ...} 或 {"id": 1, "error": "..."}
为什么做成子进程:
后端是 PyInstaller 打包的 Pythonplaywright 的 C 扩展在里面会崩;
tools/python/ 是原生 Pythonplaywright/.venv 的包都能正常运行。
"""
import json
import os
import re
import sys
import base64
import threading
from pathlib import Path
def _find_tools_dir() -> Path:
"""查找 tools 目录,优先级:
1. 环境变量 OPENMESH_TOOLS_DIR
2. 从 Python 解释器路径向上找tools/python/python.exe → tools/
3. 从脚本路径向上找
4. 当前目录的 tools/
"""
# 1. 环境变量
env_path = os.environ.get("OPENMESH_TOOLS_DIR", "")
if env_path and Path(env_path).exists():
return Path(env_path)
# 2. 从 Python 解释器向上找tools/python/python.exe → tools/
try:
exe_dir = Path(sys.executable).resolve().parent
if exe_dir.name == "python" and (exe_dir.parent / "tools").exists():
# exe 在 dist/openmesh/tools/python/
return exe_dir.parent
if exe_dir.name == "Scripts" and (exe_dir.parent.parent / "tools").exists():
# exe 在 .venv/Scripts/
return exe_dir.parent.parent / "tools"
except Exception:
pass
# 3. 从脚本路径向上找
here = Path(__file__).resolve().parent
for p in [here, here.parent, here.parent.parent, here.parent.parent.parent]:
candidate = p / "tools"
if candidate.exists():
return candidate
# 4. 兜底
return Path("tools")
# 找到 tools 目录,设置 playwright 浏览器路径
_tools_dir = _find_tools_dir()
os.environ.setdefault("PLAYWRIGHT_BROWSERS_PATH", str(_tools_dir / "playwright"))
# 把 .venv 的 site-packages 加到路径
_venv_site = _tools_dir.parent / ".venv" / "Lib" / "site-packages"
if _venv_site.exists():
sys.path.insert(0, str(_venv_site))
from playwright.sync_api import sync_playwright # noqa: E402
# ===== 页面快照 JS跟原来 browser_automation.py 里的一样)=====
_SNAPSHOT_JS = """
() => {
const visible = (el) => {
const style = window.getComputedStyle(el);
const rect = el.getBoundingClientRect();
return style && style.visibility !== 'hidden' && style.display !== 'none' && rect.width > 0 && rect.height > 0;
};
const labelFor = (el) => {
if (el.labels && el.labels.length) return Array.from(el.labels).map(l => l.innerText.trim()).filter(Boolean).join(' ');
const id = el.getAttribute('id');
if (id) {
const label = document.querySelector(`label[for="${CSS.escape(id)}"]`);
if (label) return label.innerText.trim();
}
return '';
};
const describe = (el, i) => ({
index: i,
tag: el.tagName.toLowerCase(),
type: el.getAttribute('type') || '',
id: el.getAttribute('id') || '',
name: el.getAttribute('name') || '',
role: el.getAttribute('role') || '',
aria: el.getAttribute('aria-label') || '',
label: labelFor(el),
placeholder: el.getAttribute('placeholder') || '',
text: (el.innerText || el.value || '').trim().slice(0, 200),
href: el.getAttribute('href') || '',
selectorHint: el.getAttribute('id') ? `#${CSS.escape(el.getAttribute('id'))}` : (el.getAttribute('name') ? `[name="${el.getAttribute('name')}"]` : '')
});
const controls = Array.from(document.querySelectorAll('a,button,input,textarea,select,[role="button"],[contenteditable="true"]'))
.filter(visible)
.slice(0, 120)
.map(describe);
return {
title: document.title,
url: location.href,
text: document.body ? document.body.innerText : '',
controls
};
}
"""
class BrowserWorker:
def __init__(self):
self._pw = None
self._browser = None
self._context = None
self._page = None
self._lock = threading.Lock()
def _ensure_page(self):
if self._page is not None:
return self._page
if self._pw is None:
self._pw = sync_playwright().start()
if self._browser is None:
self._browser = self._pw.chromium.launch(
headless=True,
executable_path=str(_tools_dir / "playwright" / "chromium-1234" / "chrome-win64" / "chrome.exe"),
)
if self._context is None:
self._context = self._browser.new_context(
viewport={"width": 1280, "height": 900}
)
if self._page is None:
self._page = self._context.new_page()
return self._page
def _snapshot(self, page, max_chars=20000):
data = page.evaluate(_SNAPSHOT_JS)
text = re.sub(r"\n{3,}", "\n\n", str(data.get("text") or ""))
cap = max(1, min(int(max_chars or 20000), 100000))
return {
"title": data.get("title"),
"url": data.get("url"),
"text": text[:cap],
"truncated": len(text) > cap,
"controls": (data.get("controls") or [])[:30],
}
def _target_locator(self, page, target):
target = str(target).strip()
if target.startswith("text="):
return page.get_by_text(target[5:], exact=False).first
if target.startswith("role="):
role_name = target[5:]
role, _, name = role_name.partition(":")
return page.get_by_role(role.strip(), name=name.strip() or None).first
try:
return page.locator(target).first
except Exception:
return page.get_by_text(target, exact=False).first
# ===== 动作处理 =====
def handle(self, action, params):
with self._lock:
try:
fn = getattr(self, f"_action_{action}", None)
if fn is None:
return {"error": f"unknown action: {action}"}
return fn(params or {})
except Exception as e:
return {"error": f"{type(e).__name__}: {e}"}
def _action_state(self, params):
if self._page is None:
return {"open": False, "status": "closed", "url": "", "title": "", "controls": []}
snap = self._snapshot(self._page, params.get("max_chars", 2000))
return {
"open": True,
"status": "open",
"url": self._page.url,
"title": self._page.title(),
"controls": snap.get("controls", []),
}
def _action_open_url(self, params):
url = params.get("url", "")
if not url.lower().startswith(("http://", "https://")):
return {"error": "url must start with http:// or https://"}
page = self._ensure_page()
page.goto(url, wait_until=params.get("wait_until", "domcontentloaded"), timeout=30000)
return {"ok": True, "url": page.url}
def _action_read_page(self, params):
page = self._ensure_page()
return self._snapshot(page, params.get("max_chars", 20000))
def _action_click(self, params):
page = self._ensure_page()
self._target_locator(page, params.get("target", "")).click(timeout=10000)
return {"ok": True, "url": page.url}
def _action_type(self, params):
page = self._ensure_page()
loc = self._target_locator(page, params.get("target", ""))
if params.get("clear", True):
loc.fill(params.get("text", ""), timeout=10000)
else:
loc.type(params.get("text", ""), timeout=10000)
return {"ok": True, "url": page.url}
def _action_select(self, params):
page = self._ensure_page()
self._target_locator(page, params.get("target", "")).select_option(
params.get("value", ""), timeout=10000
)
return {"ok": True, "url": page.url}
def _action_upload_file(self, params):
page = self._ensure_page()
path = params.get("path", "")
if not path or not os.path.exists(path):
return {"error": f"file not found: {path}"}
self._target_locator(page, params.get("target", "")).set_input_files(path, timeout=10000)
return {"ok": True, "path": path}
def _action_wait(self, params):
page = self._ensure_page()
target = params.get("target", "")
if target:
self._target_locator(page, target).wait_for(
timeout=max(1, int(params.get("milliseconds") or 1000))
)
else:
page.wait_for_timeout(max(1, min(int(params.get("milliseconds") or 1000), 30000)))
return {"ok": True, "url": page.url}
def _action_screenshot(self, params):
page = self._ensure_page()
path = params.get("path", "")
if path:
out_path = Path(path).resolve()
out_path.parent.mkdir(parents=True, exist_ok=True)
page.screenshot(path=str(out_path), full_page=True)
return {"ok": True, "path": str(out_path), "url": page.url}
else:
# 返回 base64
png = page.screenshot(full_page=False)
data_url = "data:image/png;base64," + base64.b64encode(png).decode("ascii")
return {"ok": True, "screenshot_data_url": data_url, "url": page.url}
def _action_close(self, params):
try:
if self._context:
self._context.close()
if self._browser:
self._browser.close()
if self._pw:
self._pw.stop()
finally:
self._page = None
self._context = None
self._browser = None
self._pw = None
return {"ok": True}
def main():
worker = BrowserWorker()
# 发送 ready 信号
print(json.dumps({"id": 0, "ok": True, "status": "ready"}), flush=True)
for line in sys.stdin:
line = line.strip()
if not line:
continue
try:
req = json.loads(line)
except json.JSONDecodeError:
print(json.dumps({"id": 0, "error": "invalid JSON"}), flush=True)
continue
req_id = req.get("id", 0)
action = req.get("action", "")
params = req.get("params", {})
result = worker.handle(action, params)
result["id"] = req_id
print(json.dumps(result, ensure_ascii=False), flush=True)
# 退出命令
if action == "close":
break
if __name__ == "__main__":
main()

View File

@@ -5931,6 +5931,7 @@ class SessionManager:
"description": r["description"],
"scope": r["scope"],
"enabled": overrides.get(r["name"], True),
"title": r.get("title", ""),
}
for r in self.skill_store.rows(workspace or None)
if r["name"] not in disabled
@@ -5950,6 +5951,7 @@ class SessionManager:
"description": entry["description"],
"scope": "coworker",
"enabled": overrides.get(name, True),
"title": entry.get("title", ""),
}
)
return {"skills": rows}

View File

@@ -23,6 +23,8 @@ class Skill:
instructions: str = "" # full body — loaded on demand
path: Optional[str] = None
allowed_tools: list[str] = field(default_factory=list)
triggers: list[str] = field(default_factory=list) # 触发关键词,用于自动路由
title: str = "" # 中文标题,用于 UI 显示(如 "办公可视化看板"
class SkillLoader:
@@ -55,15 +57,23 @@ class SkillLoader:
return self._skills.get(name)
def catalog(self) -> list[dict]:
# 按优先级排序office-viz 排最前(数据分析可视化优先),其余按字母顺序
def sort_key(s: Skill) -> tuple[int, str]:
if s.name == "office-viz":
return (0, "")
return (1, s.name)
return [
{"name": s.name, "description": s.description}
for s in self._skills.values()
{"name": s.name, "description": s.description, "title": s.title}
for s in sorted(self._skills.values(), key=sort_key)
]
def _parse_skill(md: Path) -> Skill:
text = md.read_text(encoding="utf-8")
name, description, allowed, body = md.parent.name, "", [], text
triggers: list[str] = []
title: str = ""
if text.startswith("---"):
end = text.find("\n---", 3)
if end != -1:
@@ -80,12 +90,25 @@ def _parse_skill(md: Path) -> Skill:
description = value
elif key in ("allowed-tools", "allowed_tools"):
allowed = [t.strip() for t in value.split(",") if t.strip()]
elif key == "title":
title = value
elif key in ("triggers", "trigger-keywords", "trigger_keywords"):
# 支持逗号分隔: triggers: 关键词1, 关键词2, 关键词3
raw = value
# 去掉首尾可能的引号
if raw.startswith('"') and raw.endswith('"'):
raw = raw[1:-1]
elif raw.startswith("'") and raw.endswith("'"):
raw = raw[1:-1]
triggers = [t.strip() for t in raw.split(",") if t.strip()]
return Skill(
name=name,
description=description,
instructions=body.strip(),
path=str(md.parent),
allowed_tools=allowed,
triggers=triggers,
title=title,
)
@@ -97,8 +120,19 @@ def skill_catalog_text(
]
if not catalog:
return ""
lines = [f"- {c['name']}: {c['description']}" for c in catalog]
# 在列表前加优先级提示
priority_notice = (
"⚠️ Skill 选择优先级说明:\n"
" • 数据分析 + 可视化 + 看板 → 使用 office-viz不要用 xlsx\n"
" • 纯文件读写(无分析) → 使用 xlsx\n"
" • PDF 操作 → 使用 pdf\n"
" • ...\n\n"
)
return (
priority_notice +
"Available skills — call load_skill(name) to load one's full instructions when "
"it's relevant to the task:\n" + "\n".join(lines)
)

302
coworker/skills/router.py Normal file
View File

@@ -0,0 +1,302 @@
"""
技能路由器 (Skill Router)
========================
根据用户消息的关键词,自动匹配最合适的 Skill并推荐给 Agent 加载。
设计原则:
- 每个 Skill 在 SKILL.md 的 frontmatter 里声明 triggers 字段(逗号分隔的关键词)
- 路由器读取所有 skill 的 triggers与用户消息做匹配
- 匹配度超过阈值的 skill 会被推荐给 Agent
- 用户新添加的 skill 只要声明了 triggers自动参与路由无需修改代码
匹配算法:
- 关键词命中数(加权)+ 描述相似度 + 场景上下文
- 返回按置信度排序的推荐列表
调试:
- 设置环境变量 SKILL_ROUTER_LOG=1 可输出调试日志到 workspace/skill_router.log
"""
from __future__ import annotations
import os
import re
from dataclasses import dataclass
from datetime import datetime
from pathlib import Path
from typing import Optional
@dataclass
class SkillMatch:
"""技能匹配结果"""
name: str
confidence: float # 0.0 - 1.0
matched_keywords: list[str]
reason: str = ""
class SkillRouter:
"""技能路由器"""
# 置信度阈值:超过这个值才会推荐
RECOMMEND_THRESHOLD = 0.18
# 强推荐阈值:超过这个值会强烈建议加载
STRONG_RECOMMEND_THRESHOLD = 0.35
# 自动预加载阈值:超过这个值直接把技能指令注入上下文
# 模型会认为这个技能已经加载了,直接按照指令执行
PRELOAD_THRESHOLD = 0.35
def __init__(self, loader) -> None:
"""
Args:
loader: SkillLoader 实例
"""
self._loader = loader
def match(self, user_message: str) -> list[SkillMatch]:
"""
匹配用户消息,返回推荐的技能列表(按置信度降序)
Args:
user_message: 用户的最新消息文本
Returns:
匹配结果列表,每个元素是 SkillMatch
"""
if not user_message or len(user_message.strip()) == 0:
return []
msg = user_message.lower()
results: list[SkillMatch] = []
for skill_name in self._loader.names():
skill = self._loader.get(skill_name)
if skill is None:
continue
score = 0.0
matched = []
# 1. 触发词匹配(权重最高)
triggers = getattr(skill, "triggers", [])
if triggers:
for trigger in triggers:
trigger_lower = trigger.lower()
if len(trigger_lower) < 2:
continue
# 精确包含匹配(权重高)
if trigger_lower in msg:
score += 0.20
matched.append(trigger)
continue
# 子序列匹配:所有字都在消息里且顺序一致(不一定连续)
# 中文里插入少量字是常见的(如"做个图表"≈"做图表"),权重较高
if self._is_subsequence(trigger_lower, msg):
# 触发词越长且插入字越少,权重越高
insert_ratio = len(trigger_lower) / max(len(msg), 1)
weight = 0.10 + 0.08 * insert_ratio
score += weight
matched.append(trigger + "(顺序)")
continue
# 部分匹配:触发词有 2 个字以上连续出现在消息里
hit_len = self._max_common_substring_len(trigger_lower, msg)
if hit_len >= 3:
weight = 0.05 + 0.02 * (hit_len - 3)
score += weight
matched.append(trigger + f"(部分{hit_len}字)")
# 2. 描述匹配(次要权重)
desc = skill.description.lower()
desc_keywords = self._extract_keywords(desc)
for kw in desc_keywords:
if len(kw) >= 2 and kw in msg:
score += 0.05
if kw not in matched:
matched.append(kw)
# 3. 名称匹配(小权重)
if skill_name.lower() in msg:
score += 0.1
matched.append(skill_name)
# 归一化到 0-1
confidence = min(score, 1.0)
if confidence >= self.RECOMMEND_THRESHOLD:
reason = self._build_reason(skill_name, confidence, matched)
results.append(SkillMatch(
name=skill_name,
confidence=confidence,
matched_keywords=matched,
reason=reason,
))
# 按置信度降序
results.sort(key=lambda x: x.confidence, reverse=True)
return results
def build_context_text(self, user_message: str) -> str:
"""
生成注入到系统提示的路由推荐文本
如果匹配度足够高(超过 PRELOAD_THRESHOLD直接把技能的完整指令
注入上下文,模型会认为技能已加载,直接按指令执行。
Args:
user_message: 用户最新消息
Returns:
推荐文本,如果没有匹配则返回空字符串
"""
if not user_message or len(user_message.strip()) == 0:
return ""
matches = self.match(user_message)
# 调试日志
self._log_match(user_message, matches)
if not matches:
return ""
strong = [m for m in matches if m.confidence >= self.STRONG_RECOMMEND_THRESHOLD]
preload = [m for m in matches if m.confidence >= self.PRELOAD_THRESHOLD]
normal = [m for m in matches if self.RECOMMEND_THRESHOLD <= m.confidence < self.STRONG_RECOMMEND_THRESHOLD]
lines = []
# 自动预加载:高置信度时直接注入技能完整指令
if preload:
top = preload[0]
skill = self._loader.get(top.name)
if skill and skill.instructions:
lines.append(f"🎯 自动技能路由: 已为你预加载「{skill.name}」技能")
lines.append(f" 匹配原因: {top.reason}")
lines.append(f" 置信度: {top.confidence:.0%}")
lines.append("")
lines.append(f"=== 技能: {skill.name} ===")
lines.append("以下是该技能的完整说明,请直接按照说明执行:")
lines.append("")
lines.append(skill.instructions)
lines.append("")
lines.append("=== 技能说明结束 ===")
lines.append("")
lines.append(
"📋 执行提示: 请使用上面预加载的技能来完成用户需求。"
"如果该技能有工具调用方式,请按说明调用。"
"不要使用 Python 脚本或其他方式重新实现该技能已有的功能。"
)
return "\n".join(lines)
# 低于预加载阈值时,只做推荐
lines.append("💡 Skill 推荐(根据用户消息自动匹配):")
if strong:
lines.append(" 🌟 强烈推荐:")
for m in strong:
lines.append(f" - {m.name}{m.reason}")
lines.append(f" ⚡ 请立即调用 load_skill(\"{m.name}\") 加载此技能")
lines.append(f" 不要用基础工具手动实现,直接使用该技能")
if normal:
lines.append(" 📌 可能相关:")
for m in normal[:3]:
lines.append(f" - {m.name}{m.reason}")
lines.append(f" 如需使用可调用 load_skill(\"{m.name}\")")
if strong:
lines.append(
"\n ⚡ 重要提示: 用户需求明确匹配上面的强烈推荐技能,"
"请立即调用 load_skill 加载该技能,然后按照技能说明执行。"
"不要尝试用 Python 脚本或其他方式手动实现。"
)
return "\n".join(lines)
def _extract_keywords(self, text: str) -> list[str]:
"""从文本中提取有意义的关键词(简单实现:提取中文词组和英文单词)"""
# 提取英文单词(长度 >= 3
english_words = re.findall(r'[a-zA-Z]{3,}', text.lower())
# 提取中文词组2-6 字的连续中文字符)
chinese_phrases = re.findall(r'[\u4e00-\u9fa5]{2,6}', text)
# 去掉常见停用词
stop_words = {
"", "", "", "", "", "", "", "", "", "",
"可以", "使用", "支持", "进行", "生成", "创建", "处理",
"数据", "文件", "功能", "工具", "技能", "输出", "输入",
"多种", "各种", "一个", "一些", "什么", "如何", "怎么",
}
keywords = []
for w in english_words + chinese_phrases:
if w not in stop_words and len(w) >= 2:
keywords.append(w)
# 去重并返回
return list(dict.fromkeys(keywords))
def _build_reason(self, name: str, confidence: float, matched: list[str]) -> str:
"""构建推荐理由文本"""
if matched:
show = matched[:4]
kw_text = "".join(show)
if len(matched) > 4:
kw_text += ""
return f"匹配关键词「{kw_text}"
return f"置信度 {confidence:.0%}"
def _log_match(self, user_message: str, matches: list[SkillMatch]) -> None:
"""输出调试日志(环境变量 SKILL_ROUTER_LOG=1 时启用)"""
if not os.environ.get("SKILL_ROUTER_LOG"):
return
try:
log_path = Path(os.environ.get("SKILL_ROUTER_LOG_PATH",
r"D:\project\senmeshworker-main\dist\openmesh\workspace\skill_router.log"))
log_path.parent.mkdir(parents=True, exist_ok=True)
ts = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
with open(log_path, "a", encoding="utf-8") as f:
f.write(f"\n[{ts}] 用户消息: {user_message[:100]}\n")
if matches:
f.write(f" 匹配到 {len(matches)} 个技能:\n")
for m in matches:
f.write(f" - {m.name}: {m.confidence:.1%} (关键词: {m.matched_keywords[:5]})\n")
if m.confidence >= self.PRELOAD_THRESHOLD:
f.write(f" → 自动预加载\n")
else:
f.write(" 无匹配\n")
except Exception:
pass # 日志失败不影响主流程
@staticmethod
def _is_subsequence(needle: str, haystack: str) -> bool:
"""检查 needle 是否是 haystack 的子序列(字符顺序一致,可不连续)"""
it = iter(haystack)
return all(c in it for c in needle)
@staticmethod
def _max_common_substring_len(s1: str, s2: str) -> int:
"""返回两个字符串的最长公共子串长度"""
if not s1 or not s2:
return 0
m, n = len(s1), len(s2)
max_len = 0
# 用滚动数组O(n) 空间
prev = [0] * (n + 1)
for i in range(1, m + 1):
curr = [0] * (n + 1)
for j in range(1, n + 1):
if s1[i - 1] == s2[j - 1]:
curr[j] = prev[j - 1] + 1
max_len = max(max_len, curr[j])
else:
curr[j] = 0
prev = curr
return max_len

View File

@@ -165,6 +165,7 @@ class SkillStore:
"enabled": skill.name not in disabled,
"path": str(sub),
"files": max(bundled, 0),
"title": skill.title, # 中文标题,用于 UI 显示
}
if skill.name in seen: # project copy shadows the global one
out[seen[skill.name]] = row

View File

@@ -1409,6 +1409,7 @@ export interface SessionSkillRow {
description: string;
scope: "global" | "project";
enabled: boolean; // false = muted for this session only
title?: string; // 中文标题,用于 UI 显示(如 "办公可视化看板"
}
export interface SkillUploadPreview {

View File

@@ -179,6 +179,51 @@ export function Composer(props: Props) {
setText(`/${s.name} `);
textareaRef.current?.focus();
};
// 技能选择器状态(独立于 slash popup
const [skillMenuOpen, setSkillMenuOpen] = useState(false);
const [skillOptions, setSkillOptions] = useState<SessionSkillRow[]>([]);
// 独立存储选中的技能,不受 slash popup 影响
const [selectedSkill, setSelectedSkill] = useState<SessionSkillRow | null>(null);
// 汉字转拼音首字母
const pinyinInitials = new Map(Object.entries({
"办公可视化看板": "B", "Word文档": "W", "PDF处理": "P",
"Excel操作": "E", "PPT制作": "P", "Markdown转换": "M",
"通用": "T",
}));
const getSkillBadge = (name: string, title: string) => {
const text = title || name;
if (pinyinInitials.has(text)) return pinyinInitials.get(text);
// 取中文首字或英文首字母
const match = text.match(/[\u4e00-\u9fa5]/);
return match ? match[0] : text.slice(0, 1).toUpperCase();
};
// 加载技能列表(打开下拉时)
useEffect(() => {
if (!skillMenuOpen || skillOptions.length > 0) return;
if (!props.sessionId) return;
sessionSkills(props.sessionId, props.workspace)
.then((all) => setSkillOptions(all.filter((s) => s.enabled)))
.catch(() => setSkillOptions([]));
}, [skillMenuOpen]);
// 点击技能选项
const handleSelectSkill = (s: SessionSkillRow | null) => {
setSelectedSkill(s);
if (s) {
// 选中具体技能:在文本框插入 slash 命令,触发 slash popup
setText(`/${s.name} `);
setPendingSkill(s);
textareaRef.current?.focus();
} else {
// 通用(清除技能选择)
setPendingSkill(null);
setText("");
}
setSkillMenuOpen(false);
};
const [dragging, setDragging] = useState(false);
const [attachMenuOpen, setAttachMenuOpen] = useState(false);
// UX-044: which "This session" submenu is open (bindings live server-side).
@@ -701,6 +746,7 @@ export function Composer(props: Props) {
<span className="text-[12px] text-muted tabular-nums">{recordingTime}</span>
</div>
) : props.workspace !== undefined ? (
<>
<ModeMenu
reviewerPaused={props.reviewerPaused}
mode={props.mode}
@@ -708,6 +754,62 @@ export function Composer(props: Props) {
unattended={props.unattended}
onUnattendedChange={props.onUnattendedChange}
/>
{/* 技能选择器 */}
<div className="relative">
<button
className="inline-flex items-center gap-1 px-1.5 py-0.5 rounded-lg text-[12px] text-muted hover:text-ink hover:bg-paper shrink-0 border border-line"
onClick={() => setSkillMenuOpen((v) => !v)}
aria-haspopup="menu"
aria-expanded={skillMenuOpen}
title="选择技能"
>
{selectedSkill ? (
<>
<span className="inline-flex items-center justify-center w-5 h-5 rounded text-[10px] font-bold bg-accent/10 text-accent">
{getSkillBadge(selectedSkill.name, selectedSkill.title || "")}
</span>
<span className="text-ink">{selectedSkill.title || selectedSkill.name}</span>
</>
) : (
<span></span>
)}
<Icon name="chevronDown" size={11} className="text-faint" />
</button>
{skillMenuOpen && (
<>
<div className="fixed inset-0 z-30" onClick={() => setSkillMenuOpen(false)} />
<div
className="absolute z-40 bottom-full mb-1 left-0 w-[220px] rounded-xl border border-line bg-panel shadow-2xl py-1.5"
role="menu"
>
{/* 通用选项 */}
<button
className="w-full flex items-center px-2.5 py-1.5 text-[13px] text-left hover:bg-paper"
onClick={() => handleSelectSkill(null)}
>
<span className="inline-flex items-center justify-center w-5 h-5 rounded text-[10px] font-bold bg-paper border border-line text-faint mr-2">T</span>
<span className="flex-1 text-ink"></span>
{selectedSkill === null && <span className="text-accent ml-2"></span>}
</button>
<div className="mx-2 my-1 border-t border-line" />
{skillOptions.map((s) => (
<button
key={s.name}
className="w-full flex items-center px-2.5 py-1.5 text-left hover:bg-paper"
onClick={() => handleSelectSkill(s)}
>
<span className="inline-flex items-center justify-center w-5 h-5 rounded text-[10px] font-bold bg-accent/10 text-accent mr-2">
{getSkillBadge(s.name, s.title || "")}
</span>
<span className="flex-1 text-[13px] text-ink">{s.title || s.name}</span>
{selectedSkill?.name === s.name && <span className="text-accent ml-2"></span>}
</button>
))}
</div>
</>
)}
</div>
</>
) : null}
{dictationBusy === t("composer.starting_transcribe") && <span className="text-[12px] text-accent">{dictationBusy}</span>}