Skip to content

LLM Wiki is a cross-platform desktop application that turns your documents into an organized, interlinked knowledge base — automatically. Instead of traditional RAG (retrieve-and-answer from scratch every time), the LLM incrementally builds and maintains a persistent wiki from your sources。

TypeScript

Release v0.4.11

nash_sua8cf1951d579
diff --git a/README.md b/README.md
index 6a08b16f..302617bc 100644
--- a/README.md
+++ b/README.md
@@ -42,6 +42,7 @@
- **Deep Research** — LLM-optimized search topics, multi-query web search via Tavily, SerpApi, or SearXNG, auto-ingest results into wiki
- **Async Review System** — LLM flags items for human judgment, predefined actions, pre-generated search queries
- **Chrome Web Clipper** — one-click web page capture with auto-ingest into knowledge base
+- **Local HTTP API + AI Agent Skill** — built-in `127.0.0.1:19828` JSON API (token-protected) for hybrid search, file read, graph traversal, and source rescan; ready-made [agent skill](https://github.com/nashsu/llm_wiki_skill) installs into Claude Code / Codex with one command (`npx skills add …`)
## What is this?
@@ -406,6 +407,32 @@ npm run tauri build # Production build
8. Check **Review** for items needing your attention
9. Run **Lint** periodically to maintain wiki health
+## Local HTTP API + AI Agent Skill
+
+LLM Wiki ships a built-in local HTTP API at `http://127.0.0.1:19828` (token-protected, `127.0.0.1`-only) so external tools — including AI agents like **Claude Code**, **Codex**, or any HTTP-capable script — can query your wiki:
+
+- `GET /api/v1/health` — server status (no auth)
+- `GET /api/v1/projects` — list projects
+- `GET /api/v1/projects/{id}/files` / `files/content` — read files and content
+- `POST /api/v1/projects/{id}/search` — **hybrid** retrieval (keyword + vector) returning `mode`, `tokenHits`, `vectorHits`, per-result `vectorScore`
+- `GET /api/v1/projects/{id}/graph` — wikilinks graph
+- `POST /api/v1/projects/{id}/sources/rescan` — trigger a backend rescan
+
+Enable + generate a token in **Settings → API Server**.
+
+### Plug your AI agent in with one command
+
+A ready-made **agent skill** for LLM Wiki lives in its own repo. Install it into Claude Code / Codex / any skills-compatible runtime:
+
+```bash
+npx skills add https://github.com/nashsu/llm_wiki_skill.git --skill llm_wiki_skill
+```
+
+After install, the agent can answer prompts like "what does my LLM Wiki say about X", "search my 知识库 for Y", "show the neighborhood of node Z in my wiki graph", and "rescan my wiki sources" by talking to your locally-running app — read-only by default, citing wiki page paths so you can verify in-app.
+
+- **Skill repo**: <https://github.com/nashsu/llm_wiki_skill>
+- **Trigger discipline**: it intentionally does **not** trigger on generic "search my notes" / "check my Obsidian / Notion / Logseq" — only when you explicitly name LLM Wiki / `my wiki` / `知识库`.
+
## Project Structure
```
diff --git a/README_CN.md b/README_CN.md
index 75ff6142..d2a439d0 100644
--- a/README_CN.md
+++ b/README_CN.md
@@ -42,6 +42,7 @@
- **深度研究** — LLM 智能生成搜索主题,通过 Tavily、SerpApi 或 SearXNG 进行多查询网络搜索,研究结果自动摄入 Wiki
- **异步审核系统** — LLM 在摄入时标记需人工判断的项,预定义操作,预生成搜索查询
- **Chrome 网页剪藏** — 一键捕获网页内容,自动摄入知识库
+- **本地 HTTP API + AI Agent Skill** — 内置 `127.0.0.1:19828` JSON API(Token 鉴权),支持 Hybrid 检索、文件读取、知识图谱遍历、源资料重新扫描;配套 [agent skill](https://github.com/nashsu/llm_wiki_skill) 一行命令接入 Claude Code / Codex(`npx skills add …`)
## 这是什么?
@@ -406,6 +407,32 @@ npm run tauri build # 生产构建
8. 查看 **审核** 处理需要你关注的项目
9. 定期运行 **Lint** 维护 Wiki 健康度
+## 本地 HTTP API + AI Agent Skill
+
+LLM Wiki 内置一个本地 HTTP API(监听 `http://127.0.0.1:19828`,Token 鉴权,仅本机可达),任何外部工具——包括 **Claude Code**、**Codex** 这类 AI Agent,或者任意能发 HTTP 请求的脚本——都可以直接查询你的知识库:
+
+- `GET /api/v1/health` —— 服务状态(无需鉴权)
+- `GET /api/v1/projects` —— 项目列表
+- `GET /api/v1/projects/{id}/files` / `files/content` —— 读取文件树与内容
+- `POST /api/v1/projects/{id}/search` —— **Hybrid 混合检索**(关键词 + 向量),返回 `mode`、`tokenHits`、`vectorHits`,每条结果带 `vectorScore`
+- `GET /api/v1/projects/{id}/graph` —— Wikilinks 知识图谱
+- `POST /api/v1/projects/{id}/sources/rescan` —— 触发后端重新扫描
+
+在 **设置 → API 服务** 中开启 API 并生成 Token。
+
+### 一条命令把 AI Agent 接进你的知识库
+
+LLM Wiki 配套的 **agent skill** 单独维护在另一个仓库。把它装进 Claude Code / Codex / 任意兼容 skills 的 runtime:
+
+```bash
+npx skills add https://github.com/nashsu/llm_wiki_skill.git --skill llm_wiki_skill
+```
+
+安装完成后,Agent 就能响应 "我的 LLM Wiki 里关于 X 是怎么说的"、"在我的知识库里搜 Y"、"展示我 wiki 图谱里 Z 的邻居"、"重新索引我的资料源" 等请求——直接调用本机运行的 App,默认只读,引用 wiki 页面路径方便你在 App 内核对。
+
+- **Skill 仓库**:<https://github.com/nashsu/llm_wiki_skill>
+- **触发约束**:刻意**不会**响应"搜我的笔记"/"看我的 Obsidian / Notion / Logseq"这类泛指的请求——只有你明确说 LLM Wiki / `我的 wiki` / `我的知识库` 时才会被调用。
+
## 项目结构
```
diff --git a/README_JA.md b/README_JA.md
index 00e03891..b62b3f97 100644
--- a/README_JA.md
+++ b/README_JA.md
@@ -42,6 +42,7 @@
- **Deep Research**: Tavily、SerpApi、SearXNG を使った複数クエリ検索と、結果の自動 Wiki 化。
- **非同期 Review System**: LLM が判断待ち項目を作成し、ユーザーが後から確認できます。
- **Chrome Web Clipper**: Web ページをワンクリップで取り込み、知識ベースへ自動 ingest します。
+- **ローカル HTTP API + AI Agent Skill**: `127.0.0.1:19828` の JSON API(Token 認証)で hybrid 検索 / ファイル取得 / グラフ参照 / 再スキャンを提供。専用の [agent skill](https://github.com/nashsu/llm_wiki_skill) はワンコマンドで Claude Code / Codex に追加できます(`npx skills add …`)。
## これは何ですか?
@@ -179,6 +180,32 @@ npm run tauri build # Production build
7. **Knowledge Graph** で関連を探索します。
8. **Review** と **Lint** で Wiki の品質を保ちます。
+## ローカル HTTP API + AI Agent Skill
+
+LLM Wiki は `http://127.0.0.1:19828` でローカル HTTP API を提供します(Token 認証、`127.0.0.1` のみ)。**Claude Code**、**Codex** などの AI Agent や任意の HTTP クライアントから Wiki にアクセスできます:
+
+- `GET /api/v1/health` — サーバー状態(認証不要)
+- `GET /api/v1/projects` — プロジェクト一覧
+- `GET /api/v1/projects/{id}/files` / `files/content` — ファイル / 本文取得
+- `POST /api/v1/projects/{id}/search` — **Hybrid 検索**(キーワード + ベクトル)。レスポンスに `mode` / `tokenHits` / `vectorHits` / `vectorScore` を含む
+- `GET /api/v1/projects/{id}/graph` — Wikilinks グラフ
+- `POST /api/v1/projects/{id}/sources/rescan` — バックエンド再スキャン
+
+**Settings → API Server** で API を有効化し Token を発行してください。
+
+### AI Agent をワンコマンドで接続
+
+LLM Wiki 用の **agent skill** は別リポジトリで管理しています。Claude Code / Codex などへインストール:
+
+```bash
+npx skills add https://github.com/nashsu/llm_wiki_skill.git --skill llm_wiki_skill
+```
+
+インストール後、「LLM Wiki に X について書いてある?」「知識ベースで Y を検索」「Wiki グラフで Z の近傍を表示」「ソースを再スキャン」といった依頼に対し、Agent がローカル App を直接呼び出して答えます。デフォルトは読み取り専用で、Wiki ページのパスを引用するため App 内で照合できます。
+
+- **Skill リポジトリ**:<https://github.com/nashsu/llm_wiki_skill>
+- **トリガー制約**:「メモを検索」「Obsidian / Notion / Logseq を見て」といった汎用的な依頼には**意図的に反応しません**。LLM Wiki / `my wiki` / `知識ベース` を明示した場合のみ起動します。
+
## Project Structure
```
diff --git a/package-lock.json b/package-lock.json
index f18e9929..bcefd241 100644
--- a/package-lock.json
+++ b/package-lock.json
@@ -1,12 +1,12 @@
{
"name": "llm-wiki",
- "version": "0.4.10",
+ "version": "0.4.11",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "llm-wiki",
- "version": "0.4.10",
+ "version": "0.4.11",
"dependencies": {
"@base-ui/react": "^1.3.0",
"@fontsource-variable/geist": "^5.2.8",
diff --git a/package.json b/package.json
index 05c4d65b..ac70eaf1 100644
--- a/package.json
+++ b/package.json
@@ -1,7 +1,7 @@
{
"name": "llm-wiki",
"private": true,
- "version": "0.4.10",
+ "version": "0.4.11",
"type": "module",
"scripts": {
"dev": "vite",
diff --git a/src-tauri/Cargo.lock b/src-tauri/Cargo.lock
index 733570eb..33970782 100644
--- a/src-tauri/Cargo.lock
+++ b/src-tauri/Cargo.lock
@@ -4420,7 +4420,7 @@ checksum = "11d3d7f243d5c5a8b9bb5d6dd2b1602c0cb0b9db1621bafc7ed66e35ff9fe092"
[[package]]
name = "llm-wiki"
-version = "0.4.10"
+version = "0.4.11"
dependencies = [
"arrow-array",
"arrow-schema",
@@ -4434,6 +4434,7 @@ dependencies = [
"md-5",
"notify",
"pdfium-render",
+ "reqwest 0.12.28",
"serde",
"serde_json",
"sha2",
diff --git a/src-tauri/Cargo.toml b/src-tauri/Cargo.toml
index d8a594f6..f330e15e 100644
--- a/src-tauri/Cargo.toml
+++ b/src-tauri/Cargo.toml
@@ -1,6 +1,6 @@
[package]
name = "llm-wiki"
-version = "0.4.10"
+version = "0.4.11"
description = "LLM Wiki - A personal knowledge base for LLM concepts"
authors = []
edition = "2021"
@@ -28,6 +28,7 @@ tauri-plugin-dialog = "2.7.1"
pdfium-render = "0.9"
tauri-plugin-store = "2.4.2"
tauri-plugin-http = { version = "2", features = ["unsafe-headers"] }
+reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls"] }
tiny_http = "0.12"
zip = "2"
calamine = "0.34.0"
diff --git a/src-tauri/src/api_server.rs b/src-tauri/src/api_server.rs
new file mode 100644
index 00000000..cd06302d
--- /dev/null
+++ b/src-tauri/src/api_server.rs
@@ -0,0 +1,1358 @@
+use std::collections::{BTreeMap, BTreeSet, VecDeque};
+use std::fs;
+use std::io::Read;
+use std::path::{Component, Path, PathBuf};
+use std::sync::atomic::{AtomicU8, AtomicUsize, Ordering};
+use std::sync::{Mutex, OnceLock};
+use std::thread;
+use std::time::{Duration, Instant};
+
+use serde::{Deserialize, Serialize};
+use serde_json::{json, Value};
+use tauri::{AppHandle, Manager};
+use tiny_http::{Header, Method, Response, Server, StatusCode};
+use walkdir::WalkDir;
+
+use crate::{clip_server, commands};
+
+const PORT: u16 = 19828;
+const API_PREFIX: &str = "/api/v1";
+const MAX_BODY_BYTES: usize = 1024 * 1024;
+const MAX_FILE_CONTENT_BYTES: u64 = 2 * 1024 * 1024;
+const DEFAULT_MAX_FILES: usize = 2_000;
+const HARD_MAX_FILES: usize = 10_000;
+const MAX_SEARCH_RESULTS: usize = 50;
+const BIND_RETRY_DELAY_SECS: u64 = 2;
+const MAX_BIND_RETRIES: u32 = 3;
+const APP_STATE_CACHE_TTL: Duration = Duration::from_secs(5);
+const RATE_LIMIT_WINDOW: Duration = Duration::from_secs(1);
+const RATE_LIMIT_MAX_REQUESTS: usize = 120;
+const MAX_IN_FLIGHT_REQUESTS: usize = 64;
+
+/// API status: 0=starting, 1=running, 2=port_conflict, 3=error
+static API_STATUS: AtomicU8 = AtomicU8::new(0);
+static IN_FLIGHT_REQUESTS: AtomicUsize = AtomicUsize::new(0);
+static APP_STATE_CACHE: OnceLock<Mutex<Option<CachedAppState>>> = OnceLock::new();
+static RATE_LIMIT: OnceLock<Mutex<VecDeque<Instant>>> = OnceLock::new();
+
+#[derive(Clone)]
+struct CachedAppState {
+ loaded_at: Instant,
+ value: Option<Value>,
+}
+
+pub fn get_api_status() -> &'static str {
+ match API_STATUS.load(Ordering::Relaxed) {
+ 0 => "starting",
+ 1 => "running",
+ 2 => "port_conflict",
+ _ => "error",
+ }
+}
+
+pub fn invalidate_config_cache() {
+ if let Some(lock) = APP_STATE_CACHE.get() {
+ if let Ok(mut cache) = lock.lock() {
+ *cache = None;
+ }
+ }
+}
+
+pub fn start_api_server(app: AppHandle) {
+ thread::spawn(move || loop {
+ API_STATUS.store(0, Ordering::Relaxed);
+ let server = match bind_server_with_retry() {
+ Some(server) => server,
+ None => {
+ API_STATUS.store(2, Ordering::Relaxed);
+ thread::sleep(Duration::from_secs(BIND_RETRY_DELAY_SECS));
+ continue;
+ }
+ };
+
+ API_STATUS.store(1, Ordering::Relaxed);
+ eprintln!("[API Server] Listening on http://127.0.0.1:{PORT}{API_PREFIX}");
+
+ for request in server.incoming_requests() {
+ let method = request.method().clone();
+ let url = request.url().to_string();
+ if should_rate_limit(&method, &url) && !allow_request() {
+ respond_error(request, 429, "Too many requests");
+ continue;
+ }
+ let Some(slot) = try_acquire_request_slot() else {
+ respond_error(request, 503, "API server is busy");
+ continue;
+ };
+ let app = app.clone();
+ thread::spawn(move || {
+ let _slot = slot;
+ let result = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
+ process_request(app, request);
+ }));
+ if let Err(payload) = result {
+ eprintln!("[API Server] request handler panicked: {payload:?}");
+ }
+ });
+ }
+
+ API_STATUS.store(3, Ordering::Relaxed);
+ eprintln!("[API Server] server loop exited; restarting");
+ thread::sleep(Duration::from_secs(BIND_RETRY_DELAY_SECS));
+ });
+}
+
+fn bind_server_with_retry() -> Option<Server> {
+ for attempt in 1..=MAX_BIND_RETRIES {
+ match Server::http(format!("127.0.0.1:{PORT}")) {
+ Ok(server) => return Some(server),
+ Err(err) => {
+ eprintln!(
+ "[API Server] Failed to bind 127.0.0.1:{PORT} (attempt {attempt}/{MAX_BIND_RETRIES}): {err}"
+ );
+ if attempt < MAX_BIND_RETRIES {
+ thread::sleep(Duration::from_secs(BIND_RETRY_DELAY_SECS));
+ }
+ }
+ }
+ }
+ None
+}
+
+struct RequestSlot;
+
+impl Drop for RequestSlot {
+ fn drop(&mut self) {
+ IN_FLIGHT_REQUESTS.fetch_sub(1, Ordering::Relaxed);
+ }
+}
+
+fn try_acquire_request_slot() -> Option<RequestSlot> {
+ let mut current = IN_FLIGHT_REQUESTS.load(Ordering::Relaxed);
+ loop {
+ if current >= MAX_IN_FLIGHT_REQUESTS {
+ return None;
+ }
+ match IN_FLIGHT_REQUESTS.compare_exchange_weak(
+ current,
+ current + 1,
+ Ordering::Relaxed,
+ Ordering::Relaxed,
+ ) {
+ Ok(_) => return Some(RequestSlot),
+ Err(next) => current = next,
+ }
+ }
+}
+
+fn process_request(app: AppHandle, mut request: tiny_http::Request) {
+ let method = request.method().clone();
+ let url = request.url().to_string();
+ if method == Method::Options {
+ respond_options(request);
+ return;
+ }
+
+ let headers: Vec<(String, String)> = request
+ .headers()
+ .iter()
+ .map(|header| {
+ (
+ header.field.as_str().to_ascii_lowercase().to_string(),
+ header.value.as_str().to_string(),
+ )
+ })
+ .collect();
+
+ let body = match read_body(&mut request) {
+ Ok(body) => body,
+ Err(err) => {
+ respond_error(request, 400, &err);
+ return;
+ }
+ };
+
+ let response = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
+ handle_request(&app, &method, &url, &body, &headers)
+ }))
+ .unwrap_or_else(|payload| {
+ eprintln!("[API Server] request panicked: {payload:?}");
+ err(500, "Internal API server error")
+ });
+ respond_json(request, response.status, response.body);
+}
+
+struct ApiResponse {
+ status: u16,
+ body: Value,
+}
+
+fn ok(body: Value) -> ApiResponse {
+ ApiResponse { status: 200, body }
+}
+
+fn err(status: u16, message: impl Into<String>) -> ApiResponse {
+ ApiResponse {
+ status,
+ body: json!({ "ok": false, "error": message.into() }),
+ }
+}
+
+fn handle_request(
+ app: &AppHandle,
+ method: &Method,
+ url: &str,
+ body: &str,
+ headers: &[(String, String)],
+) -> ApiResponse {
+ let (path, query) = split_url(url);
+ if path == "/health" || path == format!("{API_PREFIX}/health") {
+ // /health stays reachable even when the user has disabled the
+ // API in Settings — the desktop UI uses it to render the
+ // "Enabled / disabled / port_conflict" line, and curl-from-
+ // terminal users need a way to confirm the server is alive
+ // before they go hunting for why other endpoints 503.
+ return ok(json!({
+ "ok": true,
+ "status": get_api_status(),
+ "version": env!("CARGO_PKG_VERSION"),
+ "authRequired": api_auth_required(app),
+ "authConfigured": api_token(app).is_some(),
+ "tokenSource": api_token_source(app),
+ "enabled": api_enabled(app),
+ "allowUnauthenticated": api_allow_unauthenticated(app),
+ }));
+ }
+ if !path.starts_with(API_PREFIX) {
+ return err(404, "Not found");
+ }
+ if !api_enabled(app) {
+ // Kill-switch path: token may be configured and valid, but the
+ // user toggled the API off in Settings → API Server. 503 is
+ // the right code semantically ("temporarily unavailable")
+ // and tells well-behaved clients to back off rather than
+ // retry instantly the way 401 would.
+ return err(503, "API server is disabled in Settings → API Server");
+ }
+ if !is_authorized(app, query, headers) {
+ return err(401, "Unauthorized");
+ }
+ if !matches!(method, &Method::Get | &Method::Post) {
+ return err(405, "Method not allowed");
+ }
+
+ let parts: Vec<&str> = path
+ .trim_start_matches(API_PREFIX)
+ .trim_start_matches('/')
+ .split('/')
+ .filter(|part| !part.is_empty())
+ .collect();
+
+ match (method, parts.as_slice()) {
+ (&Method::Get, ["projects"]) => handle_projects(app),
+ (&Method::Get, ["projects", project_id, "files"]) => handle_files(app, project_id, query),
+ (&Method::Get, ["projects", project_id, "files", "content"]) => {
+ handle_file_content(app, project_id, query)
+ }
+ (&Method::Post, ["projects", project_id, "search"]) => handle_search(app, project_id, body),
+ (&Method::Get, ["projects", project_id, "graph"]) => handle_graph(app, project_id, query),
+ (&Method::Post, ["projects", project_id, "sources", "rescan"]) => {
+ handle_rescan(app, project_id)
+ }
+ (&Method::Post, ["projects", project_id, "chat"]) => {
+ let _ = project_id;
+ err(501, "Chat API is not implemented in the local Rust API server yet. The existing chat/RAG pipeline currently lives in the WebView; expose it after moving the shared chat pipeline behind a backend command.")
+ }
+ _ => err(404, "Not found"),
+ }
+}
+
+fn should_rate_limit(method: &Method, url: &str) -> bool {
+ if method == &Method::Options {
+ return false;
+ }
+ let (path, _) = split_url(url);
+ !(path == "/health" || path == format!("{API_PREFIX}/health"))
+}
+
+fn allow_request() -> bool {
+ let now = Instant::now();
+ let window_start = now - RATE_LIMIT_WINDOW;
+ let lock = RATE_LIMIT.get_or_init(|| Mutex::new(VecDeque::new()));
+ let Ok(mut hits) = lock.lock() else {
+ return false;
+ };
+ while hits.front().map(|t| *t < window_start).unwrap_or(false) {
+ hits.pop_front();
+ }
+ if hits.len() >= RATE_LIMIT_MAX_REQUESTS {
+ return false;
+ }
+ hits.push_back(now);
+ true
+}
+
+fn read_body(request: &mut tiny_http::Request) -> Result<String, String> {
+ let mut limited = request.as_reader().take(MAX_BODY_BYTES as u64 + 1);
+ let mut bytes = Vec::new();
+ limited
+ .read_to_end(&mut bytes)
+ .map_err(|e| format!("Failed to read body: {e}"))?;
+ if bytes.len() > MAX_BODY_BYTES {
+ return Err("Request body too large".to_string());
+ }
+ String::from_utf8(bytes).map_err(|_| "Request body must be UTF-8".to_string())
+}
+
+fn respond_error(request: tiny_http::Request, status: u16, message: &str) {
+ respond_json(request, status, json!({ "ok": false, "error": message }));
+}
+
+fn respond_options(request: tiny_http::Request) {
+ let mut response = Response::empty(StatusCode(204));
+ for header in cors_headers() {
+ response.add_header(header);
+ }
+ response.add_header(Header::from_bytes("Access-Control-Max-Age", "600").unwrap());
+ let _ = request.respond(response);
+}
+
+fn respond_json(request: tiny_http::Request, status: u16, body: Value) {
+ let mut response = Response::from_string(body.to_string()).with_status_code(StatusCode(status));
+ for header in cors_headers() {
+ response.add_header(header);
+ }
+ let _ = request.respond(response);
+}
+
+fn cors_headers() -> Vec<Header> {
+ vec![
+ Header::from_bytes("Access-Control-Allow-Origin", "*").unwrap(),
+ Header::from_bytes("Access-Control-Allow-Methods", "GET, POST, OPTIONS").unwrap(),
+ Header::from_bytes(
+ "Access-Control-Allow-Headers",
+ "Content-Type, Authorization, X-LLM-Wiki-Token",
+ )
+ .unwrap(),
+ Header::from_bytes("Content-Type", "application/json").unwrap(),
+ ]
+}
+
+fn split_url(url: &str) -> (String, &str) {
+ match url.split_once('?') {
+ Some((path, query)) => (path.to_string(), query),
+ None => (url.to_string(), ""),
+ }
+}
+
+fn parse_query(query: &str) -> BTreeMap<String, String> {
+ let mut out = BTreeMap::new();
+ for pair in query.split('&').filter(|s| !s.is_empty()) {
+ let (k, v) = pair.split_once('=').unwrap_or((pair, ""));
+ out.insert(percent_decode(k), percent_decode(v));
+ }
+ out
+}
+
+fn percent_decode(input: &str) -> String {
+ let bytes = input.as_bytes();
+ let mut out = Vec::with_capacity(bytes.len());
+ let mut i = 0;
+ while i < bytes.len() {
+ if bytes[i] == b'%' && i + 2 < bytes.len() {
+ if let Ok(v) = u8::from_str_radix(&input[i + 1..i + 3], 16) {
+ out.push(v);
+ i += 3;
+ continue;
+ }
+ }
+ out.push(if bytes[i] == b'+' { b' ' } else { bytes[i] });
+ i += 1;
+ }
+ String::from_utf8_lossy(&out).into_owned()
+}
+
+fn is_authorized(app: &AppHandle, query: &str, headers: &[(String, String)]) -> bool {
+ if !api_auth_required(app) {
+ return true;
+ }
+ let Some(token) = api_token(app) else {
+ return false;
+ };
+ let params = parse_query(query);
+ if params
+ .get("token")
+ .map(|v| constant_time_eq(v.as_bytes(), token.as_bytes()))
+ .unwrap_or(false)
+ {
+ return true;
+ }
+ headers.iter().any(|(key, value)| {
+ if key == "x-llm-wiki-token" {
+ return constant_time_eq(value.as_bytes(), token.as_bytes());
+ }
+ if key == "authorization" {
+ return value
+ .strip_prefix("Bearer ")
+ .map(|v| constant_time_eq(v.as_bytes(), token.as_bytes()))
+ .unwrap_or(false);
+ }
+ false
+ })
+}
+
+fn api_token(app: &AppHandle) -> Option<String> {
+ if let Ok(token) = std::env::var("LLM_WIKI_API_TOKEN") {
+ let trimmed = token.trim();
+ if !trimmed.is_empty() {
+ return Some(trimmed.to_string());
+ }
+ }
+ let parsed = load_app_state(app)?;
+ parsed
+ .get("apiConfig")
+ .and_then(|v| v.get("token"))
+ .and_then(Value::as_str)
+ .filter(|s| !s.is_empty())
+ .map(ToOwned::to_owned)
+}
+
+fn api_token_source(app: &AppHandle) -> &'static str {
+ if let Ok(token) = std::env::var("LLM_WIKI_API_TOKEN") {
+ if !token.trim().is_empty() {
+ return "env";
+ }
+ }
+ if load_app_state(app)
+ .and_then(|parsed| {
+ parsed
+ .get("apiConfig")
+ .and_then(|v| v.get("token"))
+ .and_then(Value::as_str)
+ .filter(|s| !s.is_empty())
+ .map(|_| ())
+ })
+ .is_some()
+ {
+ return "store";
+ }
+ "none"
+}
+
+fn api_auth_required(app: &AppHandle) -> bool {
+ !api_allow_unauthenticated(app)
+}
+
+fn api_allow_unauthenticated(app: &AppHandle) -> bool {
+ let Some(parsed) = load_app_state(app) else {
+ return false;
+ };
+ parsed
+ .get("apiConfig")
+ .and_then(|v| v.get("allowUnauthenticated"))
+ .and_then(Value::as_bool)
+ .unwrap_or(false)
+}
+
+/// Whether the API server should accept non-/health requests.
+///
+/// Defaults to `true` when no config has been written yet — keeps
+/// existing setups (env-token-only, hand-edited app-state.json) working
+/// after the kill-switch was introduced. New users still land in
+/// "enabled + no token = 401" which is fail-closed by virtue of the
+/// missing token, not the enable flag.
+fn api_enabled(app: &AppHandle) -> bool {
+ let Some(parsed) = load_app_state(app) else {
+ return true;
+ };
+ parsed
+ .get("apiConfig")
+ .and_then(|v| v.get("enabled"))
+ .and_then(Value::as_bool)
+ .unwrap_or(true)
+}
+
+fn constant_time_eq(left: &[u8], right: &[u8]) -> bool {
+ let max_len = left.len().max(right.len());
+ let mut diff = left.len() ^ right.len();
+ for i in 0..max_len {
+ let a = left.get(i).copied().unwrap_or(0);
+ let b = right.get(i).copied().unwrap_or(0);
+ diff |= (a ^ b) as usize;
+ }
+ diff == 0
+}
+
+fn load_app_state(app: &AppHandle) -> Option<Value> {
+ let now = Instant::now();
+ let lock = APP_STATE_CACHE.get_or_init(|| Mutex::new(None));
+ let mut previous = None;
+ if let Ok(cache) = lock.lock() {
+ if let Some(cached) = cache.as_ref() {
+ if now.duration_since(cached.loaded_at) < APP_STATE_CACHE_TTL {
+ return cached.value.clone();
+ }
+ previous = cached.value.clone();
+ }
+ }
+
+ let path = app.path().app_data_dir().ok()?.join("app-state.json");
+ let loaded = fs::read_to_string(path)
+ .ok()
+ .and_then(|raw| serde_json::from_str::<Value>(&raw).ok());
+ let value = loaded.or(previous);
+
+ if let Ok(mut cache) = lock.lock() {
+ *cache = Some(CachedAppState {
+ loaded_at: now,
+ value: value.clone(),
+ });
+ }
+ value
+}
+
+#[derive(Debug, Clone, Serialize, Deserialize)]
+struct ProjectEntry {
+ id: String,
+ name: String,
+ path: String,
+ current: bool,
+}
+
+fn handle_projects(app: &AppHandle) -> ApiResponse {
+ let projects = load_projects(app);
+ let current_project = projects.iter().find(|project| project.current).cloned();
+ ok(json!({
+ "ok": true,
+ "projects": projects,
+ "currentProject": current_project,
+ }))
+}
+
+fn load_projects(app: &AppHandle) -> Vec<ProjectEntry> {
+ let current = normalize_path(&clip_server::current_project_path());
+ let mut by_path: BTreeMap<String, ProjectEntry> = BTreeMap::new();
+
+ if let Some(parsed) = load_app_state(app) {
+ if let Some(registry) = parsed.get("projectRegistry").and_then(Value::as_object) {
+ for (id, value) in registry {
+ let path = value.get("path").and_then(Value::as_str).unwrap_or("");
+ if path.is_empty() {
+ continue;
+ }
+ let path = normalize_path(path);
+ let name = value
+ .get("name")
+ .and_then(Value::as_str)
+ .map(ToOwned::to_owned)
+ .unwrap_or_else(|| project_name_from_path(&path));
+ by_path.insert(
+ path.clone(),
+ ProjectEntry {
+ id: id.clone(),
+ name,
+ current: path == current,
+ path,
+ },
+ );
+ }
+ }
+ if let Some(recents) = parsed.get("recentProjects").and_then(Value::as_array) {
+ for value in recents {
+ let path = value.get("path").and_then(Value::as_str).unwrap_or("");
+ if path.is_empty() {
+ continue;
+ }
+ let path = normalize_path(path);
+ by_path.entry(path.clone()).or_insert_with(|| {
+ let id = read_project_id(&path).unwrap_or_else(|| path.clone());
+ let name = value
+ .get("name")
+ .and_then(Value::as_str)
+ .map(ToOwned::to_owned)
+ .unwrap_or_else(|| project_name_from_path(&path));
+ ProjectEntry {
+ id,
+ name,
+ current: path == current,
+ path,
+ }
+ });
+ }
+ }
+ }
+
+ for (name, path) in clip_server::all_projects() {
+ let path = normalize_path(&path);
+ by_path.entry(path.clone()).or_insert_with(|| ProjectEntry {
+ id: read_project_id(&path).unwrap_or_else(|| path.clone()),
+ name: if name.is_empty() {
+ project_name_from_path(&path)
+ } else {
+ name
+ },
+ current: path == current,
+ path,
+ });
+ }
+
+ if !current.is_empty() {
+ by_path
+ .entry(current.clone())
+ .or_insert_with(|| ProjectEntry {
+ id: read_project_id(&current).unwrap_or_else(|| current.clone()),
+ name: project_name_from_path(&current),
+ current: true,
+ path: current.clone(),
+ });
+ }
+
+ by_path.into_values().collect()
+}
+
+fn resolve_project(app: &AppHandle, project_id: &str) -> Result<ProjectEntry, String> {
+ let project_id = percent_decode(project_id);
+ let wants_current = project_id.eq_ignore_ascii_case("current");
+ load_projects(app)
+ .into_iter()
+ .find(|p| {
+ p.id == project_id
+ || project_path_matches(&p.path, &project_id)
+ || (wants_current && p.current)
+ })
+ .ok_or_else(|| format!("Unknown project: {project_id}"))
+}
+
+fn project_path_matches(stored_path: &str, candidate: &str) -> bool {
+ let stored = normalize_path(stored_path);
+ let candidate = normalize_path(candidate);
+ if cfg!(windows) {
+ stored.eq_ignore_ascii_case(&candidate)
+ } else {
+ stored == candidate
+ }
+}
+
+fn read_project_id(path: &str) -> Option<String> {
+ let raw = fs::read_to_string(Path::new(path).join(".llm-wiki/project.json")).ok()?;
+ let parsed: Value = serde_json::from_str(&raw).ok()?;
+ parsed
+ .get("id")
+ .and_then(Value::as_str)
+ .map(ToOwned::to_owned)
+}
+
+fn project_name_from_path(path: &str) -> String {
+ Path::new(path)
+ .file_name()
+ .and_then(|s| s.to_str())
+ .unwrap_or("Project")
+ .to_string()
+}
+
+fn normalize_path(path: &str) -> String {
+ path.replace('\\', "/").trim_end_matches('/').to_string()
+}
+
+fn handle_files(app: &AppHandle, project_id: &str, query: &str) -> ApiResponse {
+ let project = match resolve_project(app, project_id) {
+ Ok(project) => project,
+ Err(e) => return err(404, e),
+ };
+ let params = parse_query(query);
+ let root = params.get("root").map(String::as_str).unwrap_or("wiki");
+ let recursive = params
+ .get("recursive")
+ .map(|v| v != "false")
+ .unwrap_or(true);
+ let max_files = params
+ .get("maxFiles")
+ .and_then(|v| v.parse::<usize>().ok())
+ .unwrap_or(DEFAULT_MAX_FILES)
+ .clamp(1, HARD_MAX_FILES);
+ let rel = match root {
+ "wiki" => "wiki",
+ "sources" | "raw" | "raw/sources" => "raw/sources",
+ "all" | "" => "",
+ _ => return err(400, "root must be wiki, sources, or all"),
+ };
+ if rel.is_empty() {
+ return match list_public_roots(&project.path, recursive, max_files) {
+ Ok(files) => ok(json!({
+ "ok": true,
+ "projectId": project.id,
+ "root": "all",
+ "files": files,
+ "truncated": false,
+ })),
+ Err(e) => err(if e.contains("exceeds") { 413 } else { 500 }, e),
+ };
+ }
+ let dir = match safe_join(&project.path, rel) {
+ Ok(path) => path,
+ Err(e) => return err(400, e),
+ };
+ let mut count = 0;
+ match list_tree(&project.path, &dir, recursive, max_files, &mut count) {
+ Ok(files) => ok(json!({
+ "ok": true,
+ "projectId": project.id,
+ "root": rel,
+ "files": files,
+ "truncated": false,
+ })),
+ Err(e) => err(if e.contains("exceeds") { 413 } else { 500 }, e),
+ }
+}
+
+fn handle_file_content(app: &AppHandle, project_id: &str, query: &str) -> ApiResponse {
+ let project = match resolve_project(app, project_id) {
+ Ok(project) => project,
+ Err(e) => return err(404, e),
+ };
+ let params = parse_query(query);
+ let Some(rel) = params.get("path") else {
+ return err(400, "Missing path query parameter");
+ };
+ if !is_public_project_rel(rel) {
+ return err(403, "Path is not exposed by the local API");
+ }
+ if !is_text_content_rel(rel) {
+ return err(
+ 415,
+ "Only text-like project files can be read via this endpoint",
+ );
+ }
+ let path = match safe_join(&project.path, rel) {
+ Ok(path) => path,
+ Err(e) => return err(400, e),
+ };
+ let meta = match fs::metadata(&path) {
+ Ok(meta) => meta,
+ Err(e) => return err(404, format!("File not found: {e}")),
+ };
+ if meta.len() > MAX_FILE_CONTENT_BYTES {
+ return err(413, "File is too large to return via API");
+ }
+ match fs::read_to_string(&path) {
+ Ok(content) => ok(json!({
+ "ok": true,
+ "projectId": project.id,
+ "path": rel,
+ "content": content,
+ })),
+ Err(_) => err(415, "File is not valid UTF-8 text"),
+ }
+}
+
+fn safe_join(project_path: &str, rel: &str) -> Result<PathBuf, String> {
+ let root = PathBuf::from(project_path);
+ let rel = rel.trim_start_matches('/');
+ let rel_path = Path::new(rel);
+ if rel_path.is_absolute() {
+ return Err("Absolute paths are not allowed".to_string());
+ }
+ for component in rel_path.components() {
+ if matches!(
+ component,
+ Component::ParentDir | Component::Prefix(_) | Component::RootDir
+ ) {
+ return Err("Path traversal is not allowed".to_string());
+ }
+ }
+ let joined = root.join(rel_path);
+ let root_canon = root
+ .canonicalize()
+ .map_err(|e| format!("Failed to resolve project path: {e}"))?;
+ if joined.exists() {
+ let joined_canon = joined
+ .canonicalize()
+ .map_err(|e| format!("Failed to resolve path: {e}"))?;
+ if !joined_canon.starts_with(&root_canon) {
+ return Err("Resolved path escapes the project directory".to_string());
+ }
+ return Ok(joined_canon);
+ }
+ let parent = joined
+ .parent()
+ .ok_or_else(|| "Path has no parent directory".to_string())?;
+ if parent.exists() {
+ let parent_canon = parent
+ .canonicalize()
+ .map_err(|e| format!("Failed to resolve parent path: {e}"))?;
+ if !parent_canon.starts_with(&root_canon) {
+ return Err("Resolved parent escapes the project directory".to_string());
+ }
+ }
+ Ok(joined)
+}
+
+fn is_public_project_rel(rel: &str) -> bool {
+ let rel = normalize_path(rel).trim_start_matches('/').to_string();
+ if rel
+ .split('/')
+ .any(|part| part.is_empty() || part.starts_with('.'))
+ {
+ return false;
+ }
+ let lower = rel.to_lowercase();
+ lower == "purpose.md"
+ || lower == "schema.md"
+ || lower.starts_with("wiki/")
+ || lower.starts_with("raw/sources/")
+}
+
+fn is_text_content_rel(rel: &str) -> bool {
+ let rel = normalize_path(rel).to_lowercase();
+ let ext = Path::new(&rel)
+ .extension()
+ .and_then(|s| s.to_str())
+ .unwrap_or("");
+ matches!(
+ ext,
+ "md" | "mdx"
+ | "txt"
+ | "csv"
+ | "json"
+ | "yaml"
+ | "yml"
+ | "xml"
+ | "html"
+ | "htm"
+ | "rtf"
+ | "log"
+ )
+}
+
+#[derive(Serialize)]
+#[serde(rename_all = "camelCase")]
+struct ApiFileNode {
+ name: String,
+ path: String,
+ is_dir: bool,
+ size: Option<u64>,
+ children: Option<Vec<ApiFileNode>>,
+}
+
+fn list_public_roots(
+ project_path: &str,
+ recursive: bool,
+ max_files: usize,
+) -> Result<Vec<ApiFileNode>, String> {
+ let mut count = 0;
+ let mut roots = Vec::new();
+ for rel in ["purpose.md", "schema.md", "wiki", "raw/sources"] {
+ let path = safe_join(project_path, rel)?;
+ if !path.exists() {
+ continue;
+ }
+ push_file_node(
+ project_path,
+ &path,
+ recursive,
+ max_files,
+ &mut count,
+ &mut roots,
+ )?;
+ }
+ Ok(roots)
+}
+
+fn list_tree(
+ project_path: &str,
+ path: &Path,
+ recursive: bool,
+ max_files: usize,
+ count: &mut usize,
+) -> Result<Vec<ApiFileNode>, String> {
+ let mut out = Vec::new();
+ let entries = fs::read_dir(path).map_err(|e| format!("Failed to list directory: {e}"))?;
+ for entry in entries {
+ let entry = entry.map_err(|e| format!("Failed to read directory entry: {e}"))?;
+ push_file_node(
+ project_path,
+ &entry.path(),
+ recursive,
+ max_files,
+ count,
+ &mut out,
+ )?;
+ }
+ out.sort_by(|a, b| b.is_dir.cmp(&a.is_dir).then_with(|| a.name.cmp(&b.name)));
+ Ok(out)
+}
+
+fn push_file_node(
+ project_path: &str,
+ path: &Path,
+ recursive: bool,
+ max_files: usize,
+ count: &mut usize,
+ out: &mut Vec<ApiFileNode>,
+) -> Result<(), String> {
+ let name = path
+ .file_name()
+ .and_then(|s| s.to_str())
+ .unwrap_or("")
+ .to_string();
+ if name.starts_with('.') {
+ return Ok(());
+ }
+ let meta = fs::symlink_metadata(path).map_err(|e| format!("Failed to read metadata: {e}"))?;
+ let file_type = meta.file_type();
+ if file_type.is_symlink() {
+ return Ok(());
+ }
+ *count += 1;
+ if *count > max_files {
+ return Err(format!("File listing exceeds maxFiles limit ({max_files})"));
+ }
+ let is_dir = file_type.is_dir();
+ let children = if recursive && is_dir {
+ Some(list_tree(project_path, path, true, max_files, count)?)
+ } else {
+ None
+ };
+ out.push(ApiFileNode {
+ name,
+ path: relative_to_project(project_path, path),
+ is_dir,
+ size: if is_dir { None } else { Some(meta.len()) },
+ children,
+ });
+ Ok(())
+}
+
+fn relative_to_project(project_path: &str, path: &Path) -> String {
+ let root = Path::new(project_path);
+ path.strip_prefix(root)
+ .map(|p| p.to_string_lossy().replace('\\', "/"))
+ .unwrap_or_else(|_| path.to_string_lossy().replace('\\', "/"))
+}
+
+#[derive(Deserialize)]
+#[serde(rename_all = "camelCase")]
+struct SearchRequest {
+ query: String,
+ top_k: Option<usize>,
+ include_content: Option<bool>,
+ query_embedding: Option<Vec<f32>>,
+}
+
+fn handle_search(app: &AppHandle, project_id: &str, body: &str) -> ApiResponse {
+ let project = match resolve_project(app, project_id) {
+ Ok(project) => project,
+ Err(e) => return err(404, e),
+ };
+ let req: SearchRequest = match serde_json::from_str(body) {
+ Ok(req) => req,
+ Err(e) => return err(400, format!("Invalid JSON: {e}")),
+ };
+ if req.query.trim().is_empty() {
+ return err(400, "query is required");
+ }
+ let top_k = req.top_k.unwrap_or(10).clamp(1, MAX_SEARCH_RESULTS);
+ let query = req.query;
+ let query_embedding =
+ match tauri::async_runtime::block_on(commands::search::resolve_query_embedding(
+ &query,
+ req.query_embedding,
+ load_embedding_config(app),
+ )) {
+ Ok(embedding) => embedding,
+ Err(e) => return err(400, e),
+ };
+ match tauri::async_runtime::block_on(commands::search::search_project_inner(
+ project.path.clone(),
+ query,
+ top_k,
+ req.include_content.unwrap_or(false),
+ query_embedding,
+ )) {
+ Ok(search) => ok(json!({
+ "ok": true,
+ "projectId": project.id,
+ "mode": search.mode,
+ "note": "Search uses the shared backend retrieval service. When embeddingConfig is enabled, the API automatically includes LanceDB vector results; clients may also pass queryEmbedding explicitly.",
+ "tokenHits": search.token_hits,
+ "vectorHits": search.vector_hits,
+ "results": search.results,
+ })),
+ Err(e) => err(500, e),
+ }
+}
+
+fn load_embedding_config(app: &AppHandle) -> Option<commands::search::SearchEmbeddingConfig> {
+ let parsed = load_app_state(app)?;
+ let value = parsed.get("embeddingConfig")?.clone();
+ serde_json::from_value::<commands::search::SearchEmbeddingConfig>(value).ok()
+}
+
+#[derive(Debug, Clone, Serialize)]
+#[serde(rename_all = "camelCase")]
+struct ApiGraphNode {
+ id: String,
+ label: String,
+ node_type: String,
+ path: String,
+ link_count: usize,
+}
+
+#[derive(Debug, Clone, Serialize)]
+#[serde(rename_all = "camelCase")]
+struct ApiGraphEdge {
+ source: String,
+ target: String,
+ weight: f64,
+}
+
+fn handle_graph(app: &AppHandle, project_id: &str, query: &str) -> ApiResponse {
+ let project = match resolve_project(app, project_id) {
+ Ok(project) => project,
+ Err(e) => return err(404, e),
+ };
+ let params = parse_query(query);
+ let q = params.get("q").map(|s| s.to_lowercase());
+ let node_type = params.get("nodeType").map(|s| s.to_lowercase());
+ let limit = params
+ .get("limit")
+ .and_then(|s| s.parse::<usize>().ok())
+ .unwrap_or(200)
+ .clamp(1, 1000);
+
+ match build_graph(&project.path) {
+ Ok((mut nodes, edges)) => {
+ if let Some(ref q) = q {
+ nodes.retain(|n| {
+ n.id.to_lowercase().contains(q) || n.label.to_lowercase().contains(q)
+ });
+ }
+ if let Some(ref node_type) = node_type {
+ nodes.retain(|n| n.node_type == *node_type);
+ }
+ nodes.truncate(limit);
+ let ids: BTreeSet<String> = nodes.iter().map(|n| n.id.clone()).collect();
+ let edges: Vec<ApiGraphEdge> = edges
+ .into_iter()
+ .filter(|e| ids.contains(&e.source) && ids.contains(&e.target))
+ .collect();
+ ok(json!({ "ok": true, "projectId": project.id, "nodes": nodes, "edges": edges }))
+ }
+ Err(e) => err(500, e),
+ }
+}
+
+fn build_graph(project_path: &str) -> Result<(Vec<ApiGraphNode>, Vec<ApiGraphEdge>), String> {
+ let wiki_root = Path::new(project_path).join("wiki");
+ let mut raw: BTreeMap<String, (String, String, String, Vec<String>)> = BTreeMap::new();
+ for entry in WalkDir::new(&wiki_root).into_iter().filter_map(Result::ok) {
+ if !entry.file_type().is_file()
+ || entry.path().extension().and_then(|s| s.to_str()) != Some("md")
+ {
+ continue;
+ }
+ let content = match fs::read_to_string(entry.path()) {
+ Ok(content) => content,
+ Err(_) => continue,
+ };
+ let id = entry
+ .path()
+ .file_stem()
+ .and_then(|s| s.to_str())
+ .unwrap_or("")
+ .to_string();
+ if id.is_empty() {
+ continue;
+ }
+ let title =
+ commands::search::extract_title(&content, entry.file_name().to_string_lossy().as_ref());
+ let node_type = extract_type(&content);
+ let path = relative_to_project(project_path, entry.path());
+ let links = extract_wikilinks(&content);
+ raw.insert(id, (title, node_type, path, links));
+ }
+ let ids: BTreeSet<String> = raw.keys().cloned().collect();
+ let mut link_count: BTreeMap<String, usize> = raw.keys().map(|id| (id.clone(), 0)).collect();
+ let mut seen = BTreeSet::new();
+ let mut edges = Vec::new();
+ for (source, (_, _, _, links)) in &raw {
+ for link in links {
+ let Some(target) = resolve_link(link, &ids) else {
+ continue;
+ };
+ if &target == source {
+ continue;
+ }
+ let key = if source < &target {
+ format!("{source}::{target}")
+ } else {
+ format!("{target}::{source}")
+ };
+ if seen.insert(key) {
+ *link_count.entry(source.clone()).or_default() += 1;
+ *link_count.entry(target.clone()).or_default() += 1;
+ edges.push(ApiGraphEdge {
+ source: source.clone(),
+ target,
+ weight: 1.0,
+ });
+ }
+ }
+ }
+ let nodes = raw
+ .into_iter()
+ .filter(|(_, (_, node_type, _, _))| node_type != "query")
+ .map(|(id, (label, node_type, path, _))| ApiGraphNode {
+ link_count: *link_count.get(&id).unwrap_or(&0),
+ id,
+ label,
+ node_type,
+ path,
+ })
+ .collect();
+ Ok((nodes, edges))
+}
+
+fn extract_type(content: &str) -> String {
+ for line in content.lines() {
+ if let Some(value) = line.trim().strip_prefix("type:") {
+ return value
+ .trim()
+ .trim_matches('"')
+ .trim_matches('\'')
+ .to_lowercase();
+ }
+ }
+ "other".to_string()
+}
+
+fn extract_wikilinks(content: &str) -> Vec<String> {
+ let mut out = Vec::new();
+ let mut rest = content;
+ while let Some(start) = rest.find("[[") {
+ rest = &rest[start + 2..];
+ let Some(end) = rest.find("]]") else {
+ break;
+ };
+ let inner = &rest[..end];
+ let target = inner.split('|').next().unwrap_or("").trim();
+ if !target.is_empty() {
+ out.push(target.to_string());
+ }
+ rest = &rest[end + 2..];
+ }
+ out
+}
+
+fn resolve_link(raw: &str, ids: &BTreeSet<String>) -> Option<String> {
+ if ids.contains(raw) {
+ return Some(raw.to_string());
+ }
+ let normalized = raw.to_lowercase().replace(' ', "-");
+ ids.iter()
+ .find(|id| id.to_lowercase() == normalized || id.to_lowercase() == raw.to_lowercase())
+ .cloned()
+}
+
+fn handle_rescan(app: &AppHandle, project_id: &str) -> ApiResponse {
+ let project = match resolve_project(app, project_id) {
+ Ok(project) => project,
+ Err(e) => return err(404, e),
+ };
+ let source_watch_config = load_source_watch_config(app, &project.id);
+ match commands::file_sync::rescan_project_files(
+ app.clone(),
+ project.id.clone(),
+ project.path.clone(),
+ source_watch_config,
+ ) {
+ Ok(result) => ok(json!({ "ok": true, "projectId": project.id, "result": result })),
+ Err(e) => err(500, e),
+ }
+}
+
+fn load_source_watch_config(
+ app: &AppHandle,
+ project_id: &str,
+) -> Option<commands::file_sync::SourceWatchConfig> {
+ let parsed = load_app_state(app)?;
+ let settings = parsed.get("sourceWatchConfig").and_then(Value::as_object);
+ if let Some(value) = settings
+ .and_then(|s| s.get(project_id).or_else(|| s.get("default")))
+ .cloned()
+ {
+ if let Ok(config) = serde_json::from_value::<commands::file_sync::SourceWatchConfig>(value)
+ {
+ return Some(config);
+ }
+ }
+ let legacy_enabled = parsed
+ .get("projectFileSyncEnabled")
+ .and_then(Value::as_object)
+ .and_then(|settings| {
+ settings
+ .get(project_id)
+ .or_else(|| settings.get("default"))
+ .and_then(Value::as_bool)
+ });
+ legacy_enabled.and_then(|enabled| {
+ serde_json::from_value::<commands::file_sync::SourceWatchConfig>(
+ json!({ "enabled": enabled }),
+ )
+ .ok()
+ })
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use std::time::{SystemTime, UNIX_EPOCH};
+
+ fn test_project_dir() -> PathBuf {
+ let id = SystemTime::now()
+ .duration_since(UNIX_EPOCH)
+ .unwrap()
+ .as_nanos();
+ let path = std::env::temp_dir().join(format!("llm-wiki-api-test-{id}"));
+ fs::create_dir_all(path.join("wiki")).unwrap();
+ path
+ }
+
+ #[test]
+ fn safe_join_rejects_traversal() {
+ let root = test_project_dir();
+ let root_str = root.to_string_lossy();
+ assert!(safe_join(&root_str, "../secret.md").is_err());
+ assert!(safe_join(&root_str, "wiki/../../secret.md").is_err());
+ let _ = fs::remove_dir_all(root);
+ }
+
+ #[test]
+ fn safe_join_accepts_project_relative_paths() {
+ let root = test_project_dir();
+ let root_str = root.to_string_lossy();
+ let joined = safe_join(&root_str, "wiki/index.md").unwrap();
+ assert_eq!(joined, root.join("wiki/index.md"));
+ let _ = fs::remove_dir_all(root);
+ }
+
+ #[test]
+ fn query_parser_decodes_percent_and_plus() {
+ let parsed = parse_query("path=wiki%2Fhello+world.md&token=a%2Bb");
+ assert_eq!(parsed.get("path").unwrap(), "wiki/hello world.md");
+ assert_eq!(parsed.get("token").unwrap(), "a+b");
+ }
+
+ #[test]
+ fn snippet_handles_unicode_boundaries() {
+ let content = "前言。这里是关于知识图谱过滤的中文内容。后续说明。";
+ let snippet = commands::search::build_snippet(content, "知识图谱");
+ assert!(snippet.contains("知识图谱"));
+ }
+
+ #[test]
+ fn public_api_paths_exclude_internal_state() {
+ assert!(is_public_project_rel("wiki/index.md"));
+ assert!(is_public_project_rel("Wiki/index.md"));
+ assert!(is_public_project_rel("raw/sources/source.md"));
+ assert!(is_public_project_rel("Raw/Sources/source.md"));
+ assert!(!is_public_project_rel(".llm-wiki/file-change-queue.json"));
+ assert!(!is_public_project_rel("wiki/.draft.md"));
+ }
+
+ #[test]
+ fn project_path_match_normalizes_separators() {
+ assert!(project_path_matches(
+ "C:/Users/me/wiki",
+ "C:\\Users\\me\\wiki"
+ ));
+ if cfg!(windows) {
+ assert!(project_path_matches("C:/Users/me/wiki", "c:/users/me/wiki"));
+ } else {
+ assert!(!project_path_matches(
+ "C:/Users/me/wiki",
+ "c:/users/me/wiki"
+ ));
+ }
+ }
+
+ #[test]
+ fn tokenize_keeps_single_cjk_character() {
+ assert_eq!(
+ crate::commands::search::tokenize_query("图"),
+ Vec::<String>::new()
+ );
+ let tokens = crate::commands::search::tokenize_query("知识图谱");
+ assert!(tokens.contains(&"知识".to_string()));
+ }
+
+ #[test]
+ fn text_content_filter_rejects_binary_extensions() {
+ assert!(is_text_content_rel("wiki/index.md"));
+ assert!(!is_text_content_rel("wiki/media/image.png"));
+ assert!(!is_text_content_rel("raw/sources/book.pdf"));
+ }
+
+ #[test]
+ fn constant_time_eq_matches_equal_bytes_only() {
+ assert!(constant_time_eq(b"token", b"token"));
+ assert!(constant_time_eq(b"", b""));
+ assert!(!constant_time_eq(b"token", b"tokeN"));
+ assert!(!constant_time_eq(b"token", b"token-longer"));
+ }
+
+ #[test]
+ fn rate_limit_skips_health_and_options_only() {
+ assert!(!should_rate_limit(&Method::Get, "/api/v1/health"));
+ assert!(!should_rate_limit(&Method::Options, "/api/v1/projects"));
+ assert!(should_rate_limit(&Method::Get, "/wp-login"));
+ assert!(should_rate_limit(
+ &Method::Post,
+ "/api/v1/projects/current/search"
+ ));
+ }
+
+ #[test]
+ fn api_config_shape_parses_enabled_and_unauthenticated_access() {
+ // Standalone pure-function check to mirror what `api_enabled`
+ // reads off `load_app_state`. Mirrors the JS-side shape
+ // emitted by `saveApiConfig` so any rename on either side
+ // surfaces here before users hit it as a 503 in production.
+ let payload = json!({
+ "apiConfig": {
+ "enabled": false,
+ "allowUnauthenticated": true,
+ "token": "abc"
+ }
+ });
+ let enabled = payload
+ .get("apiConfig")
+ .and_then(|v| v.get("enabled"))
+ .and_then(Value::as_bool)
+ .unwrap_or(true);
+ assert!(!enabled);
+ let allow_unauthenticated = payload
+ .get("apiConfig")
+ .and_then(|v| v.get("allowUnauthenticated"))
+ .and_then(Value::as_bool)
+ .unwrap_or(false);
+ assert!(allow_unauthenticated);
+ let token_source = payload
+ .get("apiConfig")
+ .and_then(|v| v.get("token"))
+ .and_then(Value::as_str)
+ .filter(|s| !s.is_empty())
+ .map(|_| "store")
+ .unwrap_or("none");
+ assert_eq!(token_source, "store");
+
+ let missing = json!({});
+ let enabled_missing = missing
+ .get("apiConfig")
+ .and_then(|v| v.get("enabled"))
+ .and_then(Value::as_bool)
+ .unwrap_or(true);
+ // Fail-open by design — see `api_enabled` doc comment.
+ assert!(enabled_missing);
+ }
+}
diff --git a/src-tauri/src/clip_server.rs b/src-tauri/src/clip_server.rs
index 33a47f4c..b68402f0 100644
--- a/src-tauri/src/clip_server.rs
+++ b/src-tauri/src/clip_server.rs
@@ -1,5 +1,5 @@
-use std::sync::Mutex;
use std::sync::atomic::{AtomicU8, Ordering};
+use std::sync::Mutex;
use std::thread;
use tiny_http::{Header, Method, Response, Server};
@@ -26,6 +26,20 @@ pub fn get_daemon_status() -> &'static str {
}
}
+pub fn current_project_path() -> String {
+ CURRENT_PROJECT
+ .lock()
+ .map(|guard| guard.clone())
+ .unwrap_or_default()
+}
+
+pub fn all_projects() -> Vec<(String, String)> {
+ ALL_PROJECTS
+ .lock()
+ .map(|guard| guard.clone())
+ .unwrap_or_default()
+}
+
pub fn start_clip_server() {
thread::spawn(|| {
let mut restart_count: u32 = 0;
@@ -48,7 +62,9 @@ pub fn start_clip_server() {
attempt, MAX_BIND_RETRIES, e
);
if attempt < MAX_BIND_RETRIES {
- thread::sleep(std::time::Duration::from_secs(BIND_RETRY_DELAY_SECS));
+ thread::sleep(std::time::Duration::from_secs(
+ BIND_RETRY_DELAY_SECS,
+ ));
}
}
}
@@ -70,174 +86,190 @@ pub fn start_clip_server() {
restart_count = 0; // Reset on successful bind
println!("[Clip Server] Listening on http://127.0.0.1:{}", PORT);
- for mut request in server.incoming_requests() {
- let cors_headers = vec![
- Header::from_bytes("Access-Control-Allow-Origin", "*").unwrap(),
- Header::from_bytes("Access-Control-Allow-Methods", "GET, POST, OPTIONS").unwrap(),
- Header::from_bytes("Access-Control-Allow-Headers", "Content-Type").unwrap(),
- Header::from_bytes("Content-Type", "application/json").unwrap(),
- ];
-
- // Handle CORS preflight
- if request.method() == &Method::Options {
- let mut response = Response::from_string("").with_status_code(204);
- for h in &cors_headers {
- response.add_header(h.clone());
- }
- let _ = request.respond(response);
- continue;
- }
-
- let url = request.url().to_string();
-
- match (request.method(), url.as_str()) {
- (&Method::Get, "/status") => {
- let body = r#"{"ok":true,"version":"0.1.0"}"#;
- let mut response = Response::from_string(body);
+ for mut request in server.incoming_requests() {
+ let cors_headers = vec![
+ Header::from_bytes("Access-Control-Allow-Origin", "*").unwrap(),
+ Header::from_bytes("Access-Control-Allow-Methods", "GET, POST, OPTIONS")
+ .unwrap(),
+ Header::from_bytes("Access-Control-Allow-Headers", "Content-Type").unwrap(),
+ Header::from_bytes("Content-Type", "application/json").unwrap(),
+ ];
+
+ // Handle CORS preflight
+ if request.method() == &Method::Options {
+ let mut response = Response::from_string("").with_status_code(204);
for h in &cors_headers {
response.add_header(h.clone());
}
let _ = request.respond(response);
+ continue;
}
- (&Method::Get, "/project") => {
- let path = CURRENT_PROJECT.lock().unwrap().clone();
- // serde_json handles backslash escaping so a Windows
- // path that somehow still contains `\` won't break
- // the JSON parser on the client.
- let body = serde_json::json!({
- "ok": true,
- "path": path,
- }).to_string();
- let mut response = Response::from_string(body);
- for h in &cors_headers {
- response.add_header(h.clone());
+
+ let url = request.url().to_string();
+
+ match (request.method(), url.as_str()) {
+ (&Method::Get, "/status") => {
+ let body = r#"{"ok":true,"version":"0.1.0"}"#;
+ let mut response = Response::from_string(body);
+ for h in &cors_headers {
+ response.add_header(h.clone());
+ }
+ let _ = request.respond(response);
}
- let _ = request.respond(response);
- }
- (&Method::Post, "/project") => {
- let mut body = String::new();
- if let Err(e) = request.as_reader().read_to_string(&mut body) {
- let err =
- format!(r#"{{"ok":false,"error":"Failed to read body: {}"}}"#, e);
- let mut response = Response::from_string(err).with_status_code(400);
+ (&Method::Get, "/project") => {
+ let path = CURRENT_PROJECT.lock().unwrap().clone();
+ // serde_json handles backslash escaping so a Windows
+ // path that somehow still contains `\` won't break
+ // the JSON parser on the client.
+ let body = serde_json::json!({
+ "ok": true,
+ "path": path,
+ })
+ .to_string();
+ let mut response = Response::from_string(body);
for h in &cors_headers {
response.add_header(h.clone());
}
let _ = request.respond(response);
- continue;
}
+ (&Method::Post, "/project") => {
+ let mut body = String::new();
+ if let Err(e) = request.as_reader().read_to_string(&mut body) {
+ let err =
+ format!(r#"{{"ok":false,"error":"Failed to read body: {}"}}"#, e);
+ let mut response = Response::from_string(err).with_status_code(400);
+ for h in &cors_headers {
+ response.add_header(h.clone());
+ }
+ let _ = request.respond(response);
+ continue;
+ }
- let result = handle_set_project(&body);
- let status = if result.contains(r#""ok":true"#) {
- 200
- } else {
- 400
- };
- let mut response = Response::from_string(result).with_status_code(status);
- for h in &cors_headers {
- response.add_header(h.clone());
+ let result = handle_set_project(&body);
+ let status = if result.contains(r#""ok":true"#) {
+ 200
+ } else {
+ 400
+ };
+ let mut response = Response::from_string(result).with_status_code(status);
+ for h in &cors_headers {
+ response.add_header(h.clone());
+ }
+ let _ = request.respond(response);
}
- let _ = request.respond(response);
- }
- (&Method::Get, "/projects") => {
- let projects = ALL_PROJECTS.lock().unwrap().clone();
- let current = CURRENT_PROJECT.lock().unwrap().clone();
- // serde_json for proper escaping of `\`, `"`, and any
- // other characters that might appear in a project name
- // or path. Previously only `"` was escaped by hand,
- // which broke on Windows paths containing backslashes.
- let items: Vec<serde_json::Value> = projects.iter()
- .map(|(name, path)| serde_json::json!({
- "name": name,
- "path": path,
- "current": path == &current,
- }))
- .collect();
- let body = serde_json::json!({
- "ok": true,
- "projects": items,
- }).to_string();
- let mut response = Response::from_string(body);
- for h in &cors_headers { response.add_header(h.clone()); }
- let _ = request.respond(response);
- }
- (&Method::Post, "/projects") => {
- let mut body = String::new();
- if request.as_reader().read_to_string(&mut body).is_ok() {
- if let Ok(parsed) = serde_json::from_str::<serde_json::Value>(&body) {
- if let Some(arr) = parsed["projects"].as_array() {
- let mut projects = ALL_PROJECTS.lock().unwrap();
- projects.clear();
- for item in arr {
- let name = item["name"].as_str().unwrap_or("").to_string();
- let path = item["path"].as_str().unwrap_or("").to_string();
- if !path.is_empty() {
- projects.push((name, path));
+ (&Method::Get, "/projects") => {
+ let projects = ALL_PROJECTS.lock().unwrap().clone();
+ let current = CURRENT_PROJECT.lock().unwrap().clone();
+ // serde_json for proper escaping of `\`, `"`, and any
+ // other characters that might appear in a project name
+ // or path. Previously only `"` was escaped by hand,
+ // which broke on Windows paths containing backslashes.
+ let items: Vec<serde_json::Value> = projects
+ .iter()
+ .map(|(name, path)| {
+ serde_json::json!({
+ "name": name,
+ "path": path,
+ "current": path == &current,
+ })
+ })
+ .collect();
+ let body = serde_json::json!({
+ "ok": true,
+ "projects": items,
+ })
+ .to_string();
+ let mut response = Response::from_string(body);
+ for h in &cors_headers {
+ response.add_header(h.clone());
+ }
+ let _ = request.respond(response);
+ }
+ (&Method::Post, "/projects") => {
+ let mut body = String::new();
+ if request.as_reader().read_to_string(&mut body).is_ok() {
+ if let Ok(parsed) = serde_json::from_str::<serde_json::Value>(&body) {
+ if let Some(arr) = parsed["projects"].as_array() {
+ let mut projects = ALL_PROJECTS.lock().unwrap();
+ projects.clear();
+ for item in arr {
+ let name = item["name"].as_str().unwrap_or("").to_string();
+ let path = item["path"].as_str().unwrap_or("").to_string();
+ if !path.is_empty() {
+ projects.push((name, path));
+ }
}
}
}
}
+ let mut response = Response::from_string(r#"{"ok":true}"#);
+ for h in &cors_headers {
+ response.add_header(h.clone());
+ }
+ let _ = request.respond(response);
}
- let mut response = Response::from_string(r#"{"ok":true}"#);
- for h in &cors_headers { response.add_header(h.clone()); }
- let _ = request.respond(response);
- }
- (&Method::Get, "/clips/pending") => {
- let mut pending = PENDING_CLIPS.lock().unwrap();
- // Use serde_json for proper escaping of both quotes
- // and backslashes — hand-rolled escaping previously
- // produced invalid JSON on Windows paths containing
- // \r, \s, etc.
- let clips_json: Vec<serde_json::Value> = pending.iter()
- .map(|(proj, file)| serde_json::json!({
- "projectPath": proj,
- "filePath": file,
- }))
- .collect();
- let body = serde_json::json!({
- "ok": true,
- "clips": clips_json,
- }).to_string();
- pending.clear();
- let mut response = Response::from_string(body);
- for h in &cors_headers { response.add_header(h.clone()); }
- let _ = request.respond(response);
- }
- (&Method::Post, "/clip") => {
- let mut body = String::new();
- if let Err(e) = request.as_reader().read_to_string(&mut body) {
- let err =
- format!(r#"{{"ok":false,"error":"Failed to read body: {}"}}"#, e);
- let mut response = Response::from_string(err).with_status_code(400);
+ (&Method::Get, "/clips/pending") => {
+ let mut pending = PENDING_CLIPS.lock().unwrap();
+ // Use serde_json for proper escaping of both quotes
+ // and backslashes — hand-rolled escaping previously
+ // produced invalid JSON on Windows paths containing
+ // \r, \s, etc.
+ let clips_json: Vec<serde_json::Value> = pending
+ .iter()
+ .map(|(proj, file)| {
+ serde_json::json!({
+ "projectPath": proj,
+ "filePath": file,
+ })
+ })
+ .collect();
+ let body = serde_json::json!({
+ "ok": true,
+ "clips": clips_json,
+ })
+ .to_string();
+ pending.clear();
+ let mut response = Response::from_string(body);
for h in &cors_headers {
response.add_header(h.clone());
}
let _ = request.respond(response);
- continue;
}
+ (&Method::Post, "/clip") => {
+ let mut body = String::new();
+ if let Err(e) = request.as_reader().read_to_string(&mut body) {
+ let err =
+ format!(r#"{{"ok":false,"error":"Failed to read body: {}"}}"#, e);
+ let mut response = Response::from_string(err).with_status_code(400);
+ for h in &cors_headers {
+ response.add_header(h.clone());
+ }
+ let _ = request.respond(response);
+ continue;
+ }
- let result = handle_clip(&body);
- let status = if result.contains(r#""ok":true"#) {
- 200
- } else {
- 500
- };
- let mut response = Response::from_string(result).with_status_code(status);
- for h in &cors_headers {
- response.add_header(h.clone());
+ let result = handle_clip(&body);
+ let status = if result.contains(r#""ok":true"#) {
+ 200
+ } else {
+ 500
+ };
+ let mut response = Response::from_string(result).with_status_code(status);
+ for h in &cors_headers {
+ response.add_header(h.clone());
+ }
+ let _ = request.respond(response);
}
- let _ = request.respond(response);
- }
- _ => {
- let body = r#"{"ok":false,"error":"Not found"}"#;
- let mut response = Response::from_string(body).with_status_code(404);
- for h in &cors_headers {
- response.add_header(h.clone());
+ _ => {
+ let body = r#"{"ok":false,"error":"Not found"}"#;
+ let mut response = Response::from_string(body).with_status_code(404);
+ for h in &cors_headers {
+ response.add_header(h.clone());
+ }
+ let _ = request.respond(response);
}
- let _ = request.respond(response);
}
}
- }
// Server loop exited (shouldn't happen normally)
DAEMON_STATUS.store(3, Ordering::Relaxed); // error
@@ -337,7 +369,9 @@ fn handle_clip(body: &str) -> String {
let base_name = format!("{}-{}", slug, date_compact);
// Use PathBuf for cross-platform path construction
- let dir_path = std::path::Path::new(&project_path).join("raw").join("sources");
+ let dir_path = std::path::Path::new(&project_path)
+ .join("raw")
+ .join("sources");
// Ensure directory exists
if let Err(e) = std::fs::create_dir_all(&dir_path) {
@@ -373,10 +407,7 @@ fn handle_clip(body: &str) -> String {
);
if let Err(e) = std::fs::write(&file_path, &markdown) {
- return format!(
- r#"{{"ok":false,"error":"Failed to write file: {}"}}"#,
- e
- );
+ return format!(r#"{{"ok":false,"error":"Failed to write file: {}"}}"#, e);
}
// Compute relative path using Path for cross-platform separator handling
@@ -396,5 +427,6 @@ fn handle_clip(body: &str) -> String {
serde_json::json!({
"ok": true,
"path": relative_path,
- }).to_string()
+ })
+ .to_string()
}
diff --git a/src-tauri/src/commands/extract_images.rs b/src-tauri/src/commands/extract_images.rs
index b2ed8be0..db4111e6 100644
--- a/src-tauri/src/commands/extract_images.rs
+++ b/src-tauri/src/commands/extract_images.rs
@@ -111,14 +111,12 @@ pub fn extract_pdf_markdown(
let _guard = crate::commands::fs::lock_pdfium();
let pdfium = crate::commands::fs::pdfium()?;
- let doc = pdfium
- .load_pdf_from_file(path, None)
- .map_err(|e| match e {
- PdfiumError::PdfiumLibraryInternalError(
- PdfiumInternalError::PasswordError,
- ) => format!("PDF is password-protected and cannot be read: '{path}'"),
- _ => format!("Failed to open PDF '{path}': {e}"),
- })?;
+ let doc = pdfium.load_pdf_from_file(path, None).map_err(|e| match e {
+ PdfiumError::PdfiumLibraryInternalError(PdfiumInternalError::PasswordError) => {
+ format!("PDF is password-protected and cannot be read: '{path}'")
+ }
+ _ => format!("Failed to open PDF '{path}': {e}"),
+ })?;
let mut out = String::new();
let mut idx: u32 = 0;
@@ -144,9 +142,7 @@ pub fn extract_pdf_markdown(
let page_text = page
.text()
- .map_err(|e| {
- format!("Page {page_num} text extraction failed in '{path}': {e}")
- })?;
+ .map_err(|e| format!("Page {page_num} text extraction failed in '{path}': {e}"))?;
out.push_str(&page_text.all());
// Single trailing newline so the next block starts on its own
// line; the `\n\n` separator before the next `## Page` heading
@@ -170,9 +166,7 @@ pub fn extract_pdf_markdown(
let dyn_img = match image.get_raw_image() {
Ok(b) => b,
Err(e) => {
- eprintln!(
- "[extract_pdf_markdown] page {page_num} image read failed: {e}"
- );
+ eprintln!("[extract_pdf_markdown] page {page_num} image read failed: {e}");
continue;
}
};
@@ -186,9 +180,7 @@ pub fn extract_pdf_markdown(
&mut std::io::Cursor::new(&mut png_bytes),
image::ImageFormat::Png,
) {
- eprintln!(
- "[extract_pdf_markdown] page {page_num} PNG encode failed: {e}"
- );
+ eprintln!("[extract_pdf_markdown] page {page_num} PNG encode failed: {e}");
continue;
}
idx += 1;
@@ -198,9 +190,7 @@ pub fn extract_pdf_markdown(
// pass dest_dir for both args so save_one_image's
// strip_prefix is a no-op.
if let Err(e) = save_one_image(&png_bytes, dest_dir, dest_dir, &file_name) {
- eprintln!(
- "[extract_pdf_markdown] page {page_num} save failed: {e}"
- );
+ eprintln!("[extract_pdf_markdown] page {page_num} save failed: {e}");
continue;
}
total_saved += 1;
@@ -230,9 +220,7 @@ pub fn extract_pdf_markdown(
}
if media_dest_dir.is_some() {
- eprintln!(
- "[extract_pdf_markdown] '{path}' DONE — pages={page_count}, saved={total_saved}"
- );
+ eprintln!("[extract_pdf_markdown] '{path}' DONE — pages={page_count}, saved={total_saved}");
}
Ok(out)
@@ -695,7 +683,11 @@ pub fn extract_and_save_pdf_images(
filtered_too_small += 1;
eprintln!(
"[extract_and_save_pdf_images] page {} image {}x{} < min ({}x{}) — skipped",
- page_idx + 1, width, height, options.min_width, options.min_height
+ page_idx + 1,
+ width,
+ height,
+ options.min_width,
+ options.min_height
);
continue;
}
diff --git a/src-tauri/src/commands/file_sync.rs b/src-tauri/src/commands/file_sync.rs
index c34750dc..260559e5 100644
--- a/src-tauri/src/commands/file_sync.rs
+++ b/src-tauri/src/commands/file_sync.rs
@@ -266,20 +266,18 @@ pub fn start_project_file_watcher(
let root_for_overflow = root.clone();
let root_for_error = root.clone();
let mut watcher = RecommendedWatcher::new(
- move |res: notify::Result<Event>| {
- match res {
- Ok(event) => {
- for path in event.paths {
- if tx_for_watcher.try_send(path).is_err() {
- let _ = tx_for_watcher.try_send(root_for_overflow.clone());
- break;
- }
+ move |res: notify::Result<Event>| match res {
+ Ok(event) => {
+ for path in event.paths {
+ if tx_for_watcher.try_send(path).is_err() {
+ let _ = tx_for_watcher.try_send(root_for_overflow.clone());
+ break;
}
}
- Err(err) => {
- eprintln!("[file-sync] watcher error; scheduling rescan: {err}");
- let _ = tx_for_watcher.try_send(root_for_error.clone());
- }
+ }
+ Err(err) => {
+ eprintln!("[file-sync] watcher error; scheduling rescan: {err}");
+ let _ = tx_for_watcher.try_send(root_for_error.clone());
}
},
Config::default(),
@@ -441,7 +439,12 @@ fn handle_changed_paths(
if path.is_dir() {
for entry in WalkDir::new(&path).into_iter().filter_map(Result::ok) {
if entry.file_type().is_file() && !is_app_write_ignored(entry.path()) {
- if let Some(rel) = relative_watch_path(root, entry.path(), &rules, entry.metadata().ok().map(|m| m.len())) {
+ if let Some(rel) = relative_watch_path(
+ root,
+ entry.path(),
+ &rules,
+ entry.metadata().ok().map(|m| m.len()),
+ ) {
rels.insert(rel);
}
}
@@ -487,7 +490,13 @@ fn maybe_periodic_rescan(
}
*last_periodic_rescan = now_ms();
let result = std::panic::catch_unwind(AssertUnwindSafe(|| {
- rescan_watch_roots(app, root, project_id, source_watch_config, watcher_generation)
+ rescan_watch_roots(
+ app,
+ root,
+ project_id,
+ source_watch_config,
+ watcher_generation,
+ )
}));
match result {
Ok(Ok(())) => {}
@@ -538,7 +547,12 @@ fn collect_known_paths(
if path.is_dir() {
for entry in WalkDir::new(path).into_iter().filter_map(Result::ok) {
if entry.file_type().is_file() {
- if let Some(rel) = relative_watch_path(root, entry.path(), rules, entry.metadata().ok().map(|m| m.len())) {
+ if let Some(rel) = relative_watch_path(
+ root,
+ entry.path(),
+ rules,
+ entry.metadata().ok().map(|m| m.len()),
+ ) {
rels.insert(rel);
}
}
@@ -599,7 +613,12 @@ fn enqueue_rescan_changes(
let mut rels = BTreeSet::<String>::new();
for entry in WalkDir::new(root).into_iter().filter_map(Result::ok) {
if entry.file_type().is_file() {
- if let Some(rel) = relative_watch_path(root, entry.path(), &rules, entry.metadata().ok().map(|m| m.len())) {
+ if let Some(rel) = relative_watch_path(
+ root,
+ entry.path(),
+ &rules,
+ entry.metadata().ok().map(|m| m.len()),
+ ) {
rels.insert(rel);
}
}
@@ -626,20 +645,33 @@ fn enqueue_rescan_changes_for_prefixes(
for prefix in prefixes {
let path = root.join(prefix);
if path.is_file() {
- if let Some(rel) = relative_watch_path(root, &path, &rules, fs::metadata(&path).ok().map(|m| m.len())) {
+ if let Some(rel) = relative_watch_path(
+ root,
+ &path,
+ &rules,
+ fs::metadata(&path).ok().map(|m| m.len()),
+ ) {
let old = snapshot.files.get(&rel);
let fast = read_meta_fast(root, &rel)?;
- if old.map(|m| (m.size, m.mtime_ms)) != fast.as_ref().map(|m| (m.size, m.mtime_ms)) {
+ if old.map(|m| (m.size, m.mtime_ms)) != fast.as_ref().map(|m| (m.size, m.mtime_ms))
+ {
rels.insert(rel);
}
}
} else if path.exists() {
for entry in WalkDir::new(&path).into_iter().filter_map(Result::ok) {
if entry.file_type().is_file() {
- if let Some(rel) = relative_watch_path(root, entry.path(), &rules, entry.metadata().ok().map(|m| m.len())) {
+ if let Some(rel) = relative_watch_path(
+ root,
+ entry.path(),
+ &rules,
+ entry.metadata().ok().map(|m| m.len()),
+ ) {
let old = snapshot.files.get(&rel);
let fast = read_meta_fast(root, &rel)?;
- if old.map(|m| (m.size, m.mtime_ms)) != fast.as_ref().map(|m| (m.size, m.mtime_ms)) {
+ if old.map(|m| (m.size, m.mtime_ms))
+ != fast.as_ref().map(|m| (m.size, m.mtime_ms))
+ {
rels.insert(rel);
}
}
@@ -756,7 +788,11 @@ fn upsert_task(
});
}
-fn process_queue(app: &AppHandle, root: &Path, project_id: &str) -> Result<Vec<FileChangeTask>, String> {
+fn process_queue(
+ app: &AppHandle,
+ root: &Path,
+ project_id: &str,
+) -> Result<Vec<FileChangeTask>, String> {
process_queue_inner(
root,
project_id,
@@ -1050,9 +1086,7 @@ fn should_watch_rel(rel: &str, rules: &SourceWatchRules) -> bool {
}
if rel.starts_with("raw/sources/") {
let ext = extension_of(name);
- if !ext.is_empty()
- && rules.exclude_extensions.contains(ext)
- {
+ if !ext.is_empty() && rules.exclude_extensions.contains(ext) {
return false;
}
if !rules.include_extensions.is_empty()
@@ -1062,9 +1096,7 @@ fn should_watch_rel(rel: &str, rules: &SourceWatchRules) -> bool {
}
return true;
}
- rel == "purpose.md"
- || rel == "schema.md"
- || (rel.starts_with("wiki/") && rel.ends_with(".md"))
+ rel == "purpose.md" || rel == "schema.md" || (rel.starts_with("wiki/") && rel.ends_with(".md"))
}
struct SourceWatchRules<'a> {
@@ -1248,7 +1280,9 @@ fn write_json<T: Serialize>(path: PathBuf, value: &T) -> Result<(), String> {
.unwrap_or_else(|| "file-sync.json".to_string());
let tmp_path = path.with_file_name(format!(
".{file_name}.{}.tmp",
- chrono::Utc::now().timestamp_nanos_opt().unwrap_or_else(now_ms)
+ chrono::Utc::now()
+ .timestamp_nanos_opt()
+ .unwrap_or_else(now_ms)
));
fs::write(&tmp_path, text)
.map_err(|e| format!("Failed to write '{}': {e}", tmp_path.display()))?;
@@ -1458,7 +1492,8 @@ mod tests {
fs::remove_file(root.join(old)).unwrap();
fs::write(root.join(new), "new").unwrap();
- enqueue_rescan_changes_for_prefixes(&root, "p1", &["raw/sources"], &default_watch_config()).unwrap();
+ enqueue_rescan_changes_for_prefixes(&root, "p1", &["raw/sources"], &default_watch_config())
+ .unwrap();
let queue = read_queue(&root).unwrap();
let by_path = queue
.tasks
@@ -1553,9 +1588,15 @@ mod tests {
let rules = SourceWatchRules::new(&config);
assert!(should_watch_rel("raw/sources/document.docx", &rules));
assert!(should_watch_rel("wiki/concepts/topic.md", &rules));
- assert!(!should_watch_rel(".llm-wiki/file-change-queue.json", &rules));
+ assert!(!should_watch_rel(
+ ".llm-wiki/file-change-queue.json",
+ &rules
+ ));
assert!(!should_watch_rel("raw/sources/~$Document.docx", &rules));
- assert!(!should_watch_rel("raw/sources/.~lock.Document.odt#", &rules));
+ assert!(!should_watch_rel(
+ "raw/sources/.~lock.Document.odt#",
+ &rules
+ ));
assert!(!should_watch_rel("raw/sources/Thumbs.db", &rules));
assert!(!should_watch_rel("raw/sources/desktop.ini", &rules));
assert!(!should_watch_rel("raw/sources/download.crdownload", &rules));
@@ -1577,7 +1618,10 @@ mod tests {
assert!(should_watch_rel("raw/sources/final.md", &rules));
assert!(!should_watch_rel("raw/sources/data.json", &rules));
assert!(!should_watch_rel("raw/sources/drafts/final.md", &rules));
- assert!(!should_watch_rel("raw/sources/subdir/drafts/final.md", &rules));
+ assert!(!should_watch_rel(
+ "raw/sources/subdir/drafts/final.md",
+ &rules
+ ));
assert!(!should_watch_rel("raw/sources/report.private.md", &rules));
assert!(should_watch_rel("wiki/index.md", &rules));
}
@@ -1594,7 +1638,10 @@ mod tests {
let config = normalize_source_watch_config(Some(config));
let rules = SourceWatchRules::new(&config);
- assert_eq!(relative_watch_path(&root, &root.join(rel), &rules, None), None);
+ assert_eq!(
+ relative_watch_path(&root, &root.join(rel), &rules, None),
+ None
+ );
let _ = fs::remove_dir_all(root);
}
diff --git a/src-tauri/src/commands/fs.rs b/src-tauri/src/commands/fs.rs
index 770e1865..d35aa1c6 100644
--- a/src-tauri/src/commands/fs.rs
+++ b/src-tauri/src/commands/fs.rs
@@ -4,7 +4,7 @@ use std::path::Path;
use std::thread;
use std::time::Duration;
-use calamine::{Reader, open_workbook_auto, Data};
+use calamine::{open_workbook_auto, Data, Reader};
use crate::commands::file_sync;
use crate::panic_guard::run_guarded;
@@ -16,8 +16,8 @@ const IMAGE_EXTS: &[&str] = &[
"png", "jpg", "jpeg", "gif", "webp", "bmp", "ico", "tiff", "tif", "avif", "heic", "heif", "svg",
];
const MEDIA_EXTS: &[&str] = &[
- "mp4", "webm", "mov", "avi", "mkv", "flv", "wmv", "m4v",
- "mp3", "wav", "ogg", "flac", "aac", "m4a", "wma",
+ "mp4", "webm", "mov", "avi", "mkv", "flv", "wmv", "m4v", "mp3", "wav", "ogg", "flac", "aac",
+ "m4a", "wma",
];
const LEGACY_DOC_EXTS: &[&str] = &["doc", "xls", "ppt", "pages", "numbers", "key", "epub"];
@@ -113,10 +113,7 @@ pub async fn preprocess_file(path: String) -> Result<String, String> {
fn cache_path_for(original: &Path) -> std::path::PathBuf {
let parent = original.parent().unwrap_or(Path::new("."));
let cache_dir = parent.join(".cache");
- let file_name = original
- .file_name()
- .unwrap_or_default()
- .to_string_lossy();
+ let file_name = original.file_name().unwrap_or_default().to_string_lossy();
cache_dir.join(format!("{}.txt", file_name))
}
@@ -137,8 +134,7 @@ fn write_cache(original: &Path, text: &str) -> Result<(), String> {
fs::create_dir_all(parent).ok();
}
crate::commands::file_sync::mark_app_write_path(&cache_path);
- fs::write(&cache_path, text)
- .map_err(|e| format!("Failed to write cache: {}", e))
+ fs::write(&cache_path, text).map_err(|e| format!("Failed to write cache: {}", e))
}
/// Global PDFium instance — the library prefers a single binding shared
@@ -177,8 +173,7 @@ pub(crate) fn lock_pdfium() -> std::sync::MutexGuard<'static, ()> {
/// once the AppHandle is available. Lets the pdfium resolver find the
/// bundled dylib without re-implementing Tauri's platform-specific
/// resource-dir logic.
-static RESOURCE_DIR_HINT: std::sync::OnceLock<std::path::PathBuf> =
- std::sync::OnceLock::new();
+static RESOURCE_DIR_HINT: std::sync::OnceLock<std::path::PathBuf> = std::sync::OnceLock::new();
/// Called from Tauri's setup() with the resolved resource directory.
/// No-op if already set.
@@ -266,7 +261,10 @@ fn pdfium_candidate_paths() -> Vec<String> {
push(&mut v, exe_dir.join("pdfium").join("pdfium.dll"));
push(&mut v, exe_dir.join("libpdfium.dll"));
push(&mut v, exe_dir.join("resources").join("pdfium.dll"));
- push(&mut v, exe_dir.join("resources").join("pdfium").join("pdfium.dll"));
+ push(
+ &mut v,
+ exe_dir.join("resources").join("pdfium").join("pdfium.dll"),
+ );
}
#[cfg(target_os = "linux")]
@@ -274,7 +272,13 @@ fn pdfium_candidate_paths() -> Vec<String> {
push(&mut v, exe_dir.join("libpdfium.so"));
push(&mut v, exe_dir.join("pdfium").join("libpdfium.so"));
push(&mut v, exe_dir.join("resources").join("libpdfium.so"));
- push(&mut v, exe_dir.join("resources").join("pdfium").join("libpdfium.so"));
+ push(
+ &mut v,
+ exe_dir
+ .join("resources")
+ .join("pdfium")
+ .join("libpdfium.so"),
+ );
push(&mut v, exe_dir.join("../lib/libpdfium.so"));
}
}
@@ -391,8 +395,7 @@ fn extract_office_text(path: &str, ext: &str) -> Result<String, String> {
}
// PPTX and ODF: use ZIP-based parsing
- let file = fs::File::open(path)
- .map_err(|e| format!("Failed to open '{}': {}", path, e))?;
+ let file = fs::File::open(path).map_err(|e| format!("Failed to open '{}': {}", path, e))?;
let mut archive = zip::ZipArchive::new(file)
.map_err(|e| format!("Failed to read ZIP archive '{}': {}", path, e))?;
@@ -460,7 +463,9 @@ fn extract_docx_with_library(path: &str) -> Result<String, String> {
}
let text = para_text.trim().to_string();
- if text.is_empty() { continue; }
+ if text.is_empty() {
+ continue;
+ }
if is_heading {
let prefix = "#".repeat(heading_level as usize);
@@ -509,7 +514,9 @@ fn extract_docx_with_library(path: &str) -> Result<String, String> {
result.push_str(" |\n");
if i == 0 {
result.push('|');
- for _ in 0..max_cols { result.push_str(" --- |"); }
+ for _ in 0..max_cols {
+ result.push_str(" --- |");
+ }
result.push('\n');
}
}
@@ -577,7 +584,9 @@ fn extract_docx_markdown(archive: &mut zip::ZipArchive<fs::File>) -> Result<Stri
let tag_start = i;
i += 1;
let is_closing = i < len && chars[i] == '/';
- if is_closing { i += 1; }
+ if is_closing {
+ i += 1;
+ }
let mut tag_name = String::new();
while i < len && chars[i] != '>' && chars[i] != ' ' && chars[i] != '/' {
@@ -591,7 +600,9 @@ fn extract_docx_markdown(archive: &mut zip::ZipArchive<fs::File>) -> Result<Stri
tag_content.push(chars[i]);
i += 1;
}
- if i < len { i += 1; } // skip >
+ if i < len {
+ i += 1;
+ } // skip >
match tag_name.as_str() {
// Paragraph start
@@ -634,16 +645,26 @@ fn extract_docx_markdown(archive: &mut zip::ZipArchive<fs::File>) -> Result<Stri
}
}
}
- if tag_content.contains("ListParagraph") || tag_content.contains("listParagraph") {
+ if tag_content.contains("ListParagraph")
+ || tag_content.contains("listParagraph")
+ {
in_list_item = true;
}
}
// Bold
- "w:b" if !is_closing && !tag_content.contains("w:val=\"0\"") && !tag_content.contains("w:val=\"false\"") => {
+ "w:b"
+ if !is_closing
+ && !tag_content.contains("w:val=\"0\"")
+ && !tag_content.contains("w:val=\"false\"") =>
+ {
is_bold = true;
}
// Italic
- "w:i" if !is_closing && !tag_content.contains("w:val=\"0\"") && !tag_content.contains("w:val=\"false\"") => {
+ "w:i"
+ if !is_closing
+ && !tag_content.contains("w:val=\"0\"")
+ && !tag_content.contains("w:val=\"false\"") =>
+ {
is_italic = true;
}
// Run end — apply formatting
@@ -732,8 +753,16 @@ fn extract_pptx_markdown(archive: &mut zip::ZipArchive<fs::File>) -> Result<Stri
// Sort by slide number
slide_names.sort_by(|a, b| {
- let num_a = a.trim_start_matches("ppt/slides/slide").trim_end_matches(".xml").parse::<u32>().unwrap_or(0);
- let num_b = b.trim_start_matches("ppt/slides/slide").trim_end_matches(".xml").parse::<u32>().unwrap_or(0);
+ let num_a = a
+ .trim_start_matches("ppt/slides/slide")
+ .trim_end_matches(".xml")
+ .parse::<u32>()
+ .unwrap_or(0);
+ let num_b = b
+ .trim_start_matches("ppt/slides/slide")
+ .trim_end_matches(".xml")
+ .parse::<u32>()
+ .unwrap_or(0);
num_a.cmp(&num_b)
});
@@ -802,7 +831,9 @@ fn extract_spreadsheet(path: &str) -> Result<String, String> {
for sheet_name in &sheet_names {
if let Ok(range) = workbook.worksheet_range(sheet_name) {
- if range.is_empty() { continue; }
+ if range.is_empty() {
+ continue;
+ }
if sheet_names.len() > 1 {
result.push_str(&format!("## {}\n\n", sheet_name));
@@ -812,8 +843,9 @@ fn extract_spreadsheet(path: &str) -> Result<String, String> {
let mut max_cols = 0;
for row in range.rows() {
- let cells: Vec<String> = row.iter().map(|cell| {
- match cell {
+ let cells: Vec<String> = row
+ .iter()
+ .map(|cell| match cell {
Data::Empty => String::new(),
Data::String(s) => s.clone(),
Data::Float(f) => {
@@ -829,14 +861,18 @@ fn extract_spreadsheet(path: &str) -> Result<String, String> {
Data::DateTimeIso(s) => s.clone(),
Data::DurationIso(s) => s.clone(),
Data::Error(e) => format!("ERR:{:?}", e),
- }
- }).collect();
- if cells.len() > max_cols { max_cols = cells.len(); }
+ })
+ .collect();
+ if cells.len() > max_cols {
+ max_cols = cells.len();
+ }
rows.push(cells);
}
// Skip empty sheets
- if rows.is_empty() || max_cols == 0 { continue; }
+ if rows.is_empty() || max_cols == 0 {
+ continue;
+ }
for (i, row) in rows.iter().enumerate() {
let mut padded = row.clone();
@@ -849,7 +885,9 @@ fn extract_spreadsheet(path: &str) -> Result<String, String> {
if i == 0 {
result.push('|');
- for _ in 0..max_cols { result.push_str(" --- |"); }
+ for _ in 0..max_cols {
+ result.push_str(" --- |");
+ }
result.push('\n');
}
}
@@ -866,8 +904,8 @@ fn extract_spreadsheet(path: &str) -> Result<String, String> {
/// Extract OpenDocument format text (basic).
fn extract_odf_text(archive: &mut zip::ZipArchive<fs::File>) -> Result<String, String> {
- let xml = read_zip_file(archive, "content.xml")
- .ok_or_else(|| "No content.xml found".to_string())?;
+ let xml =
+ read_zip_file(archive, "content.xml").ok_or_else(|| "No content.xml found".to_string())?;
let mut result = String::new();
let mut in_tag = false;
@@ -885,7 +923,11 @@ fn extract_odf_text(archive: &mut zip::ZipArchive<fs::File>) -> Result<String, S
}
let cleaned = decode_xml_entities(&result);
- let lines: Vec<&str> = cleaned.lines().map(|l| l.trim()).filter(|l| !l.is_empty()).collect();
+ let lines: Vec<&str> = cleaned
+ .lines()
+ .map(|l| l.trim())
+ .filter(|l| !l.is_empty())
+ .collect();
if lines.is_empty() {
Ok("[Could not extract text from this file]".to_string())
@@ -937,8 +979,9 @@ pub async fn write_file_atomic(path: String, contents: String) -> Result<(), Str
file_sync::mark_app_write_path(&tmp_path);
file_sync::mark_app_write_path(p);
- fs::write(&tmp_path, contents)
- .map_err(|e| format!("Failed to write temp file '{}': {}", tmp_path.display(), e))?;
+ fs::write(&tmp_path, contents).map_err(|e| {
+ format!("Failed to write temp file '{}': {}", tmp_path.display(), e)
+ })?;
fs::rename(&tmp_path, p).map_err(|e| {
let _ = fs::remove_file(&tmp_path);
@@ -1008,11 +1051,7 @@ fn build_tree(dir: &Path, depth: usize, max_depth: usize) -> Result<Vec<FileNode
let mut nodes = Vec::new();
for entry in entries {
let entry_path = entry.path();
- let name = entry
- .file_name()
- .to_str()
- .unwrap_or("")
- .to_string();
+ let name = entry.file_name().to_str().unwrap_or("").to_string();
// Always return forward-slash paths so the TS layer can compare
// and compose paths consistently across Windows and Unix. Windows
// APIs accept forward slashes, so normalizing here is safe and
@@ -1103,9 +1142,8 @@ pub async fn copy_directory(source: String, destination: String) -> Result<Vec<S
if path.is_dir() {
copy_recursive(&path, &dest_path, files)?;
} else {
- fs::copy(&path, &dest_path).map_err(|e| {
- format!("Failed to copy '{}': {}", path.display(), e)
- })?;
+ fs::copy(&path, &dest_path)
+ .map_err(|e| format!("Failed to copy '{}': {}", path.display(), e))?;
file_sync::mark_app_write_path(&dest_path);
files.push(dest_path.to_string_lossy().replace('\\', "/"));
}
@@ -1178,7 +1216,10 @@ fn is_windows_transient_delete_error(err: &std::io::Error) -> bool {
/// Find wiki pages that reference a given source file name.
/// Scans all .md files under wiki/ for the source filename in frontmatter or content.
#[tauri::command]
-pub async fn find_related_wiki_pages(project_path: String, source_name: String) -> Result<Vec<String>, String> {
+pub async fn find_related_wiki_pages(
+ project_path: String,
+ source_name: String,
+) -> Result<Vec<String>, String> {
tauri::async_runtime::spawn_blocking(move || {
run_guarded("find_related_wiki_pages", || {
let wiki_dir = Path::new(&project_path).join("wiki");
@@ -1195,7 +1236,11 @@ pub async fn find_related_wiki_pages(project_path: String, source_name: String)
.map_err(|e| format!("find_related_wiki_pages blocking task join error: {e}"))?
}
-fn collect_related_pages(dir: &Path, source_name: &str, results: &mut Vec<String>) -> Result<(), String> {
+fn collect_related_pages(
+ dir: &Path,
+ source_name: &str,
+ results: &mut Vec<String>,
+) -> Result<(), String> {
let entries = fs::read_dir(dir).map_err(|e| e.to_string())?;
// Get just the filename without path — use Path for cross-platform separator handling
@@ -1215,7 +1260,11 @@ fn collect_related_pages(dir: &Path, source_name: &str, results: &mut Vec<String
.rev()
.collect::<Vec<_>>()
.join(".");
- let file_stem_lower = if file_stem.is_empty() { file_name_lower.clone() } else { file_stem.to_lowercase() };
+ let file_stem_lower = if file_stem.is_empty() {
+ file_name_lower.clone()
+ } else {
+ file_stem.to_lowercase()
+ };
for entry in entries.flatten() {
let path = entry.path();
@@ -1238,11 +1287,9 @@ fn collect_related_pages(dir: &Path, source_name: &str, results: &mut Vec<String
// Match 2: source summary page (wiki/sources/{stem}.md)
// Use Path component iteration to avoid hardcoded separator assumptions
- let is_in_sources_dir = path
- .components()
- .any(|c| c.as_os_str() == "sources");
- let is_source_summary = is_in_sources_dir
- && fname.to_lowercase().starts_with(&file_stem_lower);
+ let is_in_sources_dir = path.components().any(|c| c.as_os_str() == "sources");
+ let is_source_summary =
+ is_in_sources_dir && fname.to_lowercase().starts_with(&file_stem_lower);
// Match 3: the page's *sources block* mentions the
// filename. Covers the multi-line YAML list form
@@ -1281,7 +1328,10 @@ fn collect_related_pages(dir: &Path, source_name: &str, results: &mut Vec<String
// indented; an un-indented line means
// we've left the sources block for
// another top-level field.
- if line.is_empty() || line.starts_with(' ') || line.starts_with('\t') {
+ if line.is_empty()
+ || line.starts_with(' ')
+ || line.starts_with('\t')
+ {
if line.contains(&file_name_lower) {
found = true;
break;
@@ -1346,8 +1396,7 @@ pub async fn read_file_as_base64(path: String) -> Result<FileBase64, String> {
use base64::{engine::general_purpose::STANDARD as B64, Engine as _};
tauri::async_runtime::spawn_blocking(move || {
run_guarded("read_file_as_base64", || {
- let bytes = fs::read(&path)
- .map_err(|e| format!("Failed to read '{}': {}", path, e))?;
+ let bytes = fs::read(&path).map_err(|e| format!("Failed to read '{}': {}", path, e))?;
let p = Path::new(&path);
let ext = p
.extension()
@@ -1504,7 +1553,10 @@ mod tests {
let path = tmp_pdf_with_bytes(bytes);
let result = read_file(path.clone()).await;
let _ = fs::remove_file(&path);
- eprintln!("[{name}] => {:?}", result.as_ref().map(|s| &s[..s.len().min(80)]));
+ eprintln!(
+ "[{name}] => {:?}",
+ result.as_ref().map(|s| &s[..s.len().min(80)])
+ );
}
}
@@ -1544,7 +1596,8 @@ mod tests {
let p = entry.path();
if p.is_dir() {
walk(&p, out);
- } else if p.extension()
+ } else if p
+ .extension()
.and_then(|e| e.to_str())
.map(|e| e.eq_ignore_ascii_case("pdf"))
.unwrap_or(false)
@@ -1557,7 +1610,11 @@ mod tests {
walk(root, &mut pdfs);
pdfs.sort();
- eprintln!("\n[pdf_probe] found {} PDFs under {}\n", pdfs.len(), root.display());
+ eprintln!(
+ "\n[pdf_probe] found {} PDFs under {}\n",
+ pdfs.len(),
+ root.display()
+ );
let mut ok = 0usize;
let mut err = 0usize;
@@ -1572,11 +1629,23 @@ mod tests {
match result {
Ok(Ok(text)) => {
ok += 1;
- eprintln!("[{:>3}/{}] OK ({:>7} chars) {}", idx + 1, pdfs.len(), text.len(), display);
+ eprintln!(
+ "[{:>3}/{}] OK ({:>7} chars) {}",
+ idx + 1,
+ pdfs.len(),
+ text.len(),
+ display
+ );
}
Ok(Err(e)) => {
err += 1;
- eprintln!("[{:>3}/{}] ERR {} → {}", idx + 1, pdfs.len(), display, e);
+ eprintln!(
+ "[{:>3}/{}] ERR {} → {}",
+ idx + 1,
+ pdfs.len(),
+ display,
+ e
+ );
}
Err(payload) => {
panicked += 1;
@@ -1587,12 +1656,24 @@ mod tests {
} else {
"(non-string panic)".to_string()
};
- eprintln!("[{:>3}/{}] PANIC {} → {}", idx + 1, pdfs.len(), display, msg);
+ eprintln!(
+ "[{:>3}/{}] PANIC {} → {}",
+ idx + 1,
+ pdfs.len(),
+ display,
+ msg
+ );
}
}
}
- eprintln!("\n[pdf_probe] summary: {} OK / {} ERR / {} PANIC (total {})", ok, err, panicked, pdfs.len());
+ eprintln!(
+ "\n[pdf_probe] summary: {} OK / {} ERR / {} PANIC (total {})",
+ ok,
+ err,
+ panicked,
+ pdfs.len()
+ );
}
// ── collect_related_pages: regression coverage for the three match ─────
@@ -1622,7 +1703,9 @@ mod tests {
fs::create_dir_all(&dir).unwrap();
for (rel, body) in files {
let p = dir.join(rel);
- if let Some(parent) = p.parent() { fs::create_dir_all(parent).unwrap(); }
+ if let Some(parent) = p.parent() {
+ fs::create_dir_all(parent).unwrap();
+ }
fs::write(&p, body).unwrap();
}
dir
@@ -1679,10 +1762,7 @@ mod tests {
// deleted source's stem counts as the source-summary page —
// kept linked even if its sources field happens to be missing.
let wiki = make_wiki(&[
- (
- "sources/test.md",
- "---\ntitle: Test Summary\n---\nbody\n",
- ),
+ ("sources/test.md", "---\ntitle: Test Summary\n---\nbody\n"),
(
"concepts/unrelated.md",
"---\ntitle: Unrelated\nsources: [\"other.md\"]\n---\nbody\n",
@@ -1873,7 +1953,11 @@ mod tests {
assert_eq!(copied.len(), 2, "expected two files, got: {:?}", copied);
// Deep file made it across with full nesting preserved.
let leaf_dest = dest.join("a/b/c/d/e/leaf.txt");
- assert!(leaf_dest.exists(), "deep leaf.txt missing at {:?}", leaf_dest);
+ assert!(
+ leaf_dest.exists(),
+ "deep leaf.txt missing at {:?}",
+ leaf_dest
+ );
assert_eq!(std::fs::read(&leaf_dest).unwrap(), b"deep content");
// Top-level file too.
assert!(dest.join("top.md").exists());
@@ -1942,7 +2026,13 @@ mod tests {
// forward-slashed and end with the expected filename).
let names: Vec<String> = copied
.iter()
- .map(|p| Path::new(p).file_name().unwrap().to_string_lossy().to_string())
+ .map(|p| {
+ Path::new(p)
+ .file_name()
+ .unwrap()
+ .to_string_lossy()
+ .to_string()
+ })
.collect();
assert!(names.contains(&"report.pdf".to_string()));
assert!(names.contains(&"notes.md".to_string()));
diff --git a/src-tauri/src/commands/mod.rs b/src-tauri/src/commands/mod.rs
index d0a5362b..88f45203 100644
--- a/src-tauri/src/commands/mod.rs
+++ b/src-tauri/src/commands/mod.rs
@@ -4,4 +4,5 @@ pub mod extract_images;
pub mod file_sync;
pub mod fs;
pub mod project;
+pub mod search;
pub mod vectorstore;
diff --git a/src-tauri/src/commands/project.rs b/src-tauri/src/commands/project.rs
index 694dbe56..643ed756 100644
--- a/src-tauri/src/commands/project.rs
+++ b/src-tauri/src/commands/project.rs
@@ -229,7 +229,10 @@ related: []
"outgoing-link": true,
"starred": true
}"#;
- write_file_inner(root.join(".obsidian/core-plugins.json"), obsidian_core_plugins)?;
+ write_file_inner(
+ root.join(".obsidian/core-plugins.json"),
+ obsidian_core_plugins,
+ )?;
Ok(WikiProject {
name,
@@ -312,8 +315,13 @@ fn validate_wiki_project_root(root: &Path) -> Result<(), String> {
fn write_file_inner(path: std::path::PathBuf, contents: &str) -> Result<(), String> {
if let Some(parent) = path.parent() {
- fs::create_dir_all(parent)
- .map_err(|e| format!("Failed to create parent dirs for '{}': {}", path.display(), e))?;
+ fs::create_dir_all(parent).map_err(|e| {
+ format!(
+ "Failed to create parent dirs for '{}': {}",
+ path.display(),
+ e
+ )
+ })?;
}
fs::write(&path, contents)
.map_err(|e| format!("Failed to write file '{}': {}", path.display(), e))
diff --git a/src-tauri/src/commands/search.rs b/src-tauri/src/commands/search.rs
new file mode 100644
index 00000000..23b12bcd
--- /dev/null
+++ b/src-tauri/src/commands/search.rs
@@ -0,0 +1,1119 @@
+use std::collections::{BTreeMap, BTreeSet};
+use std::fs;
+use std::path::Path;
+
+use serde::{Deserialize, Serialize};
+use serde_json::{json, Value};
+use walkdir::WalkDir;
+
+use crate::commands::vectorstore;
+use crate::panic_guard::run_guarded_async;
+
+const DEFAULT_RESULTS: usize = 20;
+const MAX_RESULTS: usize = 50;
+const RRF_K: f64 = 60.0;
+const FILENAME_EXACT_BONUS: f64 = 200.0;
+const PHRASE_IN_TITLE_BONUS: f64 = 50.0;
+const PHRASE_IN_CONTENT_PER_OCC: f64 = 20.0;
+const MAX_PHRASE_OCC_COUNTED: usize = 10;
+const TITLE_TOKEN_WEIGHT: f64 = 5.0;
+const CONTENT_TOKEN_WEIGHT: f64 = 1.0;
+const SNIPPET_CONTEXT: usize = 80;
+const SEARCH_EMBEDDING_TIMEOUT_SECS: u64 = 8;
+const MAX_SEARCH_FILES: usize = 10_000;
+
+#[derive(Debug, Clone, Serialize, Deserialize)]
+#[serde(rename_all = "camelCase")]
+pub struct SearchImageRef {
+ pub url: String,
+ pub alt: String,
+}
+
+#[derive(Debug, Clone, Serialize, Deserialize)]
+#[serde(rename_all = "camelCase")]
+pub struct ProjectSearchResult {
+ pub path: String,
+ pub title: String,
+ pub snippet: String,
+ pub title_match: bool,
+ pub score: f64,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub vector_score: Option<f32>,
+ pub images: Vec<SearchImageRef>,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub content: Option<String>,
+}
+
+#[derive(Debug, Clone, Serialize, Deserialize)]
+#[serde(rename_all = "camelCase")]
+pub struct ProjectSearchResponse {
+ pub mode: String,
+ pub results: Vec<ProjectSearchResult>,
+ pub token_hits: usize,
+ pub vector_hits: usize,
+}
+
+#[derive(Debug, Clone, Serialize, Deserialize)]
+#[serde(rename_all = "camelCase")]
+pub struct SearchEmbeddingConfig {
+ pub enabled: bool,
+ pub endpoint: String,
+ pub api_key: String,
+ pub model: String,
+ pub output_dimensionality: Option<u32>,
+}
+
+#[tauri::command]
+pub async fn search_project(
+ project_path: String,
+ query: String,
+ top_k: Option<usize>,
+ include_content: Option<bool>,
+ query_embedding: Option<Vec<f32>>,
+ embedding_config: Option<SearchEmbeddingConfig>,
+) -> Result<ProjectSearchResponse, String> {
+ run_guarded_async("search_project", async move {
+ let query_embedding =
+ resolve_query_embedding(&query, query_embedding, embedding_config).await?;
+ search_project_inner(
+ project_path,
+ query,
+ top_k.unwrap_or(DEFAULT_RESULTS),
+ include_content.unwrap_or(false),
+ query_embedding,
+ )
+ .await
+ })
+ .await
+}
+
+pub async fn resolve_query_embedding(
+ query: &str,
+ explicit_embedding: Option<Vec<f32>>,
+ embedding_config: Option<SearchEmbeddingConfig>,
+) -> Result<Option<Vec<f32>>, String> {
+ if let Some(embedding) = explicit_embedding {
+ return validate_query_embedding(embedding).map(Some);
+ }
+ let Some(cfg) = embedding_config else {
+ return Ok(None);
+ };
+ if !cfg.enabled || cfg.endpoint.trim().is_empty() || cfg.model.trim().is_empty() {
+ return Ok(None);
+ }
+ match fetch_embedding(query, &cfg).await {
+ Ok(embedding) => validate_query_embedding(embedding).map(Some),
+ Err(err) => {
+ eprintln!("[Search] embedding disabled for this request: {err}");
+ Ok(None)
+ }
+ }
+}
+
+fn validate_query_embedding(embedding: Vec<f32>) -> Result<Vec<f32>, String> {
+ if embedding.is_empty() {
+ return Err("queryEmbedding must not be empty".to_string());
+ }
+ if embedding.iter().any(|v| !v.is_finite()) {
+ return Err("queryEmbedding must contain only finite numbers".to_string());
+ }
+ Ok(embedding)
+}
+
+pub async fn search_project_inner(
+ project_path: String,
+ query: String,
+ top_k: usize,
+ include_content: bool,
+ query_embedding: Option<Vec<f32>>,
+) -> Result<ProjectSearchResponse, String> {
+ if query.trim().is_empty() {
+ return Err("query is required".to_string());
+ }
+ let limit = top_k.clamp(1, MAX_RESULTS);
+ let tokens = tokenize_query(&query);
+ let effective_tokens = if tokens.is_empty() {
+ vec![query.trim().to_lowercase()]
+ } else {
+ tokens
+ };
+ let query_phrase = trim_query_punctuation(&query.to_lowercase());
+ let mut results = Vec::new();
+ let mut page_paths_by_stem = BTreeMap::new();
+
+ let wiki_root = Path::new(&project_path).join("wiki");
+ if wiki_root.exists() {
+ let mut searched_files = 0usize;
+ for entry in WalkDir::new(&wiki_root).into_iter().filter_map(Result::ok) {
+ if !entry.file_type().is_file()
+ || entry.path().extension().and_then(|s| s.to_str()) != Some("md")
+ {
+ continue;
+ }
+ searched_files += 1;
+ if searched_files > MAX_SEARCH_FILES {
+ eprintln!(
+ "[Search] stopped scanning wiki after {MAX_SEARCH_FILES} markdown files in {project_path}"
+ );
+ break;
+ }
+ let content = match fs::read_to_string(entry.path()) {
+ Ok(content) => content,
+ Err(_) => continue,
+ };
+ if let Some(stem) = entry.path().file_stem().and_then(|s| s.to_str()) {
+ let previous = page_paths_by_stem.insert(
+ stem.to_string(),
+ relative_to_project(&project_path, entry.path()),
+ );
+ if let Some(previous) = previous {
+ eprintln!(
+ "[Search] duplicate wiki page stem '{stem}': '{previous}' and '{}' share one vector page_id",
+ relative_to_project(&project_path, entry.path())
+ );
+ }
+ }
+ if let Some(hit) = score_file(
+ &project_path,
+ entry.path(),
+ &content,
+ &effective_tokens,
+ &query_phrase,
+ &query,
+ include_content,
+ ) {
+ results.push(hit);
+ }
+ }
+ }
+
+ let mut token_sorted = (0..results.len()).collect::<Vec<_>>();
+ token_sorted.sort_by(|a, b| {
+ results[*b]
+ .score
+ .partial_cmp(&results[*a].score)
+ .unwrap_or(std::cmp::Ordering::Equal)
+ .then_with(|| results[*a].path.cmp(&results[*b].path))
+ });
+ let mut token_rank = BTreeMap::new();
+ for (idx, result_idx) in token_sorted.iter().enumerate() {
+ let result = &results[*result_idx];
+ token_rank.insert(normalize_path(&result.path), idx + 1);
+ }
+
+ let mut vector_rank: BTreeMap<String, usize> = BTreeMap::new();
+ let mut vector_score: BTreeMap<String, f32> = BTreeMap::new();
+ let mut vector_hits = 0;
+ if let Some(embedding) = query_embedding {
+ if !embedding.is_empty() {
+ match search_by_embedding(&project_path, embedding, limit.max(10)).await {
+ Ok(vector_results) => {
+ vector_hits = vector_results.len();
+ for (idx, vr) in vector_results.iter().enumerate() {
+ vector_rank.insert(vr.id.clone(), idx + 1);
+ vector_score.insert(vr.id.clone(), vr.score);
+ }
+ materialize_vector_only_results(
+ &vector_results,
+ &page_paths_by_stem,
+ &project_path,
+ &mut results,
+ include_content,
+ );
+ }
+ Err(err) => {
+ eprintln!(
+ "[Search] vector search failed; falling back to keyword results: {err}"
+ );
+ }
+ }
+ }
+ }
+
+ if vector_hits == 0 {
+ results.sort_by(|a, b| {
+ b.score
+ .partial_cmp(&a.score)
+ .unwrap_or(std::cmp::Ordering::Equal)
+ .then_with(|| a.path.cmp(&b.path))
+ });
+ results.truncate(limit);
+ return Ok(ProjectSearchResponse {
+ mode: "keyword".to_string(),
+ token_hits: token_rank.len(),
+ vector_hits,
+ results,
+ });
+ }
+
+ apply_rrf_scores(&mut results, &token_rank, &vector_rank, &vector_score);
+
+ results.sort_by(|a, b| {
+ b.score
+ .partial_cmp(&a.score)
+ .unwrap_or(std::cmp::Ordering::Equal)
+ .then_with(|| a.path.cmp(&b.path))
+ });
+ results.truncate(limit);
+
+ Ok(ProjectSearchResponse {
+ mode: search_mode(token_rank.is_empty(), vector_hits).to_string(),
+ token_hits: token_rank.len(),
+ vector_hits,
+ results,
+ })
+}
+
+fn apply_rrf_scores(
+ results: &mut [ProjectSearchResult],
+ token_rank: &BTreeMap<String, usize>,
+ vector_rank: &BTreeMap<String, usize>,
+ vector_score: &BTreeMap<String, f32>,
+) {
+ for result in results {
+ let token = token_rank.get(&normalize_path(&result.path)).copied();
+ let vector = vector_rank.get(&file_stem(&result.path)).copied();
+ let mut rrf = 0.0;
+ if let Some(rank) = token {
+ rrf += 1.0 / (RRF_K + rank as f64);
+ }
+ if let Some(rank) = vector {
+ rrf += 1.0 / (RRF_K + rank as f64);
+ }
+ if let Some(score) = vector_score.get(&file_stem(&result.path)).copied() {
+ result.vector_score = Some(score);
+ }
+ result.score = rrf;
+ }
+}
+
+fn search_mode(token_rank_empty: bool, vector_hits: usize) -> &'static str {
+ if vector_hits == 0 {
+ "keyword"
+ } else if token_rank_empty {
+ "vector"
+ } else {
+ "hybrid"
+ }
+}
+
+#[derive(Debug, Clone)]
+struct PageVectorResult {
+ id: String,
+ score: f32,
+ chunk_text: String,
+ heading_path: String,
+}
+
+async fn search_by_embedding(
+ project_path: &str,
+ query_embedding: Vec<f32>,
+ top_k: usize,
+) -> Result<Vec<PageVectorResult>, String> {
+ let raw_chunks = vectorstore::vector_search_chunks(
+ project_path.to_string(),
+ query_embedding,
+ (top_k * 3).max(30),
+ )
+ .await?;
+ if raw_chunks.is_empty() {
+ return Ok(vec![]);
+ }
+
+ let mut by_page: BTreeMap<String, Vec<vectorstore::ChunkSearchResult>> = BTreeMap::new();
+ for chunk in raw_chunks {
+ by_page
+ .entry(chunk.page_id.clone())
+ .or_default()
+ .push(chunk);
+ }
+
+ let mut ranked = Vec::new();
+ for (id, mut chunks) in by_page {
+ chunks.sort_by(|a, b| {
+ b.score
+ .partial_cmp(&a.score)
+ .unwrap_or(std::cmp::Ordering::Equal)
+ .then_with(|| a.chunk_index.cmp(&b.chunk_index))
+ });
+ let top_chunk = chunks[0].clone();
+ let top = top_chunk.score;
+ let tail: f32 = chunks.iter().skip(1).map(|chunk| chunk.score).sum();
+ let blended = top + (tail * 0.3).min((1.0 - top).max(0.0));
+ ranked.push(PageVectorResult {
+ id,
+ score: blended,
+ chunk_text: top_chunk.chunk_text,
+ heading_path: top_chunk.heading_path,
+ });
+ }
+ ranked.sort_by(|a, b| {
+ b.score
+ .partial_cmp(&a.score)
+ .unwrap_or(std::cmp::Ordering::Equal)
+ .then_with(|| a.id.cmp(&b.id))
+ });
+ ranked.truncate(top_k);
+ Ok(ranked)
+}
+
+fn materialize_vector_only_results(
+ vector_results: &[PageVectorResult],
+ page_paths_by_stem: &BTreeMap<String, String>,
+ project_path: &str,
+ results: &mut Vec<ProjectSearchResult>,
+ include_content: bool,
+) {
+ let mut known: BTreeSet<String> = results.iter().map(|r| file_stem(&r.path)).collect();
+ for vr in vector_results {
+ if known.contains(&vr.id) {
+ continue;
+ }
+ if let Some(rel) = page_paths_by_stem.get(&vr.id) {
+ let path = Path::new(project_path).join(rel);
+ let Ok(content) = fs::read_to_string(&path) else {
+ continue;
+ };
+ let file_name = Path::new(&rel)
+ .file_name()
+ .and_then(|s| s.to_str())
+ .unwrap_or("");
+ let title = extract_title(&content, file_name);
+ let snippet = build_vector_snippet(vr);
+ results.push(ProjectSearchResult {
+ path: rel.clone(),
+ title,
+ snippet,
+ title_match: false,
+ score: 0.0,
+ vector_score: Some(vr.score),
+ images: extract_image_refs(&content),
+ content: include_content.then_some(content),
+ });
+ known.insert(vr.id.clone());
+ }
+ }
+}
+
+fn build_vector_snippet(result: &PageVectorResult) -> String {
+ let mut text = result.chunk_text.trim().replace('\n', " ");
+ if text.is_empty() {
+ return String::new();
+ }
+ if text.chars().count() > SNIPPET_CONTEXT * 2 {
+ text = text.chars().take(SNIPPET_CONTEXT * 2).collect::<String>();
+ text.push_str("...");
+ }
+ let heading = result.heading_path.trim();
+ if heading.is_empty() {
+ text
+ } else {
+ format!("{heading}: {text}")
+ }
+}
+
+fn score_file(
+ project_path: &str,
+ path: &Path,
+ content: &str,
+ tokens: &[String],
+ query_phrase: &str,
+ query: &str,
+ include_content: bool,
+) -> Option<ProjectSearchResult> {
+ let file_name = path.file_name().and_then(|s| s.to_str()).unwrap_or("");
+ let title = extract_title(content, file_name);
+ let title_text = format!("{title} {file_name}");
+ let title_lower = title_text.to_lowercase();
+ let content_lower = content.to_lowercase();
+ let stem = file_name.trim_end_matches(".md").to_lowercase();
+
+ let filename_exact = !query_phrase.is_empty() && stem == query_phrase;
+ let title_has_phrase = !query_phrase.is_empty() && title_lower.contains(query_phrase);
+ let content_phrase_occ =
+ count_occurrences(&content_lower, query_phrase).min(MAX_PHRASE_OCC_COUNTED);
+ let title_token_score = token_match_score(&title_text, tokens);
+ let content_token_score = token_match_score(content, tokens);
+
+ if !filename_exact
+ && !title_has_phrase
+ && content_phrase_occ == 0
+ && title_token_score == 0
+ && content_token_score == 0
+ {
+ return None;
+ }
+
+ let score = (if filename_exact {
+ FILENAME_EXACT_BONUS
+ } else {
+ 0.0
+ }) + (if title_has_phrase {
+ PHRASE_IN_TITLE_BONUS
+ } else {
+ 0.0
+ }) + content_phrase_occ as f64 * PHRASE_IN_CONTENT_PER_OCC
+ + title_token_score as f64 * TITLE_TOKEN_WEIGHT
+ + content_token_score as f64 * CONTENT_TOKEN_WEIGHT;
+
+ let snippet_anchor = if content_phrase_occ > 0 {
+ query_phrase.to_string()
+ } else {
+ tokens
+ .iter()
+ .find(|token| content_lower.contains(token.as_str()))
+ .cloned()
+ .unwrap_or_else(|| query.to_string())
+ };
+
+ Some(ProjectSearchResult {
+ path: relative_to_project(project_path, path),
+ title,
+ snippet: build_snippet(content, &snippet_anchor),
+ title_match: title_token_score > 0 || title_has_phrase,
+ score,
+ vector_score: None,
+ images: extract_image_refs(content),
+ content: include_content.then_some(content.to_string()),
+ })
+}
+
+pub fn tokenize_query(query: &str) -> Vec<String> {
+ let raw = query
+ .to_lowercase()
+ .split(is_query_separator)
+ .filter(|token| token.chars().count() > 1)
+ .filter(|token| !is_stop_word(token))
+ .map(ToOwned::to_owned)
+ .collect::<Vec<_>>();
+
+ let mut out = Vec::new();
+ for token in raw {
+ let chars = token.chars().collect::<Vec<_>>();
+ let has_cjk = chars.iter().any(|c| ('\u{3400}'..='\u{9fff}').contains(c));
+ if has_cjk && chars.len() > 2 {
+ for pair in chars.windows(2) {
+ out.push(pair.iter().collect());
+ }
+ for ch in &chars {
+ let s = ch.to_string();
+ if !is_stop_word(&s) {
+ out.push(s);
+ }
+ }
+ out.push(token);
+ } else {
+ out.push(token);
+ }
+ }
+ out.into_iter()
+ .collect::<BTreeSet<_>>()
+ .into_iter()
+ .collect()
+}
+
+fn is_query_separator(c: char) -> bool {
+ c.is_whitespace()
+ || c.is_ascii_punctuation()
+ || matches!(
+ c,
+ ',' | '。'
+ | '!'
+ | '?'
+ | '、'
+ | ';'
+ | ':'
+ | '“'
+ | '”'
+ | '‘'
+ | '’'
+ | '('
+ | ')'
+ | '·'
+ | '~'
+ | '…'
+ )
+}
+
+fn is_stop_word(token: &str) -> bool {
+ matches!(
+ token,
+ "的" | "是"
+ | "了"
+ | "什么"
+ | "在"
+ | "有"
+ | "和"
+ | "与"
+ | "对"
+ | "从"
+ | "the"
+ | "is"
+ | "a"
+ | "an"
+ | "what"
+ | "how"
+ | "are"
+ | "was"
+ | "were"
+ | "do"
+ | "does"
+ | "did"
+ | "be"
+ | "been"
+ | "being"
+ | "have"
+ | "has"
+ | "had"
+ | "it"
+ | "its"
+ | "in"
+ | "on"
+ | "at"
+ | "to"
+ | "for"
+ | "of"
+ | "with"
+ | "by"
+ | "this"
+ | "that"
+ | "these"
+ | "those"
+ )
+}
+
+fn trim_query_punctuation(value: &str) -> String {
+ value.trim_matches(is_query_separator).to_string()
+}
+
+fn token_match_score(text: &str, tokens: &[String]) -> usize {
+ let lower = text.to_lowercase();
+ tokens
+ .iter()
+ .filter(|token| lower.contains(token.as_str()))
+ .count()
+}
+
+fn count_occurrences(haystack: &str, needle: &str) -> usize {
+ if needle.is_empty() {
+ return 0;
+ }
+ haystack.match_indices(needle).count()
+}
+
+pub fn extract_title(content: &str, file_name: &str) -> String {
+ let has_frontmatter = content.starts_with("---");
+ let mut in_frontmatter = has_frontmatter;
+ let mut frontmatter_closed = false;
+ for line in content.lines().skip(if has_frontmatter { 1 } else { 0 }) {
+ let trimmed = line.trim();
+ if in_frontmatter && trimmed == "---" {
+ in_frontmatter = false;
+ frontmatter_closed = true;
+ continue;
+ }
+ if in_frontmatter && trimmed.starts_with("title:") {
+ return trimmed
+ .trim_start_matches("title:")
+ .trim()
+ .trim_matches('"')
+ .trim_matches('\'')
+ .to_string();
+ }
+ if has_frontmatter && !frontmatter_closed {
+ continue;
+ }
+ if let Some(title) = trimmed.strip_prefix("# ") {
+ return title.trim().to_string();
+ }
+ }
+ file_name.trim_end_matches(".md").replace('-', " ")
+}
+
+pub fn extract_image_refs(content: &str) -> Vec<SearchImageRef> {
+ let mut out = Vec::new();
+ let mut seen = BTreeSet::new();
+ let mut rest = content;
+ while let Some(start) = rest.find("![") {
+ rest = &rest[start + 2..];
+ let Some(alt_end) = rest.find("](") else {
+ break;
+ };
+ let alt = &rest[..alt_end];
+ rest = &rest[alt_end + 2..];
+ let Some(url_end) = rest.find(')') else {
+ break;
+ };
+ let url = &rest[..url_end];
+ if !url.trim().is_empty()
+ && !url.contains(char::is_whitespace)
+ && seen.insert(url.to_string())
+ {
+ out.push(SearchImageRef {
+ url: url.to_string(),
+ alt: alt.to_string(),
+ });
+ }
+ rest = &rest[url_end + 1..];
+ }
+ out
+}
+
+async fn fetch_embedding(text: &str, cfg: &SearchEmbeddingConfig) -> Result<Vec<f32>, String> {
+ let is_google = is_google_embedding_config(cfg);
+ let endpoint = if is_google {
+ google_embedding_endpoint(cfg)
+ } else {
+ cfg.endpoint.trim().to_string()
+ };
+ let mut req = reqwest::Client::builder()
+ .timeout(std::time::Duration::from_secs(
+ SEARCH_EMBEDDING_TIMEOUT_SECS,
+ ))
+ .build()
+ .map_err(|e| format!("Embedding HTTP client error: {e}"))?
+ .post(endpoint)
+ .header("Content-Type", "application/json");
+ if !cfg.api_key.trim().is_empty() {
+ if is_google {
+ req = req.header("x-goog-api-key", cfg.api_key.trim());
+ } else {
+ req = req.bearer_auth(cfg.api_key.trim());
+ }
+ }
+ let body = if is_google {
+ google_embedding_body(&cfg.model, text, cfg.output_dimensionality)
+ } else {
+ json!({ "model": cfg.model, "input": text })
+ };
+ let resp = req
+ .json(&body)
+ .send()
+ .await
+ .map_err(|e| format!("Embedding request failed: {e}"))?;
+ let status = resp.status();
+ let data: Value = resp
+ .json()
+ .await
+ .map_err(|e| format!("Embedding response parse failed: {e}"))?;
+ if !status.is_success() {
+ return Err(format!(
+ "Embedding API HTTP {status}: {}",
+ data.to_string().chars().take(200).collect::<String>()
+ ));
+ }
+ let values = if is_google {
+ data.get("embedding")
+ .and_then(|v| v.get("values"))
+ .and_then(Value::as_array)
+ } else {
+ data.get("data")
+ .and_then(Value::as_array)
+ .and_then(|arr| arr.first())
+ .and_then(|v| v.get("embedding"))
+ .and_then(Value::as_array)
+ }
+ .ok_or_else(|| "Embedding response missing vector".to_string())?;
+ let mut out = Vec::with_capacity(values.len());
+ for value in values {
+ let n = value
+ .as_f64()
+ .ok_or_else(|| "Embedding response contains non-number values".to_string())?;
+ if !n.is_finite() {
+ return Err("Embedding response contains non-finite values".to_string());
+ }
+ out.push(n as f32);
+ }
+ if out.is_empty() {
+ return Err("Embedding response vector is empty".to_string());
+ }
+ Ok(out)
+}
+
+fn is_google_embedding_config(cfg: &SearchEmbeddingConfig) -> bool {
+ let endpoint = cfg.endpoint.to_lowercase();
+ endpoint.contains("generativelanguage.googleapis.com") || endpoint.contains(":embedcontent")
+}
+
+fn google_embedding_endpoint(cfg: &SearchEmbeddingConfig) -> String {
+ let raw = strip_google_api_key_query(cfg.endpoint.trim())
+ .trim_end_matches('/')
+ .to_string();
+ if raw.to_lowercase().contains(":batchembedcontents") {
+ return raw
+ .replace(":batchEmbedContents", ":embedContent")
+ .replace(":batchembedcontents", ":embedContent");
+ }
+ if raw.to_lowercase().contains(":embedcontent") {
+ return raw;
+ }
+ let model = cfg.model.trim().trim_start_matches("models/");
+ if raw.to_lowercase().contains("/models/") {
+ format!("{raw}:embedContent")
+ } else {
+ format!("{raw}/models/{model}:embedContent")
+ }
+}
+
+fn strip_google_api_key_query(endpoint: &str) -> String {
+ if !endpoint.contains('?') {
+ return endpoint.to_string();
+ }
+ match reqwest::Url::parse(endpoint) {
+ Ok(mut url) => {
+ let kept = url
+ .query_pairs()
+ .filter(|(key, _)| !key.eq_ignore_ascii_case("key"))
+ .map(|(key, value)| (key.into_owned(), value.into_owned()))
+ .collect::<Vec<_>>();
+ url.query_pairs_mut().clear().extend_pairs(kept);
+ url.to_string().trim_end_matches('?').to_string()
+ }
+ Err(_) => endpoint
+ .split_once('?')
+ .map(|(base, query)| {
+ let kept = query
+ .split('&')
+ .filter(|pair| {
+ let key = pair.split_once('=').map(|(k, _)| k).unwrap_or(*pair);
+ !key.eq_ignore_ascii_case("key")
+ })
+ .collect::<Vec<_>>();
+ if kept.is_empty() {
+ base.to_string()
+ } else {
+ format!("{base}?{}", kept.join("&"))
+ }
+ })
+ .unwrap_or_else(|| endpoint.to_string()),
+ }
+}
+
+fn google_embedding_body(model: &str, text: &str, output_dimensionality: Option<u32>) -> Value {
+ let model_path = if model.trim().starts_with("models/") {
+ model.trim().to_string()
+ } else {
+ format!("models/{}", model.trim())
+ };
+ let mut body = json!({
+ "model": model_path,
+ "content": { "parts": [{ "text": text }] },
+ });
+ if let Some(dim) = output_dimensionality.filter(|dim| *dim > 0) {
+ body["output_dimensionality"] = json!(dim);
+ }
+ body
+}
+
+pub fn build_snippet(content: &str, query: &str) -> String {
+ let lower = content.to_lowercase();
+ let q = query.to_lowercase();
+ let idx = lower.find(&q).unwrap_or(0);
+ let char_positions: Vec<usize> = content.char_indices().map(|(idx, _)| idx).collect();
+ if char_positions.is_empty() {
+ return String::new();
+ }
+ let match_char = char_positions
+ .iter()
+ .position(|byte| *byte >= idx)
+ .unwrap_or(char_positions.len().saturating_sub(1));
+ let query_chars = query.chars().count().max(1);
+ let start_char = match_char.saturating_sub(SNIPPET_CONTEXT);
+ let end_char = (match_char + query_chars + SNIPPET_CONTEXT).min(char_positions.len());
+ let start = char_positions[start_char];
+ let end = if end_char < char_positions.len() {
+ char_positions[end_char]
+ } else {
+ content.len()
+ };
+ let mut snippet = content[start..end].replace('\n', " ");
+ if start > 0 {
+ snippet = format!("...{snippet}");
+ }
+ if end < content.len() {
+ snippet.push_str("...");
+ }
+ snippet
+}
+
+fn normalize_path(path: &str) -> String {
+ path.replace('\\', "/")
+}
+
+fn relative_to_project(project_path: &str, path: &Path) -> String {
+ let root = Path::new(project_path);
+ path.strip_prefix(root)
+ .map(|p| p.to_string_lossy().replace('\\', "/"))
+ .unwrap_or_else(|_| path.to_string_lossy().replace('\\', "/"))
+}
+
+fn file_stem(path: &str) -> String {
+ Path::new(path)
+ .file_stem()
+ .and_then(|s| s.to_str())
+ .unwrap_or("")
+ .to_string()
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use std::path::PathBuf;
+ use std::sync::atomic::{AtomicU64, Ordering};
+
+ fn tmp_project() -> PathBuf {
+ static COUNTER: AtomicU64 = AtomicU64::new(0);
+ let id = COUNTER.fetch_add(1, Ordering::SeqCst);
+ let path = std::env::temp_dir().join(format!("llm-wiki-search-test-{id}"));
+ let _ = fs::remove_dir_all(&path);
+ fs::create_dir_all(path.join("wiki/concepts")).unwrap();
+ path
+ }
+
+ fn write_page(root: &Path, rel: &str, content: &str) {
+ let path = root.join(rel);
+ fs::create_dir_all(path.parent().unwrap()).unwrap();
+ fs::write(path, content).unwrap();
+ }
+
+ fn result(path: &str) -> ProjectSearchResult {
+ ProjectSearchResult {
+ path: path.to_string(),
+ title: path.to_string(),
+ snippet: String::new(),
+ title_match: false,
+ score: 0.0,
+ vector_score: None,
+ images: vec![],
+ content: None,
+ }
+ }
+
+ #[test]
+ fn tokenizes_cjk_bigrams_and_chars() {
+ let tokens = tokenize_query("默会知识");
+ assert!(tokens.contains(&"默会".to_string()));
+ assert!(tokens.contains(&"知识".to_string()));
+ assert!(tokens.contains(&"默".to_string()));
+ }
+
+ #[test]
+ fn extracts_image_refs_without_duplicates() {
+ let refs = extract_image_refs("![a](wiki/media/x.png)\n![b](wiki/media/x.png)");
+ assert_eq!(refs.len(), 1);
+ assert_eq!(refs[0].alt, "a");
+ }
+
+ #[test]
+ fn extract_title_uses_frontmatter_or_heading_not_body_title_lines() {
+ let with_frontmatter = "---\ntitle: Real Title\n---\n\ntitle: Body Label\n# Heading";
+ assert_eq!(
+ extract_title(with_frontmatter, "fallback-name.md"),
+ "Real Title"
+ );
+
+ let without_frontmatter = "intro\ntitle: Body Label\n# Real Heading";
+ assert_eq!(
+ extract_title(without_frontmatter, "fallback-name.md"),
+ "Real Heading"
+ );
+
+ assert_eq!(
+ extract_title("plain body", "vector-database.md"),
+ "vector database"
+ );
+ }
+
+ #[test]
+ fn explicit_query_embedding_is_validated() {
+ assert!(validate_query_embedding(vec![0.1, 0.2]).is_ok());
+ assert!(validate_query_embedding(vec![]).is_err());
+ assert!(validate_query_embedding(vec![f32::NAN]).is_err());
+ assert!(validate_query_embedding(vec![f32::INFINITY]).is_err());
+ }
+
+ #[test]
+ fn google_embedding_endpoint_strips_key_and_normalizes_batch_endpoint() {
+ let cfg = SearchEmbeddingConfig {
+ enabled: true,
+ endpoint: "https://generativelanguage.googleapis.com/v1beta/models/gemini-embedding-001:batchEmbedContents?key=URL_KEY&alt=json".to_string(),
+ api_key: "HEADER_KEY".to_string(),
+ model: "gemini-embedding-001".to_string(),
+ output_dimensionality: Some(768),
+ };
+
+ let endpoint = google_embedding_endpoint(&cfg);
+ assert!(endpoint.contains(":embedContent"));
+ assert!(!endpoint.contains(":batchEmbedContents"));
+ assert!(!endpoint.contains("URL_KEY"));
+ assert!(endpoint.contains("alt=json"));
+
+ let body = google_embedding_body("gemini-embedding-001", "hello", Some(768));
+ assert_eq!(body["model"], "models/gemini-embedding-001");
+ assert_eq!(body["output_dimensionality"], 768);
+ }
+
+ #[test]
+ fn rrf_combines_token_and_vector_ranks_and_keeps_vector_score() {
+ let mut results = vec![
+ result("wiki/concepts/both.md"),
+ result("wiki/concepts/token-only.md"),
+ result("wiki/concepts/vector-only.md"),
+ ];
+ let token_rank = BTreeMap::from([
+ ("wiki/concepts/both.md".to_string(), 1),
+ ("wiki/concepts/token-only.md".to_string(), 2),
+ ]);
+ let vector_rank = BTreeMap::from([("both".to_string(), 1), ("vector-only".to_string(), 2)]);
+ let vector_score =
+ BTreeMap::from([("both".to_string(), 0.95), ("vector-only".to_string(), 0.8)]);
+
+ apply_rrf_scores(&mut results, &token_rank, &vector_rank, &vector_score);
+ results.sort_by(|a, b| b.score.partial_cmp(&a.score).unwrap());
+
+ assert_eq!(results[0].path, "wiki/concepts/both.md");
+ assert!((results[0].score - (1.0 / 61.0 + 1.0 / 61.0)).abs() < 0.000001);
+ assert_eq!(results[0].vector_score, Some(0.95));
+ assert!((results[1].score - (1.0 / 62.0)).abs() < 0.000001);
+ assert!((results[2].score - (1.0 / 62.0)).abs() < 0.000001);
+ }
+
+ #[test]
+ fn search_mode_distinguishes_keyword_vector_and_hybrid() {
+ assert_eq!(search_mode(false, 0), "keyword");
+ assert_eq!(search_mode(true, 3), "vector");
+ assert_eq!(search_mode(false, 3), "hybrid");
+ }
+
+ #[test]
+ fn vector_only_materialization_uses_chunk_snippet_and_any_wiki_subdir() {
+ let root = tmp_project();
+ write_page(
+ &root,
+ "wiki/custom/deep-page.md",
+ "---\ntitle: Deep Page\n---\n\n# Deep Page\n\nThe literal query is absent here.",
+ );
+ let vector_results = vec![PageVectorResult {
+ id: "deep-page".to_string(),
+ score: 0.91,
+ chunk_text: "A semantic chunk explains the actual reason for retrieval.".to_string(),
+ heading_path: "Section > Detail".to_string(),
+ }];
+ let mut results = Vec::new();
+ let pages = BTreeMap::from([(
+ "deep-page".to_string(),
+ "wiki/custom/deep-page.md".to_string(),
+ )]);
+
+ materialize_vector_only_results(
+ &vector_results,
+ &pages,
+ &root.to_string_lossy(),
+ &mut results,
+ false,
+ );
+
+ assert_eq!(results.len(), 1);
+ assert_eq!(results[0].path, "wiki/custom/deep-page.md");
+ assert_eq!(results[0].title, "Deep Page");
+ assert_eq!(results[0].vector_score, Some(0.91));
+ assert!(results[0].snippet.contains("Section > Detail"));
+ assert!(results[0].snippet.contains("semantic chunk"));
+ let _ = fs::remove_dir_all(root);
+ }
+
+ #[test]
+ fn vector_snippet_empty_chunk_does_not_echo_query() {
+ let vector = PageVectorResult {
+ id: "empty".to_string(),
+ score: 0.5,
+ chunk_text: " ".to_string(),
+ heading_path: "Heading".to_string(),
+ };
+
+ assert_eq!(build_vector_snippet(&vector), "");
+ }
+
+ #[tokio::test]
+ async fn keyword_search_prefers_filename_exact_match() {
+ let root = tmp_project();
+ write_page(
+ &root,
+ "wiki/concepts/attention.md",
+ "---\ntitle: Attention\n---\n\n# Attention\n\nbody about attention.",
+ );
+ write_page(
+ &root,
+ "wiki/concepts/random.md",
+ "---\ntitle: Random\n---\n\n# Random\n\nattention is mentioned briefly.",
+ );
+
+ let out = search_project_inner(
+ root.to_string_lossy().to_string(),
+ "attention".into(),
+ 20,
+ false,
+ None,
+ )
+ .await
+ .unwrap();
+
+ assert_eq!(out.mode, "keyword");
+ assert_eq!(out.results[0].title, "Attention");
+ assert!(out.results[0].title_match);
+ assert!(out.results[0].score > 100.0);
+ let _ = fs::remove_dir_all(root);
+ }
+
+ #[tokio::test]
+ async fn keyword_search_handles_cjk_bigram_queries() {
+ let root = tmp_project();
+ write_page(
+ &root,
+ "wiki/concepts/tacit.md",
+ "---\ntitle: 默会知识\n---\n\n# 默会知识\n\n默会知识强调难以言明的实践经验。",
+ );
+
+ let out = search_project_inner(
+ root.to_string_lossy().to_string(),
+ "默会知识".into(),
+ 20,
+ false,
+ None,
+ )
+ .await
+ .unwrap();
+
+ assert_eq!(out.results[0].title, "默会知识");
+ assert!(out.results[0].snippet.contains("默会知识"));
+ let _ = fs::remove_dir_all(root);
+ }
+
+ #[tokio::test]
+ async fn keyword_search_phrase_in_content_beats_scattered_tokens() {
+ let root = tmp_project();
+ write_page(
+ &root,
+ "wiki/concepts/phrase.md",
+ "---\ntitle: Phrase\n---\n\n# Phrase\n\nThe phrase vector database appears together.",
+ );
+ write_page(
+ &root,
+ "wiki/concepts/scattered.md",
+ "---\ntitle: Scattered\n---\n\n# Scattered\n\nvector appears here. database appears later.",
+ );
+
+ let out = search_project_inner(
+ root.to_string_lossy().to_string(),
+ "vector database".into(),
+ 20,
+ false,
+ None,
+ )
+ .await
+ .unwrap();
+
+ assert_eq!(out.results[0].title, "Phrase");
+ let _ = fs::remove_dir_all(root);
+ }
+}
diff --git a/src-tauri/src/commands/vectorstore.rs b/src-tauri/src/commands/vectorstore.rs
index 91ddab33..ded7fee0 100644
--- a/src-tauri/src/commands/vectorstore.rs
+++ b/src-tauri/src/commands/vectorstore.rs
@@ -1,10 +1,9 @@
-use lancedb::connect;
-use lancedb::query::{ExecutableQuery, QueryBase};
use arrow_array::{
- Float32Array, RecordBatch, StringArray, FixedSizeListArray, ArrayRef,
- UInt32Array,
+ ArrayRef, FixedSizeListArray, Float32Array, RecordBatch, StringArray, UInt32Array,
};
use arrow_schema::{DataType, Field, Schema};
+use lancedb::connect;
+use lancedb::query::{ExecutableQuery, QueryBase};
use serde::{Deserialize, Serialize};
use std::sync::Arc;
@@ -58,8 +57,14 @@ fn validate_page_id(page_id: &str) -> Result<(), String> {
return Err("Invalid page_id: empty or too long".to_string());
}
// Only allow alphanumeric, hyphens, underscores, dots
- if !page_id.chars().all(|c| c.is_alphanumeric() || c == '-' || c == '_' || c == '.') {
- return Err(format!("Invalid page_id: contains disallowed characters: {}", page_id));
+ if !page_id
+ .chars()
+ .all(|c| c.is_alphanumeric() || c == '-' || c == '_' || c == '.')
+ {
+ return Err(format!(
+ "Invalid page_id: contains disallowed characters: {}",
+ page_id
+ ));
}
Ok(())
}
@@ -69,28 +74,27 @@ fn make_schema(dim: i32) -> Arc<Schema> {
Field::new("page_id", DataType::Utf8, false),
Field::new(
"vector",
- DataType::FixedSizeList(
- Arc::new(Field::new("item", DataType::Float32, true)),
- dim,
- ),
+ DataType::FixedSizeList(Arc::new(Field::new("item", DataType::Float32, true)), dim),
false,
),
]))
}
-fn make_batch(schema: Arc<Schema>, page_id: &str, embedding: Vec<f32>, dim: i32) -> Result<RecordBatch, String> {
+fn make_batch(
+ schema: Arc<Schema>,
+ page_id: &str,
+ embedding: Vec<f32>,
+ dim: i32,
+) -> Result<RecordBatch, String> {
let ids: ArrayRef = Arc::new(StringArray::from(vec![page_id]));
let values = Float32Array::from(embedding);
- let vector: ArrayRef = Arc::new(
- FixedSizeListArray::new(
- Arc::new(Field::new("item", DataType::Float32, true)),
- dim,
- Arc::new(values),
- None,
- )
- );
- RecordBatch::try_new(schema, vec![ids, vector])
- .map_err(|e| format!("Batch error: {e}"))
+ let vector: ArrayRef = Arc::new(FixedSizeListArray::new(
+ Arc::new(Field::new("item", DataType::Float32, true)),
+ dim,
+ Arc::new(values),
+ None,
+ ));
+ RecordBatch::try_new(schema, vec![ids, vector]).map_err(|e| format!("Batch error: {e}"))
}
/// Upsert a page embedding into LanceDB
@@ -113,23 +117,29 @@ pub async fn vector_upsert(
let batch = make_batch(schema.clone(), &page_id, embedding, dim)?;
let data = vec![batch];
- let tables = db.table_names()
+ let tables = db
+ .table_names()
.execute()
.await
.map_err(|e| format!("List tables error: {e}"))?;
if tables.contains(&TABLE_V1.to_string()) {
- let table = db.open_table(TABLE_V1)
+ let table = db
+ .open_table(TABLE_V1)
.execute()
.await
.map_err(|e| format!("Open table error: {e}"))?;
// Delete existing entry then add new one
if let Err(e) = table.delete(&format!("page_id = '{}'", page_id)).await {
- eprintln!("[vectorstore] Warning: delete before upsert failed for '{}': {}", page_id, e);
+ eprintln!(
+ "[vectorstore] Warning: delete before upsert failed for '{}': {}",
+ page_id, e
+ );
}
- table.add(data)
+ table
+ .add(data)
.execute()
.await
.map_err(|e| format!("Add error: {e}"))?;
@@ -158,7 +168,8 @@ pub async fn vector_search(
.await
.map_err(|e| format!("DB connect error: {e}"))?;
- let tables = db.table_names()
+ let tables = db
+ .table_names()
.execute()
.await
.map_err(|e| format!("List tables error: {e}"))?;
@@ -167,7 +178,8 @@ pub async fn vector_search(
return Ok(vec![]);
}
- let table = db.open_table(TABLE_V1)
+ let table = db
+ .open_table(TABLE_V1)
.execute()
.await
.map_err(|e| format!("Open table error: {e}"))?;
@@ -214,10 +226,7 @@ pub async fn vector_search(
/// Delete a page from the vector index
#[tauri::command]
-pub async fn vector_delete(
- project_path: String,
- page_id: String,
-) -> Result<(), String> {
+pub async fn vector_delete(project_path: String, page_id: String) -> Result<(), String> {
run_guarded_async("vector_delete", async move {
validate_page_id(&page_id)?;
@@ -226,7 +235,8 @@ pub async fn vector_delete(
.await
.map_err(|e| format!("DB connect error: {e}"))?;
- let tables = db.table_names()
+ let tables = db
+ .table_names()
.execute()
.await
.map_err(|e| format!("List tables error: {e}"))?;
@@ -235,12 +245,14 @@ pub async fn vector_delete(
return Ok(());
}
- let table = db.open_table(TABLE_V1)
+ let table = db
+ .open_table(TABLE_V1)
.execute()
.await
.map_err(|e| format!("Open table error: {e}"))?;
- table.delete(&format!("page_id = '{}'", page_id))
+ table
+ .delete(&format!("page_id = '{}'", page_id))
.await
.map_err(|e| format!("Delete error: {e}"))?;
@@ -251,16 +263,15 @@ pub async fn vector_delete(
/// Get count of indexed vectors
#[tauri::command]
-pub async fn vector_count(
- project_path: String,
-) -> Result<usize, String> {
+pub async fn vector_count(project_path: String) -> Result<usize, String> {
run_guarded_async("vector_count", async move {
let db = connect(&db_path(&project_path))
.execute()
.await
.map_err(|e| format!("DB connect error: {e}"))?;
- let tables = db.table_names()
+ let tables = db
+ .table_names()
.execute()
.await
.map_err(|e| format!("List tables error: {e}"))?;
@@ -269,12 +280,14 @@ pub async fn vector_count(
return Ok(0);
}
- let table = db.open_table(TABLE_V1)
+ let table = db
+ .open_table(TABLE_V1)
.execute()
.await
.map_err(|e| format!("Open table error: {e}"))?;
- let count = table.count_rows(None)
+ let count = table
+ .count_rows(None)
.await
.map_err(|e| format!("Count error: {e}"))?;
@@ -338,10 +351,7 @@ fn make_schema_v2(dim: i32) -> Arc<Schema> {
Field::new("heading_path", DataType::Utf8, false),
Field::new(
"vector",
- DataType::FixedSizeList(
- Arc::new(Field::new("item", DataType::Float32, true)),
- dim,
- ),
+ DataType::FixedSizeList(Arc::new(Field::new("item", DataType::Float32, true)), dim),
false,
),
]))
@@ -450,10 +460,7 @@ pub async fn vector_upsert_chunks(
.await
.map_err(|e| format!("Open table error: {e}"))?;
- if let Err(e) = table
- .delete(&format!("page_id = '{}'", page_id))
- .await
- {
+ if let Err(e) = table.delete(&format!("page_id = '{}'", page_id)).await {
eprintln!(
"[vectorstore v2] Warning: delete before upsert failed for page '{}': {}",
page_id, e
@@ -570,10 +577,7 @@ pub async fn vector_search_chunks(
/// Delete every chunk belonging to a page. Used when a source document
/// is removed, or before a full re-embed of a page whose content shrank.
#[tauri::command]
-pub async fn vector_delete_page(
- project_path: String,
- page_id: String,
-) -> Result<(), String> {
+pub async fn vector_delete_page(project_path: String, page_id: String) -> Result<(), String> {
run_guarded_async("vector_delete_page", async move {
validate_page_id_for_v2(&page_id)?;
@@ -831,7 +835,9 @@ mod tests_v2 {
.unwrap();
assert_eq!(vector_count_chunks(pp.clone()).await.unwrap(), 5);
- vector_delete_page(pp.clone(), "page-a".into()).await.unwrap();
+ vector_delete_page(pp.clone(), "page-a".into())
+ .await
+ .unwrap();
assert_eq!(vector_count_chunks(pp.clone()).await.unwrap(), 2);
}
@@ -905,8 +911,12 @@ mod tests_v2 {
vector_upsert_chunks(pp.clone(), "page-a".into(), make_chunks("page-a", 2, 16))
.await
.unwrap();
- vector_delete_page(pp.clone(), "page-a".into()).await.unwrap();
- vector_delete_page(pp.clone(), "page-a".into()).await.unwrap();
+ vector_delete_page(pp.clone(), "page-a".into())
+ .await
+ .unwrap();
+ vector_delete_page(pp.clone(), "page-a".into())
+ .await
+ .unwrap();
assert_eq!(vector_count_chunks(pp).await.unwrap(), 0);
}
@@ -942,12 +952,7 @@ mod tests_v2 {
let pp = p.to_string_lossy().to_string();
// Quote would be a SQL-injection footgun for the delete filter.
- let result = vector_upsert_chunks(
- pp,
- "bad'; DROP".into(),
- make_chunks("x", 1, 16),
- )
- .await;
+ let result = vector_upsert_chunks(pp, "bad'; DROP".into(), make_chunks("x", 1, 16)).await;
assert!(result.is_err());
}
@@ -985,9 +990,13 @@ mod tests_v2 {
vector_upsert(pp.clone(), "old-page".into(), fake_embedding(0, 16))
.await
.unwrap();
- vector_upsert_chunks(pp.clone(), "new-page".into(), make_chunks("new-page", 2, 16))
- .await
- .unwrap();
+ vector_upsert_chunks(
+ pp.clone(),
+ "new-page".into(),
+ make_chunks("new-page", 2, 16),
+ )
+ .await
+ .unwrap();
assert_eq!(vector_legacy_row_count(pp.clone()).await.unwrap(), 1);
assert_eq!(vector_count_chunks(pp.clone()).await.unwrap(), 2);
diff --git a/src-tauri/src/lib.rs b/src-tauri/src/lib.rs
index 745fa87b..a786058a 100644
--- a/src-tauri/src/lib.rs
+++ b/src-tauri/src/lib.rs
@@ -1,3 +1,4 @@
+mod api_server;
mod clip_server;
mod commands;
mod panic_guard;
@@ -14,6 +15,23 @@ fn clip_server_status() -> String {
.unwrap_or_else(|e| format!("error: {e}"))
}
+#[tauri::command]
+fn api_server_status() -> String {
+ run_guarded("api_server_status", || {
+ Ok(api_server::get_api_status().to_string())
+ })
+ .unwrap_or_else(|e| format!("error: {e}"))
+}
+
+#[tauri::command]
+fn api_server_reload_config() -> String {
+ run_guarded("api_server_reload_config", || {
+ api_server::invalidate_config_cache();
+ Ok("ok".to_string())
+ })
+ .unwrap_or_else(|e| format!("error: {e}"))
+}
+
/// Apply a proxy configuration to the process env immediately, so the
/// next outbound HTTP request picks it up without needing the user to
/// restart the app. tauri-plugin-http builds a fresh
@@ -75,6 +93,7 @@ pub fn run() {
app.manage(commands::claude_cli::ClaudeCliState::default());
app.manage(commands::codex_cli::CodexCliState::default());
app.manage(commands::file_sync::FileSyncState::default());
+ api_server::start_api_server(app.handle().clone());
Ok(())
})
.invoke_handler(tauri::generate_handler![
@@ -96,7 +115,10 @@ pub fn run() {
commands::project::create_project,
commands::project::open_project,
commands::project::open_project_folder,
+ commands::search::search_project,
clip_server_status,
+ api_server_status,
+ api_server_reload_config,
commands::vectorstore::vector_upsert,
commands::vectorstore::vector_search,
commands::vectorstore::vector_delete,
@@ -159,7 +181,11 @@ pub fn run() {
.expect("error while building tauri application")
.run(|app, event| {
#[cfg(target_os = "macos")]
- if let tauri::RunEvent::Reopen { has_visible_windows, .. } = event {
+ if let tauri::RunEvent::Reopen {
+ has_visible_windows,
+ ..
+ } = event
+ {
if !has_visible_windows {
use tauri::Manager;
if let Some(window) = app.get_webview_window("main") {
diff --git a/src-tauri/src/panic_guard.rs b/src-tauri/src/panic_guard.rs
index c6320310..76a5a6c3 100644
--- a/src-tauri/src/panic_guard.rs
+++ b/src-tauri/src/panic_guard.rs
@@ -51,8 +51,7 @@ mod tests {
#[test]
fn sync_catches_string_panic() {
- let result: Result<(), String> =
- run_guarded("test", || panic!("boom from String"));
+ let result: Result<(), String> = run_guarded("test", || panic!("boom from String"));
let err = result.expect_err("panic should produce Err");
assert!(err.contains("boom from String"), "got: {err}");
assert!(err.starts_with("Internal error in test"), "got: {err}");
@@ -60,16 +59,14 @@ mod tests {
#[test]
fn sync_catches_panic_with_non_string_payload() {
- let result: Result<(), String> =
- run_guarded("test", || std::panic::panic_any(42_u32));
+ let result: Result<(), String> = run_guarded("test", || std::panic::panic_any(42_u32));
let err = result.expect_err("panic should produce Err");
assert!(err.contains("non-string panic payload"), "got: {err}");
}
#[test]
fn sync_passes_through_err() {
- let result: Result<i32, String> =
- run_guarded("test", || Err("regular error".to_string()));
+ let result: Result<i32, String> = run_guarded("test", || Err("regular error".to_string()));
assert_eq!(result.unwrap_err(), "regular error");
}
diff --git a/src-tauri/src/proxy.rs b/src-tauri/src/proxy.rs
index dfe78368..25f7019d 100644
--- a/src-tauri/src/proxy.rs
+++ b/src-tauri/src/proxy.rs
@@ -87,8 +87,7 @@ pub fn apply_proxy_env(config: &ProxyConfig) -> String {
// through the now-removed proxy. The same applies to invalid
// URLs and unsupported schemes (treat as disabled).
let url = config.url.trim();
- let invalid_scheme =
- !url.starts_with("http://") && !url.starts_with("https://");
+ let invalid_scheme = !url.starts_with("http://") && !url.starts_with("https://");
if !config.enabled || url.is_empty() || invalid_scheme {
clear_proxy_env();
@@ -315,7 +314,7 @@ mod tests {
// the URL to something we won't apply (socks5://) must clear
// any previously-applied http(s) values, not silently keep
// them.
- isolated(|| {
+ isolated(|| {
apply_proxy_env(&ProxyConfig {
enabled: true,
url: "http://127.0.0.1:7890".into(),
diff --git a/src-tauri/tauri.conf.json b/src-tauri/tauri.conf.json
index dac2889e..e67af63f 100644
--- a/src-tauri/tauri.conf.json
+++ b/src-tauri/tauri.conf.json
@@ -1,7 +1,7 @@
{
"$schema": "https://schema.tauri.app/config/2",
"productName": "LLM Wiki",
- "version": "0.4.10",
+ "version": "0.4.11",
"identifier": "com.llmwiki.app",
"build": {
"beforeDevCommand": "npm run dev",
diff --git a/src/App.tsx b/src/App.tsx
index 48f8f741..8f52d531 100644
--- a/src/App.tsx
+++ b/src/App.tsx
@@ -5,7 +5,7 @@ import { useWikiStore } from "@/stores/wiki-store"
import { useReviewStore } from "@/stores/review-store"
import { useChatStore } from "@/stores/chat-store"
import { listDirectory, openProject } from "@/commands/fs"
-import { getLastProject, getRecentProjects, saveLastProject, loadLlmConfig, loadLanguage, loadSearchApiConfig, loadEmbeddingConfig, loadMultimodalConfig, loadOutputLanguage, loadProviderConfigs, loadActivePresetId, loadProxyConfig, loadScheduledImportConfig, saveScheduledImportConfig, loadSourceWatchConfig } from "@/lib/project-store"
+import { getLastProject, getRecentProjects, saveLastProject, loadLlmConfig, loadLanguage, loadSearchApiConfig, loadEmbeddingConfig, loadMultimodalConfig, loadOutputLanguage, loadProviderConfigs, loadActivePresetId, loadProxyConfig, loadScheduledImportConfig, saveScheduledImportConfig, loadSourceWatchConfig, loadApiConfig } from "@/lib/project-store"
import { loadReviewItems, loadChatHistory } from "@/lib/persist"
import { setupAutoSave } from "@/lib/auto-save"
import { startClipWatcher } from "@/lib/clip-watcher"
@@ -222,6 +222,22 @@ function App() {
if (savedProxy) {
useWikiStore.getState().setProxyConfig(savedProxy)
}
+ // Local HTTP API server config — global (single token + enable
+ // flag for the whole install, not per-project). The Rust side
+ // reads `apiConfig.{enabled,token}` from `app-state.json`
+ // directly; this only hydrates the Zustand store so the
+ // Settings UI reflects the persisted values.
+ const savedApi = await loadApiConfig()
+ if (savedApi) {
+ useWikiStore.getState().setApiConfig({
+ enabled: typeof savedApi.enabled === "boolean" ? savedApi.enabled : true,
+ allowUnauthenticated:
+ typeof savedApi.allowUnauthenticated === "boolean"
+ ? savedApi.allowUnauthenticated
+ : false,
+ token: typeof savedApi.token === "string" ? savedApi.token : "",
+ })
+ }
const savedLang = await loadLanguage()
if (savedLang) {
await i18n.changeLanguage(savedLang)
diff --git a/src/commands/fs.ts b/src/commands/fs.ts
index a528644a..1a9a1432 100644
--- a/src/commands/fs.ts
+++ b/src/commands/fs.ts
@@ -113,3 +113,11 @@ export async function openProjectFolder(path: string): Promise<void> {
export async function clipServerStatus(): Promise<string> {
return invoke<string>("clip_server_status")
}
+
+export async function apiServerStatus(): Promise<string> {
+ return invoke<string>("api_server_status")
+}
+
+export async function apiServerReloadConfig(): Promise<string> {
+ return invoke<string>("api_server_reload_config")
+}
diff --git a/src/components/settings/sections/about-section.tsx b/src/components/settings/sections/about-section.tsx
index bcebf3df..a93a30a7 100644
--- a/src/components/settings/sections/about-section.tsx
+++ b/src/components/settings/sections/about-section.tsx
@@ -2,15 +2,24 @@ import { useEffect, useState, useCallback } from "react"
import { Download, RefreshCw, CheckCircle2, Sparkles } from "lucide-react"
import { useTranslation } from "react-i18next"
import { openUrl } from "@tauri-apps/plugin-opener"
-import { clipServerStatus } from "@/commands/fs"
+import { apiServerStatus, clipServerStatus } from "@/commands/fs"
import { Button } from "@/components/ui/button"
+import { API_SERVER_HEALTH_URL, API_SERVER_PORT } from "@/lib/api-server-constants"
import { useUpdateStore, hasAvailableUpdate } from "@/stores/update-store"
import { checkForUpdates, toLatestReleaseUrl } from "@/lib/update-check"
import { saveUpdateCheckState } from "@/lib/project-store"
+interface ApiHealth {
+ enabled?: boolean
+ authConfigured?: boolean
+ allowUnauthenticated?: boolean
+}
+
export function AboutSection() {
const { t } = useTranslation()
const [clipStatus, setClipStatus] = useState<string>("...")
+ const [apiStatus, setApiStatus] = useState<string>("...")
+ const [apiHealth, setApiHealth] = useState<ApiHealth | null>(null)
const updateStore = useUpdateStore()
useEffect(() => {
@@ -22,6 +31,21 @@ export function AboutSection() {
.catch(() => {
if (alive) setClipStatus("unknown")
})
+ apiServerStatus()
+ .then((s) => {
+ if (alive) setApiStatus(s)
+ })
+ .catch(() => {
+ if (alive) setApiStatus("unknown")
+ })
+ fetch(API_SERVER_HEALTH_URL)
+ .then((res) => res.json() as Promise<ApiHealth>)
+ .then((value) => {
+ if (alive) setApiHealth(value)
+ })
+ .catch(() => {
+ if (alive) setApiHealth(null)
+ })
return () => {
alive = false
}
@@ -67,9 +91,22 @@ export function AboutSection() {
})
}, [])
+ const apiStatusDisplay = (() => {
+ if (apiStatus === "running" && apiHealth?.enabled === false) {
+ return t("settings.sections.about.apiDisabled")
+ }
+ if (apiStatus === "running" && apiHealth?.allowUnauthenticated) {
+ return t("settings.sections.about.apiOpen")
+ }
+ if (apiStatus === "running" && apiHealth?.authConfigured === false) {
+ return t("settings.sections.about.apiNoToken")
+ }
+ return apiStatus
+ })()
const rows: Array<{ label: string; value: string; mono?: boolean }> = [
{ label: t("settings.sections.about.version"), value: `v${__APP_VERSION__}`, mono: true },
{ label: t("settings.sections.about.clipServer"), value: `${clipStatus} @ 127.0.0.1:19827`, mono: true },
+ { label: t("settings.sections.about.apiServer"), value: `${apiStatusDisplay} @ 127.0.0.1:${API_SERVER_PORT}`, mono: true },
]
// About panel = user-initiated navigation. They came here on
diff --git a/src/components/settings/sections/api-server-section.tsx b/src/components/settings/sections/api-server-section.tsx
new file mode 100644
index 00000000..840928c8
--- /dev/null
+++ b/src/components/settings/sections/api-server-section.tsx
@@ -0,0 +1,464 @@
+import { useCallback, useEffect, useMemo, useState } from "react"
+import {
+ Copy,
+ Eye,
+ EyeOff,
+ ExternalLink,
+ RefreshCw,
+ Server,
+ ShieldAlert,
+} from "lucide-react"
+import { useTranslation } from "react-i18next"
+import { openUrl } from "@tauri-apps/plugin-opener"
+import { apiServerStatus } from "@/commands/fs"
+import { Button } from "@/components/ui/button"
+import { Input } from "@/components/ui/input"
+import { Label } from "@/components/ui/label"
+import { API_SERVER_BASE_URL, API_SERVER_HEALTH_URL } from "@/lib/api-server-constants"
+import { generateApiToken } from "@/lib/api-token"
+import { useWikiStore } from "@/stores/wiki-store"
+import type { SettingsDraft, DraftSetter } from "../settings-types"
+
+interface Props {
+ draft: SettingsDraft
+ setDraft: DraftSetter
+}
+
+interface ApiHealth {
+ ok?: boolean
+ status?: string
+ enabled?: boolean
+ authRequired?: boolean
+ authConfigured?: boolean
+ allowUnauthenticated?: boolean
+ tokenSource?: "env" | "store" | "none"
+}
+
+/**
+ * Documented endpoint surface. Kept in lock-step with
+ * `src-tauri/src/api_server.rs::handle_request`. When you add or remove
+ * a route there, update this list — it's the only place users discover
+ * the API contract until we ship a proper OpenAPI doc.
+ */
+const ENDPOINTS: Array<{ method: "GET" | "POST"; path: string; noteKey: string }> = [
+ { method: "GET", path: "/api/v1/health", noteKey: "endpointHealthNote" },
+ { method: "GET", path: "/api/v1/projects", noteKey: "endpointProjectsNote" },
+ { method: "GET", path: "/api/v1/projects/{id}/files", noteKey: "endpointFilesNote" },
+ { method: "GET", path: "/api/v1/projects/{id}/files/content", noteKey: "endpointContentNote" },
+ { method: "POST", path: "/api/v1/projects/{id}/search", noteKey: "endpointSearchNote" },
+ { method: "GET", path: "/api/v1/projects/{id}/graph", noteKey: "endpointGraphNote" },
+ { method: "POST", path: "/api/v1/projects/{id}/sources/rescan", noteKey: "endpointRescanNote" },
+ { method: "POST", path: "/api/v1/projects/{id}/chat", noteKey: "endpointChatNote" },
+]
+
+export function ApiServerSection({ draft, setDraft }: Props) {
+ const { t } = useTranslation()
+ const [showToken, setShowToken] = useState(false)
+ const [copiedField, setCopiedField] = useState<"token" | "curl" | null>(null)
+ const [serverStatus, setServerStatus] = useState<string>("...")
+ const [health, setHealth] = useState<ApiHealth | null>(null)
+ const persistedApiConfig = useWikiStore((s) => s.apiConfig)
+
+ useEffect(() => {
+ let alive = true
+ apiServerStatus()
+ .then((s) => {
+ if (alive) setServerStatus(s)
+ })
+ .catch(() => {
+ if (alive) setServerStatus("unknown")
+ })
+ fetch(API_SERVER_HEALTH_URL)
+ .then((res) => res.json() as Promise<ApiHealth>)
+ .then((value) => {
+ if (alive) setHealth(value)
+ })
+ .catch(() => {
+ if (alive) setHealth(null)
+ })
+ return () => {
+ alive = false
+ }
+ }, [])
+
+ const handleGenerate = useCallback(() => {
+ setDraft("apiToken", generateApiToken())
+ setShowToken(true)
+ }, [setDraft])
+
+ const handleCopyToken = useCallback(async () => {
+ if (!draft.apiToken) return
+ try {
+ await navigator.clipboard.writeText(draft.apiToken)
+ setCopiedField("token")
+ setTimeout(() => setCopiedField(null), 1500)
+ } catch (err) {
+ console.error("[api-settings] copy token failed:", err)
+ }
+ }, [draft.apiToken])
+
+ const sampleCurl = useMemo(() => {
+ if (draft.apiAllowUnauthenticated) {
+ return `curl ${API_SERVER_BASE_URL}/api/v1/projects`
+ }
+ // Show the user a complete, paste-runnable example. The Bearer
+ // header is the recommended auth (never put the token in URL
+ // query — it leaks into logs / shell history / Referer).
+ const tokenForExample = draft.apiToken || "<your-token>"
+ return `curl -H 'Authorization: Bearer ${tokenForExample}' ${API_SERVER_BASE_URL}/api/v1/projects`
+ }, [draft.apiAllowUnauthenticated, draft.apiToken])
+
+ const hasUnsavedApiConfig =
+ persistedApiConfig.enabled !== draft.apiEnabled ||
+ persistedApiConfig.allowUnauthenticated !== draft.apiAllowUnauthenticated ||
+ persistedApiConfig.token !== draft.apiToken.trim()
+
+ const handleCopyCurl = useCallback(async () => {
+ try {
+ await navigator.clipboard.writeText(sampleCurl)
+ setCopiedField("curl")
+ setTimeout(() => setCopiedField(null), 1500)
+ } catch (err) {
+ console.error("[api-settings] copy curl failed:", err)
+ }
+ }, [sampleCurl])
+
+ const handleOpenHealth = useCallback(() => {
+ void openUrl(API_SERVER_HEALTH_URL).catch((err) => {
+ console.error("[api-settings] open health failed:", err)
+ })
+ }, [])
+
+ const statusLabel = useMemo(() => {
+ if (!draft.apiEnabled) {
+ return t("settings.sections.apiServer.statusDisabled", { defaultValue: "Disabled" })
+ }
+ if (health?.allowUnauthenticated || draft.apiAllowUnauthenticated) {
+ return t("settings.sections.apiServer.statusOpen", { defaultValue: "Running, no auth" })
+ }
+ if (serverStatus === "running" && health?.authConfigured === false && !draft.apiToken) {
+ return t("settings.sections.apiServer.statusNoToken", { defaultValue: "Running, no token" })
+ }
+ switch (serverStatus) {
+ case "running":
+ return t("settings.sections.apiServer.statusRunning", { defaultValue: "Running" })
+ case "starting":
+ return t("settings.sections.apiServer.statusStarting", { defaultValue: "Starting…" })
+ case "port_conflict":
+ return t("settings.sections.apiServer.statusPortConflict", {
+ defaultValue: "Port 19828 in use",
+ })
+ case "error":
+ return t("settings.sections.apiServer.statusError", { defaultValue: "Error" })
+ case "unknown":
+ return t("settings.sections.apiServer.statusUnknown", { defaultValue: "Unknown" })
+ default:
+ return serverStatus
+ }
+ }, [draft.apiAllowUnauthenticated, draft.apiEnabled, draft.apiToken, health, serverStatus, t])
+
+ const statusToneClass =
+ !draft.apiEnabled
+ ? "text-muted-foreground"
+ : (health?.allowUnauthenticated || draft.apiAllowUnauthenticated)
+ ? "text-amber-700 dark:text-amber-400"
+ : serverStatus === "running"
+ ? "text-emerald-600 dark:text-emerald-400"
+ : serverStatus === "starting"
+ ? "text-muted-foreground"
+ : serverStatus === "unknown"
+ ? "text-muted-foreground"
+ : "text-destructive"
+
+ const tokenStrength: "unused" | "missing" | "weak" | "ok" = draft.apiAllowUnauthenticated
+ ? "unused"
+ : !draft.apiToken
+ ? "missing"
+ : draft.apiToken.length < 16
+ ? "weak"
+ : "ok"
+
+ return (
+ <div className="space-y-6">
+ <div>
+ <h2 className="text-xl font-semibold">
+ {t("settings.sections.apiServer.title", { defaultValue: "API Server" })}
+ </h2>
+ <p className="mt-1 text-sm text-muted-foreground">
+ {t("settings.sections.apiServer.description", {
+ defaultValue:
+ "A local HTTP API at 127.0.0.1:19828 exposes project files, search, graph, and rescan to your own scripts and tools. Bind your favorite editor or shell to read the wiki without going through the UI.",
+ })}
+ </p>
+ </div>
+
+ {/* ── Enable + status ───────────────────────────────────────── */}
+ <div className="space-y-4 rounded-lg border border-border/60 bg-muted/20 p-4">
+ <label className="flex items-start gap-3">
+ <input
+ type="checkbox"
+ checked={draft.apiEnabled}
+ onChange={(event) => setDraft("apiEnabled", event.target.checked)}
+ className="mt-1 h-4 w-4"
+ />
+ <div className="space-y-1">
+ <div className="flex items-center gap-2 text-sm font-semibold">
+ <Server className="h-4 w-4 text-muted-foreground" />
+ {t("settings.sections.apiServer.enable", {
+ defaultValue: "Enable local HTTP API",
+ })}
+ </div>
+ <p className="text-xs leading-relaxed text-muted-foreground">
+ {t("settings.sections.apiServer.enableHint", {
+ defaultValue:
+ "Disable to make every non-/health endpoint return 503 even if a token is configured. Useful as a kill-switch without unsetting the token.",
+ })}
+ </p>
+ </div>
+ </label>
+
+ <label className="flex items-start gap-3 rounded-md border border-amber-200 bg-amber-50 px-3 py-2 dark:border-amber-900/50 dark:bg-amber-950/30">
+ <input
+ type="checkbox"
+ checked={draft.apiAllowUnauthenticated}
+ onChange={(event) => setDraft("apiAllowUnauthenticated", event.target.checked)}
+ className="mt-1 h-4 w-4"
+ />
+ <div className="space-y-1">
+ <div className="text-sm font-semibold text-amber-900 dark:text-amber-200">
+ {t("settings.sections.apiServer.allowUnauthenticated", {
+ defaultValue: "Allow access without a token",
+ })}
+ </div>
+ <p className="text-xs leading-relaxed text-amber-900 dark:text-amber-200">
+ {t("settings.sections.apiServer.allowUnauthenticatedHint", {
+ defaultValue:
+ "Use only for trusted local agents. Any process or browser page on this machine can call the API while this is enabled.",
+ })}
+ </p>
+ </div>
+ </label>
+
+ <div className="grid grid-cols-1 gap-3 rounded-md border border-border/60 bg-background/40 p-3 text-sm sm:grid-cols-2">
+ <div className="space-y-0.5">
+ <div className="text-[11px] uppercase tracking-wide text-muted-foreground">
+ {t("settings.sections.apiServer.status", { defaultValue: "Status" })}
+ </div>
+ <div className={`font-mono text-xs ${statusToneClass}`}>{statusLabel}</div>
+ </div>
+ <div className="space-y-0.5">
+ <div className="text-[11px] uppercase tracking-wide text-muted-foreground">
+ {t("settings.sections.apiServer.baseUrl", { defaultValue: "Base URL" })}
+ </div>
+ <div className="font-mono text-xs">{API_SERVER_BASE_URL}</div>
+ </div>
+ </div>
+
+ <div className="flex items-center gap-2">
+ <Button variant="outline" size="sm" onClick={handleOpenHealth} className="gap-1.5">
+ <ExternalLink className="h-3.5 w-3.5" />
+ {t("settings.sections.apiServer.openHealth", { defaultValue: "Open /health" })}
+ </Button>
+ <span className="text-[11px] text-muted-foreground">
+ {t("settings.sections.apiServer.openHealthHint", {
+ defaultValue: "Opens in your system browser. /health is the only unauthenticated endpoint.",
+ })}
+ </span>
+ </div>
+ </div>
+
+ {/* ── Token ─────────────────────────────────────────────────── */}
+ <div className="space-y-3 rounded-lg border border-border/60 bg-muted/20 p-4">
+ <div>
+ <h3 className="text-sm font-semibold">
+ {t("settings.sections.apiServer.token", { defaultValue: "Access token" })}
+ </h3>
+ <p className="mt-1 text-xs leading-relaxed text-muted-foreground">
+ {t("settings.sections.apiServer.tokenHint", {
+ defaultValue:
+ "Required unless unauthenticated access is enabled. Send as `Authorization: Bearer <token>` or `X-LLM-Wiki-Token: <token>`. The environment variable LLM_WIKI_API_TOKEN overrides this field if set.",
+ })}
+ </p>
+ </div>
+
+ <Label htmlFor="api-token-input" className="sr-only">
+ {t("settings.sections.apiServer.token", { defaultValue: "Access token" })}
+ </Label>
+ <div className="flex gap-2">
+ <Input
+ id="api-token-input"
+ type={showToken ? "text" : "password"}
+ value={draft.apiToken}
+ onChange={(event) => setDraft("apiToken", event.target.value)}
+ placeholder={t("settings.sections.apiServer.tokenPlaceholder", {
+ defaultValue: "Paste an existing token or click Generate",
+ })}
+ className="font-mono"
+ autoComplete="off"
+ spellCheck={false}
+ />
+ <Button
+ type="button"
+ variant="outline"
+ size="icon"
+ onClick={() => setShowToken((value) => !value)}
+ title={
+ showToken
+ ? t("settings.sections.apiServer.hide", { defaultValue: "Hide" })
+ : t("settings.sections.apiServer.show", { defaultValue: "Show" })
+ }
+ aria-label={
+ showToken
+ ? t("settings.sections.apiServer.hide", { defaultValue: "Hide" })
+ : t("settings.sections.apiServer.show", { defaultValue: "Show" })
+ }
+ >
+ {showToken ? <EyeOff className="h-4 w-4" /> : <Eye className="h-4 w-4" />}
+ </Button>
+ <Button
+ type="button"
+ variant="outline"
+ size="icon"
+ onClick={handleCopyToken}
+ disabled={!draft.apiToken}
+ title={t("settings.sections.apiServer.copy", { defaultValue: "Copy" })}
+ aria-label={t("settings.sections.apiServer.copy", { defaultValue: "Copy" })}
+ >
+ <Copy className="h-4 w-4" />
+ </Button>
+ </div>
+
+ <div className="flex items-center gap-2">
+ <Button type="button" variant="outline" size="sm" onClick={handleGenerate} className="gap-1.5">
+ <RefreshCw className="h-3.5 w-3.5" />
+ {t("settings.sections.apiServer.generate", { defaultValue: "Generate new token" })}
+ </Button>
+ {copiedField === "token" && (
+ <span className="text-xs text-emerald-600 dark:text-emerald-400">
+ {t("settings.sections.apiServer.copied", { defaultValue: "Copied" })}
+ </span>
+ )}
+ {tokenStrength === "missing" && (
+ <span className="text-xs text-amber-700 dark:text-amber-400">
+ {t("settings.sections.apiServer.tokenMissing", {
+ defaultValue: "No token — every endpoint will return 401",
+ })}
+ </span>
+ )}
+ {tokenStrength === "unused" && (
+ <span className="text-xs text-amber-700 dark:text-amber-400">
+ {t("settings.sections.apiServer.tokenUnused", {
+ defaultValue: "Token is not used while unauthenticated access is enabled",
+ })}
+ </span>
+ )}
+ {health?.tokenSource === "env" && (
+ <span className="text-xs text-amber-700 dark:text-amber-400">
+ {t("settings.sections.apiServer.envTokenActive", {
+ defaultValue: "LLM_WIKI_API_TOKEN is active and overrides this field",
+ })}
+ </span>
+ )}
+ {hasUnsavedApiConfig && (
+ <span className="text-xs text-muted-foreground">
+ {t("settings.sections.apiServer.saveFirst", {
+ defaultValue: "Save settings to apply API changes",
+ })}
+ </span>
+ )}
+ {tokenStrength === "weak" && (
+ <span className="text-xs text-amber-700 dark:text-amber-400">
+ {t("settings.sections.apiServer.tokenWeak", {
+ defaultValue: "Token is short — consider Generate for 256-bit entropy",
+ })}
+ </span>
+ )}
+ </div>
+
+ <div className="flex items-start gap-2 rounded-md border border-amber-200 bg-amber-50 px-3 py-2 text-xs leading-relaxed text-amber-900 dark:border-amber-900/50 dark:bg-amber-950/40 dark:text-amber-200">
+ <ShieldAlert className="mt-0.5 h-3.5 w-3.5 shrink-0" />
+ <div className="space-y-1">
+ <div>
+ {t("settings.sections.apiServer.tokenWarning", {
+ defaultValue:
+ "Keep this token secret. Anyone with the token on this machine can read your project files via localhost.",
+ })}
+ </div>
+ <div>
+ {t("settings.sections.apiServer.tokenQueryWarning", {
+ defaultValue:
+ "Prefer the Authorization header — passing ?token=… via URL leaks the value into shell history, logs, and Referer headers.",
+ })}
+ </div>
+ </div>
+ </div>
+ </div>
+
+ {/* ── Sample curl ───────────────────────────────────────────── */}
+ <div className="space-y-3 rounded-lg border border-border/60 bg-muted/20 p-4">
+ <div className="flex items-center justify-between gap-2">
+ <h3 className="text-sm font-semibold">
+ {t("settings.sections.apiServer.sample", { defaultValue: "Example request" })}
+ </h3>
+ <Button
+ type="button"
+ variant="outline"
+ size="sm"
+ onClick={handleCopyCurl}
+ disabled={hasUnsavedApiConfig}
+ className="gap-1.5"
+ >
+ <Copy className="h-3.5 w-3.5" />
+ {copiedField === "curl"
+ ? t("settings.sections.apiServer.copied", { defaultValue: "Copied" })
+ : t("settings.sections.apiServer.copy", { defaultValue: "Copy" })}
+ </Button>
+ </div>
+ <pre className="overflow-x-auto whitespace-pre-wrap break-all rounded-md bg-background/60 px-3 py-2 text-[11px] font-mono leading-relaxed">
+ {hasUnsavedApiConfig
+ ? t("settings.sections.apiServer.saveFirstExample", {
+ defaultValue: "Save settings first, then copy an example request.",
+ })
+ : sampleCurl}
+ </pre>
+ </div>
+
+ {/* ── Endpoint catalog ──────────────────────────────────────── */}
+ <div className="space-y-2 rounded-lg border border-border/60 bg-muted/20 p-4">
+ <h3 className="text-sm font-semibold">
+ {t("settings.sections.apiServer.endpoints", { defaultValue: "Endpoints" })}
+ </h3>
+ <p className="text-xs leading-relaxed text-muted-foreground">
+ {t("settings.sections.apiServer.endpointsHint", {
+ defaultValue: "Replace {id} with a project UUID, a project filesystem path, or the literal 'current'.",
+ })}
+ </p>
+ <div className="space-y-1 text-xs">
+ {ENDPOINTS.map((endpoint) => {
+ const note = t(`settings.sections.apiServer.${endpoint.noteKey}`, {
+ defaultValue: "",
+ })
+ const methodClass =
+ endpoint.method === "GET"
+ ? "bg-blue-500/10 text-blue-700 dark:text-blue-400"
+ : "bg-emerald-500/10 text-emerald-700 dark:text-emerald-400"
+ return (
+ <div
+ key={`${endpoint.method} ${endpoint.path}`}
+ className="flex flex-wrap items-baseline gap-2 rounded border border-border/40 bg-background/50 px-2 py-1"
+ >
+ <span
+ className={`inline-block rounded px-1.5 py-0.5 text-[10px] font-semibold ${methodClass}`}
+ >
+ {endpoint.method}
+ </span>
+ <span className="font-mono">{endpoint.path}</span>
+ {note && <span className="text-muted-foreground">— {note}</span>}
+ </div>
+ )
+ })}
+ </div>
+ </div>
+ </div>
+ )
+}
diff --git a/src/components/settings/settings-types.ts b/src/components/settings/settings-types.ts
index 3f73525c..1e7968f0 100644
--- a/src/components/settings/settings-types.ts
+++ b/src/components/settings/settings-types.ts
@@ -62,6 +62,11 @@ export interface SettingsDraft {
// Source folder auto watch
sourceWatchConfig: SourceWatchConfig
+
+ // Local HTTP API server
+ apiEnabled: boolean
+ apiAllowUnauthenticated: boolean
+ apiToken: string
}
export type DraftSetter = <K extends keyof SettingsDraft>(
diff --git a/src/components/settings/settings-view.tsx b/src/components/settings/settings-view.tsx
index d1af73ed..c0e7bbd0 100644
--- a/src/components/settings/settings-view.tsx
+++ b/src/components/settings/settings-view.tsx
@@ -12,6 +12,7 @@ import {
Wrench,
Clock,
FolderSync,
+ Server,
} from "lucide-react"
import { useTranslation } from "react-i18next"
import { invoke } from "@tauri-apps/api/core"
@@ -32,6 +33,7 @@ import { InterfaceSection } from "./sections/interface-section"
import { NetworkSection } from "./sections/network-section"
import { ScheduledImportSection } from "./sections/scheduled-import-section"
import { SourceWatchSection } from "./sections/source-watch-section"
+import { ApiServerSection } from "./sections/api-server-section"
import { ChangelogSection } from "./sections/changelog-section"
import { MaintenanceSection } from "./sections/maintenance-section"
import { AboutSection } from "./sections/about-section"
@@ -44,6 +46,7 @@ type CategoryId =
| "network"
| "source-watch"
| "scheduled-import"
+ | "api-server"
| "output"
| "interface"
| "maintenance"
@@ -67,6 +70,7 @@ const CATEGORIES: Category[] = [
{ id: "network", labelKey: "settings.categories.network", icon: Network },
{ id: "source-watch", labelKey: "settings.categories.sourceWatch", icon: FolderSync },
{ id: "scheduled-import", labelKey: "settings.categories.scheduledImport", icon: Clock },
+ { id: "api-server", labelKey: "settings.categories.apiServer", icon: Server },
{ id: "output", labelKey: "settings.categories.output", icon: Languages },
{ id: "interface", labelKey: "settings.categories.interface", icon: Palette },
{ id: "maintenance", labelKey: "settings.categories.maintenance", icon: Wrench },
@@ -82,6 +86,7 @@ function initialDraft(
proxy: ReturnType<typeof useWikiStore.getState>["proxyConfig"],
scheduledImport: ReturnType<typeof useWikiStore.getState>["scheduledImportConfig"],
sourceWatch: ReturnType<typeof useWikiStore.getState>["sourceWatchConfig"],
+ apiConfig: ReturnType<typeof useWikiStore.getState>["apiConfig"],
maxHistoryMessages: number,
uiLanguage: string,
projectPath?: string,
@@ -131,6 +136,9 @@ function initialDraft(
scheduledImportPath: displayPath,
scheduledImportInterval: scheduledImport.interval,
sourceWatchConfig: normalizeSourceWatchConfig(sourceWatch),
+ apiEnabled: apiConfig.enabled,
+ apiAllowUnauthenticated: apiConfig.allowUnauthenticated,
+ apiToken: apiConfig.token,
uiLanguage,
}
}
@@ -152,6 +160,8 @@ export function SettingsView() {
const setScheduledImportConfig = useWikiStore((s) => s.setScheduledImportConfig)
const sourceWatchConfig = useWikiStore((s) => s.sourceWatchConfig)
const setSourceWatchConfig = useWikiStore((s) => s.setSourceWatchConfig)
+ const apiConfig = useWikiStore((s) => s.apiConfig)
+ const setApiConfig = useWikiStore((s) => s.setApiConfig)
const maxHistoryMessages = useChatStore((s) => s.maxHistoryMessages)
const setMaxHistoryMessages = useChatStore((s) => s.setMaxHistoryMessages)
// Drives the red dot next to the "About" row in the settings
@@ -175,6 +185,7 @@ export function SettingsView() {
proxyConfig,
scheduledImportConfig,
sourceWatchConfig,
+ apiConfig,
maxHistoryMessages,
i18n.language,
project?.path,
@@ -217,6 +228,7 @@ export function SettingsView() {
proxyConfig,
scheduledImportConfig,
sourceWatchConfig,
+ apiConfig,
maxHistoryMessages,
prev.uiLanguage,
project?.path,
@@ -230,6 +242,7 @@ export function SettingsView() {
proxyConfig,
scheduledImportConfig,
sourceWatchConfig,
+ apiConfig,
maxHistoryMessages,
project,
])
@@ -247,6 +260,7 @@ export function SettingsView() {
saveProxyConfig,
saveScheduledImportConfig,
saveSourceWatchConfig,
+ saveApiConfig,
} = await import("@/lib/project-store")
const newLlm = {
@@ -348,6 +362,23 @@ export function SettingsView() {
setMaxHistoryMessages(draft.maxHistoryMessages)
+ // ── API server: persist + push to store. The Rust side reads
+ // `apiConfig.{enabled,token}` from this same `app-state.json` on
+ // every request via a 5s cache, so saved changes propagate
+ // within that window without any IPC round-trip.
+ const newApiConfig = {
+ enabled: draft.apiEnabled,
+ allowUnauthenticated: draft.apiAllowUnauthenticated,
+ token: draft.apiToken.trim(),
+ }
+ setApiConfig(newApiConfig)
+ await saveApiConfig(newApiConfig)
+ try {
+ await invoke<string>("api_server_reload_config")
+ } catch (err) {
+ console.warn("[api] failed to reload API server config cache:", err)
+ }
+
if (draft.uiLanguage !== i18n.language) {
await i18n.changeLanguage(draft.uiLanguage)
await saveLanguage(draft.uiLanguage)
@@ -364,6 +395,7 @@ export function SettingsView() {
setProxyConfig,
setScheduledImportConfig,
setSourceWatchConfig,
+ setApiConfig,
scheduledImportConfig,
setMaxHistoryMessages,
outputLanguage,
@@ -388,6 +420,8 @@ export function SettingsView() {
return <SourceWatchSection draft={draft} setDraft={setDraft} projectReady={!!project} />
case "scheduled-import":
return <ScheduledImportSection draft={draft} setDraft={setDraft} />
+ case "api-server":
+ return <ApiServerSection draft={draft} setDraft={setDraft} />
case "output":
return <OutputSection draft={draft} setDraft={setDraft} />
case "interface":
diff --git a/src/i18n/en.json b/src/i18n/en.json
index 80c4f436..27c7d3b0 100644
--- a/src/i18n/en.json
+++ b/src/i18n/en.json
@@ -184,6 +184,7 @@
"network": "Network",
"sourceWatch": "Source Watch",
"scheduledImport": "Scheduled Import",
+ "apiServer": "API Server",
"output": "Output",
"interface": "Interface",
"maintenance": "Maintenance",
@@ -357,6 +358,53 @@
"lastScan": "Last scan: {{time}}",
"never": "Never"
},
+ "apiServer": {
+ "title": "API Server",
+ "description": "A local HTTP API at 127.0.0.1:19828 exposes project files, search, graph, and rescan to your own scripts and tools. Bind your favorite editor or shell to read the wiki without going through the UI.",
+ "enable": "Enable local HTTP API",
+ "enableHint": "Disable to make every non-/health endpoint return 503 even if a token is configured. Useful as a kill-switch without unsetting the token.",
+ "status": "Status",
+ "statusRunning": "Running",
+ "statusOpen": "Running, no auth",
+ "statusNoToken": "Running, no token",
+ "statusDisabled": "Disabled",
+ "statusStarting": "Starting…",
+ "statusPortConflict": "Port 19828 in use",
+ "statusError": "Error",
+ "statusUnknown": "Unknown",
+ "baseUrl": "Base URL",
+ "openHealth": "Open /health",
+ "openHealthHint": "Opens in your system browser. /health is the only unauthenticated endpoint.",
+ "allowUnauthenticated": "Allow access without a token",
+ "allowUnauthenticatedHint": "Use only for trusted local agents. Any process or browser page on this machine can call the API while this is enabled.",
+ "token": "Access token",
+ "tokenHint": "Required unless unauthenticated access is enabled. Send as `Authorization: Bearer <token>` or `X-LLM-Wiki-Token: <token>`. The environment variable LLM_WIKI_API_TOKEN overrides this field if set.",
+ "tokenPlaceholder": "Paste an existing token or click Generate",
+ "generate": "Generate new token",
+ "show": "Show",
+ "hide": "Hide",
+ "copy": "Copy",
+ "copied": "Copied",
+ "tokenMissing": "No token — every endpoint will return 401",
+ "tokenUnused": "Token is not used while unauthenticated access is enabled",
+ "envTokenActive": "LLM_WIKI_API_TOKEN is active and overrides this field",
+ "tokenWeak": "Token is short — consider Generate for 256-bit entropy",
+ "saveFirst": "Save settings to apply API changes",
+ "saveFirstExample": "Save settings first, then copy an example request.",
+ "tokenWarning": "Keep this token secret. Anyone with the token on this machine can read your project files via localhost.",
+ "tokenQueryWarning": "Prefer the Authorization header — passing ?token=… via URL leaks the value into shell history, logs, and Referer headers.",
+ "sample": "Example request",
+ "endpoints": "Endpoints",
+ "endpointsHint": "Replace {id} with a project UUID, a project filesystem path, or the literal 'current'.",
+ "endpointHealthNote": "Liveness probe. No auth.",
+ "endpointProjectsNote": "List known projects with their IDs and paths.",
+ "endpointFilesNote": "?root=wiki|sources|all; ?recursive=false; ?maxFiles=N (≤10000).",
+ "endpointContentNote": "?path=wiki/index.md — text files only, 2 MB max.",
+ "endpointSearchNote": "JSON body: { query, topK?, includeContent?, queryEmbedding? }. Uses shared keyword + vector retrieval when embeddings are configured.",
+ "endpointGraphNote": "?q=…&nodeType=…&limit=N — wikilinks graph derived from wiki/*.md.",
+ "endpointRescanNote": "Triggers a backend rescan using your Source Watch config.",
+ "endpointChatNote": "Not implemented yet (returns 501)."
+ },
"output": {
"title": "Output Preferences",
"description": "Controls for generated content — language, conversation history length.",
@@ -409,6 +457,10 @@
"description": "Build info and runtime status.",
"version": "Version",
"clipServer": "Clip Server",
+ "apiServer": "API Server",
+ "apiDisabled": "disabled",
+ "apiOpen": "running (no auth)",
+ "apiNoToken": "running (no token)",
"updateCheck": "Update Check",
"lastChecked": "Last checked",
"lastCheckedNever": "Never",
diff --git a/src/i18n/zh.json b/src/i18n/zh.json
index fce47791..20fbff83 100644
--- a/src/i18n/zh.json
+++ b/src/i18n/zh.json
@@ -184,6 +184,7 @@
"network": "网络",
"sourceWatch": "资料文件夹监控",
"scheduledImport": "定时导入",
+ "apiServer": "API 服务",
"output": "输出偏好",
"interface": "界面",
"maintenance": "维护",
@@ -357,6 +358,53 @@
"lastScan": "上次扫描:{{time}}",
"never": "从未"
},
+ "apiServer": {
+ "title": "API 服务",
+ "description": "本地在 127.0.0.1:19828 暴露一个 HTTP API,把项目文件、搜索、知识图谱、重新扫描接口开放给你自己的脚本和工具。无需通过 UI 即可读取 wiki。",
+ "enable": "启用本地 HTTP API",
+ "enableHint": "关闭后,即使已配置 token,除 /health 外的所有接口都会返回 503。可作为不清空 token 的临时关闭开关。",
+ "status": "状态",
+ "statusRunning": "运行中",
+ "statusOpen": "运行中,无需鉴权",
+ "statusNoToken": "运行中,未配置 token",
+ "statusDisabled": "已禁用",
+ "statusStarting": "启动中…",
+ "statusPortConflict": "端口 19828 被占用",
+ "statusError": "错误",
+ "statusUnknown": "未知",
+ "baseUrl": "服务地址",
+ "openHealth": "打开 /health",
+ "openHealthHint": "在系统浏览器中打开。/health 是唯一不需要 token 的接口。",
+ "allowUnauthenticated": "允许无 token 访问",
+ "allowUnauthenticatedHint": "仅用于可信的本地智能体。启用后,这台机器上的任何进程或浏览器页面都可以调用 API。",
+ "token": "访问令牌",
+ "tokenHint": "除非启用了无认证访问,否则必须配置。通过 `Authorization: Bearer <token>` 或 `X-LLM-Wiki-Token: <token>` 发送。设置了环境变量 LLM_WIKI_API_TOKEN 时优先生效。",
+ "tokenPlaceholder": "粘贴已有 token,或点击生成",
+ "generate": "生成新 token",
+ "show": "显示",
+ "hide": "隐藏",
+ "copy": "复制",
+ "copied": "已复制",
+ "tokenMissing": "未配置 token —— 所有接口都会返回 401",
+ "tokenUnused": "启用无认证访问时,token 不会参与鉴权",
+ "envTokenActive": "LLM_WIKI_API_TOKEN 正在生效,并会覆盖此处填写的 token",
+ "tokenWeak": "token 长度偏短 —— 建议使用「生成新 token」获得 256 位熵",
+ "saveFirst": "保存设置后 API 改动才会生效",
+ "saveFirstExample": "请先保存设置,然后再复制示例请求。",
+ "tokenWarning": "请妥善保管。任何能访问这台机器 localhost 的进程拿到 token 后,都能通过 API 读取你的项目内容。",
+ "tokenQueryWarning": "推荐使用 Authorization header —— 将 token 放在 ?token=… 查询参数里会泄露到 shell 历史、日志和 Referer。",
+ "sample": "示例请求",
+ "endpoints": "接口列表",
+ "endpointsHint": "把 {id} 替换为项目 UUID、项目文件系统路径,或字面量 'current'。",
+ "endpointHealthNote": "存活探针。无需鉴权。",
+ "endpointProjectsNote": "列出所有已知项目及其 ID 和路径。",
+ "endpointFilesNote": "?root=wiki|sources|all;?recursive=false;?maxFiles=N(≤10000)。",
+ "endpointContentNote": "?path=wiki/index.md —— 仅文本文件,最大 2 MB。",
+ "endpointSearchNote": "JSON body:{ query, topK?, includeContent?, queryEmbedding? }。配置嵌入后使用统一的关键词 + 向量检索。",
+ "endpointGraphNote": "?q=…&nodeType=…&limit=N —— 基于 wiki/*.md 的双链图谱。",
+ "endpointRescanNote": "使用你的资料文件夹监控配置触发后端重新扫描。",
+ "endpointChatNote": "尚未实现(返回 501)。"
+ },
"output": {
"title": "输出偏好",
"description": "影响生成内容的语言、对话历史长度等参数。",
@@ -409,6 +457,10 @@
"description": "构建信息和运行时状态。",
"version": "版本",
"clipServer": "Clip 服务",
+ "apiServer": "API 服务",
+ "apiDisabled": "已禁用",
+ "apiOpen": "运行中(无认证)",
+ "apiNoToken": "运行中(未配置 token)",
"updateCheck": "更新检查",
"lastChecked": "上次检查",
"lastCheckedNever": "从未",
diff --git a/src/lib/api-server-constants.ts b/src/lib/api-server-constants.ts
new file mode 100644
index 00000000..0bc963bc
--- /dev/null
+++ b/src/lib/api-server-constants.ts
@@ -0,0 +1,3 @@
+export const API_SERVER_PORT = 19828
+export const API_SERVER_BASE_URL = `http://127.0.0.1:${API_SERVER_PORT}`
+export const API_SERVER_HEALTH_URL = `${API_SERVER_BASE_URL}/api/v1/health`
diff --git a/src/lib/api-server.real-llm.test.ts b/src/lib/api-server.real-llm.test.ts
new file mode 100644
index 00000000..b4ad2bb4
--- /dev/null
+++ b/src/lib/api-server.real-llm.test.ts
@@ -0,0 +1,802 @@
+/**
+ * Real local-API tests against a running LLM Wiki desktop app.
+ *
+ * These tests intentionally perform real HTTP calls to
+ * http://127.0.0.1:19828/api/v1 and use a real project id. They are gated
+ * behind RUN_API_TESTS=1 (or RUN_LLM_TESTS=1 for the repo's shared real-test
+ * script) because they require the app to be running and the API server to be
+ * enabled in Settings -> API Server.
+ *
+ * WARNING: the auth/config tests mutate the live app-state.json to verify
+ * enabled=false and unauthenticated mode. They restore the original file in
+ * afterAll, but run this against a dedicated test project when possible.
+ *
+ * Example:
+ * RUN_API_TESTS=1 \
+ * API_PROJECT_ID=a0e90b29-fcf3-4364-9502-8bd1272de820 \
+ * API_TOKEN=<token-if-required> \
+ * npx vitest run src/lib/api-server.real-llm.test.ts
+ */
+import { describe, expect, it } from "vitest"
+import { afterAll, beforeAll } from "vitest"
+import fs from "node:fs/promises"
+import os from "node:os"
+import path from "node:path"
+import { API_SERVER_BASE_URL } from "./api-server-constants"
+
+const ENABLED = process.env.RUN_API_TESTS === "1" || process.env.RUN_LLM_TESTS === "1"
+const BASE_URL = process.env.API_BASE_URL ?? API_SERVER_BASE_URL
+const PROJECT_ID =
+ process.env.API_PROJECT_ID ?? "a0e90b29-fcf3-4364-9502-8bd1272de820"
+const API_TOKEN = process.env.API_TOKEN ?? process.env.LLM_WIKI_API_TOKEN ?? ""
+const TEST_TOKEN = process.env.API_TEST_TOKEN ?? "llm-wiki-real-api-test-token"
+
+const TEST_TIMEOUT_MS = 30_000
+const RESCAN_TIMEOUT_MS = 60_000
+
+interface ApiEnvelope {
+ ok?: boolean
+ error?: string
+ [key: string]: unknown
+}
+
+interface ApiHealth extends ApiEnvelope {
+ status?: string
+ enabled?: boolean
+ authRequired?: boolean
+ authConfigured?: boolean
+ allowUnauthenticated?: boolean
+ tokenSource?: "env" | "store" | "none"
+}
+
+interface ApiProject {
+ id: string
+ name: string
+ path: string
+ current: boolean
+}
+
+interface ApiFileNode {
+ name: string
+ path: string
+ isDir: boolean
+ size?: number
+ children?: ApiFileNode[]
+}
+
+interface SearchHit {
+ path: string
+ title: string
+ snippet: string
+ score: number
+ content?: string
+}
+
+interface GraphNode {
+ id: string
+ label: string
+ nodeType: string
+ path: string
+ linkCount: number
+}
+
+interface GraphEdge {
+ source: string
+ target: string
+ weight: number
+}
+
+interface RescanResult {
+ queue?: {
+ version?: number
+ tasks?: unknown[]
+ }
+ changedTasks?: unknown[]
+}
+
+let serverUnavailableReason: string | null = null
+let initialHealth: ApiHealth | null = null
+let appStatePath: string | null = null
+let originalAppStateRaw: string | null = null
+let appStateMutated = false
+
+function endpoint(path: string): string {
+ return `${BASE_URL}${path}`
+}
+
+function authHeaders(): Record<string, string> {
+ return API_TOKEN ? { Authorization: `Bearer ${API_TOKEN}` } : {}
+}
+
+async function api<T extends ApiEnvelope>(
+ path: string,
+ init: RequestInit = {},
+ options: { auth?: "default" | "none" | "bearer" | "xToken"; token?: string } = {},
+): Promise<{ status: number; body: T }> {
+ const authMode = options.auth ?? "default"
+ const token = options.token ?? API_TOKEN
+ const auth =
+ authMode === "none"
+ ? {}
+ : authMode === "bearer"
+ ? { Authorization: `Bearer ${token}` }
+ : authMode === "xToken"
+ ? { "X-LLM-Wiki-Token": token }
+ : authHeaders()
+ const headers = {
+ ...auth,
+ ...(init.body ? { "Content-Type": "application/json" } : {}),
+ ...(init.headers as Record<string, string> | undefined),
+ }
+ const response = await fetch(endpoint(path), { ...init, headers })
+ const text = await response.text()
+ const body = (text ? JSON.parse(text) : {}) as T
+ return { status: response.status, body }
+}
+
+async function health(): Promise<ApiHealth> {
+ const { status, body } = await api<ApiHealth>("/api/v1/health", {}, { auth: "none" })
+ if (status !== 200 || body.ok !== true) {
+ throw new Error(`API health check failed with HTTP ${status}: ${JSON.stringify(body)}`)
+ }
+ return body
+}
+
+function flatten(nodes: ApiFileNode[]): ApiFileNode[] {
+ const out: ApiFileNode[] = []
+ const visit = (items: ApiFileNode[]) => {
+ for (const item of items) {
+ out.push(item)
+ if (item.children) visit(item.children)
+ }
+ }
+ visit(nodes)
+ return out
+}
+
+function firstTextFile(nodes: ApiFileNode[]): ApiFileNode | undefined {
+ return flatten(nodes).find(
+ (node) =>
+ !node.isDir &&
+ /(^|\/)(purpose|schema)\.md$|\.(md|mdx|txt|json|yaml|yml|csv|html|htm|xml|rtf|log)$/i.test(
+ node.path,
+ ),
+ )
+}
+
+function firstBinaryFile(nodes: ApiFileNode[]): ApiFileNode | undefined {
+ return flatten(nodes).find(
+ (node) =>
+ !node.isDir &&
+ /^wiki\/media\//i.test(node.path) &&
+ !/\.(md|mdx|txt|json|yaml|yml|csv|html|htm|xml|rtf|log)$/i.test(node.path),
+ )
+}
+
+function searchQueryFromContent(content: string, fallback: string): string {
+ const title =
+ content.match(/^title:\s*["']?([^"'\n]+)["']?/m)?.[1]?.trim() ??
+ content.match(/^#\s+(.+)$/m)?.[1]?.trim()
+ if (title && title.length >= 2) return title
+ const candidates = content
+ .replace(/```[\s\S]*?```/g, " ")
+ .split(/[^\p{L}\p{N}_-]+/u)
+ .map((s) => s.trim())
+ .filter(
+ (s) =>
+ s.length >= 2 &&
+ ![
+ "title",
+ "type",
+ "sources",
+ "the",
+ "and",
+ "for",
+ "that",
+ "with",
+ "this",
+ "from",
+ ].includes(s.toLowerCase()),
+ )
+ return candidates[0] ?? fallback
+}
+
+async function requireUsableApi(): Promise<ApiHealth> {
+ const h = await health()
+ expect(h.status).toBe("running")
+ expect(h.enabled).toBe(true)
+ if (h.authRequired && !h.allowUnauthenticated && !API_TOKEN) {
+ throw new Error(
+ "API requires auth. Re-run with API_TOKEN=<token> or enable Settings -> API Server -> Allow access without a token.",
+ )
+ }
+ return h
+}
+
+function appStateCandidates(): string[] {
+ const explicit = process.env.API_APP_STATE_PATH
+ const home = os.homedir()
+ const candidates = explicit ? [explicit] : []
+ if (process.platform === "darwin") {
+ candidates.push(
+ path.join(home, "Library/Application Support/com.llmwiki.app/app-state.json"),
+ path.join(home, "Library/Application Support/LLM Wiki/app-state.json"),
+ )
+ } else if (process.platform === "win32") {
+ const appData = process.env.APPDATA ?? path.join(home, "AppData/Roaming")
+ candidates.push(
+ path.join(appData, "com.llmwiki.app/app-state.json"),
+ path.join(appData, "LLM Wiki/app-state.json"),
+ )
+ } else {
+ candidates.push(
+ path.join(process.env.XDG_DATA_HOME ?? path.join(home, ".local/share"), "com.llmwiki.app/app-state.json"),
+ path.join(process.env.XDG_CONFIG_HOME ?? path.join(home, ".config"), "com.llmwiki.app/app-state.json"),
+ )
+ }
+ return [...new Set(candidates)]
+}
+
+async function resolveAppStatePath(): Promise<string | null> {
+ for (const candidate of appStateCandidates()) {
+ try {
+ await fs.access(candidate)
+ return candidate
+ } catch {
+ // try next candidate
+ }
+ }
+ return null
+}
+
+async function readAppState(): Promise<Record<string, unknown>> {
+ if (!appStatePath) throw new Error("app-state.json path is not available")
+ const raw = await fs.readFile(appStatePath, "utf8")
+ if (originalAppStateRaw === null) originalAppStateRaw = raw
+ return JSON.parse(raw) as Record<string, unknown>
+}
+
+async function writeApiConfig(config: {
+ enabled: boolean
+ allowUnauthenticated: boolean
+ token: string
+}): Promise<void> {
+ const state = await readAppState()
+ state.apiConfig = config
+ await fs.writeFile(appStatePath!, `${JSON.stringify(state, null, 2)}\n`, "utf8")
+ appStateMutated = true
+}
+
+async function waitForHealth(
+ predicate: (value: ApiHealth) => boolean,
+ label: string,
+ timeoutMs = 8_000,
+): Promise<ApiHealth> {
+ const deadline = Date.now() + timeoutMs
+ let last: ApiHealth | null = null
+ while (Date.now() < deadline) {
+ try {
+ last = await health()
+ if (predicate(last)) return last
+ } catch {
+ // keep polling
+ }
+ await new Promise((resolve) => setTimeout(resolve, 250))
+ }
+ throw new Error(`Timed out waiting for API health condition: ${label}. Last health: ${JSON.stringify(last)}`)
+}
+
+function ensureServer(ctx: { skip: () => void }) {
+ if (serverUnavailableReason) {
+ console.warn(serverUnavailableReason)
+ ctx.skip()
+ }
+}
+
+function ensureMutableAppState(ctx: { skip: () => void }) {
+ ensureServer(ctx)
+ if (!appStatePath) {
+ console.warn(
+ "Skipping API config mutation tests because app-state.json was not found. Set API_APP_STATE_PATH to run them.",
+ )
+ ctx.skip()
+ }
+}
+
+describe.skipIf(!ENABLED)("local API v1 against real project", () => {
+ beforeAll(async () => {
+ try {
+ initialHealth = await waitForHealth(() => true, "server reachable", 2_000)
+ } catch (err) {
+ serverUnavailableReason = `App is not running or the local API is unreachable at ${BASE_URL}. Launch LLM Wiki first, then re-run. ${String(err)}`
+ return
+ }
+ appStatePath = await resolveAppStatePath()
+ }, 5_000)
+
+ afterAll(async () => {
+ if (appStateMutated && appStatePath && originalAppStateRaw !== null) {
+ await fs.writeFile(appStatePath, originalAppStateRaw, "utf8")
+ await waitForHealth(
+ (h) =>
+ h.enabled === initialHealth?.enabled &&
+ h.allowUnauthenticated === initialHealth?.allowUnauthenticated,
+ "original API config restored",
+ ).catch((err) => {
+ console.warn("[api-real-test] failed to observe restored API config:", err)
+ })
+ }
+ }, 12_000)
+
+ it(
+ "reports server health and auth mode",
+ async (ctx) => {
+ ensureServer(ctx)
+ const h = await requireUsableApi()
+ expect(typeof h.authRequired).toBe("boolean")
+ expect(typeof h.authConfigured).toBe("boolean")
+ expect(typeof h.allowUnauthenticated).toBe("boolean")
+ expect(["env", "store", "none"]).toContain(h.tokenSource)
+ },
+ TEST_TIMEOUT_MS,
+ )
+
+ it(
+ "enforces the token auth matrix when auth is required",
+ async (ctx) => {
+ ensureMutableAppState(ctx)
+ await writeApiConfig({ enabled: true, allowUnauthenticated: false, token: TEST_TOKEN })
+ const h = await waitForHealth(
+ (value) => value.enabled === true && value.allowUnauthenticated === false && value.authRequired === true,
+ "auth required",
+ )
+ const effectiveToken = h.tokenSource === "env" ? API_TOKEN : TEST_TOKEN
+ if (!effectiveToken) {
+ console.warn("Skipping positive auth checks because the running app uses LLM_WIKI_API_TOKEN and API_TOKEN was not provided.")
+ }
+
+ const noToken = await api<ApiEnvelope>("/api/v1/projects", {}, { auth: "none" })
+ expect(noToken.status).toBe(401)
+
+ const wrongBearer = await api<ApiEnvelope>("/api/v1/projects", {}, { auth: "bearer", token: "wrong-token" })
+ expect(wrongBearer.status).toBe(401)
+
+ const wrongXToken = await api<ApiEnvelope>("/api/v1/projects", {}, { auth: "xToken", token: "wrong-token" })
+ expect(wrongXToken.status).toBe(401)
+
+ if (effectiveToken) {
+ const bearer = await api<ApiEnvelope & { projects: ApiProject[]; currentProject?: ApiProject | null }>(
+ "/api/v1/projects",
+ {},
+ { auth: "bearer", token: effectiveToken },
+ )
+ expect(bearer.status).toBe(200)
+ expect(Array.isArray(bearer.body.projects)).toBe(true)
+ expect(bearer.body.currentProject == null || bearer.body.currentProject.current).toBe(true)
+
+ const xToken = await api<ApiEnvelope & { projects: ApiProject[] }>(
+ "/api/v1/projects",
+ {},
+ { auth: "xToken", token: effectiveToken },
+ )
+ expect(xToken.status).toBe(200)
+
+ const query = await api<ApiEnvelope & { projects: ApiProject[] }>(
+ `/api/v1/projects?token=${encodeURIComponent(effectiveToken)}`,
+ {},
+ { auth: "none" },
+ )
+ expect(query.status).toBe(200)
+ }
+ },
+ TEST_TIMEOUT_MS,
+ )
+
+ it(
+ "honors the enabled=false kill switch",
+ async (ctx) => {
+ ensureMutableAppState(ctx)
+ await writeApiConfig({ enabled: false, allowUnauthenticated: false, token: TEST_TOKEN })
+ await waitForHealth((value) => value.enabled === false, "API disabled")
+ const resp = await api<ApiEnvelope>(
+ "/api/v1/projects",
+ {},
+ { auth: "bearer", token: API_TOKEN || TEST_TOKEN },
+ )
+ expect(resp.status).toBe(503)
+ expect(resp.body.error).toContain("disabled")
+ },
+ TEST_TIMEOUT_MS,
+ )
+
+ it(
+ "allows no-token access when unauthenticated mode is enabled",
+ async (ctx) => {
+ ensureMutableAppState(ctx)
+ await writeApiConfig({ enabled: true, allowUnauthenticated: true, token: "" })
+ await waitForHealth(
+ (value) => value.enabled === true && value.allowUnauthenticated === true && value.authRequired === false,
+ "unauthenticated mode",
+ )
+ const resp = await api<ApiEnvelope & { projects: ApiProject[]; currentProject?: ApiProject | null }>("/api/v1/projects", {}, { auth: "none" })
+ expect(resp.status).toBe(200)
+ expect(resp.body.ok).toBe(true)
+ expect(Array.isArray(resp.body.projects)).toBe(true)
+ expect(resp.body.currentProject == null || resp.body.currentProject.current).toBe(true)
+ },
+ TEST_TIMEOUT_MS,
+ )
+
+ it(
+ "lists projects and includes the target project id",
+ async (ctx) => {
+ ensureServer(ctx)
+ await requireUsableApi()
+ const { status, body } = await api<ApiEnvelope & { projects: ApiProject[]; currentProject?: ApiProject | null }>("/api/v1/projects")
+ expect(status).toBe(200)
+ expect(body.ok).toBe(true)
+ expect(Array.isArray(body.projects)).toBe(true)
+ const project = body.projects.find((p) => p.id === PROJECT_ID)
+ expect(project, `project ${PROJECT_ID} should be known to the API`).toBeTruthy()
+ expect(project!.path).toBeTruthy()
+ const currentFromList = body.projects.find((p) => p.current)
+ if (currentFromList) {
+ expect(body.currentProject).toMatchObject({
+ id: currentFromList.id,
+ path: currentFromList.path,
+ current: true,
+ })
+ } else {
+ expect(body.currentProject ?? null).toBeNull()
+ }
+ },
+ TEST_TIMEOUT_MS,
+ )
+
+ it(
+ "returns wiki, source, and public all-file trees",
+ async (ctx) => {
+ ensureServer(ctx)
+ await requireUsableApi()
+ for (const root of ["wiki", "sources", "all"] as const) {
+ const { status, body } = await api<ApiEnvelope & { files: ApiFileNode[] }>(
+ `/api/v1/projects/${PROJECT_ID}/files?root=${root}&recursive=true&maxFiles=10000`,
+ )
+ expect(status).toBe(200)
+ expect(body.ok).toBe(true)
+ expect(Array.isArray(body.files)).toBe(true)
+ }
+ },
+ TEST_TIMEOUT_MS,
+ )
+
+ it(
+ "reads real file content from the project",
+ async (ctx) => {
+ ensureServer(ctx)
+ await requireUsableApi()
+ const filesResp = await api<ApiEnvelope & { files: ApiFileNode[] }>(
+ `/api/v1/projects/${PROJECT_ID}/files?root=all&recursive=true&maxFiles=10000`,
+ )
+ expect(filesResp.status).toBe(200)
+ const file = firstTextFile(filesResp.body.files)
+ expect(file, "expected at least one text-like public project file").toBeTruthy()
+
+ const contentResp = await api<ApiEnvelope & { path: string; content: string }>(
+ `/api/v1/projects/${PROJECT_ID}/files/content?path=${encodeURIComponent(file!.path)}`,
+ )
+ expect(contentResp.status).toBe(200)
+ expect(contentResp.body.ok).toBe(true)
+ expect(contentResp.body.path).toBe(file!.path)
+ expect(typeof contentResp.body.content).toBe("string")
+ },
+ TEST_TIMEOUT_MS,
+ )
+
+ it(
+ "searches the real wiki and returns ranked results",
+ async (ctx) => {
+ ensureServer(ctx)
+ await requireUsableApi()
+ const filesResp = await api<ApiEnvelope & { files: ApiFileNode[] }>(
+ `/api/v1/projects/${PROJECT_ID}/files?root=wiki&recursive=true&maxFiles=10000`,
+ )
+ expect(filesResp.status).toBe(200)
+ const file = firstTextFile(filesResp.body.files)
+ expect(file, "expected at least one wiki text file for search seed").toBeTruthy()
+
+ const contentResp = await api<ApiEnvelope & { content: string }>(
+ `/api/v1/projects/${PROJECT_ID}/files/content?path=${encodeURIComponent(file!.path)}`,
+ )
+ expect(contentResp.status).toBe(200)
+ const query = searchQueryFromContent(contentResp.body.content, file!.name.replace(/\.[^.]+$/, ""))
+
+ const searchResp = await api<ApiEnvelope & { mode: string; results: SearchHit[] }>(
+ `/api/v1/projects/${PROJECT_ID}/search`,
+ {
+ method: "POST",
+ body: JSON.stringify({ query, topK: 10, includeContent: true }),
+ },
+ )
+ expect(searchResp.status).toBe(200)
+ expect(searchResp.body.ok).toBe(true)
+ expect(["keyword", "vector", "hybrid"]).toContain(searchResp.body.mode)
+ expect(searchResp.body.results.length).toBeGreaterThan(0)
+ expect(searchResp.body.results[0].score).toBeGreaterThan(0)
+ expect(searchResp.body.results[0].path).toBeTruthy()
+ expect(searchResp.body.results[0].content).toBeTruthy()
+ },
+ TEST_TIMEOUT_MS,
+ )
+
+ it(
+ "returns the real wiki graph",
+ async (ctx) => {
+ ensureServer(ctx)
+ await requireUsableApi()
+ const graphResp = await api<ApiEnvelope & { nodes: GraphNode[]; edges: GraphEdge[] }>(
+ `/api/v1/projects/${PROJECT_ID}/graph?limit=1000`,
+ )
+ expect(graphResp.status).toBe(200)
+ expect(graphResp.body.ok).toBe(true)
+ expect(Array.isArray(graphResp.body.nodes)).toBe(true)
+ expect(Array.isArray(graphResp.body.edges)).toBe(true)
+ expect(graphResp.body.nodes.length, "real project should have graph nodes").toBeGreaterThan(0)
+ expect(graphResp.body.nodes[0].id).toBeTruthy()
+ expect(graphResp.body.nodes[0].path).toMatch(/^wiki\//)
+ expect(typeof graphResp.body.nodes[0].linkCount).toBe("number")
+ if (graphResp.body.edges.length > 0) {
+ expect(typeof graphResp.body.edges[0].weight).toBe("number")
+ }
+ },
+ TEST_TIMEOUT_MS,
+ )
+
+ it(
+ "supports graph q filtering when a node exists",
+ async (ctx) => {
+ ensureServer(ctx)
+ await requireUsableApi()
+ const graphResp = await api<ApiEnvelope & { nodes: GraphNode[] }>(
+ `/api/v1/projects/${PROJECT_ID}/graph?limit=1000`,
+ )
+ expect(graphResp.status).toBe(200)
+ if (graphResp.body.nodes.length === 0) {
+ console.warn("Skipping graph q filter assertion because this project has no graph nodes.")
+ ctx.skip()
+ }
+ const q = encodeURIComponent(graphResp.body.nodes[0].label.slice(0, 4))
+ const filtered = await api<ApiEnvelope & { nodes: GraphNode[] }>(
+ `/api/v1/projects/${PROJECT_ID}/graph?q=${q}&limit=1000`,
+ )
+ expect(filtered.status).toBe(200)
+ expect(filtered.body.ok).toBe(true)
+ expect(filtered.body.nodes.length).toBeGreaterThan(0)
+ },
+ TEST_TIMEOUT_MS,
+ )
+
+ it(
+ "runs a real source rescan through the API",
+ async (ctx) => {
+ ensureServer(ctx)
+ await requireUsableApi()
+ const rescanResp = await api<ApiEnvelope & { result: RescanResult }>(
+ `/api/v1/projects/${PROJECT_ID}/sources/rescan`,
+ { method: "POST" },
+ )
+ expect(rescanResp.status).toBe(200)
+ expect(rescanResp.body.ok).toBe(true)
+ expect(rescanResp.body.result).toMatchObject({
+ queue: { version: expect.any(Number), tasks: expect.any(Array) },
+ changedTasks: expect.any(Array),
+ })
+ },
+ RESCAN_TIMEOUT_MS,
+ )
+
+ it(
+ "reports chat as not implemented in API v1",
+ async (ctx) => {
+ ensureServer(ctx)
+ await requireUsableApi()
+ const chatResp = await api<ApiEnvelope>(`/api/v1/projects/${PROJECT_ID}/chat`, {
+ method: "POST",
+ body: JSON.stringify({ message: "hello" }),
+ })
+ expect(chatResp.status).toBe(501)
+ expect(chatResp.body.ok).toBe(false)
+ },
+ TEST_TIMEOUT_MS,
+ )
+
+ it(
+ "rejects path traversal",
+ async (ctx) => {
+ ensureServer(ctx)
+ await requireUsableApi()
+ const traversal = await api<ApiEnvelope>(
+ `/api/v1/projects/${PROJECT_ID}/files/content?path=${encodeURIComponent("../app-state.json")}`,
+ )
+ expect(traversal.status).toBe(403)
+ },
+ TEST_TIMEOUT_MS,
+ )
+
+ it(
+ "rejects binary content reads with 415",
+ async (ctx) => {
+ ensureServer(ctx)
+ await requireUsableApi()
+ const filesResp = await api<ApiEnvelope & { files: ApiFileNode[] }>(
+ `/api/v1/projects/${PROJECT_ID}/files?root=all&recursive=true&maxFiles=10000`,
+ )
+ expect(filesResp.status).toBe(200)
+ const binaryFile = firstBinaryFile(filesResp.body.files)
+ if (!binaryFile) {
+ console.warn("Skipping binary content assertion because no wiki/media binary file is present.")
+ ctx.skip()
+ }
+
+ const binary = await api<ApiEnvelope>(
+ `/api/v1/projects/${PROJECT_ID}/files/content?path=${encodeURIComponent(binaryFile!.path)}`,
+ )
+ expect(binary.status).toBe(415)
+ },
+ TEST_TIMEOUT_MS,
+ )
+
+ it(
+ "rejects empty search requests with 400",
+ async (ctx) => {
+ ensureServer(ctx)
+ await requireUsableApi()
+ const badSearch = await api<ApiEnvelope>(`/api/v1/projects/${PROJECT_ID}/search`, {
+ method: "POST",
+ body: JSON.stringify({ query: "" }),
+ })
+ expect(badSearch.status).toBe(400)
+ },
+ TEST_TIMEOUT_MS,
+ )
+
+ it(
+ "returns 404 for unknown project ids",
+ async (ctx) => {
+ ensureServer(ctx)
+ await requireUsableApi()
+ const unknownProject = await api<ApiEnvelope>(
+ `/api/v1/projects/00000000-0000-0000-0000-000000000000/files`,
+ )
+ expect(unknownProject.status).toBe(404)
+ },
+ TEST_TIMEOUT_MS,
+ )
+
+ it(
+ "returns 404 for missing routes",
+ async (ctx) => {
+ ensureServer(ctx)
+ await requireUsableApi()
+ const missingRoute = await api<ApiEnvelope>("/api/v1/not-a-route")
+ expect(missingRoute.status).toBe(404)
+ },
+ TEST_TIMEOUT_MS,
+ )
+
+ it(
+ "returns 405 for unsupported methods on API routes",
+ async (ctx) => {
+ ensureServer(ctx)
+ await requireUsableApi()
+ const resp = await api<ApiEnvelope>("/api/v1/projects", { method: "PUT" })
+ expect(resp.status).toBe(405)
+ },
+ TEST_TIMEOUT_MS,
+ )
+
+ it(
+ "rejects invalid file roots with 400",
+ async (ctx) => {
+ ensureServer(ctx)
+ await requireUsableApi()
+ const resp = await api<ApiEnvelope>(`/api/v1/projects/${PROJECT_ID}/files?root=invalid`)
+ expect(resp.status).toBe(400)
+ },
+ TEST_TIMEOUT_MS,
+ )
+
+ it(
+ "supports current and filesystem-path project identifiers",
+ async (ctx) => {
+ ensureServer(ctx)
+ await requireUsableApi()
+ const projectsResp = await api<ApiEnvelope & { projects: ApiProject[]; currentProject?: ApiProject | null }>("/api/v1/projects")
+ expect(projectsResp.status).toBe(200)
+ const target = projectsResp.body.projects.find((p) => p.id === PROJECT_ID)
+ expect(target).toBeTruthy()
+
+ const byPath = await api<ApiEnvelope & { files: ApiFileNode[] }>(
+ `/api/v1/projects/${encodeURIComponent(target!.path)}/files?root=wiki&recursive=false`,
+ )
+ expect(byPath.status).toBe(200)
+
+ const current = projectsResp.body.projects.find((p) => p.current)
+ if (!current) {
+ console.warn("Skipping current project id assertion because no current project is registered.")
+ ctx.skip()
+ }
+ expect(projectsResp.body.currentProject).toMatchObject({
+ id: current!.id,
+ path: current!.path,
+ current: true,
+ })
+ const currentResp = await api<ApiEnvelope & { files: ApiFileNode[] }>(
+ "/api/v1/projects/current/files?root=wiki&recursive=false",
+ )
+ expect(currentResp.status).toBe(200)
+
+ const currentCaseResp = await api<ApiEnvelope & { files: ApiFileNode[] }>(
+ "/api/v1/projects/Current/files?root=wiki&recursive=false",
+ )
+ expect(currentCaseResp.status).toBe(200)
+ },
+ TEST_TIMEOUT_MS,
+ )
+
+ it(
+ "clamps oversized search and graph limits",
+ async (ctx) => {
+ ensureServer(ctx)
+ await requireUsableApi()
+ const searchResp = await api<ApiEnvelope & { results: SearchHit[] }>(
+ `/api/v1/projects/${PROJECT_ID}/search`,
+ {
+ method: "POST",
+ body: JSON.stringify({ query: "a", topK: 999, includeContent: false }),
+ },
+ )
+ expect(searchResp.status).toBe(200)
+ expect(searchResp.body.results.length).toBeLessThanOrEqual(50)
+
+ const graphResp = await api<ApiEnvelope & { nodes: GraphNode[] }>(
+ `/api/v1/projects/${PROJECT_ID}/graph?limit=999999`,
+ )
+ expect(graphResp.status).toBe(200)
+ expect(graphResp.body.nodes.length).toBeLessThanOrEqual(1000)
+ },
+ TEST_TIMEOUT_MS,
+ )
+
+ it(
+ "returns 400 for oversized request bodies",
+ async (ctx) => {
+ ensureServer(ctx)
+ await requireUsableApi()
+ const hugeQuery = "x".repeat(1024 * 1024 + 1)
+ const resp = await api<ApiEnvelope>(`/api/v1/projects/${PROJECT_ID}/search`, {
+ method: "POST",
+ body: JSON.stringify({ query: hugeQuery }),
+ })
+ expect(resp.status).toBe(400)
+ },
+ TEST_TIMEOUT_MS,
+ )
+
+ it(
+ "returns 413 when the requested file tree exceeds maxFiles",
+ async (ctx) => {
+ ensureServer(ctx)
+ await requireUsableApi()
+ const resp = await api<ApiEnvelope & { files?: ApiFileNode[] }>(
+ `/api/v1/projects/${PROJECT_ID}/files?root=all&recursive=true&maxFiles=1`,
+ )
+ if (resp.status !== 413) {
+ console.warn("Skipping maxFiles 413 assertion because this project has one or fewer public nodes.")
+ ctx.skip()
+ }
+ expect(resp.status).toBe(413)
+ },
+ TEST_TIMEOUT_MS,
+ )
+})
diff --git a/src/lib/api-token.test.ts b/src/lib/api-token.test.ts
new file mode 100644
index 00000000..565b7981
--- /dev/null
+++ b/src/lib/api-token.test.ts
@@ -0,0 +1,26 @@
+import { describe, expect, it } from "vitest"
+import { generateApiToken } from "@/lib/api-token"
+
+describe("generateApiToken", () => {
+ it("produces URL-safe base64 with no padding", () => {
+ for (let i = 0; i < 32; i++) {
+ const token = generateApiToken()
+ expect(token).toMatch(/^[A-Za-z0-9_-]+$/)
+ expect(token).not.toContain("=")
+ expect(token).not.toContain("+")
+ expect(token).not.toContain("/")
+ }
+ })
+
+ it("emits at least 32 bytes of entropy (>= 43 chars base64url)", () => {
+ const token = generateApiToken()
+ // 32 bytes -> 43 chars unpadded base64url.
+ expect(token.length).toBeGreaterThanOrEqual(43)
+ })
+
+ it("is overwhelmingly unique across calls", () => {
+ const tokens = new Set<string>()
+ for (let i = 0; i < 100; i++) tokens.add(generateApiToken())
+ expect(tokens.size).toBe(100)
+ })
+})
diff --git a/src/lib/api-token.ts b/src/lib/api-token.ts
new file mode 100644
index 00000000..1161b87b
--- /dev/null
+++ b/src/lib/api-token.ts
@@ -0,0 +1,23 @@
+/**
+ * Generate a cryptographically random API token using the Web Crypto API.
+ *
+ * Uses 32 bytes (256 bits) of randomness and base64url-encodes them so the
+ * result is URL-safe and free of `+` / `/` / `=` — paste-safe into shell,
+ * env vars, headers, query strings, and chat tools without escaping.
+ */
+export function generateApiToken(): string {
+ const bytes = new Uint8Array(32)
+ crypto.getRandomValues(bytes)
+ return base64UrlEncode(bytes)
+}
+
+function base64UrlEncode(bytes: Uint8Array): string {
+ let binary = ""
+ for (let i = 0; i < bytes.length; i++) {
+ binary += String.fromCharCode(bytes[i])
+ }
+ return btoa(binary)
+ .replace(/\+/g, "-")
+ .replace(/\//g, "_")
+ .replace(/=+$/, "")
+}
diff --git a/src/lib/project-store.ts b/src/lib/project-store.ts
index b5f2e978..bfcb10ae 100644
--- a/src/lib/project-store.ts
+++ b/src/lib/project-store.ts
@@ -1,6 +1,6 @@
import { load } from "@tauri-apps/plugin-store"
import type { WikiProject } from "@/types/wiki"
-import type { LlmConfig, SearchApiConfig, EmbeddingConfig, MultimodalConfig, OutputLanguage, ProviderConfigs, ProxyConfig, ScheduledImportConfig, SourceWatchConfig } from "@/stores/wiki-store"
+import type { ApiConfig, LlmConfig, SearchApiConfig, EmbeddingConfig, MultimodalConfig, OutputLanguage, ProviderConfigs, ProxyConfig, ScheduledImportConfig, SourceWatchConfig } from "@/stores/wiki-store"
import { normalizeSourceWatchConfig } from "@/lib/source-watch-config"
import { normalizePath } from "@/lib/path-utils"
@@ -135,6 +135,28 @@ export async function loadProxyConfig(): Promise<ProxyConfig | null> {
return (await store.get<ProxyConfig>(PROXY_CONFIG_KEY)) ?? null
}
+// Local API server config. KEY MUST stay `apiConfig` — the Rust
+// `api_server` module reads `parsed.get("apiConfig")` from this same
+// `app-state.json` on every request (5s cache). Rename one side and
+// the API silently goes back to "no token configured = 401 forever".
+const API_CONFIG_KEY = "apiConfig"
+
+export async function saveApiConfig(config: ApiConfig): Promise<void> {
+ const store = await getStore()
+ await store.set(API_CONFIG_KEY, config)
+ // Force-flush. The 100ms debounce default is fine for cosmetic
+ // settings, but the API token is on a security hot path — a user
+ // generates one, hits Save, then immediately curls the API from
+ // another terminal. We want the disk file to match in-memory
+ // state before the next request reads it.
+ await store.save()
+}
+
+export async function loadApiConfig(): Promise<ApiConfig | null> {
+ const store = await getStore()
+ return (await store.get<ApiConfig>(API_CONFIG_KEY)) ?? null
+}
+
const SCHEDULED_IMPORT_KEY_PREFIX = "scheduledImportConfig:"
function scheduledImportKey(projectPath: string): string {
diff --git a/src/lib/search-rrf.test.ts b/src/lib/search-rrf.test.ts
index ebd24c26..5f6fc282 100644
--- a/src/lib/search-rrf.test.ts
+++ b/src/lib/search-rrf.test.ts
@@ -1,58 +1,22 @@
/**
- * Hybrid retrieval (RRF) tests for searchWiki.
- *
- * search.scenarios.test.ts disables embedding to test token search in
- * isolation. THIS file does the inverse: scripts both token search
- * input (real materialized wiki dir) AND vector search output (via
- * a stubbed searchByEmbedding) so we can pin the exact RRF math
- * and ranking semantics.
- *
- * Why RRF matters and what these tests guard:
- * Before this refactor, vector results contributed `vr.score * 5`
- * to the existing token score. Token-only signals (FILENAME_EXACT
- * = 200, PHRASE_IN_TITLE = 50, …) drowned out the cosine sim ≤ 1.0
- * range, so a page with NO keyword overlap but perfect semantic
- * match would max out at score 5 and never make top-10 against any
- * keyword-matching page. This file pins that this is no longer true.
+ * The RRF scoring implementation now lives in Rust
+ * (`commands::search`). These TS tests only guard the WebView wrapper:
+ * it should pass embedding config to the shared backend command and map
+ * backend-relative result paths back to absolute project paths for the editor.
*/
-import { describe, it, expect, vi, beforeEach } from "vitest"
-import path from "node:path"
-import fs from "node:fs/promises"
-import { realFs, createTempProject } from "@/test-helpers/fs-temp"
+import { beforeEach, describe, expect, it, vi } from "vitest"
+import { useWikiStore } from "@/stores/wiki-store"
-vi.mock("@/commands/fs", () => realFs)
+const mockInvoke = vi.fn()
-// Stub searchByEmbedding so each test scripts the exact ranked list
-// of vector results coming from LanceDB, with no real model calls.
-const mockSearchByEmbedding =
- vi.fn<(...args: unknown[]) => Promise<Array<{ id: string; score: number }>>>()
-vi.mock("./embedding", () => ({
- searchByEmbedding: (...args: unknown[]) => mockSearchByEmbedding(...args),
+vi.mock("@tauri-apps/api/core", () => ({
+ invoke: (...args: unknown[]) => mockInvoke(...args),
}))
-import { searchWiki } from "./search"
-import { useWikiStore } from "@/stores/wiki-store"
-
-interface Ctx {
- tmp: { path: string; cleanup: () => Promise<void> }
-}
-let ctx: Ctx | undefined
-
-/** Lay out a small project on disk: just the wiki/* files we need. */
-async function setupProject(files: Record<string, string>): Promise<Ctx> {
- const tmp = await createTempProject("search-rrf")
- for (const [rel, content] of Object.entries(files)) {
- const full = path.join(tmp.path, rel)
- await fs.mkdir(path.dirname(full), { recursive: true })
- await fs.writeFile(full, content, "utf-8")
- }
- return { tmp }
-}
+import { searchWiki, tokenizeQuery } from "./search"
beforeEach(() => {
- mockSearchByEmbedding.mockReset()
- // Default: embedding ENABLED (the whole point of this file). Tests
- // that want token-only override locally.
+ mockInvoke.mockReset()
useWikiStore.getState().setEmbeddingConfig({
enabled: true,
endpoint: "http://test/v1/embeddings",
@@ -61,193 +25,66 @@ beforeEach(() => {
})
})
-afterEach(async () => {
- if (ctx) {
- await ctx.tmp.cleanup()
- ctx = undefined
- }
-})
-
-// Vitest's `afterEach` import — declare here to keep the import block
-// near the top tidy and use the same lifecycle hooks shape as
-// search.scenarios.test.ts.
-import { afterEach } from "vitest"
-
-describe("searchWiki — RRF fusion of token + vector lists", () => {
- it("a vector-only match (no keyword overlap) surfaces ahead of a token-only weak match", async () => {
- // The query mentions "memory bandwidth"; flash-attention.md doesn't
- // contain that phrase verbatim but is semantically about it. A
- // separate page literally contains the word "memory" in unrelated
- // context. Old behavior: token-match page wins. RRF behavior:
- // vector match wins because it ranks #1 in vector list while the
- // token match ranks deep in token list.
- ctx = await setupProject({
- "wiki/concepts/flash-attention.md":
- "---\ntitle: Flash Attention\n---\n\n# Flash Attention\n\nIO-aware tiled attention.",
- "wiki/concepts/memory-leak.md":
- "---\ntitle: Memory Leak Diagnosis\n---\n\n# Memory Leak\n\nRSS growing over time.",
+describe("searchWiki backend wrapper", () => {
+ it("passes embeddingConfig to the shared backend search command and absolutizes paths", async () => {
+ mockInvoke.mockResolvedValueOnce({
+ mode: "hybrid",
+ tokenHits: 1,
+ vectorHits: 1,
+ results: [
+ {
+ path: "wiki/concepts/attention.md",
+ title: "Attention",
+ snippet: "Attention",
+ titleMatch: true,
+ score: 1 / 61,
+ images: [],
+ },
+ ],
})
- // Token search will find "memory-leak" (literal "Memory" hit) and
- // possibly nothing for flash-attention. Vector search ranks
- // flash-attention #1.
- mockSearchByEmbedding.mockResolvedValueOnce([
- { id: "flash-attention", score: 0.85 },
- { id: "memory-leak", score: 0.4 },
- ])
+ const out = await searchWiki("/tmp/project", "attention")
- const out = await searchWiki(ctx.tmp.path, "GPU memory bandwidth optimization for attention")
-
- // The vector match must be in the top result. Without RRF, the
- // keyword-only "memory-leak" page would dominate.
- expect(out[0].title).toBe("Flash Attention")
- // Both pages still appear (RRF surfaces both lists' contents).
- const titles = out.map((r) => r.title)
- expect(titles).toContain("Memory Leak Diagnosis")
- })
-
- it("a page in BOTH lists outranks a page in only one (combined contribution)", async () => {
- // Construct three pages:
- // - A: matches token search (rank 1) AND vector search (rank 1)
- // - B: only matches token search (rank 1's neighbor at rank 2)
- // - C: only matches vector search (rank 2)
- //
- // RRF expectation:
- // A: 1/61 + 1/61 = 0.0328
- // B: 1/62 + 0 = 0.0161
- // C: 0 + 1/62 = 0.0161
- //
- // → A wins clearly; B and C tie (broken by alphabetical path).
- ctx = await setupProject({
- "wiki/concepts/aaa.md": "---\ntitle: AAA\n---\n\n# AAA\n\nrope rope rope.",
- "wiki/concepts/bbb.md": "---\ntitle: BBB\n---\n\n# BBB\n\nrope.",
- "wiki/concepts/ccc.md": "---\ntitle: CCC\n---\n\n# CCC\n\nunrelated.",
+ expect(mockInvoke).toHaveBeenCalledWith("search_project", {
+ projectPath: "/tmp/project",
+ query: "attention",
+ topK: 20,
+ includeContent: false,
+ queryEmbedding: null,
+ embeddingConfig: expect.objectContaining({ enabled: true, model: "test-embed" }),
})
-
- mockSearchByEmbedding.mockResolvedValueOnce([
- { id: "aaa", score: 0.95 },
- { id: "ccc", score: 0.7 },
- ])
-
- const out = await searchWiki(ctx.tmp.path, "rope")
-
- expect(out[0].title).toBe("AAA")
- // Score: aaa got contributions from BOTH; bbb and ccc each from ONE.
- expect(out[0].score).toBeCloseTo(1 / 61 + 1 / 61, 6)
- // The runner-up is bbb or ccc — both at rank 2 of their list, both
- // contributing 1/62. Tie broken by path → alphabetical. concepts/bbb
- // < concepts/ccc, so bbb is #2.
- expect(out[1].title).toBe("BBB")
- expect(out[2].title).toBe("CCC")
- expect(out[1].score).toBeCloseTo(1 / 62, 6)
- expect(out[2].score).toBeCloseTo(1 / 62, 6)
+ expect(out[0].path).toBe("/tmp/project/wiki/concepts/attention.md")
})
- it("vector list is empty (embedding disabled) → behaves like pure token rank", async () => {
+ it("passes disabled embedding config through for backend keyword-only search", async () => {
useWikiStore.getState().setEmbeddingConfig({
enabled: false,
endpoint: "",
apiKey: "",
model: "",
})
-
- ctx = await setupProject({
- "wiki/concepts/attention.md":
- "---\ntitle: Attention\n---\n\n# Attention\n\nbody about attention.",
- "wiki/concepts/random.md":
- "---\ntitle: Random\n---\n\n# Random\n\nattention is mentioned briefly.",
+ mockInvoke.mockResolvedValueOnce({
+ mode: "keyword",
+ tokenHits: 1,
+ vectorHits: 0,
+ results: [],
})
- const out = await searchWiki(ctx.tmp.path, "attention")
- // Token search puts attention.md (filename exact match → 200 bonus)
- // at rank 1, random.md at rank 2.
- expect(out[0].title).toBe("Attention")
- expect(out[0].score).toBeCloseTo(1 / 61, 6)
- expect(out[1].title).toBe("Random")
- expect(out[1].score).toBeCloseTo(1 / 62, 6)
- // searchByEmbedding must NOT have been called at all.
- expect(mockSearchByEmbedding).not.toHaveBeenCalled()
- })
-
- it("token list empty (no keyword match anywhere) → vector ranks alone determine order", async () => {
- ctx = await setupProject({
- "wiki/concepts/foo.md": "---\ntitle: Foo\n---\n\n# Foo\n\nfoo body.",
- "wiki/concepts/bar.md": "---\ntitle: Bar\n---\n\n# Bar\n\nbar body.",
- })
-
- // Query has no token overlap with either page; vector finds them.
- mockSearchByEmbedding.mockResolvedValueOnce([
- { id: "bar", score: 0.7 },
- { id: "foo", score: 0.3 },
- ])
+ await searchWiki("/tmp/project", "attention")
- const out = await searchWiki(ctx.tmp.path, "completely unrelated query terms")
- // Order matches vector ranks: bar (rank 1) > foo (rank 2).
- expect(out.map((r) => r.title)).toEqual(["Bar", "Foo"])
- expect(out[0].score).toBeCloseTo(1 / 61, 6)
- expect(out[1].score).toBeCloseTo(1 / 62, 6)
+ expect(mockInvoke).toHaveBeenCalledWith(
+ "search_project",
+ expect.objectContaining({
+ queryEmbedding: null,
+ embeddingConfig: expect.objectContaining({ enabled: false }),
+ }),
+ )
})
- it("vector contribution is rank-based, NOT score-magnitude — fixes the score-dwarfing bug", async () => {
- // This is the precise regression we're guarding. Old code:
- // existing.score += vr.score * 5
- // → vector cos_sim 0.95 boosts by only 4.75
- // → buried under any token PHRASE_IN_TITLE_BONUS (50)
- //
- // Build: page A is the ONLY token match (low-quality match,
- // rank 1 of token but at the absolute floor of token scoring);
- // page B is vector rank 1 (no token hits at all).
- // Expected: B (in top-1 of vector) MUST appear at the top
- // alongside A — under the old behavior, B never made it in.
- ctx = await setupProject({
- "wiki/concepts/quick-mention.md":
- "---\ntitle: Quick Mention\n---\n\n# Quick Mention\n\nrope is mentioned once.",
- "wiki/concepts/positional-encoding.md":
- "---\ntitle: Positional Encoding\n---\n\n# Positional Encoding\n\nstrong semantic match.",
- })
-
- // Token search will hit "rope" only in quick-mention.md.
- // Vector search puts positional-encoding at rank 1.
- mockSearchByEmbedding.mockResolvedValueOnce([
- { id: "positional-encoding", score: 0.92 },
- ])
-
- const out = await searchWiki(ctx.tmp.path, "rope")
-
- // Both must be present.
- const ids = out.map((r) => r.path.split("/").pop())
- expect(ids).toContain("quick-mention.md")
- expect(ids).toContain("positional-encoding.md")
- // positional-encoding contributes 1/61 (vector rank 1) and
- // quick-mention contributes 1/61 (token rank 1). They tie on
- // score; alphabetical tie-break gives positional-encoding (p < q).
- expect(out[0].title).toBe("Positional Encoding")
- expect(out[1].title).toBe("Quick Mention")
- // Both scores should be 1/61 — proving vector rank 1 carries
- // the same weight as token rank 1, NOT 5/200th of it.
- expect(out[0].score).toBeCloseTo(1 / 61, 6)
- expect(out[1].score).toBeCloseTo(1 / 61, 6)
- })
-
- it("vector results referencing nonexistent page ids are silently dropped (post-source-delete safety)", async () => {
- // After a source-delete cascade, LanceDB *should* have its chunks
- // gone too (we just fixed that bug), but be defensive: if vector
- // search returns an id whose .md file doesn't exist, the search
- // pipeline must not crash and must not surface a phantom result.
- ctx = await setupProject({
- "wiki/concepts/exists.md":
- "---\ntitle: Exists\n---\n\n# Exists\n\nbody.",
- })
-
- mockSearchByEmbedding.mockResolvedValueOnce([
- { id: "exists", score: 0.9 },
- { id: "ghost", score: 0.85 }, // no file with this slug
- ])
-
- const out = await searchWiki(ctx.tmp.path, "anything")
- // Only the real page surfaces. Ghost is dropped during
- // materialization (every `tryPath` readFile throws ENOENT).
- expect(out).toHaveLength(1)
- expect(out[0].title).toBe("Exists")
+ it("keeps CJK tokenization behavior for image caption filtering", () => {
+ const tokens = tokenizeQuery("默会知识")
+ expect(tokens).toContain("默会")
+ expect(tokens).toContain("知识")
+ expect(tokens).toContain("默")
})
})
diff --git a/src/lib/search.scenarios.test.ts b/src/lib/search.scenarios.test.ts
index 19c85a63..d745bdb8 100644
--- a/src/lib/search.scenarios.test.ts
+++ b/src/lib/search.scenarios.test.ts
@@ -1,40 +1,22 @@
/**
- * Scenario-driven tests for searchWiki.
- *
- * Each scenario materializes a wiki dir + a query string, then asserts
- * the ranked result list from searchWiki.
+ * Search ranking scenarios moved to Rust with the shared backend search
+ * service (`src-tauri/src/commands/search.rs`). The WebView now only
+ * wraps that command, so this file guards the command contract from
+ * the TS side instead of duplicating ranking logic in Node.
*/
-import { describe, it, expect, beforeAll, beforeEach, afterEach, vi } from "vitest"
-import path from "node:path"
-import fs from "node:fs/promises"
-import { realFs, createTempProject } from "@/test-helpers/fs-temp"
-import { materializeScenario, copyDir } from "@/test-helpers/scenarios/materialize"
-import { searchScenarios } from "@/test-helpers/scenarios/search-scenarios"
-import type { SearchScenario } from "@/test-helpers/scenarios/types"
-
-vi.mock("@/commands/fs", () => realFs)
-
-import { searchWiki } from "./search"
+import { beforeEach, describe, expect, it, vi } from "vitest"
import { useWikiStore } from "@/stores/wiki-store"
-const FIXTURES_ROOT = path.join(
- process.cwd(),
- "tests",
- "fixtures",
- "scenarios-search",
-)
+const mockInvoke = vi.fn()
-beforeAll(async () => {
- await fs.rm(FIXTURES_ROOT, { recursive: true, force: true })
- await fs.mkdir(FIXTURES_ROOT, { recursive: true })
- for (const s of searchScenarios) {
- await materializeScenario(s, FIXTURES_ROOT)
- }
-})
+vi.mock("@tauri-apps/api/core", () => ({
+ invoke: (...args: unknown[]) => mockInvoke(...args),
+}))
+
+import { searchWiki } from "./search"
beforeEach(() => {
- // Disable embedding/vector search — we only want to exercise the
- // BM25-style text search path.
+ mockInvoke.mockReset()
useWikiStore.getState().setEmbeddingConfig({
enabled: false,
endpoint: "",
@@ -43,94 +25,34 @@ beforeEach(() => {
})
})
-interface Ctx {
- tmp: { path: string; cleanup: () => Promise<void> }
-}
-let ctx: Ctx | undefined
-
-async function setup(scenario: SearchScenario): Promise<Ctx> {
- const tmp = await createTempProject(
- `search-${scenario.name.replace(/\//g, "-")}`,
- )
- const initialWikiDir = path.join(FIXTURES_ROOT, scenario.name, "initial-wiki")
- await copyDir(initialWikiDir, tmp.path)
- return { tmp }
-}
-
-afterEach(async () => {
- if (ctx) {
- await ctx.tmp.cleanup()
- ctx = undefined
- }
-})
-
-describe("search scenarios (fixture-driven)", () => {
- it.each(searchScenarios.map((s) => [s.name, s]))(
- "%s",
- async (_name, scenario) => {
- ctx = await setup(scenario)
-
- const results = await searchWiki(ctx.tmp.path, scenario.query)
-
- // Helper: does a scenario-relative path (e.g. "wiki/attention.md")
- // match any result in the list?
- function resultFor(relPath: string) {
- // Results have absolute paths; compare by suffix-ending-in-relPath
- return results.find((r) =>
- r.path.endsWith("/" + relPath) || r.path.endsWith(relPath),
- )
- }
-
- try {
- // 1. Top results in the expected order
- for (let i = 0; i < scenario.expected.topResultPaths.length; i++) {
- const expectedPath = scenario.expected.topResultPaths[i]
- const actual = results[i]
- expect(
- actual,
- `result #${i} missing (expected ${expectedPath}); got ${results.length} results`,
- ).toBeTruthy()
- expect(
- actual.path.endsWith("/" + expectedPath) ||
- actual.path.endsWith(expectedPath),
- `result #${i} path mismatch — expected ending with ${expectedPath}, got ${actual.path}`,
- ).toBe(true)
- }
-
- // 2. Excluded paths
- if (scenario.expected.excludedPaths) {
- for (const excluded of scenario.expected.excludedPaths) {
- expect(
- resultFor(excluded),
- `excluded path found in results: ${excluded}`,
- ).toBeFalsy()
- }
- }
-
- // 3. Paths that must have titleMatch=true
- if (scenario.expected.titleMatchPaths) {
- for (const p of scenario.expected.titleMatchPaths) {
- const r = resultFor(p)
- expect(r, `title-match page ${p} missing from results`).toBeTruthy()
- expect(r!.titleMatch, `${p} should have titleMatch=true`).toBe(true)
- }
- }
- } catch (err) {
- // eslint-disable-next-line no-console
- console.error(
- `\n[search: ${scenario.name}] query="${scenario.query}" — actual results:\n` +
- JSON.stringify(
- results.map((r) => ({
- path: r.path,
- titleMatch: r.titleMatch,
- score: r.score,
- })),
- null,
- 2,
- ),
- )
- throw err
- }
- },
- )
+describe("searchWiki backend command contract", () => {
+ it("delegates ranking to search_project and maps relative wiki paths to absolute paths", async () => {
+ mockInvoke.mockResolvedValueOnce({
+ mode: "keyword",
+ tokenHits: 1,
+ vectorHits: 0,
+ results: [
+ {
+ path: "wiki/concepts/attention.md",
+ title: "Attention",
+ snippet: "body",
+ titleMatch: true,
+ score: 1 / 61,
+ images: [],
+ },
+ ],
+ })
+
+ const results = await searchWiki("/tmp/project", "attention")
+
+ expect(mockInvoke).toHaveBeenCalledWith("search_project", {
+ projectPath: "/tmp/project",
+ query: "attention",
+ topK: 20,
+ includeContent: false,
+ queryEmbedding: null,
+ embeddingConfig: expect.objectContaining({ enabled: false }),
+ })
+ expect(results[0].path).toBe("/tmp/project/wiki/concepts/attention.md")
+ })
})
diff --git a/src/lib/search.ts b/src/lib/search.ts
index 967e4880..a73f81c4 100644
--- a/src/lib/search.ts
+++ b/src/lib/search.ts
@@ -1,17 +1,7 @@
-import { readFile, listDirectory } from "@/commands/fs"
-import type { FileNode } from "@/types/wiki"
-import { normalizePath, getFileStem } from "@/lib/path-utils"
+import { invoke } from "@tauri-apps/api/core"
+import { normalizePath } from "@/lib/path-utils"
+import { useWikiStore } from "@/stores/wiki-store"
-/**
- * One image reference extracted from a matched page's markdown.
- *
- * `url` is verbatim what was inside the `![](...)` parens — this is
- * a forward-slash path that the markdown image resolver knows how
- * to map to a renderable URL. We deliberately keep it pre-resolution
- * so the search-result UI can filter by URL prefix (e.g. "only show
- * images from this source's media dir") before paying the cost of
- * `convertFileSrc`.
- */
export interface ImageRef {
url: string
alt: string
@@ -23,59 +13,18 @@ export interface SearchResult {
snippet: string
titleMatch: boolean
score: number
- /**
- * Image references found inside this result's markdown. Populated
- * even when the query doesn't match the alt text — the UI splits
- * "alt-matches-query" from "image just lives on this matched
- * page" itself, so both views need the full set. May be empty.
- */
+ vectorScore?: number
images: ImageRef[]
}
-const MAX_RESULTS = 20
-const SNIPPET_CONTEXT = 80
-
-// ── Reciprocal Rank Fusion ─────────────────────────────────────────────────
-// Token search and vector search produce two independently-ranked lists.
-// Their absolute scores are incommensurable (token score: 1-400, vector
-// cosine: 0-1), so summing them privileges whichever list happens to use
-// the larger numbers. RRF sidesteps that by fusing on RANK only:
-//
-// fused(p) = sum over lists L of 1 / (K + rank_L(p))
-//
-// A page that ranks #1 in BOTH lists wins handily. A page that's only in
-// one list still surfaces if it ranks high there, but a page in BOTH a
-// little lower can outrank it — exactly what we want for hybrid retrieval.
-//
-// K=60 is the canonical constant from Cormack et al. (SIGIR 2009), large
-// enough that small rank differences near the top don't dominate but
-// small enough that being deep in either list still falls off quickly.
-const RRF_K = 60
-
-// ── Scoring weights ────────────────────────────────────────────────────────
-// Exact lexical matches dominate everything else. The rationale: when a
-// user types "attention", the page literally named `attention.md` MUST
-// rank first, regardless of how many other pages also mention the word.
-//
-// filename == query (e.g. `attention.md` for query "attention")
-// → FILENAME_EXACT_BONUS — large enough that nothing short of an
-// equally-exact match can outrank it.
-//
-// title or content contains the raw query as a substring
-// → PHRASE_IN_TITLE_BONUS / PHRASE_IN_CONTENT_PER_OCC — phrase
-// presence is worth far more than individual token presence, and
-// in content it rewards repetition (with a cap to avoid runaway).
-//
-// per-token matches (existing behavior, but now smaller weight)
-// → TITLE_TOKEN_WEIGHT / CONTENT_TOKEN_WEIGHT. These used to
-// dominate via a flat +10 title bonus regardless of how many
-// tokens matched; now each matched token counts individually.
-const FILENAME_EXACT_BONUS = 200
-const PHRASE_IN_TITLE_BONUS = 50
-const PHRASE_IN_CONTENT_PER_OCC = 20
-const MAX_PHRASE_OCC_COUNTED = 10 // cap to avoid runaway on huge logs
-const TITLE_TOKEN_WEIGHT = 5
-const CONTENT_TOKEN_WEIGHT = 1
+interface BackendSearchResponse {
+ // Reserved for result badges/debug UI. The backend already returns these
+ // signals so API and WebView search share the same retrieval contract.
+ mode: "keyword" | "vector" | "hybrid"
+ results: SearchResult[]
+ tokenHits: number
+ vectorHits: number
+}
const STOP_WORDS = new Set([
"的", "是", "了", "什么", "在", "有", "和", "与", "对", "从",
@@ -86,7 +35,6 @@ const STOP_WORDS = new Set([
])
export function tokenizeQuery(query: string): string[] {
- // Split by whitespace and punctuation
const rawTokens = query
.toLowerCase()
.split(/[\s,,。!?、;:""''()()\-_/\\·~~…]+/)
@@ -94,402 +42,41 @@ export function tokenizeQuery(query: string): string[] {
.filter((t) => !STOP_WORDS.has(t))
const tokens: string[] = []
-
for (const token of rawTokens) {
- // Check if token contains CJK characters
const hasCJK = /[\u4e00-\u9fff\u3400-\u4dbf]/.test(token)
-
if (hasCJK && token.length > 2) {
- // For CJK text: split into individual characters AND overlapping bigrams
- // "默会知识" → ["默会", "会知", "知识", "默", "会", "知", "识"]
const chars = [...token]
- // Add bigrams (most useful for Chinese)
- for (let i = 0; i < chars.length - 1; i++) {
- tokens.push(chars[i] + chars[i + 1])
- }
- // Also add individual chars (for single-char matches)
+ for (let i = 0; i < chars.length - 1; i++) tokens.push(chars[i] + chars[i + 1])
for (const ch of chars) {
- if (!STOP_WORDS.has(ch)) {
- tokens.push(ch)
- }
+ if (!STOP_WORDS.has(ch)) tokens.push(ch)
}
- // Keep the original token too (for exact phrase match)
tokens.push(token)
} else {
tokens.push(token)
}
}
-
- // Deduplicate
return [...new Set(tokens)]
}
-function tokenMatchScore(text: string, tokens: readonly string[]): number {
- const lower = text.toLowerCase()
- let score = 0
- for (const token of tokens) {
- if (lower.includes(token)) score += 1
- }
- return score
-}
-
-function countOccurrences(haystackLower: string, needleLower: string): number {
- if (!needleLower || needleLower.length === 0) return 0
- let count = 0
- let pos = 0
- while (true) {
- const idx = haystackLower.indexOf(needleLower, pos)
- if (idx === -1) break
- count++
- pos = idx + needleLower.length
- }
- return count
-}
-
-function flattenMdFiles(nodes: FileNode[]): FileNode[] {
- const files: FileNode[] = []
- for (const node of nodes) {
- if (node.is_dir && node.children) {
- files.push(...flattenMdFiles(node.children))
- } else if (!node.is_dir && node.name.endsWith(".md")) {
- files.push(node)
- }
- }
- return files
-}
-
-function extractTitle(content: string, fileName: string): string {
- // Try YAML frontmatter title
- const frontmatterMatch = content.match(/^---\n[\s\S]*?^title:\s*["']?(.+?)["']?\s*$/m)
- if (frontmatterMatch) return frontmatterMatch[1].trim()
-
- // Try first heading
- const headingMatch = content.match(/^#\s+(.+)$/m)
- if (headingMatch) return headingMatch[1].trim()
-
- // Fall back to filename
- return fileName.replace(/\.md$/, "").replace(/-/g, " ")
-}
-
-/**
- * Markdown image-reference regex. Matches `![alt](url)` capturing
- * groups 1=alt, 2=url. Identical to the regex in
- * `image-caption-pipeline.ts` — kept duplicated rather than shared
- * because the two modules have very different lifetimes (search
- * runs every keystroke; the pipeline runs at ingest), and a shared
- * symbol there would tie them together for no benefit.
- *
- * Excludes:
- * - HTML `<img src=...>` (we don't generate these)
- * - Reference-style `![alt][ref]` (we don't generate these either)
- */
-const IMAGE_REF_RE = /!\[([^\]]*)\]\(([^)\s]+)\)/g
-
-function extractImageRefs(content: string): ImageRef[] {
- const seen = new Set<string>()
- const out: ImageRef[] = []
- for (const m of content.matchAll(IMAGE_REF_RE)) {
- const url = m[2]
- // De-dupe within a single page: the same image may be
- // referenced both inline (LLM-preserved) AND in the safety-net
- // "## Embedded Images" section. Showing it twice in the
- // results UI would just be visual noise.
- if (seen.has(url)) continue
- seen.add(url)
- out.push({ url, alt: m[1] })
- }
- return out
-}
-
-function buildSnippet(content: string, query: string): string {
- const lower = content.toLowerCase()
- const lowerQuery = query.toLowerCase()
- const idx = lower.indexOf(lowerQuery)
- if (idx === -1) return content.slice(0, SNIPPET_CONTEXT * 2).replace(/\n/g, " ")
-
- const start = Math.max(0, idx - SNIPPET_CONTEXT)
- const end = Math.min(content.length, idx + query.length + SNIPPET_CONTEXT)
- let snippet = content.slice(start, end).replace(/\n/g, " ")
- if (start > 0) snippet = "..." + snippet
- if (end < content.length) snippet = snippet + "..."
- return snippet
-}
-
export async function searchWiki(
projectPath: string,
query: string,
): Promise<SearchResult[]> {
if (!query.trim()) return []
const pp = normalizePath(projectPath)
-
- const tokens = tokenizeQuery(query)
- // Fallback: if all tokens were filtered out, use the trimmed query as a single token
- const effectiveTokens = tokens.length > 0 ? tokens : [query.trim().toLowerCase()]
- const results: SearchResult[] = []
-
- // Search wiki pages.
- //
- // We deliberately do NOT also search `raw/sources/` here anymore.
- // Previously this section walked every file under raw/sources/
- // (including PDFs / DOCX / PPTX) and called `readFile` on each,
- // which triggers the heavy pdfium / office text-extraction path
- // — even on cache hits, that's an IPC round-trip per file plus
- // a cache file read of the now-large combined-markdown output
- // (text + per-page image refs after the unified extractor
- // landed). On a project with ~50 PDFs this added 5-15s per
- // search, which the user reported as "very, very slow."
- //
- // The content lost: nothing material. Each ingested raw source
- // produces a `wiki/sources/<slug>.md` summary which is included
- // in the wiki/ search below; the full extracted text lives in
- // the embedding chunks and is reachable via vector search. The
- // raw-files token pass added recall only for raw files that had
- // never been ingested (and thus had no wiki summary), which is
- // not a workflow we want to optimize at the cost of every other
- // search call.
- try {
- const t0 = performance.now()
- const wikiTree = await listDirectory(`${pp}/wiki`)
- const wikiFiles = flattenMdFiles(wikiTree)
- const tList = Math.round(performance.now() - t0)
- const t1 = performance.now()
- await searchFiles(wikiFiles, effectiveTokens, query, results)
- const tRead = Math.round(performance.now() - t1)
- console.log(
- `[Search:token] wiki/ ${wikiFiles.length} files | list=${tList}ms read+match=${tRead}ms`,
- )
- } catch {
- // no wiki directory
- }
-
- // ── Build the token-side ranking (still based on the score field
- // populated by searchFiles above). Snapshot it BEFORE the vector
- // step, so adding vector-only pages doesn't shift token ranks.
- const tokenSorted = [...results].sort((a, b) => b.score - a.score)
- const tokenRank = new Map<string, number>()
- tokenSorted.forEach((r, i) => {
- tokenRank.set(normalizePath(r.path), i + 1) // 1-indexed
- })
-
- // ── Vector search: collect ranked list of page-ids and materialize
- // pages that token search missed. We do NOT add to results' score
- // here — that's done in the RRF step below.
- let vectorRank = new Map<string, number>()
- let vectorCount = 0
- try {
- const { useWikiStore } = await import("@/stores/wiki-store")
- const embCfg = useWikiStore.getState().embeddingConfig
- console.log(`[Vector Search] Config: enabled=${embCfg.enabled}, model="${embCfg.model}"`)
- if (embCfg.enabled && embCfg.model) {
- const t0 = performance.now()
- const { searchByEmbedding } = await import("@/lib/embedding")
- const vectorResults = await searchByEmbedding(pp, query, embCfg, 10)
- const vectorMs = Math.round(performance.now() - t0)
- vectorCount = vectorResults.length
-
- console.log(
- `[Vector Search] query="${query}" | ${vectorResults.length} results in ${vectorMs}ms | model=${embCfg.model}` +
- (vectorResults.length > 0
- ? ` | top: ${vectorResults.slice(0, 5).map((r) => `${r.id}(${r.score.toFixed(3)})`).join(", ")}`
- : "")
- )
-
- // Build vectorRank by page_id (slug); searchByEmbedding returns
- // results pre-sorted by descending similarity.
- vectorResults.forEach((vr, i) => vectorRank.set(vr.id, i + 1))
-
- // Materialize any vector-result page that token search didn't
- // already include — without this, `results` has no entry for
- // them and they can't surface even with a top vector rank.
- const knownIds = new Set(results.map((r) => getFileStem(r.path)))
- let added = 0
- for (const vr of vectorResults) {
- if (knownIds.has(vr.id)) continue
- const dirs = ["entities", "concepts", "sources", "synthesis", "comparison", "queries"]
- for (const dir of dirs) {
- const tryPath = `${pp}/wiki/${dir}/${vr.id}.md`
- try {
- const content = await readFile(tryPath)
- const title = extractTitle(content, `${vr.id}.md`)
- results.push({
- path: tryPath,
- title,
- snippet: buildSnippet(content, query),
- titleMatch: false,
- score: 0, // overwritten by RRF below
- images: extractImageRefs(content),
- })
- knownIds.add(vr.id)
- added++
- break
- } catch {
- // not in this directory
- }
- }
- }
- if (added > 0) {
- console.log(`[Vector Search] Added ${added} vector-only pages to candidate set`)
- }
- }
- } catch (err) {
- console.log(`[Vector Search] Skipped: ${err instanceof Error ? err.message : "not available"}`)
- vectorRank = new Map()
- }
-
- // ── RRF fusion: replace each result's score with
- // 1/(K + token_rank) + 1/(K + vector_rank)
- //
- // Pages absent from either list contribute 0 from that side.
- // Pages absent from BOTH never make it here (we only iterate the
- // results array, which already contains every candidate).
- for (const r of results) {
- const tRank = tokenRank.get(normalizePath(r.path))
- const vRank = vectorRank.get(getFileStem(r.path))
- let rrf = 0
- if (tRank !== undefined) rrf += 1 / (RRF_K + tRank)
- if (vRank !== undefined) rrf += 1 / (RRF_K + vRank)
- r.score = rrf
- }
-
- // Sort by RRF score descending. Ties (e.g. two pages both at vector
- // rank 1 but neither in token list) are broken by alphabetical path
- // order so output is deterministic for tests.
- results.sort((a, b) => {
- if (b.score !== a.score) return b.score - a.score
- return a.path.localeCompare(b.path)
+ const embCfg = useWikiStore.getState().embeddingConfig
+
+ const response = await invoke<BackendSearchResponse>("search_project", {
+ projectPath: pp,
+ query,
+ topK: 20,
+ includeContent: false,
+ queryEmbedding: null,
+ embeddingConfig: embCfg,
})
- const tokenHits = tokenRank.size
- console.log(
- `[Search] query="${query}" | RRF fused: ${tokenHits} token + ${vectorCount} vector → ${results.length} unique`,
- )
-
- return results.slice(0, MAX_RESULTS)
+ return response.results.map((result) => ({
+ ...result,
+ path: `${pp}/${normalizePath(result.path).replace(/^\/+/, "")}`,
+ }))
}
-
-/**
- * Bound on concurrent `readFile` calls during search. Going wider
- * than this saturates the Tauri IPC channel and starts QUEUING
- * work behind the search request — the wider you go, the SLOWER
- * a single search gets past about 16-32 in-flight reads (measured
- * in dev against a 200-file project). 16 is a comfortable middle
- * ground that gives near-linear speedup over sequential without
- * choking the IPC layer.
- */
-const SEARCH_READ_CONCURRENCY = 16
-
-/**
- * Punctuation pattern shared between token splitting and the
- * phrase-bonus normalization below. Matched at start AND end of
- * the query — internal punctuation (`U.S.A.`, `2024-Q3`) stays
- * because it might be load-bearing in legitimate phrase matches.
- */
-const TRIM_PUNCT_RE =
- /^[\s,,。!?、;:""''()()\-_/\\·~~…]+|[\s,,。!?、;:""''()()\-_/\\·~~…]+$/g
-
-async function searchFiles(
- files: FileNode[],
- tokens: readonly string[],
- query: string,
- results: SearchResult[],
-): Promise<void> {
- // Strip leading / trailing punctuation from the query before using
- // it as a phrase-bonus probe. Without this, `query="总资产。"`
- // tries to substring-match `总资产。` inside titles / content that
- // never have the period at that spot — the phrase-bonus signal
- // (worth +50 in titles, +20 per occurrence in content) silently
- // goes to zero and the page's RRF rank slides. Same surface that
- // bit the search-view image filter, fixed there in token space;
- // here we apply the analog in phrase space.
- //
- // Internal punctuation is preserved on purpose: queries like
- // "2024-Q3" or domain names should still phrase-match exactly.
- const queryPhrase = query.trim().toLowerCase().replace(TRIM_PUNCT_RE, "")
-
- // Process files in fixed-size concurrent batches. Promise.all over
- // the entire list would work but spawns N IPC calls simultaneously
- // — tested at N=200, that's where we saw the slowdown above.
- for (let i = 0; i < files.length; i += SEARCH_READ_CONCURRENCY) {
- const batch = files.slice(i, i + SEARCH_READ_CONCURRENCY)
- const batchResults = await Promise.all(
- batch.map(async (file) => {
- let content: string
- try {
- content = await readFile(file.path)
- } catch {
- return null
- }
- return scoreFile(file, content, tokens, queryPhrase, query)
- }),
- )
- for (const r of batchResults) {
- if (r) results.push(r)
- }
- }
-}
-
-/**
- * Pure scoring pass — no IO. Extracted so `searchFiles` can run
- * the IO and the matching independently and so this function can
- * be unit-tested without mocking readFile.
- */
-function scoreFile(
- file: FileNode,
- content: string,
- tokens: readonly string[],
- queryPhrase: string,
- query: string,
-): SearchResult | null {
- const title = extractTitle(content, file.name)
- const titleText = `${title} ${file.name}`
- const titleLower = titleText.toLowerCase()
- const contentLower = content.toLowerCase()
- const fileStem = file.name.replace(/\.md$/, "").toLowerCase()
-
- // Exact-match signals (strongest)
- const filenameExact = fileStem === queryPhrase
- const titleHasPhrase =
- queryPhrase.length > 0 && titleLower.includes(queryPhrase)
- const contentPhraseOcc = Math.min(
- countOccurrences(contentLower, queryPhrase),
- MAX_PHRASE_OCC_COUNTED,
- )
-
- // Token-level signals (fallback / density)
- const titleTokenScore = tokenMatchScore(titleText, tokens)
- const contentTokenScore = tokenMatchScore(content, tokens)
-
- if (
- !filenameExact &&
- !titleHasPhrase &&
- contentPhraseOcc === 0 &&
- titleTokenScore === 0 &&
- contentTokenScore === 0
- ) {
- return null
- }
-
- const score =
- (filenameExact ? FILENAME_EXACT_BONUS : 0) +
- (titleHasPhrase ? PHRASE_IN_TITLE_BONUS : 0) +
- contentPhraseOcc * PHRASE_IN_CONTENT_PER_OCC +
- titleTokenScore * TITLE_TOKEN_WEIGHT +
- contentTokenScore * CONTENT_TOKEN_WEIGHT
-
- const isTitleMatch = titleTokenScore > 0 || titleHasPhrase
-
- const snippetAnchor =
- contentPhraseOcc > 0
- ? queryPhrase
- : tokens.find((t) => contentLower.includes(t)) ?? query
-
- return {
- path: file.path,
- title,
- snippet: buildSnippet(content, snippetAnchor),
- titleMatch: isTitleMatch,
- score,
- images: extractImageRefs(content),
- }
-}
-
diff --git a/src/stores/wiki-store.ts b/src/stores/wiki-store.ts
index b19a5894..b5952c1a 100644
--- a/src/stores/wiki-store.ts
+++ b/src/stores/wiki-store.ts
@@ -142,6 +142,26 @@ interface ScheduledImportConfig {
lastScan: number | null // 上次扫描时间戳
}
+/**
+ * Local HTTP API server config. Read by the Rust `api_server` module on
+ * every request via `load_app_state` (5s cache). The Rust side is the
+ * source of truth at request time; this struct is the persisted form
+ * the UI edits.
+ *
+ * - `enabled` gates all non-/health endpoints. Default `true` so an
+ * env-token-only setup keeps working after the toggle is added.
+ * - `allowUnauthenticated` lets local agents call the API without a
+ * token. It is explicit and default-off.
+ * - `token` is the bearer secret. Empty + auth required =
+ * every non-/health request returns 401. The env var
+ * `LLM_WIKI_API_TOKEN` overrides this field at the backend.
+ */
+interface ApiConfig {
+ enabled: boolean
+ allowUnauthenticated: boolean
+ token: string
+}
+
interface SourceWatchConfig {
enabled: boolean
autoIngest: boolean
@@ -247,6 +267,7 @@ interface WikiState {
proxyConfig: ProxyConfig
scheduledImportConfig: ScheduledImportConfig
sourceWatchConfig: SourceWatchConfig
+ apiConfig: ApiConfig
dataVersion: number
setProject: (project: WikiProject | null) => void
@@ -266,6 +287,7 @@ interface WikiState {
setProxyConfig: (config: ProxyConfig) => void
setScheduledImportConfig: (config: ScheduledImportConfig) => void
setSourceWatchConfig: (config: SourceWatchConfig) => void
+ setApiConfig: (config: ApiConfig) => void
bumpDataVersion: () => void
}
@@ -348,6 +370,17 @@ export const useWikiStore = create<WikiState>((set) => ({
sourceWatchConfig: DEFAULT_SOURCE_WATCH_CONFIG,
+ // Default `enabled: true` preserves the pre-toggle behavior: anyone
+ // who already had `LLM_WIKI_API_TOKEN` set or `apiConfig.token`
+ // hand-edited keeps their working API. New users land in
+ // "enabled + no token = 401 on every endpoint" — fail-closed by
+ // virtue of the token being empty.
+ apiConfig: {
+ enabled: true,
+ allowUnauthenticated: false,
+ token: "",
+ },
+
setLlmConfig: (llmConfig) => set({ llmConfig }),
setProviderConfigs: (providerConfigs) => set({ providerConfigs }),
setActivePresetId: (activePresetId) => set({ activePresetId }),
@@ -358,7 +391,8 @@ export const useWikiStore = create<WikiState>((set) => ({
setProxyConfig: (proxyConfig) => set({ proxyConfig }),
setScheduledImportConfig: (scheduledImportConfig) => set({ scheduledImportConfig }),
setSourceWatchConfig: (sourceWatchConfig) => set({ sourceWatchConfig }),
+ setApiConfig: (apiConfig) => set({ apiConfig }),
bumpDataVersion: () => set((state) => ({ dataVersion: state.dataVersion + 1 })),
}))
-export type { WikiState, LlmConfig, SearchApiConfig, EmbeddingConfig, MultimodalConfig, OutputLanguage, ProxyConfig, ScheduledImportConfig, SourceWatchConfig }
+export type { WikiState, LlmConfig, SearchApiConfig, EmbeddingConfig, MultimodalConfig, OutputLanguage, ProxyConfig, ScheduledImportConfig, SourceWatchConfig, ApiConfig }