An open-source, privacy-first, self-hosted knowledge workspace where humans and AI agents work together 开源、隐私优先、自托管的知识工作空间,让人与智能体在此协作
TypeScript agentic-aiai-agentknowledge-baselocal-firstmarkdownmcpmindmapnote-takingself-hostedsiyuanworkspace
:bug: Guard document sources before sync and snapshot checkout https://github.com/siyuan-note/siyuan/issues/19938
Daniel30de520fabb7
diff --git a/docs/DOCUMENT-ADDRESSING.md b/docs/DOCUMENT-ADDRESSING.mdindex 12004490cb..5aeaf0892e 100644--- a/docs/DOCUMENT-ADDRESSING.md+++ b/docs/DOCUMENT-ADDRESSING.md@@ -2,7 +2,7 @@[中文](DOCUMENT-ADDRESSING.zh-CN.md)-Related issues: [Improve hPath by separating display text from document addressing](https://github.com/siyuan-note/siyuan/issues/19938), [Improve content preservation when converting headings to documents](https://github.com/siyuan-note/siyuan/issues/19235)+Related issues: [Support `/` in document titles](https://github.com/siyuan-note/siyuan/issues/19938), [Improve content preservation when converting headings to documents](https://github.com/siyuan-note/siyuan/issues/19235), [Improve document titles, notebook names, and plugin compatibility](https://github.com/siyuan-note/siyuan/issues/20837)This document defines document titles, hierarchy addressing, readable paths, APIs, and migration behavior. Development implementation and release requirements are described separately; development capabilities do not establish availability in a released version. See [Document rename and path indexes](DOCUMENT-HPATH.md) (Chinese) for the current rename and index update mechanism.@@ -20,7 +20,7 @@ Administrator notebook configuration reads expose `documentCapabilities`, distinDocument IDs determine identity and hierarchy membership; titles express content; readable paths provide presentation. Creation, moves, and references must not recover document identity by parsing display text. Once the affected entry points have migrated and compatibility has been verified, document titles can contain the ASCII slash `/` while remaining distinct from existing fullwidth slashes `/`.-Future support for inline elements in document titles is a required extension capability of this design. Addressing, title presentation, plain-text extraction, and storage representation must remain separate. The design must not permanently constrain titles to a single plain-text string interpreted directly by every consumer. A complete rich title format, editing interactions, and the supported element set are outside the current scope.+Future support for inline elements in document titles is a required extension capability of this design. Addressing, title presentation, plain-text extraction, and storage representation must remain separate. The design must not permanently constrain titles to a single plain-text string interpreted directly by every consumer. A complete rich title format, editing interactions, and the supported element set are outside the current scope. Issue #20837 tracks inline titles and bidirectional conversion, literal `/` in notebook names, and verification of actual third-party plugin versions; older-client protection and acceptance for this release remain in #19938.The design covers desktop, browser, mobile, kernel APIs, and plugin-visible data, subject to the respective access boundaries of ordinary and encrypted notebooks. Notebook names participate in path presentation as separate display fields. Their character restrictions and rich text support require a separate assessment of configuration formats and older clients; document title rules do not automatically apply to notebook names.@@ -388,6 +388,8 @@ Raw file endpoints apply the same source-format checks to paths composed of noteDetermine the minimum compatible version through read, write, and rejection tests against actual releases. Clients predating the protective release that ignore capability markers cannot be constrained automatically and are outside the safe mixed-version promise. Before using the feature release, all sync devices must run at least the protective release. Offline device state cannot be proven; release notes and user guides must explain this limit. Incomplete protection prevents a claim of release readiness or safe mixed-version use; documentation or a user switch cannot substitute for protection.+An isolated probe of the published `v3.8.9-alpha.4` source rejects direct reads of Spec 6 documents but can still read legacy-specification documents in mode `2` or write a stale editing snapshot. That release cannot be recorded as verified protection, and the result does not support a blanket claim that older versions cannot open the data. Local protective candidate builds and regressions establish behavior of the tested source, not of an actual release package. After publishing the protective version, verify the package, source, and compiled capabilities, then validate ordinary and encrypted data compatibility, mode rejection, stale writes, sync, and recovery before recording `minimumReaderVersion` and completing release acceptance for this issue.+Automatic migration to mode `2` has no switch back to `1`. Even after current titles no longer contain `/`, history and backups can retain new formats. Older clients may use a verified compatible export copy or a complete pre-upgrade backup restored into a separate workspace; do not overwrite the original workspace with old configuration and continue syncing. Any future downgrade converter requires its own conversion manifest, conflict policy, and recovery design while preserving original data.### Mode records and conflict recovery@@ -400,6 +402,8 @@ Settings export remains an archive containing one JSON file. When confirmed capaThe protective release recognizes both configuration and capability floor records, rejecting unsupported modes before processing source files. Sync application and snapshot restoration check affected notebooks' capability requirements before replacing configuration or files. Once mode `2` is confirmed, incoming old configuration, missing new fields, or deletion of a capability record cannot lower the effective mode; restore configuration projections and floor records before reopening access. If checks can occur only after files are persisted, or the dependency cannot expose required configuration before application, stage A is incomplete. Complete sync and recovery entry points first; index rebuilding does not substitute for pre-application protection.+Mode markers do not replace document-source checks. Before any file in a sync or snapshot batch is written or removed, validate incoming ordinary document sources and refuse to overwrite or partially delete existing sources identified during preflight as requiring unsupported specifications, title representations, or title capabilities. Missing mode markers must not allow incompatible sources to be persisted before indexing detects them. This check follows the existing recovery process; it does not make ordinary snapshot restoration an atomic transaction isolated from arbitrary external file changes. Valid history may still restore generally corrupted ordinary documents, whole-notebook deletion retains its existing semantics, and `.sy` files inside asset or plugin directories are not interpreted as documents. Locked encrypted notebooks continue transferring original ciphertext under the existing protocol without requiring an unlock for title preflight. Repository chunk verification does not authenticate notebook contents; ciphertext authentication and source-format checks remain within the existing protected read and recovery boundaries.+Deleting an entire notebook retains existing semantics; do not recreate it merely to restore a capability record. Keep the locally confirmed floor for that identity and apply it when the same identity is restored. Native imports of whole notebooks validate the source identity and capability record before allocating or reusing the destination identity under existing import rules, then bind the floor to that destination identity. Identity remapping must not lower the source notebook's confirmed mode. Importing selected documents into an existing notebook instead checks the complete operation set described above without copying the source notebook's mode configuration.Automatic migration first persists `{version: 1, notebook, phase: "records"}` in `conf/document-addressing-recovery/<notebookID>.json`, then writes the local floor, notebook floor, and configuration mode marker in that order, and finally removes the recovery record. After interruption, the next access continues the same migration without retracting a written requirement. The protective release also interprets valid recovery records as requiring at least mode `2`. Preserve unknown configuration fields and opaque encrypted payload values, without scanning or rewriting `.sy` files. Raw file APIs cannot overwrite the recovery directory.@@ -478,6 +482,8 @@ Build a subtree with an ordinary root title but descendants containing `/` or aSimulate old-configuration replacement, missing or corrupt capability records, interrupted sync, and source documents already requiring mode `2` while indexes remain unavailable. Verify that effective mode never decreases, unknown formats preserve original data, automatic migration works without a setting or ready index, and restart continues the same recovery task. Inject failures at every migration write, cover deletion and restoration of the same notebook identity, and preserve capability floors when imports remap identities. Inspect new payload persistence and locked responses and caches for absence of plaintext target objects, and verify old payload reads and recovery after backups and password changes.+`TestDocumentAddressingCheckoutSourcePreflight` uses actual repository snapshots to cover unmarked future specifications, unknown title representations, incompatible current sources, and mode-check precedence. Rejection occurs before additions, replacements, or removals in the batch, preserving source bytes, timestamps, and incoming snapshot contents. `TestDocumentAddressingCheckoutSourceRemoval` distinguishes partial removal from whole-notebook deletion. `TestDocumentAddressingCheckoutPreservesOpaqueEncryptedSource` uses historical ciphertext fixtures to verify byte-preserving transfer without unlocking, with configuration and recovery material unchanged. Path regressions cover backslashes, duplicate destinations, and Windows trailing-dot, trailing-space, and default-data-stream aliases so the actual write target cannot bypass source checks. Retain regressions for BOM-prefixed sources, valid slash titles, recovery of generally corrupted sources, and asset `.sy` files so source protection does not introduce unrelated file-operation restrictions.+Retain `TestHeadingDocParagraph` and `TestHeadingDocParagraphTextMark` so foundational separation continues covering images, links, and original heading content preserved in the body. Stage E additionally verifies supported nodes in title sources and both conversion directions without deleting user paragraphs based on equal plain text. Template document trees cover grouped undo and redo, partial commits, name or ID conflicts on redo, and user edits after creation, not just successful creation.Actual HTTP regressions verify strict request types, required fields versus `null`, result ordering, truncated candidates, error data, administrator and writable access, encrypted response leases, and published-visitor rejection. Assert old compatibility behavior separately from new endpoints instead of replacing legacy expectations. Frontend tests cover old template configuration round-trips, new segment evaluation, one time snapshot, partial-success presentation, and mobile entry points.diff --git a/docs/DOCUMENT-ADDRESSING.zh-CN.md b/docs/DOCUMENT-ADDRESSING.zh-CN.mdindex 0981645f65..5ede984dcf 100644--- a/docs/DOCUMENT-ADDRESSING.zh-CN.md+++ b/docs/DOCUMENT-ADDRESSING.zh-CN.md@@ -2,7 +2,7 @@[English](DOCUMENT-ADDRESSING.md)-关联问题:[Improve hPath by separating display text from document addressing](https://github.com/siyuan-note/siyuan/issues/19938)、[Improve content preservation when converting headings to documents](https://github.com/siyuan-note/siyuan/issues/19235)+关联问题:[Support `/` in document titles](https://github.com/siyuan-note/siyuan/issues/19938)、[Improve content preservation when converting headings to documents](https://github.com/siyuan-note/siyuan/issues/19235)、[Improve document titles, notebook names, and plugin compatibility](https://github.com/siyuan-note/siyuan/issues/20837)本文规定文档标题、层级定位、可读路径、接口及迁移的目标行为。当前开发实现和发布条件分别说明,不能将开发分支能力视为已发布能力。当前重命名与路径索引的更新机制见 [文档重命名与路径索引](DOCUMENT-HPATH.md)。@@ -20,7 +20,7 @@文档使用 ID 确定身份和所属层级,标题用于表达内容,可读路径用于展示。创建、移动和引用不得依赖从展示文本反向解析文档身份。完成相关入口迁移与兼容验证后,文档标题可以包含半角斜杠 `/`,并与原有全角斜杠 `/` 保持区别。-未来文档标题支持行级元素是本方案必须保留的扩展能力。寻址、标题显示、纯文本提取和存储表示必须分离,不能将标题永久限定为一个供所有模块直接解释的纯文本字符串。本次不定义完整的富文本标题格式、编辑器交互或元素支持清单。+未来文档标题支持行级元素是本方案必须保留的扩展能力。寻址、标题显示、纯文本提取和存储表示必须分离,不能将标题永久限定为一个供所有模块直接解释的纯文本字符串。本次不定义完整的富文本标题格式、编辑器交互或元素支持清单。行级标题与双向转换、笔记本名称保留 `/`,以及第三方插件实际版本的适配验证由 #20837 继续跟踪;旧客户端保护与本轮发布验收仍属于 #19938。方案覆盖桌面端、浏览器端、移动端、内核接口和插件可见数据,并遵守普通笔记本与加密笔记本各自的访问边界。笔记本名称作为独立显示字段参与路径展示;笔记本名称的字符限制与富文本支持需要单独评估配置格式和旧客户端兼容,不能由文档标题规则自动推导。@@ -388,6 +388,8 @@最低兼容版本由真实发布版本的读写及拒绝测试确定。保护版本之前忽略能力标记的客户端无法由新字段自动约束,不在安全混用承诺内。使用功能版本前要求所有同步设备至少升级到保护版本;系统无法证明离线设备状态,发布说明及用户指南须说明这一限制。如果保护仍存在缺口,不能宣称已达到对外发布或安全混用条件,也不能以文档提示或用户开关代替必要保护。+已发布 `v3.8.9-alpha.4` 源码的隔离验证表明:它拒绝直接读取 Spec 6 文档,但仍可能读取模式 `2` 中保留旧规格的文档,或将旧编辑快照写回。因此,该版本不能登记为已验证的保护版本,也不能概括为“旧版本一定无法打开”。本地保护候选构建及其回归只证明相应源码的行为,不代替真实发行包验证;实际保护版本发布后,须核对发行包、源码与构建能力,再验证普通及加密数据的兼容读取、模式拒绝、旧缓存写入、同步和恢复,才登记 `minimumReaderVersion` 并完成本议题的发布验收。+自动迁移到模式 `2` 后不提供改回 `1` 的开关。即使当前标题已不含 `/`,历史和备份仍可能包含新格式。回退客户端只能使用经验证的兼容导出副本,或恢复升级前的完整备份到独立工作空间;不能将旧配置覆盖回原工作空间并继续同步。若未来提供降级转换,应另行设计转换清单、冲突处理和恢复机制,保留原数据。### 模式记录与冲突恢复@@ -400,6 +402,8 @@保护版本同时识别配置及能力下界记录;不支持的模式在实际处理源文件前拒绝。同步应用和快照恢复先核对受影响笔记本的能力要求,再替换配置或文件。已确认模式 `2` 时,传入旧配置、缺少新字段或删除能力记录都不能降低有效模式;恢复配置投影和下界记录后再开放访问。若只能在文件落盘后检查,或依赖层不能在应用前提供所需配置数据,阶段 A 尚未完成,须先补齐同步和恢复入口,不能以重建索引代替前置保护。+模式标记不能代替文档源检查。同步和快照恢复在整批文件写入或删除前,检查待应用的普通文档源,并拒绝以旧快照覆盖或部分删除预检时已确认使用不支持规格、标题表示或标题能力的现存源;即使来源缺少模式标记,也不能先落盘再依赖索引发现错误。该检查沿用现有恢复过程,不将普通快照恢复变为可隔离任意外部文件修改的原子事务。合法历史源仍可恢复一般损坏的普通文档,整本删除保留既有语义,附件及插件目录中的 `.sy` 不解释为文档。锁定加密笔记本仍按既有协议传输原始密文字节,不为标题预检要求解锁,也不把仓库分块校验当作笔记本内容认证;密文的认证及源格式检查继续在原有受保护读取和恢复边界内完成。+删除整个笔记本仍按现有语义执行,不为补齐能力记录而重新创建已删除的笔记本;本机保留该身份的已确认下界,恢复同一身份时继续生效。完整笔记本的原生导入先校验源身份与能力记录,再按既有导入规则分配或复用目标身份,并将能力下界绑定到目标身份;不得借身份重映射降低源笔记本已确认的模式。向既有笔记本导入部分文档则按前述完整操作集合检查,不直接复制源笔记本的模式配置。自动迁移先在 `conf/document-addressing-recovery/<notebookID>.json` 持久化 `{version: 1, notebook, phase: "records"}`,再依次写入本机下界、笔记本下界及配置中的模式标记,最后删除该恢复记录。任一步骤中断后,下次访问继续同一迁移,不撤回已写入的模式要求;保护版本也必须将有效恢复记录视为至少需要模式 `2`。配置的未知字段及加密载荷原值保留,不扫描或批量重写 `.sy` 文件,原始文件接口不能覆盖该恢复目录。@@ -478,6 +482,8 @@模拟旧配置覆盖、能力记录缺失或损坏、同步中断,以及源文档已经需要模式 `2` 但索引尚不可用的场景。验证有效模式不下降、未知格式保留原数据、无开关的自动迁移可在索引未就绪时进行、重启继续同一恢复任务。覆盖迁移每一步的写入故障、整本删除后按同一身份恢复,以及完整导入重映射笔记本身份时保留能力下界。对新配置载荷检查磁盘及锁定后的接口和缓存均无新目标对象明文,并验证旧载荷读取及备份、改密后的恢复。+`TestDocumentAddressingCheckoutSourcePreflight` 使用实际仓库快照,覆盖缺少模式标记的未来规格、未知标题表示、现存不兼容源及模式检查优先级;拒绝时同批新增、替换和删除均不开始,源字节、时间及快照来源保持不变。`TestDocumentAddressingCheckoutSourceRemoval` 区分部分删除与整本删除;`TestDocumentAddressingCheckoutPreservesOpaqueEncryptedSource` 使用历史密文样本验证无需解锁的原字节传输,并检查配置及恢复材料不变。路径回归覆盖反斜杠、重复目标,以及 Windows 的尾点、尾空格和默认数据流别名,不能让实际落盘目标绕过来源检查。另保留 BOM、合法斜杠标题、一般损坏源恢复和附件 `.sy` 的回归,避免来源保护扩大为不兼容的文件操作限制。+保留 `TestHeadingDocParagraph` 与 `TestHeadingDocParagraphTextMark`,基础解耦继续覆盖图片、链接和原标题内容进入正文的行为;阶段 E 再验证支持的节点进入标题源内容及双向转换,不按纯文本相等删除用户段落。模板文档树覆盖整组撤销和重做、部分提交、重做时的名称或 ID 冲突及创建后用户编辑,不能只测试新建成功。实际 HTTP 回归检查严格请求类型、必填与 `null` 区别、返回顺序、候选截断、错误数据、管理员与可写保护、加密响应租约及发布访问拒绝。旧接口的兼容回归与新接口回归分开断言,避免用新预期覆盖旧行为。前端验证旧模板配置往返、新模板逐段求值、同一时间快照、部分成功提示以及移动端入口。diff --git a/kernel/model/document_addressing.go b/kernel/model/document_addressing.goindex 4db7aca24d..94cb5a4e37 100644--- a/kernel/model/document_addressing.go+++ b/kernel/model/document_addressing.go@@ -1,16 +1,20 @@package modelimport (+ "bytes""encoding/json""errors"+ "fmt""os""path/filepath"+ "runtime""strings""github.com/88250/lute/ast""github.com/siyuan-note/dejavu""github.com/siyuan-note/filelock""github.com/siyuan-note/siyuan/kernel/documentaddressing"+ "github.com/siyuan-note/siyuan/kernel/treenode""github.com/siyuan-note/siyuan/kernel/util")@@ -26,16 +30,139 @@ func documentAddressingPreflight(repo *dejavu.Repo) func(*dejavu.CheckoutPlan) eupserts := make([]string, 0, len(plan.Upserts))removes := make([]string, 0, len(plan.Removes))paths := map[string]string{}+ upsertBoxes, removedPaths := map[string]bool{}, map[string]bool{}+ seen := map[string]bool{}+ // 仓库索引统一使用前导斜杠和正斜杠;拒绝别名及重复目标,避免覆盖计划遮盖来源。+ checkPath := func(name string) error {+ if !strings.HasPrefix(name, "/") || strings.Contains(name, "\\") {+ return documentaddressing.ErrInvalid+ }+ key := name+ if runtime.GOOS == "windows" {+ for _, part := range strings.Split(name[1:], "/") {+ if strings.TrimRight(part, ". ") != part || strings.Contains(part, ":") {+ return documentaddressing.ErrInvalid+ }+ }+ key = strings.ToLower(key)+ }+ if seen[key] {+ return documentaddressing.ErrInvalid+ }+ seen[key] = true+ return nil+ }for _, file := range plan.Upserts {+ if err := checkPath(file.Path); err != nil {+ return err+ }upserts = append(upserts, file.Path)- paths[strings.TrimPrefix(file.Path, "/")] = file.Path+ name := strings.TrimPrefix(file.Path, "/")+ paths[name] = file.Path+ boxID, _, _ := strings.Cut(name, "/")+ upsertBoxes[boxID] = true}for _, file := range plan.Removes {+ if err := checkPath(file.Path); err != nil {+ return err+ }removes = append(removes, file.Path)+ removedPaths[strings.TrimPrefix(file.Path, "/")] = true}- return documentaddressing.CheckBatch(repo.DataPath, util.ConfDir, upserts, removes,- func(name string) ([]byte, error) { return plan.ReadFile(paths[name]) })+ if err := documentaddressing.CheckBatch(repo.DataPath, util.ConfDir, upserts, removes,+ func(name string) ([]byte, error) { return plan.ReadFile(paths[name]) }); err != nil {+ return err+ }+ for _, file := range plan.Upserts {+ name := strings.TrimPrefix(file.Path, "/")+ if !isDocumentAddressingSource([]string{name}) {+ continue+ }+ data, err := plan.ReadFile(file.Path)+ if err == nil {+ err = checkDocumentAddressingCheckoutSource(data)+ }+ if err != nil {+ return fmt.Errorf("document [%s]: %w", name, err)+ }+ if err = checkDocumentAddressingCheckoutCurrent(repo.DataPath, name); err != nil {+ return err+ }+ }+ deletedBoxes := map[string]bool{}+ for _, file := range plan.Removes {+ name := strings.TrimPrefix(file.Path, "/")+ if !isDocumentAddressingSource([]string{name}) {+ continue+ }+ boxID, _, _ := strings.Cut(name, "/")+ deleted, checked := deletedBoxes[boxID]+ if !checked {+ var err error+ if !upsertBoxes[boxID] {+ deleted, err = documentAddressingCheckoutRemovesNotebook(repo.DataPath, boxID, removedPaths)+ }+ if err != nil {+ return err+ }+ deletedBoxes[boxID] = deleted+ }+ if !deleted {+ if err := checkDocumentAddressingCheckoutCurrent(repo.DataPath, name); err != nil {+ return err+ }+ }+ }+ return nil+ }+}++func checkDocumentAddressingCheckoutCurrent(dataDir, name string) error {+ data, err := filelock.ReadFile(filepath.Join(dataDir, filepath.FromSlash(name)))+ if errors.Is(err, os.ErrNotExist) {+ return nil+ }+ if err != nil {+ return err+ }+ // 已识别的不支持版本或标题不能被覆盖或部分删除;普通损坏源仍可用合法历史恢复。+ if err = checkDocumentAddressingCheckoutSource(data); errors.Is(err, treenode.ErrSpecTooNew) ||+ errors.Is(err, documentaddressing.ErrModeConflict) || errors.Is(err, documentaddressing.ErrInvalidTitle) {+ return fmt.Errorf("document [%s]: %w", name, err)+ }+ return nil+}++// documentAddressingCheckoutRemovesNotebook 仅在整本现存文件全部移除时保留无需解释内容的删除语义。+func documentAddressingCheckoutRemovesNotebook(dataDir, boxID string, removes map[string]bool) (bool, error) {+ deleted := true+ err := filepath.WalkDir(filepath.Join(dataDir, boxID), func(name string, entry os.DirEntry, err error) error {+ if errors.Is(err, os.ErrNotExist) {+ return nil+ }+ if err != nil || entry.IsDir() {+ return err+ }+ relative, err := filepath.Rel(dataDir, name)+ if err != nil {+ return err+ }+ if !removes[filepath.ToSlash(relative)] {+ deleted = false+ return filepath.SkipAll+ }+ return nil+ })+ return deleted, err+}++// checkDocumentAddressingCheckoutSource 只解释普通源;密文同步保持原字节,不要求解锁或声明内容已认证。+// 仓库分块认证不代替笔记本认证,密文的格式及标题仍在解锁读取时检查。+func checkDocumentAddressingCheckoutSource(data []byte) error {+ if util.IsCiphertext(data) {+ return nil}+ return treenode.CheckSpecJSONForMode(bytes.TrimPrefix(data, []byte("\xef\xbb\xbf")), documentaddressing.DefaultMode())}func documentAddressingCheckoutHooks(repo *dejavu.Repo) dejavu.CheckoutHooks {diff --git a/kernel/model/document_addressing_preflight_test.go b/kernel/model/document_addressing_preflight_test.gonew file mode 100644index 0000000000..655d57e2be--- /dev/null+++ b/kernel/model/document_addressing_preflight_test.go@@ -0,0 +1,371 @@+package model++import (+ "bytes"+ "crypto/sha256"+ "encoding/hex"+ "encoding/json"+ "errors"+ "os"+ "path/filepath"+ "reflect"+ "runtime"+ "strings"+ "testing"+ "time"++ "github.com/siyuan-note/dejavu"+ "github.com/siyuan-note/dejavu/entity"+ "github.com/siyuan-note/siyuan/kernel/documentaddressing"+ "github.com/siyuan-note/siyuan/kernel/filesys"+ "github.com/siyuan-note/siyuan/kernel/treenode"+ "github.com/siyuan-note/siyuan/kernel/util"+)++const checkoutSourceBox = "20261024000000-checkbx"+const checkoutSourceDocument = "20261024000001-checkdc.sy"++func TestDocumentAddressingCheckoutRejectsWindowsPathAliases(t *testing.T) {+ if runtime.GOOS != "windows" {+ t.Skip("Windows filename aliases")+ }+ for _, alias := range []string{+ checkoutSourceBox + "/" + checkoutSourceDocument + ".",+ checkoutSourceBox + "/" + checkoutSourceDocument + " ",+ checkoutSourceBox + "." + "/" + checkoutSourceDocument,+ checkoutSourceBox + "/" + checkoutSourceDocument + "::$DATA",+ } {+ t.Run(alias, func(t *testing.T) {+ repo := newCheckoutSourceTestRepo(t)+ writeCheckoutSourceTestFile(t, repo, checkoutSourceBox+"/"+documentaddressing.ConfigPath, []byte(`{}`), 10)+ writeCheckoutSourceTestFile(t, repo, checkoutSourceBox+"/"+checkoutSourceDocument,+ []byte(`{"Spec":"999","Properties":{"title":"Future"}}`), 10)+ original, err := os.Stat(filepath.Join(repo.DataPath, checkoutSourceBox, checkoutSourceDocument))+ if err != nil {+ t.Fatal(err)+ }+ aliased, err := os.Stat(filepath.Join(repo.DataPath, filepath.FromSlash(alias)))+ if err != nil || !os.SameFile(original, aliased) {+ t.Fatal("fixture did not reproduce a Windows alias of the document", err)+ }+ plan := &dejavu.CheckoutPlan{Removes: []*entity.File{{Path: "/" + alias}}}+ if err = documentAddressingPreflight(repo)(plan); !errors.Is(err, documentaddressing.ErrInvalid) {+ t.Fatal("Windows alias bypassed source validation", err)+ }+ })+ }+}++func TestDocumentAddressingCheckoutNativeAttachmentNames(t *testing.T) {+ for _, name := range []string{"assets/report.", "assets/report ", "assets/space /report.txt", "assets/report:data"} {+ t.Run(name, func(t *testing.T) {+ repo := newCheckoutSourceTestRepo(t)+ plan := &dejavu.CheckoutPlan{Upserts: []*entity.File{{Path: "/" + name}}}+ err := documentAddressingPreflight(repo)(plan)+ if runtime.GOOS == "windows" {+ if !errors.Is(err, documentaddressing.ErrInvalid) {+ t.Fatal("noncanonical Windows target was accepted", err)+ }+ } else if err != nil {+ t.Fatal("native attachment name was rejected", err)+ }+ })+ }+}++func TestDocumentAddressingCheckoutRejectsNoncanonicalPaths(t *testing.T) {+ for _, operation := range []string{"upsert", "remove", "metadata alias", "missing slash", "duplicate target", "upsert and remove"} {+ t.Run(operation, func(t *testing.T) {+ repo := newCheckoutSourceTestRepo(t)+ writeCheckoutSourceTestFile(t, repo, checkoutSourceBox+"/"+documentaddressing.ConfigPath, []byte(`{}`), 10)+ writeCheckoutSourceTestFile(t, repo, checkoutSourceBox+"/"+checkoutSourceDocument,+ []byte(`{"Spec":"999","Properties":{"title":"Future"}}`), 10)+ file := &entity.File{Path: "/" + checkoutSourceBox + "\\" + checkoutSourceDocument}+ plan := &dejavu.CheckoutPlan{Upserts: []*entity.File{file}}+ if operation == "remove" {+ plan.Upserts, plan.Removes = nil, []*entity.File{file}+ } else if operation == "metadata alias" {+ file.Path = "/" + checkoutSourceBox + "\\.siyuan\\conf.json"+ plan.Upserts = append(plan.Upserts, &entity.File{Path: "/" + checkoutSourceBox + "/" + documentaddressing.ConfigPath})+ } else if operation == "missing slash" {+ file.Path = checkoutSourceBox + "/" + checkoutSourceDocument+ } else if operation == "duplicate target" || operation == "upsert and remove" {+ file.Path = "/" + checkoutSourceBox + "/" + checkoutSourceDocument+ if operation == "duplicate target" {+ plan.Upserts = append(plan.Upserts, &entity.File{Path: file.Path})+ } else {+ plan.Removes = []*entity.File{{Path: file.Path}}+ }+ }+ if err := documentAddressingPreflight(repo)(plan); !errors.Is(err, documentaddressing.ErrInvalid) {+ t.Fatal("noncanonical checkout path bypassed source or metadata validation", err)+ }+ })+ }+}++func newCheckoutSourceTestRepo(t *testing.T) *dejavu.Repo {+ t.Helper()+ previousConf, previousConfDir := Conf, util.ConfDir+ Conf = &AppConf{Lang: "en"}+ base := t.TempDir()+ util.ConfDir = filepath.Join(base, "conf")+ t.Cleanup(func() { Conf, util.ConfDir = previousConf, previousConfDir })+ repo, err := dejavu.NewRepoWithOptions(dejavu.Options{+ DataPath: filepath.Join(base, "data"), RepoPath: filepath.Join(base, "repo"),+ HistoryPath: filepath.Join(base, "history"), TempPath: filepath.Join(base, "temp"),+ DeviceID: "source-preflight", DeviceName: "source-preflight", DeviceOS: "windows",+ AESKey: []byte("0123456789abcdef0123456789abcdef"), HiddenDirectoryNames: []string{".siyuan"},+ })+ if err != nil {+ t.Fatal(err)+ }+ repo.SetCheckoutHooks(documentAddressingCheckoutHooks(repo))+ return repo+}++func writeCheckoutSourceTestFile(t *testing.T, repo *dejavu.Repo, name string, data []byte, tick int64) {+ t.Helper()+ name = filepath.Join(repo.DataPath, filepath.FromSlash(name))+ if err := os.MkdirAll(filepath.Dir(name), 0755); err != nil {+ t.Fatal(err)+ }+ if err := os.WriteFile(name, data, 0600); err != nil {+ t.Fatal(err)+ }+ stamp := time.Unix(1700000000+tick, 0)+ if err := os.Chtimes(name, stamp, stamp); err != nil {+ t.Fatal(err)+ }+}++func TestDocumentAddressingCheckoutSourcePreflight(t *testing.T) {+ ordinary := `{"Spec":"2","Properties":{"title":"Ordinary"}}`+ future := `{"Spec":"999","Properties":{"title":"Future"}}`+ title := `{"Spec":"2","Properties":{"title":{"format":"future"}}}`+ slash := `{"Spec":"6","Properties":{"title":"Parent/Child"}}`+ var slashError error+ if documentaddressing.DefaultMode() < 2 {+ slashError = documentaddressing.ErrModeConflict+ }+ for _, test := range []struct {+ name, incoming, current, config, floor, document string+ wantErr error+ }{+ {name: "incoming future spec", incoming: future, current: ordinary, wantErr: treenode.ErrSpecTooNew},+ {name: "incoming future title", incoming: title, current: ordinary, wantErr: documentaddressing.ErrInvalidTitle},+ {name: "current future spec", incoming: ordinary, current: future, wantErr: treenode.ErrSpecTooNew},+ {name: "current future title", incoming: ordinary, current: title, wantErr: documentaddressing.ErrInvalidTitle},+ {name: "mode before source", incoming: future, current: ordinary, config: `{"documentAddressingVersion":3}`, wantErr: documentaddressing.ErrUnsupported},+ {name: "floor before source", incoming: future, current: ordinary,+ floor: `{"version":1,"notebook":"` + checkoutSourceBox + `","minimumMode":3}`, wantErr: documentaddressing.ErrUnsupported},+ {name: "ordinary", incoming: ordinary, current: `{"Spec":"2","Properties":{"title":"Old"}}`},+ {name: "literal slash", incoming: slash, current: ordinary, wantErr: slashError},+ {name: "BOM source", incoming: "\xef\xbb\xbf" + ordinary, current: ordinary},+ {name: "recover corrupt source", incoming: ordinary, current: `{"Spec":"2","Properties":`},+ {name: "asset sy", incoming: future, current: ordinary, document: "assets/" + checkoutSourceDocument},+ } {+ t.Run(test.name, func(t *testing.T) {+ repo := newCheckoutSourceTestRepo(t)+ document := test.document+ if document == "" {+ document = checkoutSourceBox + "/" + checkoutSourceDocument+ }+ config := test.config+ if config == "" {+ config = `{}`+ }+ writeCheckoutSourceTestFile(t, repo, checkoutSourceBox+"/"+documentaddressing.ConfigPath, []byte(config), 10)+ if test.floor != "" {+ writeCheckoutSourceTestFile(t, repo, checkoutSourceBox+"/"+documentaddressing.FloorPath, []byte(test.floor), 10)+ }+ writeCheckoutSourceTestFile(t, repo, document, []byte(test.incoming), 10)+ writeCheckoutSourceTestFile(t, repo, "assets/add.txt", []byte("incoming addition"), 10)+ target, err := repo.Index("incoming source", false, nil)+ if err != nil {+ t.Fatal(err)+ }+ writeCheckoutSourceTestFile(t, repo, document, []byte(test.current), 20)+ writeCheckoutSourceTestFile(t, repo, checkoutSourceBox+"/"+documentaddressing.ConfigPath, []byte(`{}`), 20)+ if test.floor != "" {+ if err = os.Remove(filepath.Join(repo.DataPath, checkoutSourceBox, documentaddressing.FloorPath)); err != nil {+ t.Fatal(err)+ }+ }+ if err = os.Remove(filepath.Join(repo.DataPath, "assets", "add.txt")); err != nil {+ t.Fatal(err)+ }+ writeCheckoutSourceTestFile(t, repo, "assets/keep.txt", []byte("retained local file"), 20)+ before, err := archiveDirectoryFiles(repo.DataPath)+ if err != nil {+ t.Fatal(err)+ }+ beforeDocument, err := os.Stat(filepath.Join(repo.DataPath, filepath.FromSlash(document)))+ if err != nil {+ t.Fatal(err)+ }+ _, _, err = repo.Checkout(target.ID, nil)+ if !errors.Is(err, test.wantErr) {+ t.Fatalf("source preflight: %v, expected %v", err, test.wantErr)+ }+ if test.wantErr != nil {+ after, readErr := archiveDirectoryFiles(repo.DataPath)+ if readErr != nil || !reflect.DeepEqual(before, after) {+ t.Fatal("rejected batch changed a source, addition, or removal", readErr)+ }+ afterDocument, statErr := os.Stat(filepath.Join(repo.DataPath, filepath.FromSlash(document)))+ if statErr != nil || !beforeDocument.ModTime().Equal(afterDocument.ModTime()) {+ t.Fatal("rejected batch changed the source modification time", statErr)+ }+ } else {+ actual, readErr := os.ReadFile(filepath.Join(repo.DataPath, filepath.FromSlash(document)))+ if readErr != nil || !bytes.Equal(actual, []byte(test.incoming)) {+ t.Fatal("checkout changed source bytes", readErr)+ }+ if _, err = os.Stat(filepath.Join(repo.DataPath, "assets", "add.txt")); err != nil {+ t.Fatal("accepted batch did not apply its addition", err)+ }+ if _, err = os.Stat(filepath.Join(repo.DataPath, "assets", "keep.txt")); !os.IsNotExist(err) {+ t.Fatal("accepted batch did not apply its removal", err)+ }+ }+ // 拒绝或采用均不改写仓库中的恢复来源。+ files, err := repo.GetFiles(target)+ if err != nil {+ t.Fatal(err)+ }+ var stored []byte+ for _, file := range files {+ if strings.TrimPrefix(file.Path, "/") == document {+ stored, err = repo.OpenFile(file)+ break+ }+ }+ if err != nil || !bytes.Equal(stored, []byte(test.incoming)) {+ t.Fatal("checkout changed the incoming snapshot", err)+ }+ })+ }+}++func TestDocumentAddressingCheckoutSourceRemoval(t *testing.T) {+ for _, whole := range []bool{false, true} {+ t.Run(map[bool]string{false: "partial", true: "whole notebook"}[whole], func(t *testing.T) {+ repo := newCheckoutSourceTestRepo(t)+ writeCheckoutSourceTestFile(t, repo, "assets/retained.txt", []byte("unchanged"), 10)+ if !whole {+ writeCheckoutSourceTestFile(t, repo, checkoutSourceBox+"/"+documentaddressing.ConfigPath, []byte(`{}`), 10)+ }+ writeCheckoutSourceTestFile(t, repo, "assets/add.txt", []byte("incoming addition"), 10)+ target, err := repo.Index("remove unsupported source", false, nil)+ if err != nil {+ t.Fatal(err)+ }+ writeCheckoutSourceTestFile(t, repo, checkoutSourceBox+"/"+documentaddressing.ConfigPath, []byte(`{}`), 20)+ writeCheckoutSourceTestFile(t, repo, checkoutSourceBox+"/"+checkoutSourceDocument,+ []byte(`{"Spec":"999","Properties":{"title":"Future"}}`), 20)+ if err = os.Remove(filepath.Join(repo.DataPath, "assets", "add.txt")); err != nil {+ t.Fatal(err)+ }+ writeCheckoutSourceTestFile(t, repo, "assets/keep.txt", []byte("retained until acceptance"), 20)+ before, err := archiveDirectoryFiles(repo.DataPath)+ if err != nil {+ t.Fatal(err)+ }+ _, _, err = repo.Checkout(target.ID, nil)+ if whole {+ if err != nil {+ t.Fatal("whole notebook deletion was blocked", err)+ }+ if _, err = os.Stat(filepath.Join(repo.DataPath, checkoutSourceBox, checkoutSourceDocument)); !os.IsNotExist(err) {+ t.Fatal("whole notebook deletion retained its document", err)+ }+ } else {+ if !errors.Is(err, treenode.ErrSpecTooNew) {+ t.Fatal("partial deletion accepted an unsupported source", err)+ }+ after, readErr := archiveDirectoryFiles(repo.DataPath)+ if readErr != nil || !reflect.DeepEqual(before, after) {+ t.Fatal("rejected partial deletion changed workspace files", readErr)+ }+ }+ })+ }+}++func TestDocumentAddressingCheckoutPreservesOpaqueEncryptedSource(t *testing.T) {+ for _, futureEnvelope := range []bool{false, true} {+ t.Run(map[bool]string{false: "released ciphertext", true: "opaque future envelope"}[futureEnvelope], func(t *testing.T) {+ repo := newCheckoutSourceTestRepo(t)+ fixtureDir := filepath.Join("testdata", "structured_titles_legacy")+ manifestData, err := os.ReadFile(filepath.Join(fixtureDir, "manifest.json"))+ if err != nil {+ t.Fatal(err)+ }+ var manifest struct {+ SHA256 map[string]string `json:"sha256"`+ }+ if err = json.Unmarshal(manifestData, &manifest); err != nil {+ t.Fatal(err)+ }+ const boxID = "20261023180038-mbrj58v"+ const document = boxID + "/20261023180038-pwkjtsl.sy"+ for name, expected := range manifest.SHA256 {+ if !strings.HasPrefix(name, "data/"+boxID+"/") && !strings.HasPrefix(name, "data/.siyuan/") {+ continue+ }+ data, err := os.ReadFile(filepath.Join(fixtureDir, filepath.FromSlash(name)))+ if err != nil {+ t.Fatal(err)+ }+ digest := sha256.Sum256(data)+ if hex.EncodeToString(digest[:]) != expected {+ t.Fatal("released encrypted fixture changed", name)+ }+ writeCheckoutSourceTestFile(t, repo, strings.TrimPrefix(name, "data/"), data, 10)+ }+ original, err := os.ReadFile(filepath.Join(repo.DataPath, filepath.FromSlash(document)))+ if err != nil || !util.IsCiphertext(original) {+ t.Fatal("released ciphertext fixture is unavailable", err)+ }+ incoming := bytes.Clone(original)+ if futureEnvelope {+ incoming[4] = 255+ writeCheckoutSourceTestFile(t, repo, document, incoming, 10)+ }+ target, err := repo.Index("opaque encrypted source", false, nil)+ if err != nil {+ t.Fatal(err)+ }+ current := bytes.Clone(original)+ current[len(current)-1] ^= 1+ writeCheckoutSourceTestFile(t, repo, document, current, 20)+ before, err := archiveDirectoryFiles(repo.DataPath)+ if err != nil {+ t.Fatal(err)+ }+ previousProvider := filesys.DEKProvider+ filesys.DEKProvider = func(string) ([]byte, error) {+ t.Error("opaque checkout requested a notebook key")+ return nil, errors.New("locked notebook")+ }+ t.Cleanup(func() { filesys.DEKProvider = previousProvider })+ if _, _, err = repo.Checkout(target.ID, nil); err != nil {+ t.Fatal("opaque ciphertext checkout required an unlocked notebook", err)+ }+ actual, err := os.ReadFile(filepath.Join(repo.DataPath, filepath.FromSlash(document)))+ if err != nil || !bytes.Equal(actual, incoming) {+ t.Fatal("opaque checkout changed ciphertext", err)+ }+ after, err := archiveDirectoryFiles(repo.DataPath)+ if err != nil {+ t.Fatal(err)+ }+ delete(before, document)+ delete(after, document)+ if !reflect.DeepEqual(before, after) {+ t.Fatal("opaque checkout changed configuration or key recovery material")+ }+ })+ }+}