Memos is an open-source, self-hosted note-taking app that brings your everyday thoughts together in a personal timeline. Your data stays yours, and you choose what to keep private or share.
Go dockerfossgomarkdownmemomicroblognote-takingnotesopen-sourceown-your-datareactself-hostedsqlitetimeline
feat(store): add Cloudflare D1 database driver (#6330)
Johnny2e3db14741a4
diff --git a/.github/workflows/backend-tests.yml b/.github/workflows/backend-tests.ymlindex a1eca1e56..732407ab9 100644--- a/.github/workflows/backend-tests.yml+++ b/.github/workflows/backend-tests.yml@@ -65,7 +65,7 @@ jobs:run: |case "${{ matrix.test-group }}" instore)- # Run store tests for all drivers (sqlite, mysql, postgres)+ # Run store tests for all drivers (sqlite, mysql, postgres, d1)go test -v -coverprofile=coverage.out -covermode=atomic ./store/...;;server)diff --git a/cmd/memos/main.go b/cmd/memos/main.goindex 4f213a4a6..432d75075 100644--- a/cmd/memos/main.go+++ b/cmd/memos/main.go@@ -60,7 +60,7 @@ func init() {rootCmd.Flags().Int("port", 8081, "port of server")rootCmd.Flags().String("unix-sock", "", "path to the unix socket, overrides --addr and --port")rootCmd.Flags().String("data", "", "data directory")- rootCmd.Flags().String("driver", "sqlite", "database driver")+ rootCmd.Flags().String("driver", "sqlite", "database driver (sqlite, mysql, postgres, d1)")rootCmd.Flags().String("dsn", "", "database source name (DSN)")rootCmd.Flags().String("instance-url", "", "canonical external URL of the Memos instance")rootCmd.Flags().Bool("allow-private-webhooks", false, "allow webhooks to access any private/reserved IP address")diff --git a/docs/design/cloudflare-d1.md b/docs/design/cloudflare-d1.mdnew file mode 100644index 000000000..415b15a45--- /dev/null+++ b/docs/design/cloudflare-d1.md@@ -0,0 +1,113 @@+# Cloudflare D1 Driver Design++Status: Accepted++Date: 2026-09-15++Existing domain language: [Memos context](../../CONTEXT.md)++Related: [Multi-Spaces Design](multi-spaces.md)++## Summary++Memos gains a fourth database engine, Cloudflare D1, selected with+`--driver d1`. D1 runs SQLite, but it is reached over Cloudflare's REST API+rather than a local file or socket, it exposes no interactive transactions,+and it accepts no custom SQL functions. Those three facts make it a different+engine from the SQLite driver rather than a variant of it, so it lives in its+own package, `store/db/d1`, with its own migration directory, its own filter+dialect, and its own test harness, exactly like MySQL and PostgreSQL.++## Goals++- Run an unmodified Memos binary against a D1 database from any host.+- Keep every store invariant the other drivers enforce: no partial writes,+ no stale-read overwrites, the same error values for the same conditions.+- Share no Go code with the SQLite driver so either can change freely.+- Run the full store test suite without a Cloudflare account.++## Non-goals++- Running Memos inside Cloudflare Workers. The server is a Go binary; D1 is+ only its database.+- Hiding D1's limits. Statements bind at most 100 parameters, a row or blob+ is at most 2 MB, and every query is an HTTP round trip. The driver respects+ the first, surfaces the second as an error, and does nothing about the third.+- General regular-expression filters. D1 has no `REGEXP` operator and cannot+ load one. `matches()` works only for literal text with optional `^` and `$`+ anchors, rendered as case-sensitive substring, prefix, suffix, or equality+ tests; any other pattern is rejected at compile time.++## Configuration++```+--driver d1 --dsn 'd1://<account_id>/<database_id>?token=<api_token>'+```++The token may instead come from `CLOUDFLARE_API_TOKEN` so it stays out of+process listings. An `endpoint` query parameter overrides the API base URL and+exists for the test emulator. The token needs the D1 Edit permission.++## Transport++The driver holds one HTTP client for the database. Reads and single writes+go to the `/raw` endpoint, which returns rows as arrays with a column list.+Bound parameters travel as JSON numbers, strings, and nulls; booleans become+integers. Binary values cannot travel in JSON, so blob columns are written+with `unhex(?)` bound to hex text and read back through `hex(blob)`.++A thin `database/sql` adapter wraps the same client so `Driver.GetDB()` keeps+working for the migrator and for tests. Its transactions buffer writes and+send them as one atomic request on commit. A read after a buffered write is+refused, because the write is not visible until the batch runs.++## Atomicity without transactions++D1 offers two atomic units: a single statement, and a `batch` request whose+statements run in one transaction. Every driver method is shaped around them.++1. Validation reads run first, outside any transaction, and produce the+ store's documented errors (`ErrSpacePermissionDenied`,+ `ErrMemoMutationConflict`, and so on).+2. The writes are collected into one batch and committed together.+3. Preconditions the reads established are re-asserted inside the batch with+ guard statements. A guard inserts the value `0` into `d1_guard`, a table+ whose CHECK constraint rejects anything but `1`, but only when the guarded+ condition is false. The constraint failure aborts and rolls back the whole+ batch, and the driver maps it to the conflict error the caller expects.+ `d1_guard` never holds rows.++A batch cannot read its own writes, so an id generated by an `INSERT` cannot+feed a later statement. Methods that need it use the unique natural key+instead: a membership created with a new space references+`(SELECT id FROM space WHERE uid = ?)`, and the space is read back by uid+after the commit.++## Schema and migrations++`store/migration/d1/LATEST.sql` is written for D1: the SQLite schema plus the+`d1_guard` table. D1 enforces foreign keys by default, as MySQL and PostgreSQL+do, so the `memo_share` cascade is declared. The directory carries no+incremental migrations: a D1 database can never be older than the schema+that introduced the driver, so `LATEST.sql` is the whole history. The+migrator reports the highest version any driver's migration directory+defines for such a driver, and future migrations are added as plain SQL+files under `store/migration/d1/<version>/` alongside the other drivers'.+They must avoid the `PRAGMA foreign_keys` toggles the SQLite migrations use,+which D1 ignores.++## Filter dialect++`filter.DialectD1` renders like SQLite except that case folding uses the+built-in `LOWER()` (ASCII only, because the Unicode fold used by the SQLite+driver is a registered Go function) and `matches()` accepts only literal+patterns with optional anchors.++## Testing++`store/db/d1/d1test` emulates the D1 REST API in-process on a private SQLite+database: the same endpoints, request bodies, response shapes, implicit+transaction per request, atomic batches, foreign-key enforcement, and the+100-parameter limit, with no custom functions registered. `DRIVER=d1` runs+the whole store suite against it. Setting `D1_DSN` points the suite at a real+database for spot checks.diff --git a/filter/engine_test.go b/filter/engine_test.goindex e7b734a16..5678de330 100644--- a/filter/engine_test.go+++ b/filter/engine_test.go@@ -80,7 +80,7 @@ func TestRenderSpaceFilters(t *testing.T) {for _, schema := range []Schema{NewSchema(), NewAttachmentSchema()} {engine, err := NewEngine(schema)require.NoError(t, err)- for _, dialect := range []DialectName{DialectSQLite, DialectMySQL, DialectPostgres} {+ for _, dialect := range []DialectName{DialectSQLite, DialectD1, DialectMySQL, DialectPostgres} {assigned, err := engine.CompileToStatement(context.Background(),`space == "spaces/team"`,@@ -96,7 +96,7 @@ func TestRenderSpaceFilters(t *testing.T) {require.Empty(t, unassigned.Args, schema.Name, dialect)}- for _, dialect := range []DialectName{DialectSQLite, DialectMySQL, DialectPostgres} {+ for _, dialect := range []DialectName{DialectSQLite, DialectD1, DialectMySQL, DialectPostgres} {assigned, err := engine.CompileToStatement(context.Background(), `space != null`, RenderOptions{Dialect: dialect})require.NoError(t, err, schema.Name, dialect)require.Contains(t, assigned.SQL, "IS NOT NULL", schema.Name, dialect)@@ -283,7 +283,7 @@ func TestRenderEmptyTagListIsFalse(t *testing.T) {engine, err := NewEngine(NewSchema())require.NoError(t, err)- for _, dialect := range []DialectName{DialectSQLite, DialectMySQL, DialectPostgres} {+ for _, dialect := range []DialectName{DialectSQLite, DialectD1, DialectMySQL, DialectPostgres} {stmt, err := engine.CompileToStatement(context.Background(), `tag in []`, RenderOptions{Dialect: dialect})require.NoError(t, err, dialect)require.Equal(t, "1 = 0", stmt.SQL, dialect)@@ -412,6 +412,7 @@ func TestRenderStartsWithPerDialect(t *testing.T) {fragments []string}{{DialectSQLite, []string{"memos_unicode_lower(", "`memo`.`content`", `ESCAPE '\'`}},+ {DialectD1, []string{"LOWER(`memo`.`content`) LIKE LOWER(?)", `ESCAPE '\'`}},{DialectPostgres, []string{"memo.content ILIKE $1"}},{DialectMySQL, []string{"`memo`.`content` LIKE ?"}},}@@ -650,3 +651,40 @@ func TestHasLocationSQLiteBehavior(t *testing.T) {require.Equal(t, tc.want, selectMemoIDs(t, db, stmt), tc.expr)}}++func TestRenderRegexLiteralFallbackOnD1(t *testing.T) {+ t.Parallel()++ engine, err := NewEngine(NewSchema())+ require.NoError(t, err)++ cases := []struct {+ expression string+ sql string+ args []any+ }{+ {`content.matches("^release")`, "instr(`memo`.`content`, ?) = 1", []any{"release"}},+ {`content.matches("notes$")`, "substr(`memo`.`content`, -length(?)) = ?", []any{"notes", "notes"}},+ {`content.matches("^done$")`, "`memo`.`content` = ?", []any{"done"}},+ {`content.matches("v1\\.2")`, "instr(`memo`.`content`, ?) > 0", []any{"v1.2"}},+ // An anchor alone matches everything and renders as no condition.+ {`content.matches("^")`, "", []any{}},+ }+ for _, tc := range cases {+ stmt, err := engine.CompileToStatement(context.Background(), tc.expression, RenderOptions{Dialect: DialectD1})+ require.NoError(t, err, tc.expression)+ require.Equal(t, tc.sql, stmt.SQL, tc.expression)+ require.Equal(t, tc.args, stmt.Args, tc.expression)+ }++ for _, expression := range []string{+ `content.matches("v[0-9]+")`,+ `content.matches("cat|mouse")`,+ `content.matches("a.b")`,+ `content.matches("\\d+")`,+ `content.matches("x*")`,+ } {+ _, err := engine.CompileToStatement(context.Background(), expression, RenderOptions{Dialect: DialectD1})+ require.ErrorContains(t, err, "not supported on Cloudflare D1", expression)+ }+}diff --git a/filter/render.go b/filter/render.goindex 1a4880895..6d1668da1 100644--- a/filter/render.go+++ b/filter/render.go@@ -3,6 +3,7 @@ package filterimport ("fmt""strings"+ "unicode""github.com/pkg/errors")@@ -185,7 +186,7 @@ func (r *renderer) renderComparison(cond *ComparisonCondition) (renderResult, er// accessorSpec maps a CEL timestamp accessor to per-dialect SQL date-part tokens// and the offset to subtract so the result matches CEL's base (e.g. CEL months// are 0-based but every dialect reports 1-based, so off=1). off is indexed-// [sqlite, postgres, mysql].+// [sqlite, postgres, mysql]; D1 shares the SQLite entry.type accessorSpec struct {sqlite string // strftime format specifierpg string // EXTRACT field@@ -236,7 +237,7 @@ func (r *renderer) timestampAccessorExpr(field Field, accessor string) (string,var base stringvar off intswitch r.dialect {- case DialectSQLite:+ case DialectSQLite, DialectD1:base = fmt.Sprintf("CAST(strftime('%s', %s, 'unixepoch') AS INTEGER)", spec.sqlite, col)off = spec.off[0]case DialectPostgres:@@ -370,7 +371,7 @@ func (r *renderer) renderJSONBoolComparison(field Field, op ComparisonOperator,jsonExpr := jsonExtractExpr(r.dialect, field)switch r.dialect {- case DialectSQLite:+ case DialectSQLite, DialectD1:switch op {case CompareEq:if field.Name == "has_task_list" {@@ -554,11 +555,75 @@ func (r *renderer) renderRegex(cond *RegexCondition) (renderResult, error) {case DialectMySQL, DialectSQLite:// MySQL has a native REGEXP operator; SQLite uses the registered regexp() function.return renderResult{sql: fmt.Sprintf("%s REGEXP %s", column, r.addArg(cond.Pattern))}, nil+ case DialectD1:+ return r.renderLiteralRegex(column, cond.Pattern)default:return renderResult{}, errors.Errorf("unsupported dialect %s", r.dialect)}}+// renderLiteralRegex renders matches() for Cloudflare D1, which exposes no+// REGEXP operator and accepts no custom functions. Only patterns that are a+// literal string with optional ^ and $ anchors can be expressed, as+// case-sensitive substring, prefix, suffix, or equality tests; any other+// pattern is rejected at compile time.+func (r *renderer) renderLiteralRegex(column, pattern string) (renderResult, error) {+ literal, anchoredStart, anchoredEnd, ok := literalRegexPattern(pattern)+ if !ok {+ return renderResult{}, errors.Errorf("regular expression %q is not supported on Cloudflare D1: only literal text with optional ^ and $ anchors can be matched", pattern)+ }+ switch {+ case anchoredStart && anchoredEnd:+ return renderResult{sql: fmt.Sprintf("%s = %s", column, r.addArg(literal))}, nil+ case literal == "":+ return renderResult{sql: "1 = 1", trivial: true}, nil+ case anchoredStart:+ return renderResult{sql: fmt.Sprintf("instr(%s, %s) = 1", column, r.addArg(literal))}, nil+ case anchoredEnd:+ lengthValue := r.addArg(literal)+ compareValue := r.addArg(literal)+ return renderResult{sql: fmt.Sprintf("substr(%s, -length(%s)) = %s", column, lengthValue, compareValue)}, nil+ default:+ return renderResult{sql: fmt.Sprintf("instr(%s, %s) > 0", column, r.addArg(literal))}, nil+ }+}++// literalRegexPattern reports whether pattern is a literal string with+// optional ^ and $ anchors and returns the literal text. Backslash-escaped+// punctuation is accepted as the punctuation character; every other+// metacharacter or escape sequence makes the pattern non-literal.+func literalRegexPattern(pattern string) (literal string, anchoredStart bool, anchoredEnd bool, ok bool) {+ if strings.HasPrefix(pattern, "^") {+ anchoredStart = true+ pattern = pattern[1:]+ }+ var builder strings.Builder+ runes := []rune(pattern)+ for i := 0; i < len(runes); i++ {+ c := runes[i]+ switch {+ case c == '\\':+ if i+1 >= len(runes) {+ return "", false, false, false+ }+ next := runes[i+1]+ if unicode.IsLetter(next) || unicode.IsDigit(next) {+ // \d, \w, \n and similar are classes or controls, not literals.+ return "", false, false, false+ }+ builder.WriteRune(next)+ i+++ case c == '$' && i == len(runes)-1:+ anchoredEnd = true+ case strings.ContainsRune(`.*+?()[]{}|^$`, c):+ return "", false, false, false+ default:+ builder.WriteRune(c)+ }+ }+ return builder.String(), anchoredStart, anchoredEnd, true+}+// foldedLike renders a case-insensitive LIKE comparison of colExpr against a// (already metacharacter-escaped) pattern, using each dialect's case-folding.func (r *renderer) foldedLike(colExpr, pattern string) string {@@ -567,6 +632,10 @@ func (r *renderer) foldedLike(colExpr, pattern string) string {// memos_unicode_lower gives Unicode-aware folding; ESCAPE '\' is required// because SQLite has no default LIKE escape character.return fmt.Sprintf(`memos_unicode_lower(%s) LIKE memos_unicode_lower(%s) ESCAPE '\'`, colExpr, r.addArg(pattern))+ case DialectD1:+ // D1 cannot register custom functions, so folding is limited to the+ // ASCII range covered by SQLite's built-in LOWER.+ return fmt.Sprintf(`LOWER(%s) LIKE LOWER(%s) ESCAPE '\'`, colExpr, r.addArg(pattern))case DialectPostgres:// ILIKE is case-insensitive; backslash is the default escape character.return fmt.Sprintf("%s ILIKE %s", colExpr, r.addArg(pattern))@@ -619,7 +688,7 @@ func (r *renderer) renderTagComprehension(field Field, pred PredicateExpr, kindvar elements, length stringswitch r.dialect {- case DialectSQLite:+ case DialectSQLite, DialectD1:arrayExpr = fmt.Sprintf("COALESCE(%s, JSON_ARRAY())", arrayExpr)elements = fmt.Sprintf("json_each(%s) AS tag_item", arrayExpr)length = fmt.Sprintf("json_array_length(%s)", arrayExpr)@@ -672,7 +741,7 @@ func (r *renderer) tagElementPredicateSQL(element string, pred PredicateExpr) (scase *EqualsPredicate:placeholder := r.addArg(p.Value)switch r.dialect {- case DialectSQLite:+ case DialectSQLite, DialectD1:return fmt.Sprintf("(%s COLLATE BINARY) = (%s COLLATE BINARY)", element, placeholder), nilcase DialectMySQL:return fmt.Sprintf("CAST(%s AS BINARY) = CAST(%s AS BINARY)", element, placeholder), nil@@ -697,7 +766,7 @@ func (r *renderer) tagElementTextMatch(element string, mode TextMatchMode, value}switch r.dialect {- case DialectSQLite:+ case DialectSQLite, DialectD1:switch mode {case TextMatchPrefix:return fmt.Sprintf("instr(%s, %s) = 1", element, r.addArg(value)), nil@@ -739,7 +808,7 @@ func tagLikePattern(mode TextMatchMode, value string) string {func (r *renderer) jsonExistsSQL(field Field) (string, error) {expr := jsonExtractExpr(r.dialect, field)switch r.dialect {- case DialectSQLite, DialectPostgres:+ case DialectSQLite, DialectD1, DialectPostgres:// SQLite's JSON_EXTRACT and Postgres' terminal ->> fold both a missing// key and a JSON null to SQL NULL.return fmt.Sprintf("%s IS NOT NULL", expr), nil@@ -779,7 +848,7 @@ func (r *renderer) renderJSONExistsComparison(field Field, op ComparisonOperatorfunc (r *renderer) jsonBoolPredicate(field Field) (string, error) {expr := jsonExtractExpr(r.dialect, field)switch r.dialect {- case DialectSQLite:+ case DialectSQLite, DialectD1:return fmt.Sprintf("%s IS TRUE", expr), nilcase DialectMySQL:return fmt.Sprintf("COALESCE(%s, CAST('false' AS JSON)) = CAST('true' AS JSON)", expr), nil@@ -832,7 +901,7 @@ func (r *renderer) addArg(value any) string {func (r *renderer) addBoolArg(value bool) string {var v anyswitch r.dialect {- case DialectSQLite:+ case DialectSQLite, DialectD1:if value {v = 1} else {@@ -913,7 +982,7 @@ func jsonPath(field Field) string {func jsonExtractExpr(d DialectName, field Field) string {column := qualifyColumn(d, field.Column)switch d {- case DialectSQLite, DialectMySQL:+ case DialectSQLite, DialectD1, DialectMySQL:return fmt.Sprintf("JSON_EXTRACT(%s, '%s')", column, jsonPath(field))case DialectPostgres:return buildPostgresJSONAccessor(column, field.JSONPath, true)@@ -925,7 +994,7 @@ func jsonExtractExpr(d DialectName, field Field) string {func jsonArrayExpr(d DialectName, field Field) string {column := qualifyColumn(d, field.Column)switch d {- case DialectSQLite, DialectMySQL:+ case DialectSQLite, DialectD1, DialectMySQL:return fmt.Sprintf("JSON_EXTRACT(%s, '%s')", column, jsonPath(field))case DialectPostgres:return buildPostgresJSONAccessor(column, field.JSONPath, false)@@ -937,7 +1006,7 @@ func jsonArrayExpr(d DialectName, field Field) string {func jsonArrayLengthExpr(d DialectName, field Field) string {arrayExpr := jsonArrayExpr(d, field)switch d {- case DialectSQLite:+ case DialectSQLite, DialectD1:return fmt.Sprintf("JSON_ARRAY_LENGTH(COALESCE(%s, JSON_ARRAY()))", arrayExpr)case DialectMySQL:return fmt.Sprintf("JSON_LENGTH(COALESCE(%s, JSON_ARRAY()))", arrayExpr)diff --git a/filter/schema.go b/filter/schema.goindex 5e4a7b652..e86f0443d 100644--- a/filter/schema.go+++ b/filter/schema.go@@ -14,6 +14,8 @@ const (DialectSQLite DialectName = "sqlite"DialectMySQL DialectName = "mysql"DialectPostgres DialectName = "postgres"+ // DialectD1 targets Cloudflare D1: SQLite syntax without custom functions.+ DialectD1 DialectName = "d1")// FieldType represents the logical type of a field.@@ -105,6 +107,7 @@ func NewSchema() Schema {Column: Column{Table: "memo_creator", Name: "username"},Expressions: map[DialectName]string{DialectSQLite: "('users/' || %s)",+ DialectD1: "('users/' || %s)",DialectMySQL: "CONCAT('users/', %s)",DialectPostgres: "('users/' || %s)",},@@ -135,6 +138,7 @@ func NewSchema() Schema {// PostgreSQL and SQLite store created_ts as BIGINT (epoch), no conversion neededDialectPostgres: "%s",DialectSQLite: "%s",+ DialectD1: "%s",},},"updated_ts": {@@ -148,6 +152,7 @@ func NewSchema() Schema {// PostgreSQL and SQLite store updated_ts as BIGINT (epoch), no conversion neededDialectPostgres: "%s",DialectSQLite: "%s",+ DialectD1: "%s",},},"pinned": {@@ -179,6 +184,7 @@ func NewSchema() Schema {Column: Column{Table: "memo_space", Name: "uid"},Expressions: map[DialectName]string{DialectSQLite: "CASE WHEN `memo`.`space_id` IS NULL THEN NULL WHEN %[1]s IS NULL THEN '' ELSE ('spaces/' || %[1]s) END",+ DialectD1: "CASE WHEN `memo`.`space_id` IS NULL THEN NULL WHEN %[1]s IS NULL THEN '' ELSE ('spaces/' || %[1]s) END",DialectMySQL: "CASE WHEN `memo`.`space_id` IS NULL THEN NULL WHEN %[1]s IS NULL THEN '' ELSE CONCAT('spaces/', %[1]s) END",DialectPostgres: "CASE WHEN memo.space_id IS NULL THEN NULL WHEN %[1]s IS NULL THEN '' ELSE ('spaces/' || %[1]s) END",},@@ -315,6 +321,7 @@ func NewAttachmentSchema() Schema {// PostgreSQL and SQLite store created_ts as BIGINT (epoch), no conversion neededDialectPostgres: "%s",DialectSQLite: "%s",+ DialectD1: "%s",},},"memo_id": {@@ -335,6 +342,7 @@ func NewAttachmentSchema() Schema {Column: Column{Table: "attachment_space", Name: "uid"},Expressions: map[DialectName]string{DialectSQLite: "CASE WHEN `attachment`.`memo_id` IS NULL OR `memo`.`space_id` IS NULL THEN NULL WHEN %[1]s IS NULL THEN '' ELSE ('spaces/' || %[1]s) END",+ DialectD1: "CASE WHEN `attachment`.`memo_id` IS NULL OR `memo`.`space_id` IS NULL THEN NULL WHEN %[1]s IS NULL THEN '' ELSE ('spaces/' || %[1]s) END",DialectMySQL: "CASE WHEN `attachment`.`memo_id` IS NULL OR `memo`.`space_id` IS NULL THEN NULL WHEN %[1]s IS NULL THEN '' ELSE CONCAT('spaces/', %[1]s) END",DialectPostgres: "CASE WHEN attachment.memo_id IS NULL OR memo.space_id IS NULL THEN NULL WHEN %[1]s IS NULL THEN '' ELSE ('spaces/' || %[1]s) END",},diff --git a/internal/profile/profile.go b/internal/profile/profile.goindex e3293267c..3ff0d9e35 100644--- a/internal/profile/profile.go+++ b/internal/profile/profile.go@@ -26,8 +26,7 @@ type Profile struct {Data string// DSN points to where memos stores its own dataDSN string- // Driver is the database driver- // sqlite, mysql+ // Driver is the database driver: sqlite, mysql, postgres, or d1.Driver string// Version is the current version of serverVersion stringdiff --git a/server/api/v1/user_service_memo_views.go b/server/api/v1/user_service_memo_views.goindex 680bfb49f..5b4cd0a99 100644--- a/server/api/v1/user_service_memo_views.go+++ b/server/api/v1/user_service_memo_views.go@@ -261,6 +261,8 @@ func (s *APIV1Service) filterDialect() filter.DialectName {return filter.DialectMySQLcase "postgres":return filter.DialectPostgres+ case "d1":+ return filter.DialectD1default:return filter.DialectSQLite}diff --git a/store/db/d1/attachment.go b/store/db/d1/attachment.gonew file mode 100644index 000000000..71ab852f3--- /dev/null+++ b/store/db/d1/attachment.go@@ -0,0 +1,307 @@+package d1++import (+ "context"+ "database/sql"+ "encoding/hex"+ "fmt"+ "strings"++ "github.com/pkg/errors"+ "google.golang.org/protobuf/encoding/protojson"++ "github.com/usememos/memos/filter"+ storepb "github.com/usememos/memos/proto/gen/store"+ "github.com/usememos/memos/store"+)++// CreateAttachment inserts an attachment, checking the memo write policy when one is set.+func (d *DB) CreateAttachment(ctx context.Context, create *store.Attachment) (*store.Attachment, error) {+ columns, values, args, err := attachmentInsertValues(create)+ if err != nil {+ return nil, err+ }+ if create.Policy == nil {+ stmt := "INSERT INTO attachment (" + strings.Join(columns, ", ") + ") VALUES (" + strings.Join(values, ", ") + ") RETURNING id, created_ts, updated_ts"+ if err := d.db.QueryRowContext(ctx, stmt, args...).Scan(&create.ID, &create.CreatedTs, &create.UpdatedTs); err != nil {+ return nil, err+ }+ return create, nil+ }+ if create.MemoID == nil || create.CreatorID != create.Policy.ActorUserID {+ return nil, store.ErrMemoPermissionDenied+ }+ if err := validateMemoWritePolicy(ctx, d.db, *create.MemoID, create.Policy, nil); err != nil {+ return nil, err+ }+ // Without a transaction the validated memo can vanish before the insert;+ // selecting the values from the memo row makes the insert conditional on+ // its continued existence, so a vanished memo yields no row instead of an+ // orphaned attachment.+ args = append(args, *create.MemoID)+ stmt := "INSERT INTO attachment (" + strings.Join(columns, ", ") + ") SELECT " + strings.Join(values, ", ") + " FROM memo WHERE memo.id = ? RETURNING id, created_ts, updated_ts"+ if err := d.db.QueryRowContext(ctx, stmt, args...).Scan(&create.ID, &create.CreatedTs, &create.UpdatedTs); err != nil {+ if errors.Is(err, sql.ErrNoRows) {+ return nil, store.ErrMemoMutationConflict+ }+ return nil, err+ }+ return create, nil+}++// attachmentInsertValues renders the column list, the value expressions, and+// their bindings for an attachment insert. The blob travels as hex text+// through unhex(?) because the D1 JSON transport cannot carry raw bytes.+func attachmentInsertValues(create *store.Attachment) (columns []string, values []string, args []any, err error) {+ storageType := ""+ if create.StorageType != storepb.AttachmentStorageType_ATTACHMENT_STORAGE_TYPE_UNSPECIFIED {+ storageType = create.StorageType.String()+ }+ payloadString := "{}"+ if create.Payload != nil {+ bytes, err := protojson.Marshal(create.Payload)+ if err != nil {+ return nil, nil, nil, errors.Wrap(err, "failed to marshal attachment payload")+ }+ payloadString = string(bytes)+ }+ blobExpression, blobArg := "?", any(nil)+ if create.Blob != nil {+ blobExpression, blobArg = "unhex(?)", hex.EncodeToString(create.Blob)+ }+ columns = []string{"uid", "filename", "blob", "type", "size", "creator_id", "memo_id", "storage_type", "reference", "payload"}+ values = []string{"?", "?", blobExpression, "?", "?", "?", "?", "?", "?", "?"}+ args = []any{create.UID, create.Filename, blobArg, create.Type, create.Size, create.CreatorID, create.MemoID, storageType, create.Reference, payloadString}+ return columns, values, args, nil+}++// ListAttachments returns the attachments matching find.+func (d *DB) ListAttachments(ctx context.Context, find *store.FindAttachment) ([]*store.Attachment, error) {+ where, args := []string{"1 = 1"}, []any{}++ if v := find.ID; v != nil {+ where, args = append(where, "attachment.id = ?"), append(args, *v)+ }+ if v := find.UID; v != nil {+ where, args = append(where, "attachment.uid = ?"), append(args, *v)+ }+ if v := find.CreatorID; v != nil {+ where, args = append(where, "attachment.creator_id = ?"), append(args, *v)+ }+ if v := find.Filename; v != nil {+ where, args = append(where, "attachment.filename = ?"), append(args, *v)+ }+ if v := find.FilenameSearch; v != nil {+ where, args = append(where, "attachment.filename LIKE ?"), append(args, fmt.Sprintf("%%%s%%", *v))+ }+ if v := find.MemoID; v != nil {+ where, args = append(where, "attachment.memo_id = ?"), append(args, *v)+ }+ if len(find.MemoIDList) > 0 {+ clause, memoArgs := inClause(find.MemoIDList)+ where, args = append(where, "attachment.memo_id IN "+clause), append(args, memoArgs...)+ }+ if find.HasRelatedMemo {+ where = append(where, "attachment.memo_id IS NOT NULL")+ }+ if len(find.Filters) > 0 {+ engine, err := filter.DefaultAttachmentEngine()+ if err != nil {+ return nil, errors.Wrap(err, "failed to get filter engine")+ }+ if err := filter.AppendConditions(ctx, engine, find.Filters, filter.DialectD1, &where, &args); err != nil {+ return nil, errors.Wrap(err, "failed to append filter conditions")+ }+ }+ if access := find.Access; access != nil {+ scopeClauses := []string{}+ if access.UserID != nil {+ scopeClauses = append(scopeClauses, "(attachment.memo_id IS NULL AND EXISTS (SELECT 1 FROM user AS attachment_user WHERE attachment_user.id = ? AND attachment_user.row_status = 'NORMAL') AND attachment.creator_id = ?)")+ args = append(args, *access.UserID, *access.UserID)+ }+ scopeClauses = append(scopeClauses, "(attachment.memo_id IS NOT NULL AND "+memoAccessPredicate(access, "memo", "attachment_member", &args)+")")+ where = append(where, "("+strings.Join(scopeClauses, " OR ")+")")+ }++ fields := []string{+ "attachment.id AS id",+ "attachment.uid AS uid",+ "attachment.filename AS filename",+ "attachment.type AS type",+ "attachment.size AS size",+ "attachment.creator_id AS creator_id",+ "attachment.created_ts AS created_ts",+ "attachment.updated_ts AS updated_ts",+ "attachment.memo_id AS memo_id",+ "attachment.storage_type AS storage_type",+ "attachment.reference AS reference",+ "attachment.payload AS payload",+ "CASE WHEN memo.uid IS NOT NULL THEN memo.uid ELSE NULL END AS memo_uid",+ }+ if find.GetBlob {+ // hex(NULL) is the empty string, so keep NULL explicit to tell an+ // absent blob from an empty one.+ fields = append(fields, "CASE WHEN attachment.blob IS NULL THEN NULL ELSE hex(attachment.blob) END AS blob")+ }++ query := "SELECT " + strings.Join(fields, ", ") + " FROM attachment" ++ " LEFT JOIN memo ON attachment.memo_id = memo.id" ++ " LEFT JOIN space AS attachment_space ON memo.space_id = attachment_space.id" ++ " WHERE " + strings.Join(where, " AND ") ++ " ORDER BY attachment.updated_ts DESC"+ query = appendLimit(query, find.Limit, find.Offset)++ rows, err := d.db.QueryContext(ctx, query, args...)+ if err != nil {+ return nil, err+ }+ defer rows.Close()++ list := make([]*store.Attachment, 0)+ for rows.Next() {+ attachment := store.Attachment{}+ var memoID sql.NullInt32+ var storageType string+ var payloadBytes []byte+ var blobHex sql.NullString+ dests := []any{+ &attachment.ID,+ &attachment.UID,+ &attachment.Filename,+ &attachment.Type,+ &attachment.Size,+ &attachment.CreatorID,+ &attachment.CreatedTs,+ &attachment.UpdatedTs,+ &memoID,+ &storageType,+ &attachment.Reference,+ &payloadBytes,+ &attachment.MemoUID,+ }+ if find.GetBlob {+ dests = append(dests, &blobHex)+ }+ if err := rows.Scan(dests...); err != nil {+ return nil, err+ }+ if blobHex.Valid {+ blob, err := hex.DecodeString(blobHex.String)+ if err != nil {+ return nil, errors.Wrap(err, "failed to decode attachment blob")+ }+ attachment.Blob = blob+ }+ if memoID.Valid {+ attachment.MemoID = &memoID.Int32+ }+ attachment.StorageType = storepb.AttachmentStorageType(storepb.AttachmentStorageType_value[storageType])+ payload := &storepb.AttachmentPayload{}+ if err := protojsonUnmarshaler.Unmarshal(payloadBytes, payload); err != nil {+ return nil, err+ }+ attachment.Payload = payload+ list = append(list, &attachment)+ }+ if err := rows.Err(); err != nil {+ return nil, err+ }+ return list, nil+}++// UpdateAttachment applies the given attachment changes.+func (d *DB) UpdateAttachment(ctx context.Context, update *store.UpdateAttachment) error {+ set, args, err := attachmentUpdateSet(update)+ if err != nil {+ return err+ }+ if len(set) == 0 {+ return nil+ }+ stmt := "UPDATE attachment SET " + strings.Join(set, ", ") + " WHERE id = ?"+ args = append(args, update.ID)+ if update.Policy == nil {+ if _, err := d.execOne(ctx, stmt, args...); err != nil {+ return errors.Wrap(err, "failed to update attachment")+ }+ return nil+ }+ if update.MemoID != nil {+ return store.ErrMemoMutationConflict+ }+ attachmentIDs := []int32{update.ID}+ attachments, err := listAttachmentSnapshots(ctx, d.db, attachmentIDs)+ if err != nil {+ return err+ }+ memoIDs, err := store.ValidateAttachmentMutationTargets(update.Policy.ActorUserID, attachmentIDs, attachments)+ if err != nil {+ return err+ }+ if err := authorizeAttachmentMutation(ctx, d.db, update.Policy.ActorUserID, memoIDs, nil); err != nil {+ return err+ }+ // The actor was validated above; the guard re-asserts it at commit time.+ b := newBatch()+ b.guard(activeUserCondition, update.Policy.ActorUserID)+ b.add(stmt, args...)+ if _, err := b.commit(ctx, d); err != nil {+ return errors.Wrap(guardError(err, store.ErrMemoPermissionDenied), "failed to update attachment")+ }+ return nil+}++// attachmentUpdateSet renders the SET clauses and bindings of an attachment update.+func attachmentUpdateSet(update *store.UpdateAttachment) ([]string, []any, error) {+ set, args := []string{}, []any{}+ if v := update.UID; v != nil {+ set, args = append(set, "uid = ?"), append(args, *v)+ }+ if v := update.UpdatedTs; v != nil {+ set, args = append(set, "updated_ts = ?"), append(args, *v)+ }+ if v := update.Filename; v != nil {+ set, args = append(set, "filename = ?"), append(args, *v)+ }+ if v := update.MemoID; v != nil {+ set, args = append(set, "memo_id = ?"), append(args, *v)+ }+ if v := update.Payload; v != nil {+ bytes, err := protojson.Marshal(v)+ if err != nil {+ return nil, nil, errors.Wrap(err, "failed to marshal attachment payload")+ }+ set, args = append(set, "payload = ?"), append(args, string(bytes))+ }+ return set, args, nil+}++// DeleteAttachment removes one attachment.+func (d *DB) DeleteAttachment(ctx context.Context, delete *store.DeleteAttachment) error {+ return d.DeleteAttachments(ctx, []*store.DeleteAttachment{delete})+}++// DeleteAttachments removes the rows in one atomic batch so a failure leaves+// either every attachment or none deleted, matching a transaction.+func (d *DB) DeleteAttachments(ctx context.Context, deletes []*store.DeleteAttachment) error {+ if len(deletes) == 0 {+ return nil+ }+ b := newBatch()+ for _, delete := range deletes {+ b.add("DELETE FROM attachment WHERE id = ?", delete.ID)+ }+ if _, err := b.commit(ctx, d); err != nil {+ return errors.Wrap(err, "failed to delete attachments")+ }+ return nil+}++// GetAttachmentStorageUsage returns the total persisted attachment size for a creator in bytes.+func (d *DB) GetAttachmentStorageUsage(ctx context.Context, creatorID int32) (int64, error) {+ var size int64+ if err := d.db.QueryRowContext(ctx, "SELECT COALESCE(SUM(size), 0) FROM attachment WHERE creator_id = ?", creatorID).Scan(&size); err != nil {+ return 0, errors.Wrap(err, "failed to sum attachment sizes")+ }+ return size, nil+}diff --git a/store/db/d1/attachment_delete.go b/store/db/d1/attachment_delete.gonew file mode 100644index 000000000..348bcdb00--- /dev/null+++ b/store/db/d1/attachment_delete.go@@ -0,0 +1,45 @@+package d1++import (+ "context"++ "github.com/pkg/errors"++ "github.com/usememos/memos/store"+)++// DeleteAttachmentsWithPolicy authorizes the actor against every linked memo+// and then deletes the rows in one atomic batch. Each delete is preceded by a+// guard on the row's existence so a row that vanished after the validation+// reads aborts the whole batch as a conflict, mirroring the per-row+// RowsAffected check a transaction would perform.+func (d *DB) DeleteAttachmentsWithPolicy(ctx context.Context, policy *store.AttachmentDeletionPolicy, attachmentIDs []int32) error {+ if policy == nil || policy.ActorUserID <= 0 {+ return store.ErrMemoPermissionDenied+ }+ attachments, err := listAttachmentSnapshots(ctx, d.db, attachmentIDs)+ if err != nil {+ return errors.Wrap(err, "failed to read attachment delete targets")+ }+ memoIDs, err := store.ValidateAttachmentMutationTargets(policy.ActorUserID, attachmentIDs, attachments)+ if err != nil {+ return err+ }+ if err := store.ValidateAttachmentDeletionMemoSnapshots(memoIDs, policy.ExpectedMemoContents); err != nil {+ return err+ }+ if err := authorizeAttachmentMutation(ctx, d.db, policy.ActorUserID, memoIDs, policy.ExpectedMemoContents); err != nil {+ return err+ }++ b := newBatch()+ b.guard(activeUserCondition, policy.ActorUserID)+ for _, attachmentID := range attachmentIDs {+ b.guard("EXISTS (SELECT 1 FROM attachment WHERE id = ?)", attachmentID)+ b.add("DELETE FROM attachment WHERE id = ?", attachmentID)+ }+ if _, err := b.commit(ctx, d); err != nil {+ return guardError(err, store.ErrMemoMutationConflict)+ }+ return nil+}diff --git a/store/db/d1/auth_config.go b/store/db/d1/auth_config.gonew file mode 100644index 000000000..aba612e8b--- /dev/null+++ b/store/db/d1/auth_config.go@@ -0,0 +1,105 @@+package d1++import (+ "context"+ "database/sql"++ "github.com/pkg/errors"++ "github.com/usememos/memos/store"+)++// errAuthStateChanged reports that the authentication state changed between+// validation and the write; the store retries the mutation.+var errAuthStateChanged = errors.New("d1: authentication configuration changed during mutation")++// IsRetryableAuthenticationMutationError reports whether err is a transient+// D1 failure (rate limit, overload, or a busy database) or a mutation that+// lost the race against a concurrent authentication change; both are worth+// retrying.+func (*DB) IsRetryableAuthenticationMutationError(err error) bool {+ return errors.Is(err, errAuthStateChanged) || isRetryable(err)+}++// ApplyAuthenticationConfigMutation validates the mutation against the+// current authentication state and applies it. D1 has no serializable+// transaction to hold between the reads and the write, so the batch guards+// that the GENERAL setting and the identity provider set are exactly what was+// validated; a concurrent change aborts the write and the store retries.+func (d *DB) ApplyAuthenticationConfigMutation(ctx context.Context, mutation *store.AuthenticationConfigMutation) error {+ state, err := authLoadState(ctx, d.db)+ if err != nil {+ return err+ }+ if mutation.Validate != nil {+ if err := mutation.Validate(state); err != nil {+ return err+ }+ }+ b := newBatch()+ authGuardState(b, state)+ switch {+ case mutation.UpsertGeneralSetting != nil:+ setting := mutation.UpsertGeneralSetting+ b.add(settingUpsertStatement, setting.Name, setting.Value, setting.Description)+ case mutation.DeleteIdentityProviderID != nil:+ b.add("DELETE FROM idp WHERE id = ?", *mutation.DeleteIdentityProviderID)+ default:+ return errors.New("authentication configuration mutation has no operation")+ }+ if _, err := b.commit(ctx, d); err != nil {+ return errors.Wrap(guardError(err, errAuthStateChanged), "failed to apply authentication configuration mutation")+ }+ return nil+}++// authGuardState asserts that the GENERAL setting and the identity provider+// ids are unchanged since state was read.+func authGuardState(b *batch, state *store.AuthenticationConfigState) {+ generalCount, generalValue := 0, ""+ if state.GeneralSetting != nil {+ generalCount, generalValue = 1, state.GeneralSetting.Value+ }+ b.guard("(SELECT COUNT(*) FROM system_setting WHERE name = 'GENERAL') = ?", generalCount)+ b.guard("COALESCE((SELECT value FROM system_setting WHERE name = 'GENERAL'), '') = ?", generalValue)++ ids := make([]int32, 0, len(state.IdentityProviders))+ for _, provider := range state.IdentityProviders {+ ids = append(ids, provider.ID)+ }+ b.guard("(SELECT COUNT(*) FROM idp) = ?", len(ids))+ if len(ids) > 0 {+ b.guard("NOT EXISTS (SELECT 1 FROM idp WHERE id NOT IN " + intList(ids) + ")")+ }+}++// authLoadState reads the GENERAL setting and the identity provider ids.+func authLoadState(ctx context.Context, q querier) (*store.AuthenticationConfigState, error) {+ state := &store.AuthenticationConfigState{}+ general := &store.InstanceSetting{}+ err := q.QueryRowContext(ctx, "SELECT name, value, description FROM system_setting WHERE name = ?", "GENERAL").Scan(+ &general.Name, &general.Value, &general.Description,+ )+ if err == nil {+ state.GeneralSetting = general+ } else if !errors.Is(err, sql.ErrNoRows) {+ return nil, errors.Wrap(err, "failed to read GENERAL setting")+ }++ rows, err := q.QueryContext(ctx, "SELECT id, uid FROM idp ORDER BY id")+ if err != nil {+ return nil, errors.Wrap(err, "failed to read identity providers")+ }+ defer rows.Close()+ for rows.Next() {+ provider := &store.IdentityProvider{}+ if err := rows.Scan(&provider.ID, &provider.UID); err != nil {+ return nil, errors.Wrap(err, "failed to scan identity provider")+ }+ state.IdentityProviders = append(state.IdentityProviders, provider)+ }+ if err := rows.Err(); err != nil {+ return nil, errors.Wrap(err, "failed to iterate identity providers")+ }+ return state, nil+}diff --git a/store/db/d1/client.go b/store/db/d1/client.gonew file mode 100644index 000000000..dce511851--- /dev/null+++ b/store/db/d1/client.go@@ -0,0 +1,462 @@+package d1++import (+ "bytes"+ "context"+ "database/sql/driver"+ "encoding/json"+ "fmt"+ "io"+ "net"+ "net/http"+ "net/url"+ "os"+ "strings"+ "time"++ "github.com/pkg/errors"+)++// DefaultEndpoint is the Cloudflare REST API base URL.+const DefaultEndpoint = "https://api.cloudflare.com/client/v4"++// maxBoundParameters is D1's documented per-statement bind limit.+const maxBoundParameters = 100++// Config is the parsed form of a D1 DSN.+//+// DSN grammar:+//+// d1://<account_id>/<database_id>?token=<api_token>[&endpoint=<base_url>]+//+// The token may be omitted from the DSN and supplied through the+// CLOUDFLARE_API_TOKEN environment variable instead so it stays out of+// process listings and logs.+type Config struct {+ AccountID string+ DatabaseID string+ Token string+ Endpoint string+}++// ParseDSN parses a d1:// DSN.+func ParseDSN(dsn string) (*Config, error) {+ parsed, err := url.Parse(dsn)+ if err != nil {+ return nil, errors.Wrap(err, "failed to parse d1 dsn")+ }+ if parsed.Scheme != "d1" {+ return nil, errors.Errorf("d1 dsn must use the d1:// scheme, got %q", parsed.Scheme)+ }+ config := &Config{+ AccountID: parsed.Host,+ DatabaseID: strings.Trim(parsed.Path, "/"),+ Token: parsed.Query().Get("token"),+ Endpoint: strings.TrimRight(parsed.Query().Get("endpoint"), "/"),+ }+ if config.AccountID == "" {+ return nil, errors.New("d1 dsn is missing the account id: d1://<account_id>/<database_id>")+ }+ if config.DatabaseID == "" || strings.Contains(config.DatabaseID, "/") {+ return nil, errors.New("d1 dsn is missing the database id: d1://<account_id>/<database_id>")+ }+ if config.Token == "" {+ config.Token = os.Getenv("CLOUDFLARE_API_TOKEN")+ }+ if config.Token == "" {+ return nil, errors.New("d1 api token is required: add ?token=<api_token> to the dsn or set CLOUDFLARE_API_TOKEN")+ }+ if config.Endpoint == "" {+ config.Endpoint = DefaultEndpoint+ }+ if err := validateEndpoint(config.Endpoint); err != nil {+ return nil, err+ }+ return config, nil+}++// validateEndpoint rejects endpoints that would send the bearer token in the+// clear. Plain HTTP is allowed only towards loopback, where the test+// emulator listens.+func validateEndpoint(endpoint string) error {+ parsed, err := url.Parse(endpoint)+ if err != nil {+ return errors.Wrap(err, "invalid d1 endpoint")+ }+ if parsed.Host == "" {+ return errors.Errorf("d1 endpoint %q has no host", endpoint)+ }+ switch parsed.Scheme {+ case "https":+ return nil+ case "http":+ if isLoopbackHost(parsed.Hostname()) {+ return nil+ }+ return errors.Errorf("d1 endpoint %q must use https unless it points at loopback", endpoint)+ default:+ return errors.Errorf("d1 endpoint %q must use https", endpoint)+ }+}++func isLoopbackHost(host string) bool {+ if host == "localhost" {+ return true+ }+ ip := net.ParseIP(host)+ return ip != nil && ip.IsLoopback()+}++// Error is a failure reported by the D1 REST API.+type Error struct {+ // Status is the HTTP status code of the response.+ Status int+ // Code is the Cloudflare API error code, when the response carried one.+ Code int+ // Message is the human readable error text.+ Message string+}++func (e *Error) Error() string {+ if e.Code != 0 {+ return fmt.Sprintf("d1: %s (code %d, http %d)", e.Message, e.Code, e.Status)+ }+ return fmt.Sprintf("d1: %s (http %d)", e.Message, e.Status)+}++// statement is one parameterized SQL statement.+type statement struct {+ SQL string+ Args []any+}++// result is the outcome of one statement.+type result struct {+ Columns []string+ Rows [][]any+ Changes int64+ LastRowID int64+}++// client talks to one D1 database through the Cloudflare REST API.+type client struct {+ httpClient *http.Client+ config *Config+}++func newClient(config *Config) *client {+ return &client{+ httpClient: &http.Client{Timeout: 60 * time.Second},+ config: config,+ }+}++type requestStatement struct {+ SQL string `json:"sql"`+ Params []any `json:"params,omitempty"`+}++type responseInfo struct {+ Code int `json:"code"`+ Message string `json:"message"`+}++type responseMeta struct {+ Changes json.Number `json:"changes"`+ LastRowID json.Number `json:"last_row_id"`+}++type rawResults struct {+ Columns []string `json:"columns"`+ Rows [][]json.RawMessage `json:"rows"`+}++type queryResult struct {+ Success bool `json:"success"`+ Meta responseMeta `json:"meta"`+ Results json.RawMessage `json:"results"`+}++type envelope struct {+ Success bool `json:"success"`+ Errors []responseInfo `json:"errors"`+ Result []queryResult `json:"result"`+}++// exec runs one statement and returns its rows and change counters.+func (c *client) exec(ctx context.Context, stmt statement) (*result, error) {+ params, err := encodeArgs(stmt.Args)+ if err != nil {+ return nil, err+ }+ results, err := c.raw(ctx, map[string]any{"sql": stmt.SQL, "params": params})+ if err != nil {+ return nil, err+ }+ if len(results) != 1 {+ return nil, errors.Errorf("d1: expected 1 result, got %d", len(results))+ }+ return results[0], nil+}++// script runs a parameter-free, possibly multi-statement SQL text as one+// atomic batch. It backs schema migrations.+func (c *client) script(ctx context.Context, sql string) error {+ _, err := c.raw(ctx, map[string]any{"sql": sql})+ return err+}++// batch runs the statements in order as one transaction: either every+// statement commits or none does.+func (c *client) batch(ctx context.Context, stmts []statement) ([]*result, error) {+ if len(stmts) == 0 {+ return nil, nil+ }+ if len(stmts) == 1 {+ res, err := c.exec(ctx, stmts[0])+ if err != nil {+ return nil, err+ }+ return []*result{res}, nil+ }+ body := make([]requestStatement, 0, len(stmts))+ for _, stmt := range stmts {+ params, err := encodeArgs(stmt.Args)+ if err != nil {+ return nil, err+ }+ body = append(body, requestStatement{SQL: stmt.SQL, Params: params})+ }+ results, err := c.raw(ctx, map[string]any{"batch": body})+ if err != nil {+ return nil, err+ }+ if len(results) != len(stmts) {+ return nil, errors.Errorf("d1: expected %d results, got %d", len(stmts), len(results))+ }+ return results, nil+}++func (c *client) raw(ctx context.Context, body map[string]any) ([]*result, error) {+ payload, err := json.Marshal(body)+ if err != nil {+ return nil, errors.Wrap(err, "failed to encode d1 request")+ }+ responseBody, status, err := c.do(ctx, http.MethodPost, c.databaseURL()+"/raw", payload)+ if err != nil {+ return nil, err+ }+ var env envelope+ if err := decodeJSON(responseBody, &env); err != nil {+ return nil, errors.Wrap(err, "failed to decode d1 response")+ }+ if status != http.StatusOK || !env.Success {+ return nil, envelopeError(status, env.Errors)+ }+ results := make([]*result, 0, len(env.Result))+ for _, item := range env.Result {+ if !item.Success {+ return nil, &Error{Status: status, Message: "statement reported failure"}+ }+ res, err := decodeResult(item)+ if err != nil {+ return nil, err+ }+ results = append(results, res)+ }+ return results, nil+}++// databaseSize returns the database file size reported by the D1 metadata endpoint.+func (c *client) databaseSize(ctx context.Context) (int64, error) {+ responseBody, status, err := c.do(ctx, http.MethodGet, c.databaseURL()+"?fields=file_size", nil)+ if err != nil {+ return -1, err+ }+ var env struct {+ Success bool `json:"success"`+ Errors []responseInfo `json:"errors"`+ Result struct {+ FileSize json.Number `json:"file_size"`+ } `json:"result"`+ }+ if err := decodeJSON(responseBody, &env); err != nil {+ return -1, errors.Wrap(err, "failed to decode d1 database metadata")+ }+ if status != http.StatusOK || !env.Success {+ return -1, envelopeError(status, env.Errors)+ }+ size, err := env.Result.FileSize.Int64()+ if err != nil {+ return -1, errors.Wrap(err, "failed to parse d1 database size")+ }+ return size, nil+}++func (c *client) databaseURL() string {+ return fmt.Sprintf("%s/accounts/%s/d1/database/%s", c.config.Endpoint, url.PathEscape(c.config.AccountID), url.PathEscape(c.config.DatabaseID))+}++func (c *client) do(ctx context.Context, method, target string, payload []byte) ([]byte, int, error) {+ var reader io.Reader+ if payload != nil {+ reader = bytes.NewReader(payload)+ }+ request, err := http.NewRequestWithContext(ctx, method, target, reader)+ if err != nil {+ return nil, 0, errors.Wrap(err, "failed to build d1 request")+ }+ request.Header.Set("Authorization", "Bearer "+c.config.Token)+ request.Header.Set("Accept", "application/json")+ if payload != nil {+ request.Header.Set("Content-Type", "application/json")+ }+ response, err := c.httpClient.Do(request)+ if err != nil {+ return nil, 0, errors.Wrap(err, "d1 request failed")+ }+ defer response.Body.Close()+ responseBody, err := io.ReadAll(response.Body)+ if err != nil {+ return nil, response.StatusCode, errors.Wrap(err, "failed to read d1 response")+ }+ return responseBody, response.StatusCode, nil+}++func envelopeError(status int, infos []responseInfo) error {+ if len(infos) == 0 {+ return &Error{Status: status, Message: http.StatusText(status)}+ }+ messages := make([]string, 0, len(infos))+ for _, info := range infos {+ messages = append(messages, info.Message)+ }+ return &Error{Status: status, Code: infos[0].Code, Message: strings.Join(messages, "; ")}+}++func decodeJSON(data []byte, target any) error {+ decoder := json.NewDecoder(bytes.NewReader(data))+ decoder.UseNumber()+ return decoder.Decode(target)+}++func decodeResult(item queryResult) (*result, error) {+ res := &result{}+ if item.Meta.Changes != "" {+ changes, err := item.Meta.Changes.Int64()+ if err != nil {+ return nil, errors.Wrap(err, "failed to parse d1 change count")+ }+ res.Changes = changes+ }+ if item.Meta.LastRowID != "" {+ lastRowID, err := item.Meta.LastRowID.Int64()+ if err != nil {+ return nil, errors.Wrap(err, "failed to parse d1 last row id")+ }+ res.LastRowID = lastRowID+ }+ if len(item.Results) == 0 || string(item.Results) == "null" {+ return res, nil+ }+ var raw rawResults+ if err := decodeJSON(item.Results, &raw); err != nil {+ return nil, errors.Wrap(err, "failed to decode d1 rows")+ }+ res.Columns = raw.Columns+ res.Rows = make([][]any, 0, len(raw.Rows))+ for _, rawRow := range raw.Rows {+ row := make([]any, 0, len(rawRow))+ for _, rawValue := range rawRow {+ value, err := decodeValue(rawValue)+ if err != nil {+ return nil, err+ }+ row = append(row, value)+ }+ res.Rows = append(res.Rows, row)+ }+ return res, nil+}++// decodeValue maps one JSON cell onto the driver.Value domain.+func decodeValue(raw json.RawMessage) (any, error) {+ if len(raw) == 0 {+ return nil, nil+ }+ switch raw[0] {+ case 'n':+ return nil, nil+ case 't':+ return int64(1), nil+ case 'f':+ return int64(0), nil+ case '"':+ var text string+ if err := json.Unmarshal(raw, &text); err != nil {+ return nil, errors.Wrap(err, "failed to decode d1 text cell")+ }+ return text, nil+ case '[':+ // Blobs arrive as arrays of byte values.+ var bytesValue []byte+ var numbers []json.Number+ if err := decodeJSON(raw, &numbers); err != nil {+ return nil, errors.Wrap(err, "failed to decode d1 blob cell")+ }+ bytesValue = make([]byte, 0, len(numbers))+ for _, number := range numbers {+ b, err := number.Int64()+ if err != nil || b < 0 || b > 255 {+ return nil, errors.Errorf("d1: invalid blob byte %q", number)+ }+ bytesValue = append(bytesValue, byte(b))+ }+ return bytesValue, nil+ case '{':+ return string(raw), nil+ default:+ number := json.Number(bytes.TrimSpace(raw))+ if integer, err := number.Int64(); err == nil {+ return integer, nil+ }+ float, err := number.Float64()+ if err != nil {+ return nil, errors.Wrapf(err, "d1: invalid numeric cell %q", string(raw))+ }+ return float, nil+ }+}++// encodeArgs converts Go arguments to the JSON parameter list D1 accepts.+// Binary values are rejected: blob columns are written with unhex(?) and a+// hex-encoded string because JSON cannot carry raw bytes.+func encodeArgs(args []any) ([]any, error) {+ if len(args) > maxBoundParameters {+ return nil, errors.Errorf("d1: statement binds %d parameters, limit is %d", len(args), maxBoundParameters)+ }+ params := make([]any, 0, len(args))+ for _, arg := range args {+ value, err := driver.DefaultParameterConverter.ConvertValue(arg)+ if err != nil {+ return nil, errors.Wrap(err, "d1: unsupported parameter")+ }+ switch v := value.(type) {+ case nil, int64, float64, string:+ params = append(params, v)+ case bool:+ if v {+ params = append(params, int64(1))+ } else {+ params = append(params, int64(0))+ }+ case []byte:+ return nil, errors.New("d1: binary parameters are not supported; bind hex text through unhex(?)")+ case time.Time:+ params = append(params, v.UTC().Format(time.RFC3339Nano))+ default:+ return nil, errors.Errorf("d1: unsupported parameter type %T", value)+ }+ }+ return params, nil+}diff --git a/store/db/d1/d1.go b/store/db/d1/d1.gonew file mode 100644index 000000000..8487f43c1--- /dev/null+++ b/store/db/d1/d1.go@@ -0,0 +1,66 @@+// Package d1 implements the store driver for Cloudflare D1.+//+// D1 is reached through the Cloudflare REST API rather than a socket, and it+// exposes no interactive transactions. Each driver method therefore performs+// its validation reads first and then commits its writes as one atomic batch.+// Preconditions that must hold at commit time are re-checked inside the batch+// with guard statements (see guard.go) so a stale read never yields a partial+// write.+package d1++import (+ "context"+ "database/sql"++ "github.com/pkg/errors"++ "github.com/usememos/memos/internal/profile"+ "github.com/usememos/memos/store"+)++// DB is the D1 store driver.+type DB struct {+ client *client+ db *sql.DB+ profile *profile.Profile+}++// NewDB opens a D1 database described by the profile DSN.+func NewDB(profile *profile.Profile) (store.Driver, error) {+ config, err := ParseDSN(profile.DSN)+ if err != nil {+ return nil, err+ }+ c := newClient(config)+ return &DB{client: c, db: openSQLDB(c), profile: profile}, nil+}++// GetDB exposes a database/sql handle over the REST client for the migrator+// and tests. See sqldriver.go for its transaction semantics.+func (d *DB) GetDB() *sql.DB {+ return d.db+}++// Close releases the HTTP connection pool.+func (d *DB) Close() error {+ return d.db.Close()+}++// IsInitialized reports whether the schema has been created.+func (d *DB) IsInitialized(ctx context.Context) (bool, error) {+ var exists bool+ err := d.db.QueryRowContext(ctx, "SELECT EXISTS(SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = 'memo')").Scan(&exists)+ if err != nil {+ return false, errors.Wrap(err, "failed to check if database is initialized")+ }+ return exists, nil+}++// GetDatabaseSize returns the database size reported by the D1 metadata endpoint.+func (d *DB) GetDatabaseSize(ctx context.Context) (int64, error) {+ size, err := d.client.databaseSize(ctx)+ if err != nil {+ return -1, errors.Wrap(err, "failed to read d1 database size")+ }+ return size, nil+}diff --git a/store/db/d1/d1_test.go b/store/db/d1/d1_test.gonew file mode 100644index 000000000..ec709c541--- /dev/null+++ b/store/db/d1/d1_test.go@@ -0,0 +1,176 @@+package d1++import (+ "context"+ "errors"+ "testing"++ "github.com/stretchr/testify/require"++ "github.com/usememos/memos/internal/profile"+ "github.com/usememos/memos/store/db/d1/d1test"+)++var errConflict = errors.New("conflict")++func newTestDB(t *testing.T) *DB {+ t.Helper()+ server := d1test.New(t)+ driver, err := NewDB(&profile.Profile{Driver: "d1", DSN: server.DSN()})+ require.NoError(t, err)+ db, ok := driver.(*DB)+ require.True(t, ok)+ t.Cleanup(func() { require.NoError(t, db.Close()) })+ return db+}++func TestParseDSN(t *testing.T) {+ config, err := ParseDSN("d1://acct/dbid?token=secret&endpoint=http://127.0.0.1:1/")+ require.NoError(t, err)+ require.Equal(t, "acct", config.AccountID)+ require.Equal(t, "dbid", config.DatabaseID)+ require.Equal(t, "secret", config.Token)+ require.Equal(t, "http://127.0.0.1:1", config.Endpoint)++ for _, endpoint := range []string{"http://127.0.0.1:8787", "http://localhost:8787", "http://[::1]:8787", "https://api.example.com/v4"} {+ _, err = ParseDSN("d1://acct/dbid?token=secret&endpoint=" + endpoint)+ require.NoError(t, err, endpoint)+ }+ for _, endpoint := range []string{"http://api.example.com", "ftp://api.example.com", "api.example.com"} {+ _, err = ParseDSN("d1://acct/dbid?token=secret&endpoint=" + endpoint)+ require.Error(t, err, endpoint)+ }++ _, err = ParseDSN("sqlite://x")+ require.ErrorContains(t, err, "d1:// scheme")+ _, err = ParseDSN("d1:///dbid?token=t")+ require.ErrorContains(t, err, "account id")+ t.Setenv("CLOUDFLARE_API_TOKEN", "")+ _, err = ParseDSN("d1://acct/dbid")+ require.ErrorContains(t, err, "token")+ t.Setenv("CLOUDFLARE_API_TOKEN", "from-env")+ config, err = ParseDSN("d1://acct/dbid")+ require.NoError(t, err)+ require.Equal(t, "from-env", config.Token)+ require.Equal(t, DefaultEndpoint, config.Endpoint)+}++func TestTransportRoundTrip(t *testing.T) {+ ctx := context.Background()+ db := newTestDB(t)++ initialized, err := db.IsInitialized(ctx)+ require.NoError(t, err)+ require.False(t, initialized)++ // Multi-statement script through the sql adapter transaction, as the migrator does.+ tx, err := db.GetDB().Begin()+ require.NoError(t, err)+ _, err = tx.ExecContext(ctx, `+ -- schema+ CREATE TABLE d1_guard (ok INTEGER NOT NULL CONSTRAINT d1_guard_ok CHECK (ok = 1));+ CREATE TABLE memo (id INTEGER PRIMARY KEY AUTOINCREMENT, uid TEXT NOT NULL UNIQUE, content TEXT NOT NULL DEFAULT 'a;b', blob BLOB);+ `)+ require.NoError(t, err)+ require.NoError(t, tx.Commit())++ initialized, err = db.IsInitialized(ctx)+ require.NoError(t, err)+ require.True(t, initialized)++ // Parameter binding and RETURNING through database/sql.+ var id int64+ var content string+ require.NoError(t, db.GetDB().QueryRowContext(ctx, "INSERT INTO memo (uid, blob) VALUES (?, unhex(?)) RETURNING id, content", "m1", "00ff").Scan(&id, &content))+ require.Equal(t, int64(1), id)+ require.Equal(t, "a;b", content)++ var hexBlob string+ require.NoError(t, db.GetDB().QueryRowContext(ctx, "SELECT hex(blob) FROM memo WHERE id = ?", id).Scan(&hexBlob))+ require.Equal(t, "00FF", hexBlob)++ res, err := db.GetDB().ExecContext(ctx, "UPDATE memo SET content = ? WHERE id = ?", "changed", id)+ require.NoError(t, err)+ affected, err := res.RowsAffected()+ require.NoError(t, err)+ require.Equal(t, int64(1), affected)++ // Unique violations are recognizable.+ _, err = db.execOne(ctx, "INSERT INTO memo (uid) VALUES (?)", "m1")+ require.Error(t, err)+ require.True(t, isUniqueViolation(err), err.Error())+ require.False(t, isRetryable(err))++ // A batch commits atomically and reports per-statement changes.+ b := newBatch()+ first := b.add("INSERT INTO memo (uid) VALUES (?)", "m2")+ second := b.add("UPDATE memo SET content = ? WHERE uid IN (?, ?)", "both", "m1", "m2")+ results, err := b.commit(ctx, db)+ require.NoError(t, err)+ require.Len(t, results, 2)+ require.Equal(t, int64(1), results[first].Changes)+ require.Equal(t, int64(2), results[second].Changes)+ require.NoError(t, requireChanges(results, second, 2, errConflict))+ require.ErrorIs(t, requireChanges(results, second, 1, errConflict), errConflict)++ // A failing guard rolls the whole batch back.+ b = newBatch()+ b.add("INSERT INTO memo (uid) VALUES (?)", "m3")+ b.guard("EXISTS (SELECT 1 FROM memo WHERE uid = ?)", "missing")+ _, err = b.commit(ctx, db)+ require.Error(t, err)+ require.True(t, isGuardFailure(err), err.Error())+ require.ErrorIs(t, guardError(err, errConflict), errConflict)+ var count int+ require.NoError(t, db.GetDB().QueryRowContext(ctx, "SELECT COUNT(*) FROM memo").Scan(&count))+ require.Equal(t, 2, count)++ // A passing guard leaves the batch intact and the guard table empty.+ b = newBatch()+ b.guard("EXISTS (SELECT 1 FROM memo WHERE uid = ?)", "m1")+ b.add("INSERT INTO memo (uid) VALUES (?)", "m3")+ _, err = b.commit(ctx, db)+ require.NoError(t, err)+ require.NoError(t, db.GetDB().QueryRowContext(ctx, "SELECT COUNT(*) FROM d1_guard").Scan(&count))+ require.Equal(t, 0, count)++ // Buffered transactions with parameters commit as one batch, and reads+ // after a buffered write are refused.+ tx, err = db.GetDB().BeginTx(ctx, nil)+ require.NoError(t, err)+ _, err = tx.ExecContext(ctx, "UPDATE memo SET content = ? WHERE uid = ?", "tx", "m1")+ require.NoError(t, err)+ err = tx.QueryRowContext(ctx, "SELECT 1").Scan(&count)+ require.ErrorIs(t, err, errUnsupportedQueryInTx)+ require.NoError(t, tx.Commit())+ require.NoError(t, db.GetDB().QueryRowContext(ctx, "SELECT content FROM memo WHERE uid = ?", "m1").Scan(&content))+ require.Equal(t, "tx", content)++ // Bind limit is enforced before the request leaves the process.+ args := make([]any, maxBoundParameters+1)+ _, err = db.execOne(ctx, "SELECT 1", args...)+ require.ErrorContains(t, err, "limit is 100")++ size, err := db.GetDatabaseSize(ctx)+ require.NoError(t, err)+ require.Greater(t, size, int64(0))+}++func TestNullAndTypedScans(t *testing.T) {+ ctx := context.Background()+ db := newTestDB(t)+ _, err := db.execOne(ctx, "CREATE TABLE t (id INTEGER PRIMARY KEY, n INTEGER, f REAL, s TEXT, ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')))")+ require.NoError(t, err)+ var nilInt *int32+ _, err = db.execOne(ctx, "INSERT INTO t (n, f, s) VALUES (?, ?, ?)", nilInt, 1.5, "x")+ require.NoError(t, err)+ var n *int32+ var f float64+ var s string+ var ts int64+ require.NoError(t, db.GetDB().QueryRowContext(ctx, "SELECT n, f, s, ts FROM t").Scan(&n, &f, &s, &ts))+ require.Nil(t, n)+ require.Equal(t, 1.5, f)+ require.Equal(t, "x", s)+ require.Greater(t, ts, int64(0))+}diff --git a/store/db/d1/d1test/server.go b/store/db/d1/d1test/server.gonew file mode 100644index 000000000..7a15b91df--- /dev/null+++ b/store/db/d1/d1test/server.go@@ -0,0 +1,413 @@+// Package d1test runs an in-process emulation of the Cloudflare D1 REST API+// for tests. It speaks the same JSON protocol the driver uses (the /raw and+// /query endpoints, batch bodies, the database metadata endpoint) on top of a+// private SQLite database, and enforces the D1 constraints that matter to the+// driver: foreign keys are on, statements are wrapped in an implicit+// transaction, a batch is atomic, and a statement may bind at most 100+// parameters. It registers no custom SQL functions, exactly like D1.+package d1test++import (+ "context"+ "database/sql"+ "encoding/json"+ "fmt"+ "net/http"+ "net/http/httptest"+ "strings"+ "sync"+ "testing"+ "time"++ // SQLite engine behind the emulated service.+ _ "modernc.org/sqlite"+)++const (+ accountID = "test-account"+ databaseID = "test-database"+ token = "test-token"++ maxBoundParameters = 100+)++// Server is one emulated D1 database reachable over HTTP.+type Server struct {+ http *httptest.Server+ db *sql.DB+ conn *sql.Conn+ mu sync.Mutex+}++// New starts a fresh emulated database that is torn down with the test.+func New(t testing.TB) *Server {+ t.Helper()+ db, err := sql.Open("sqlite", ":memory:?_pragma=foreign_keys(1)")+ if err != nil {+ t.Fatalf("d1test: open sqlite: %v", err)+ }+ db.SetMaxOpenConns(1)+ conn, err := db.Conn(context.Background())+ if err != nil {+ t.Fatalf("d1test: acquire sqlite connection: %v", err)+ }+ s := &Server{db: db, conn: conn}+ s.http = httptest.NewServer(http.HandlerFunc(s.handle))+ t.Cleanup(s.Close)+ return s+}++// DSN returns the driver DSN pointing at this server.+func (s *Server) DSN() string {+ return fmt.Sprintf("d1://%s/%s?token=%s&endpoint=%s", accountID, databaseID, token, s.http.URL)+}++// Close stops the server and discards the database.+func (s *Server) Close() {+ s.http.Close()+ _ = s.conn.Close()+ _ = s.db.Close()+}++type requestStatement struct {+ SQL string `json:"sql"`+ Params []any `json:"params"`+}++type requestBody struct {+ SQL *string `json:"sql"`+ Params []any `json:"params"`+ Batch []requestStatement `json:"batch"`+}++type apiError struct {+ Code int `json:"code"`+ Message string `json:"message"`+}++func (s *Server) handle(w http.ResponseWriter, r *http.Request) {+ if r.Header.Get("Authorization") != "Bearer "+token {+ writeErrors(w, http.StatusForbidden, 10000, "Authentication error")+ return+ }+ prefix := fmt.Sprintf("/accounts/%s/d1/database/%s", accountID, databaseID)+ if !strings.HasPrefix(r.URL.Path, prefix) {+ writeErrors(w, http.StatusNotFound, 7404, "database not found")+ return+ }+ switch rest := strings.TrimPrefix(r.URL.Path, prefix); {+ case rest == "" && r.Method == http.MethodGet:+ s.handleMetadata(w)+ case rest == "/raw" && r.Method == http.MethodPost:+ s.handleQuery(w, r, true)+ case rest == "/query" && r.Method == http.MethodPost:+ s.handleQuery(w, r, false)+ default:+ writeErrors(w, http.StatusNotFound, 7404, "unknown route")+ }+}++func (s *Server) handleMetadata(w http.ResponseWriter) {+ s.mu.Lock()+ defer s.mu.Unlock()+ var pageCount, pageSize int64+ if err := s.conn.QueryRowContext(context.Background(), "PRAGMA page_count").Scan(&pageCount); err != nil {+ writeErrors(w, http.StatusInternalServerError, 7500, err.Error())+ return+ }+ if err := s.conn.QueryRowContext(context.Background(), "PRAGMA page_size").Scan(&pageSize); err != nil {+ writeErrors(w, http.StatusInternalServerError, 7500, err.Error())+ return+ }+ writeJSON(w, http.StatusOK, map[string]any{+ "success": true,+ "errors": []any{},+ "result": map[string]any{+ "uuid": databaseID,+ "name": "memos-test",+ "file_size": pageCount * pageSize,+ },+ })+}++func (s *Server) handleQuery(w http.ResponseWriter, r *http.Request, rawRows bool) {+ var body requestBody+ decoder := json.NewDecoder(r.Body)+ decoder.UseNumber()+ if err := decoder.Decode(&body); err != nil {+ writeErrors(w, http.StatusBadRequest, 7400, "invalid request body: "+err.Error())+ return+ }+ var entries []requestStatement+ switch {+ case body.SQL != nil:+ entries = []requestStatement{{SQL: *body.SQL, Params: body.Params}}+ case len(body.Batch) > 0:+ entries = body.Batch+ default:+ writeErrors(w, http.StatusBadRequest, 7400, "request must carry sql or batch")+ return+ }++ // Expand every entry into single statements; a parameterized entry must+ // hold exactly one statement because D1 documents no binding rule for+ // several.+ var stmts []requestStatement+ for _, entry := range entries {+ parts := splitStatements(entry.SQL)+ if len(parts) == 0 {+ writeErrors(w, http.StatusBadRequest, 7400, "empty sql")+ return+ }+ if len(parts) > 1 && len(entry.Params) > 0 {+ writeErrors(w, http.StatusBadRequest, 7400, "params cannot be bound to multiple statements")+ return+ }+ if len(entry.Params) > maxBoundParameters {+ writeErrors(w, http.StatusBadRequest, 7400, fmt.Sprintf("too many bound parameters: %d", len(entry.Params)))+ return+ }+ for _, part := range parts {+ stmts = append(stmts, requestStatement{SQL: part, Params: entry.Params})+ }+ }++ results, err := s.execute(r.Context(), stmts, rawRows)+ if err != nil {+ writeErrors(w, http.StatusBadRequest, 7500, "D1_ERROR: "+err.Error())+ return+ }+ writeJSON(w, http.StatusOK, map[string]any{+ "success": true,+ "errors": []any{},+ "result": results,+ })+}++// execute runs stmts inside one transaction, mirroring D1's implicit+// transaction per request and its atomic batches.+func (s *Server) execute(ctx context.Context, stmts []requestStatement, rawRows bool) ([]map[string]any, error) {+ s.mu.Lock()+ defer s.mu.Unlock()++ tx, err := s.conn.BeginTx(ctx, nil)+ if err != nil {+ return nil, err+ }+ results := make([]map[string]any, 0, len(stmts))+ for _, stmt := range stmts {+ res, err := runStatement(ctx, tx, stmt, rawRows)+ if err != nil {+ _ = tx.Rollback()+ return nil, err+ }+ results = append(results, res)+ }+ if err := tx.Commit(); err != nil {+ return nil, err+ }+ return results, nil+}++func runStatement(ctx context.Context, tx *sql.Tx, stmt requestStatement, rawRows bool) (map[string]any, error) {+ args := make([]any, 0, len(stmt.Params))+ for _, param := range stmt.Params {+ args = append(args, decodeParam(param))+ }+ // changes() reports the last write even after a read, so the statement's+ // own count is the total_changes() delta across its execution.+ var before int64+ if err := tx.QueryRowContext(ctx, "SELECT total_changes()").Scan(&before); err != nil {+ return nil, err+ }+ columns, arrayRows, objectRows, err := collectRows(ctx, tx, stmt.SQL, args, rawRows)+ if err != nil {+ return nil, err+ }+ var after, lastRowID int64+ if err := tx.QueryRowContext(ctx, "SELECT total_changes(), last_insert_rowid()").Scan(&after, &lastRowID); err != nil {+ return nil, err+ }+ changes := after - before+ var results any+ if rawRows {+ results = map[string]any{"columns": columns, "rows": arrayRows}+ } else {+ results = objectRows+ }+ return map[string]any{+ "success": true,+ "meta": map[string]any{+ "changed_db": changes > 0,+ "changes": changes,+ "last_row_id": lastRowID,+ "duration": 0,+ },+ "results": results,+ }, nil+}++// collectRows executes one statement and drains its result set in the+// requested shape. Statements without a result set yield empty slices.+func collectRows(ctx context.Context, tx *sql.Tx, query string, args []any, rawRows bool) ([]string, [][]any, []map[string]any, error) {+ rows, err := tx.QueryContext(ctx, query, args...)+ if err != nil {+ return nil, nil, nil, err+ }+ defer rows.Close()+ columns, err := rows.Columns()+ if err != nil {+ return nil, nil, nil, err+ }+ arrayRows := [][]any{}+ objectRows := []map[string]any{}+ for rows.Next() {+ cells := make([]any, len(columns))+ targets := make([]any, len(columns))+ for i := range cells {+ targets[i] = &cells[i]+ }+ if err := rows.Scan(targets...); err != nil {+ return nil, nil, nil, err+ }+ encoded := make([]any, len(columns))+ for i, cell := range cells {+ encoded[i] = encodeCell(cell)+ }+ if rawRows {+ arrayRows = append(arrayRows, encoded)+ continue+ }+ object := make(map[string]any, len(columns))+ for i, column := range columns {+ object[column] = encoded[i]+ }+ objectRows = append(objectRows, object)+ }+ if err := rows.Err(); err != nil {+ return nil, nil, nil, err+ }+ if err := rows.Close(); err != nil {+ return nil, nil, nil, err+ }+ return columns, arrayRows, objectRows, nil+}++// decodeParam maps a JSON parameter onto a SQLite bind value.+func decodeParam(param any) any {+ switch v := param.(type) {+ case json.Number:+ if integer, err := v.Int64(); err == nil {+ return integer+ }+ if float, err := v.Float64(); err == nil {+ return float+ }+ return v.String()+ case bool:+ if v {+ return int64(1)+ }+ return int64(0)+ default:+ return v+ }+}++// encodeCell maps a SQLite value onto its D1 JSON representation. Blobs are+// emitted as arrays of byte values.+func encodeCell(cell any) any {+ switch v := cell.(type) {+ case []byte:+ values := make([]int, len(v))+ for i, b := range v {+ values[i] = int(b)+ }+ return values+ case time.Time:+ return v.UTC().Format(time.RFC3339Nano)+ default:+ return v+ }+}++// splitStatements divides SQL text at top-level semicolons, honoring string+// literals, quoted identifiers, and comments.+func splitStatements(text string) []string {+ var statements []string+ var current strings.Builder+ flush := func() {+ stmt := strings.TrimSpace(current.String())+ current.Reset()+ if stmt != "" && !isCommentOnly(stmt) {+ statements = append(statements, stmt)+ }+ }+ for i := 0; i < len(text); i++ {+ c := text[i]+ switch {+ case c == '-' && i+1 < len(text) && text[i+1] == '-':+ end := strings.IndexByte(text[i:], '\n')+ if end < 0 {+ i = len(text)+ } else {+ i += end+ }+ case c == '/' && i+1 < len(text) && text[i+1] == '*':+ end := strings.Index(text[i+2:], "*/")+ if end < 0 {+ i = len(text)+ } else {+ i += end + 3+ }+ case c == '\'' || c == '"' || c == '`':+ end := closingQuote(text, i, c)+ current.WriteString(text[i : end+1])+ i = end+ case c == ';':+ flush()+ default:+ current.WriteByte(c)+ }+ }+ flush()+ return statements+}++func closingQuote(text string, start int, quote byte) int {+ for i := start + 1; i < len(text); i++ {+ if text[i] != quote {+ continue+ }+ if i+1 < len(text) && text[i+1] == quote {+ i+++ continue+ }+ return i+ }+ return len(text) - 1+}++func isCommentOnly(stmt string) bool {+ for _, line := range strings.Split(stmt, "\n") {+ line = strings.TrimSpace(line)+ if line != "" && !strings.HasPrefix(line, "--") {+ return false+ }+ }+ return true+}++func writeErrors(w http.ResponseWriter, status, code int, message string) {+ writeJSON(w, status, map[string]any{+ "success": false,+ "errors": []apiError{{Code: code, Message: message}},+ "result": nil,+ })+}++func writeJSON(w http.ResponseWriter, status int, body any) {+ w.Header().Set("Content-Type", "application/json")+ w.WriteHeader(status)+ _ = json.NewEncoder(w).Encode(body)+}diff --git a/store/db/d1/errors.go b/store/db/d1/errors.gonew file mode 100644index 000000000..27e8db1d4--- /dev/null+++ b/store/db/d1/errors.go@@ -0,0 +1,36 @@+package d1++import (+ "errors"+ "net/http"+ "strings"+)++// isUniqueViolation reports whether err is a UNIQUE or PRIMARY KEY constraint+// failure. D1 surfaces SQLite errors as text, so the message is inspected.+func isUniqueViolation(err error) bool {+ var d1Err *Error+ if !errors.As(err, &d1Err) {+ return false+ }+ message := d1Err.Message+ return strings.Contains(message, "UNIQUE constraint failed") ||+ strings.Contains(message, "SQLITE_CONSTRAINT_UNIQUE") ||+ strings.Contains(message, "SQLITE_CONSTRAINT_PRIMARYKEY")+}++// isRetryable reports whether err is a transient D1 failure worth retrying:+// rate limiting, a busy or overloaded database, or a server-side error.+func isRetryable(err error) bool {+ var d1Err *Error+ if !errors.As(err, &d1Err) {+ return false+ }+ if d1Err.Status == http.StatusTooManyRequests || d1Err.Status >= http.StatusInternalServerError {+ return true+ }+ message := d1Err.Message+ return strings.Contains(message, "overloaded") ||+ strings.Contains(message, "SQLITE_BUSY") ||+ strings.Contains(message, "database is locked")+}diff --git a/store/db/d1/executor.go b/store/db/d1/executor.gonew file mode 100644index 000000000..c42f3e785--- /dev/null+++ b/store/db/d1/executor.go@@ -0,0 +1,126 @@+package d1++import (+ "context"+ "database/sql"+ "fmt"+ "strconv"+ "strings"++ "github.com/pkg/errors"+ "google.golang.org/protobuf/encoding/protojson"+)++var protojsonUnmarshaler = protojson.UnmarshalOptions{+ DiscardUnknown: true,+}++// inClauseBatchSize keeps every IN (...) list under D1's 100 bound parameter+// limit, leaving room for the other bindings a statement carries.+const inClauseBatchSize = 90++// querier is the read surface shared by the driver methods.+type querier interface {+ QueryRowContext(context.Context, string, ...any) *sql.Row+ QueryContext(context.Context, string, ...any) (*sql.Rows, error)+}++// batch is an ordered set of statements committed atomically. Statements are+// appended with add and conditions with guard; commit sends them in one+// request.+type batch struct {+ stmts []statement+}++func newBatch() *batch {+ return &batch{}+}++// add appends a statement and returns its index in the results.+func (b *batch) add(sql string, args ...any) int {+ b.stmts = append(b.stmts, statement{SQL: sql, Args: args})+ return len(b.stmts) - 1+}++// guard appends an assertion that aborts the whole batch when condition is+// false. condition is a SQL boolean expression using ? placeholders.+func (b *batch) guard(condition string, args ...any) {+ b.stmts = append(b.stmts, guardStatement(condition, args...))+}++// commit executes the batch atomically and returns one result per statement.+func (b *batch) commit(ctx context.Context, d *DB) ([]*result, error) {+ return d.client.batch(ctx, b.stmts)+}++// execOne runs a single write outside any batch.+func (d *DB) execOne(ctx context.Context, sql string, args ...any) (*result, error) {+ return d.client.exec(ctx, statement{SQL: sql, Args: args})+}++// placeholders returns n comma separated ? markers.+func placeholders(n int) string {+ if n <= 0 {+ return ""+ }+ return strings.TrimSuffix(strings.Repeat("?, ", n), ", ")+}++// inClause renders "(?, ?, ...)" for values and appends them to args.+func inClause[T any](values []T) (string, []any) {+ args := make([]any, 0, len(values))+ for _, value := range values {+ args = append(args, value)+ }+ return "(" + placeholders(len(values)) + ")", args+}++// intList renders trusted integer ids as a literal IN list. Inlining them+// keeps long id lists under D1's 100 bound parameter limit.+func intList(ids []int32) string {+ parts := make([]string, 0, len(ids))+ for _, id := range ids {+ parts = append(parts, strconv.FormatInt(int64(id), 10))+ }+ return "(" + strings.Join(parts, ", ") + ")"+}++// chunk splits values into slices of at most size elements.+func chunk[T any](values []T, size int) [][]T {+ if len(values) == 0 {+ return nil+ }+ if size <= 0 {+ size = len(values)+ }+ chunks := make([][]T, 0, (len(values)+size-1)/size)+ for start := 0; start < len(values); start += size {+ end := min(start+size, len(values))+ chunks = append(chunks, values[start:end])+ }+ return chunks+}++// appendLimit adds LIMIT/OFFSET clauses when present.+func appendLimit(query string, limit, offset *int) string {+ if limit != nil {+ query = fmt.Sprintf("%s LIMIT %d", query, *limit)+ if offset != nil {+ query = fmt.Sprintf("%s OFFSET %d", query, *offset)+ }+ }+ return query+}++// requireChanges returns ErrMemoMutationConflict-style failures via the+// supplied error when a batch statement touched a different number of rows+// than expected.+func requireChanges(results []*result, index int, expected int64, conflict error) error {+ if index < 0 || index >= len(results) {+ return errors.Errorf("d1: batch result %d is out of range", index)+ }+ if results[index].Changes != expected {+ return conflict+ }+ return nil+}diff --git a/store/db/d1/guard.go b/store/db/d1/guard.gonew file mode 100644index 000000000..09e4265eb--- /dev/null+++ b/store/db/d1/guard.go@@ -0,0 +1,44 @@+package d1++import (+ "errors"+ "strings"+)++// guardTable backs conditional writes inside atomic batches. Inserting the+// value 0 violates its CHECK constraint, which aborts and rolls back the+// batch; the table never holds rows. Driver methods use it to re-verify a+// precondition read before the batch so a concurrent change cannot slip a+// partial write through.+const guardTable = "d1_guard"++// guardConstraint names the CHECK so a guard failure is recognizable in the+// error message returned by D1.+const guardConstraint = "d1_guard_ok"++// guardStatement builds the assertion statement for condition. The insert+// selects a violating row only when the condition is false.+func guardStatement(condition string, args ...any) statement {+ return statement{+ SQL: "INSERT INTO " + guardTable + " (ok) SELECT 0 WHERE NOT (" + condition + ")",+ Args: args,+ }+}++// isGuardFailure reports whether err is a batch aborted by a guard statement.+func isGuardFailure(err error) bool {+ var d1Err *Error+ if !errors.As(err, &d1Err) {+ return false+ }+ return strings.Contains(d1Err.Message, guardConstraint)+}++// guardError maps a batch error to conflict when the batch was aborted by a+// guard and returns the original error otherwise.+func guardError(err error, conflict error) error {+ if isGuardFailure(err) {+ return conflict+ }+ return err+}diff --git a/store/db/d1/idp.go b/store/db/d1/idp.gonew file mode 100644index 000000000..6d3e2a21a--- /dev/null+++ b/store/db/d1/idp.go@@ -0,0 +1,101 @@+package d1++import (+ "context"+ "strings"++ storepb "github.com/usememos/memos/proto/gen/store"+ "github.com/usememos/memos/store"+)++const idpColumns = "id, uid, name, type, identifier_filter, config"++// CreateIdentityProvider inserts an identity provider.+func (d *DB) CreateIdentityProvider(ctx context.Context, create *store.IdentityProvider) (*store.IdentityProvider, error) {+ stmt := "INSERT INTO idp (uid, name, type, identifier_filter, config) VALUES (?, ?, ?, ?, ?) RETURNING id"+ if err := d.db.QueryRowContext(ctx, stmt, create.UID, create.Name, create.Type.String(), create.IdentifierFilter, create.Config).Scan(&create.ID); err != nil {+ return nil, err+ }+ return create, nil+}++// ListIdentityProviders returns the identity providers matching find.+func (d *DB) ListIdentityProviders(ctx context.Context, find *store.FindIdentityProvider) ([]*store.IdentityProvider, error) {+ where, args := []string{"1 = 1"}, []any{}+ if v := find.ID; v != nil {+ where, args = append(where, "id = ?"), append(args, *v)+ }+ if v := find.UID; v != nil {+ where, args = append(where, "uid = ?"), append(args, *v)+ }++ rows, err := d.db.QueryContext(ctx, "SELECT "+idpColumns+" FROM idp WHERE "+strings.Join(where, " AND ")+" ORDER BY id ASC", args...)+ if err != nil {+ return nil, err+ }+ defer rows.Close()++ var identityProviders []*store.IdentityProvider+ for rows.Next() {+ identityProvider, err := idpScan(rows)+ if err != nil {+ return nil, err+ }+ identityProviders = append(identityProviders, identityProvider)+ }+ if err := rows.Err(); err != nil {+ return nil, err+ }+ return identityProviders, nil+}++// idpScanner is satisfied by *sql.Row and *sql.Rows.+type idpScanner interface {+ Scan(dest ...any) error+}++// idpScan reads one row selected with idpColumns.+func idpScan(scanner idpScanner) (*store.IdentityProvider, error) {+ var identityProvider store.IdentityProvider+ var typeString string+ if err := scanner.Scan(+ &identityProvider.ID,+ &identityProvider.UID,+ &identityProvider.Name,+ &typeString,+ &identityProvider.IdentifierFilter,+ &identityProvider.Config,+ ); err != nil {+ return nil, err+ }+ identityProvider.Type = storepb.IdentityProvider_Type(storepb.IdentityProvider_Type_value[typeString])+ return &identityProvider, nil+}++// UpdateIdentityProvider applies the given identity provider changes.+func (d *DB) UpdateIdentityProvider(ctx context.Context, update *store.UpdateIdentityProvider) (*store.IdentityProvider, error) {+ set, args := []string{}, []any{}+ if v := update.Name; v != nil {+ set, args = append(set, "name = ?"), append(args, *v)+ }+ if v := update.IdentifierFilter; v != nil {+ set, args = append(set, "identifier_filter = ?"), append(args, *v)+ }+ if v := update.Config; v != nil {+ set, args = append(set, "config = ?"), append(args, *v)+ }+ if len(set) == 0 {+ // Nothing to change: report the current row rather than sending an+ // empty SET clause.+ return idpScan(d.db.QueryRowContext(ctx, "SELECT "+idpColumns+" FROM idp WHERE id = ?", update.ID))+ }+ args = append(args, update.ID)+ stmt := "UPDATE idp SET " + strings.Join(set, ", ") + " WHERE id = ? RETURNING " + idpColumns+ return idpScan(d.db.QueryRowContext(ctx, stmt, args...))+}++// DeleteIdentityProvider removes an identity provider.+func (d *DB) DeleteIdentityProvider(ctx context.Context, delete *store.DeleteIdentityProvider) error {+ _, err := d.execOne(ctx, "DELETE FROM idp WHERE id = ?", delete.ID)+ return err+}diff --git a/store/db/d1/inbox.go b/store/db/d1/inbox.gonew file mode 100644index 000000000..38496ccde--- /dev/null+++ b/store/db/d1/inbox.go@@ -0,0 +1,122 @@+package d1++import (+ "context"+ "strings"++ "github.com/pkg/errors"+ "google.golang.org/protobuf/encoding/protojson"++ storepb "github.com/usememos/memos/proto/gen/store"+ "github.com/usememos/memos/store"+)++const inboxColumns = "id, created_ts, sender_id, receiver_id, status, message"++// CreateInbox inserts an inbox message.+func (d *DB) CreateInbox(ctx context.Context, create *store.Inbox) (*store.Inbox, error) {+ messageString := "{}"+ if create.Message != nil {+ bytes, err := protojson.Marshal(create.Message)+ if err != nil {+ return nil, errors.Wrap(err, "failed to marshal inbox message")+ }+ messageString = string(bytes)+ }++ stmt := "INSERT INTO inbox (sender_id, receiver_id, status, message) VALUES (?, ?, ?, ?) RETURNING id, created_ts"+ if err := d.db.QueryRowContext(ctx, stmt, create.SenderID, create.ReceiverID, create.Status, messageString).Scan(+ &create.ID,+ &create.CreatedTs,+ ); err != nil {+ return nil, err+ }+ return create, nil+}++// ListInboxes returns the inbox messages matching find.+func (d *DB) ListInboxes(ctx context.Context, find *store.FindInbox) ([]*store.Inbox, error) {+ where, args := []string{"1 = 1"}, []any{}++ if find.ID != nil {+ where, args = append(where, "id = ?"), append(args, *find.ID)+ }+ if find.SenderID != nil {+ where, args = append(where, "sender_id = ?"), append(args, *find.SenderID)+ }+ if find.ReceiverID != nil {+ where, args = append(where, "receiver_id = ?"), append(args, *find.ReceiverID)+ }+ if find.Status != nil {+ where, args = append(where, "status = ?"), append(args, *find.Status)+ }+ if find.MessageType != nil {+ // The message column stores protojson, whose type field is the enum+ // name; D1 ships SQLite's JSON functions so JSON_EXTRACT works here.+ if *find.MessageType == storepb.InboxMessage_TYPE_UNSPECIFIED {+ where, args = append(where, "(JSON_EXTRACT(message, '$.type') IS NULL OR JSON_EXTRACT(message, '$.type') = ?)"), append(args, find.MessageType.String())+ } else {+ where, args = append(where, "JSON_EXTRACT(message, '$.type') = ?"), append(args, find.MessageType.String())+ }+ }++ query := "SELECT " + inboxColumns + " FROM inbox WHERE " + strings.Join(where, " AND ") + " ORDER BY created_ts DESC"+ query = appendLimit(query, find.Limit, find.Offset)+ rows, err := d.db.QueryContext(ctx, query, args...)+ if err != nil {+ return nil, err+ }+ defer rows.Close()++ list := []*store.Inbox{}+ for rows.Next() {+ inbox, err := inboxScan(rows)+ if err != nil {+ return nil, err+ }+ list = append(list, inbox)+ }+ if err := rows.Err(); err != nil {+ return nil, err+ }+ return list, nil+}++// inboxScanner is satisfied by *sql.Row and *sql.Rows.+type inboxScanner interface {+ Scan(dest ...any) error+}++// inboxScan reads one row selected with inboxColumns.+func inboxScan(scanner inboxScanner) (*store.Inbox, error) {+ inbox := &store.Inbox{}+ var messageBytes []byte+ if err := scanner.Scan(+ &inbox.ID,+ &inbox.CreatedTs,+ &inbox.SenderID,+ &inbox.ReceiverID,+ &inbox.Status,+ &messageBytes,+ ); err != nil {+ return nil, err+ }+ message := &storepb.InboxMessage{}+ if err := protojsonUnmarshaler.Unmarshal(messageBytes, message); err != nil {+ return nil, err+ }+ inbox.Message = message+ return inbox, nil+}++// UpdateInbox changes the status of an inbox message.+func (d *DB) UpdateInbox(ctx context.Context, update *store.UpdateInbox) (*store.Inbox, error) {+ query := "UPDATE inbox SET status = ? WHERE id = ? RETURNING " + inboxColumns+ return inboxScan(d.db.QueryRowContext(ctx, query, update.Status.String(), update.ID))+}++// DeleteInbox removes an inbox message.+func (d *DB) DeleteInbox(ctx context.Context, delete *store.DeleteInbox) error {+ _, err := d.execOne(ctx, "DELETE FROM inbox WHERE id = ?", delete.ID)+ return err+}diff --git a/store/db/d1/instance_setting.go b/store/db/d1/instance_setting.gonew file mode 100644index 000000000..2f52ef81a--- /dev/null+++ b/store/db/d1/instance_setting.go@@ -0,0 +1,65 @@+package d1++import (+ "context"+ "strings"++ "github.com/pkg/errors"++ "github.com/usememos/memos/store"+)++// CreateInstanceSettingIfNotExists atomically creates the setting when its name is absent and reports whether it inserted the row.+func (d *DB) CreateInstanceSettingIfNotExists(ctx context.Context, create *store.InstanceSetting) (bool, error) {+ result, err := d.execOne(ctx, `INSERT INTO system_setting (name, value, description) VALUES (?, ?, ?) ON CONFLICT(name) DO NOTHING`,+ create.Name, create.Value, create.Description)+ if err != nil {+ return false, errors.Wrap(err, "failed to conditionally create instance setting")+ }+ return result.Changes == 1, nil+}++// UpsertInstanceSetting inserts or replaces an instance setting.+func (d *DB) UpsertInstanceSetting(ctx context.Context, upsert *store.InstanceSetting) (*store.InstanceSetting, error) {+ if _, err := d.execOne(ctx, settingUpsertStatement, upsert.Name, upsert.Value, upsert.Description); err != nil {+ return nil, err+ }+ return upsert, nil+}++// settingUpsertStatement inserts or replaces one system_setting row.+const settingUpsertStatement = `INSERT INTO system_setting (name, value, description) VALUES (?, ?, ?)+ ON CONFLICT(name) DO UPDATE SET value = EXCLUDED.value, description = EXCLUDED.description`++// ListInstanceSettings returns the instance settings matching find.+func (d *DB) ListInstanceSettings(ctx context.Context, find *store.FindInstanceSetting) ([]*store.InstanceSetting, error) {+ where, args := []string{"1 = 1"}, []any{}+ if find.Name != "" {+ where, args = append(where, "name = ?"), append(args, find.Name)+ }++ rows, err := d.db.QueryContext(ctx, "SELECT name, value, description FROM system_setting WHERE "+strings.Join(where, " AND "), args...)+ if err != nil {+ return nil, err+ }+ defer rows.Close()++ list := []*store.InstanceSetting{}+ for rows.Next() {+ setting := &store.InstanceSetting{}+ if err := rows.Scan(&setting.Name, &setting.Value, &setting.Description); err != nil {+ return nil, err+ }+ list = append(list, setting)+ }+ if err := rows.Err(); err != nil {+ return nil, err+ }+ return list, nil+}++// DeleteInstanceSetting removes an instance setting.+func (d *DB) DeleteInstanceSetting(ctx context.Context, delete *store.DeleteInstanceSetting) error {+ _, err := d.execOne(ctx, "DELETE FROM system_setting WHERE name = ?", delete.Name)+ return err+}diff --git a/store/db/d1/memo.go b/store/db/d1/memo.gonew file mode 100644index 000000000..eabd262ea--- /dev/null+++ b/store/db/d1/memo.go@@ -0,0 +1,446 @@+package d1++import (+ "context"+ "database/sql"+ "strings"++ "github.com/pkg/errors"+ "google.golang.org/protobuf/encoding/protojson"++ "github.com/usememos/memos/filter"+ storepb "github.com/usememos/memos/proto/gen/store"+ "github.com/usememos/memos/store"+)++// memoState is the memo row state read before a write batch. Guards built+// from it re-assert the row inside the batch so a concurrent change aborts+// the commit instead of being overwritten.+type memoState struct {+ id int32+ creatorID int32+ rowStatus store.RowStatus+ visibility store.Visibility+ spaceID *int32+ content string+}++// memoInlineUIDThreshold is the uid list length above which ListMemos resolves+// uids to ids before querying, leaving bind capacity for the other filters.+const memoInlineUIDThreshold = 40++// memoResolveUIDs maps uids to memo ids in chunks that respect the bind limit.+// Unknown uids are dropped; -1 keeps the IN list valid when none resolve.+func memoResolveUIDs(ctx context.Context, q querier, uids []string) ([]int32, error) {+ ids := make([]int32, 0, len(uids))+ for _, batch := range chunk(uids, inClauseBatchSize) {+ clause, args := inClause(batch)+ resolved, err := listMemoIDs(ctx, q, "SELECT id FROM memo WHERE uid IN "+clause, args...)+ if err != nil {+ return nil, err+ }+ ids = append(ids, resolved...)+ }+ if len(ids) == 0 {+ ids = append(ids, -1)+ }+ return ids, nil+}++// loadMemoState reads the memo row; a missing memo yields sql.ErrNoRows.+func loadMemoState(ctx context.Context, q querier, memoID int32) (*memoState, error) {+ state := &memoState{}+ var spaceID sql.NullInt64+ if err := q.QueryRowContext(ctx, "SELECT id, creator_id, row_status, visibility, space_id, content FROM memo WHERE id = ?", memoID).Scan(+ &state.id, &state.creatorID, &state.rowStatus, &state.visibility, &spaceID, &state.content,+ ); err != nil {+ return nil, err+ }+ state.spaceID = store.NullInt32Pointer(spaceID)+ return state, nil+}++// unchangedCondition is a guard condition asserting the memo still has the+// ownership, lifecycle, audience, and placement that were read. "IS ?" makes+// a NULL space compare equal to a NULL binding.+func (s *memoState) unchangedCondition() (string, []any) {+ return "EXISTS (SELECT 1 FROM memo WHERE id = ? AND creator_id = ? AND row_status = ? AND visibility = ? AND space_id IS ?)",+ []any{s.id, s.creatorID, s.rowStatus, s.visibility, s.spaceID}+}++// contentUnchangedCondition extends unchangedCondition with the content the+// caller expects, for mutations that were computed from that content.+func (s *memoState) contentUnchangedCondition() (string, []any) {+ return "EXISTS (SELECT 1 FROM memo WHERE id = ? AND creator_id = ? AND row_status = ? AND visibility = ? AND space_id IS ? AND content = ?)",+ []any{s.id, s.creatorID, s.rowStatus, s.visibility, s.spaceID, s.content}+}++// CreateMemo inserts a memo after validating its creator and Space placement.+func (d *DB) CreateMemo(ctx context.Context, create *store.Memo) (*store.Memo, error) {+ if err := memoValidateCreate(ctx, d.db, create); err != nil {+ return nil, err+ }+ b := newBatch()+ memoGuardCreate(b, create)+ if err := memoAddInsert(b, create); err != nil {+ return nil, err+ }+ if _, err := b.commit(ctx, d); err != nil {+ // The guards mirror memoValidateCreate; a failure means the creator+ // or the space membership changed between the read and the commit.+ return nil, guardError(err, store.ErrMemoSpaceMembershipRequired)+ }+ if err := memoLoadCreated(ctx, d.db, create); err != nil {+ return nil, err+ }+ return create, nil+}++// memoValidateCreate checks that the creator is active and, for a Space memo,+// an active member of an existing Space.+func memoValidateCreate(ctx context.Context, q querier, create *store.Memo) error {+ if err := requireActiveUser(ctx, q, create.CreatorID, store.ErrMemoSpaceMembershipRequired); err != nil {+ return err+ }+ if create.SpaceID == nil {+ return nil+ }+ exists, member, err := spaceState(ctx, q, *create.SpaceID, create.CreatorID)+ if err != nil {+ return err+ }+ if !exists {+ return store.ErrMemoSpaceNotWritable+ }+ if !member {+ return store.ErrMemoSpaceMembershipRequired+ }+ return nil+}++// memoGuardCreate re-asserts memoValidateCreate inside the batch.+func memoGuardCreate(b *batch, create *store.Memo) {+ b.guard(activeUserCondition, create.CreatorID)+ if create.SpaceID != nil {+ b.guard("EXISTS (SELECT 1 FROM space WHERE id = ?)", *create.SpaceID)+ b.guard(activeSpaceMemberCondition, *create.SpaceID, create.CreatorID)+ }+}++// memoAddInsert appends the memo INSERT. The uid is always written explicitly+// because later statements in the same batch can only find the new row by it.+func memoAddInsert(b *batch, create *store.Memo) error {+ payload := "{}"+ if create.Payload != nil {+ payloadBytes, err := protojson.Marshal(create.Payload)+ if err != nil {+ return errors.Wrap(err, "failed to marshal memo payload")+ }+ payload = string(payloadBytes)+ }+ columns := []string{"uid", "creator_id", "content", "visibility", "payload", "space_id"}+ args := []any{create.UID, create.CreatorID, create.Content, create.Visibility, payload, create.SpaceID}+ if create.CreatedTs != 0 {+ columns = append(columns, "created_ts")+ args = append(args, create.CreatedTs)+ }+ if create.UpdatedTs != 0 {+ columns = append(columns, "updated_ts")+ args = append(args, create.UpdatedTs)+ }+ b.add("INSERT INTO memo ("+strings.Join(columns, ", ")+") VALUES ("+placeholders(len(args))+")", args...)+ return nil+}++// memoLoadCreated fills the generated columns of a memo inserted by a batch.+// A batch cannot return them, so the row is found again through its uid.+func memoLoadCreated(ctx context.Context, q querier, create *store.Memo) error {+ if err := q.QueryRowContext(ctx, "SELECT id, created_ts, updated_ts, row_status FROM memo WHERE uid = ?", create.UID).Scan(+ &create.ID, &create.CreatedTs, &create.UpdatedTs, &create.RowStatus,+ ); err != nil {+ return errors.Wrap(err, "failed to load created memo")+ }+ return nil+}++// ListMemos returns the memos matching find.+func (d *DB) ListMemos(ctx context.Context, find *store.FindMemo) ([]*store.Memo, error) {+ where, args, err := memoListConditions(ctx, d.db, find)+ if err != nil {+ return nil, err+ }++ fields := []string{+ "memo.id AS id",+ "memo.uid AS uid",+ "memo.creator_id AS creator_id",+ "memo.created_ts AS created_ts",+ "memo.updated_ts AS updated_ts",+ "memo.row_status AS row_status",+ "memo.visibility AS visibility",+ "memo.pinned AS pinned",+ "memo.payload AS payload",+ "memo.space_id AS space_id",+ `(SELECT parent_memo.uid+ FROM memo_relation AS parent_relation+ JOIN memo AS parent_memo ON parent_memo.id = parent_relation.related_memo_id+ WHERE parent_relation.memo_id = memo.id AND parent_relation.type = 'COMMENT'+ ORDER BY parent_memo.id LIMIT 1) AS parent_uid`,+ }+ if !find.ExcludeContent {+ fields = append(fields, "memo.content AS content")+ }+ // The creator and space joins are referenced by filter expressions.+ query := "SELECT " + strings.Join(fields, ", ") + " FROM memo " ++ "LEFT JOIN user AS memo_creator ON memo.creator_id = memo_creator.id " ++ "LEFT JOIN space AS memo_space ON memo.space_id = memo_space.id " ++ "WHERE " + strings.Join(where, " AND ") + " " ++ "ORDER BY " + strings.Join(memoListOrder(find), ", ")+ query = appendLimit(query, find.Limit, find.Offset)++ rows, err := d.db.QueryContext(ctx, query, args...)+ if err != nil {+ return nil, err+ }+ defer rows.Close()++ list := make([]*store.Memo, 0)+ for rows.Next() {+ memo, err := memoScanRow(rows, find.ExcludeContent)+ if err != nil {+ return nil, err+ }+ list = append(list, memo)+ }+ if err := rows.Err(); err != nil {+ return nil, err+ }+ return list, nil+}++// memoListConditions renders the WHERE clauses for find. q serves the uid+// resolution a long uid list needs.+func memoListConditions(ctx context.Context, q querier, find *store.FindMemo) ([]string, []any, error) {+ where, args := []string{"1 = 1"}, []any{}++ engine, err := filter.DefaultEngine()+ if err != nil {+ return nil, nil, err+ }+ if err := filter.AppendConditions(ctx, engine, find.Filters, filter.DialectD1, &where, &args); err != nil {+ return nil, nil, err+ }+ if v := find.ID; v != nil {+ where, args = append(where, "memo.id = ?"), append(args, *v)+ }+ if len(find.IDList) > 0 {+ where = append(where, "memo.id IN "+intList(find.IDList))+ }+ if v := find.UID; v != nil {+ where, args = append(where, "memo.uid = ?"), append(args, *v)+ }+ if len(find.UIDList) > 0 {+ // A long uid list would exhaust D1's bind limit together with the+ // other filters, so it is resolved to ids first and inlined.+ if len(find.UIDList) > memoInlineUIDThreshold {+ ids, err := memoResolveUIDs(ctx, q, find.UIDList)+ if err != nil {+ return nil, nil, err+ }+ where = append(where, "memo.id IN "+intList(ids))+ } else {+ clause, uidArgs := inClause(find.UIDList)+ where, args = append(where, "memo.uid IN "+clause), append(args, uidArgs...)+ }+ }+ if v := find.CreatorID; v != nil {+ where, args = append(where, "memo.creator_id = ?"), append(args, *v)+ }+ if v := find.RowStatus; v != nil {+ where, args = append(where, "memo.row_status = ?"), append(args, *v)+ }+ if len(find.VisibilityList) > 0 {+ visibilities := make([]string, 0, len(find.VisibilityList))+ for _, visibility := range find.VisibilityList {+ visibilities = append(visibilities, visibility.String())+ }+ clause, visibilityArgs := inClause(visibilities)+ where, args = append(where, "memo.visibility IN "+clause), append(args, visibilityArgs...)+ }+ if v := find.CommentContextMemoID; v != nil {+ where, args = append(where, `EXISTS (+ SELECT 1 FROM memo_relation AS comment_context+ WHERE comment_context.memo_id = memo.id+ AND comment_context.related_memo_id = ?+ AND comment_context.type = 'COMMENT'+ )`), append(args, *v)+ }+ if access := find.Access; access != nil {+ where = append(where, memoAccessPredicate(access, "memo", "access_member", &args))+ }+ if find.ExcludeComments {+ where = append(where, `NOT EXISTS (+ SELECT 1 FROM memo_relation AS comment_relation+ WHERE comment_relation.memo_id = memo.id AND comment_relation.type = 'COMMENT'+ )`)+ }+ return where, args, nil+}++// memoListOrder renders the ORDER BY terms for find, with id as tie-breaker.+func memoListOrder(find *store.FindMemo) []string {+ order := "DESC"+ if find.OrderByTimeAsc {+ order = "ASC"+ }+ orderBy := []string{}+ if find.OrderByPinned {+ orderBy = append(orderBy, "pinned DESC")+ }+ if find.OrderByUpdatedTs {+ orderBy = append(orderBy, "updated_ts "+order)+ } else {+ orderBy = append(orderBy, "created_ts "+order)+ }+ return append(orderBy, "id DESC")+}++// memoScanRow reads one ListMemos row.+func memoScanRow(rows *sql.Rows, excludeContent bool) (*store.Memo, error) {+ memo := &store.Memo{}+ var payloadBytes []byte+ dests := []any{+ &memo.ID,+ &memo.UID,+ &memo.CreatorID,+ &memo.CreatedTs,+ &memo.UpdatedTs,+ &memo.RowStatus,+ &memo.Visibility,+ &memo.Pinned,+ &payloadBytes,+ &memo.SpaceID,+ &memo.ParentUID,+ }+ if !excludeContent {+ dests = append(dests, &memo.Content)+ }+ if err := rows.Scan(dests...); err != nil {+ return nil, err+ }+ payload := &storepb.MemoPayload{}+ if len(payloadBytes) > 0 {+ if err := protojsonUnmarshaler.Unmarshal(payloadBytes, payload); err != nil {+ return nil, errors.Wrap(err, "failed to unmarshal payload")+ }+ }+ memo.Payload = payload+ return memo, nil+}++// UpdateMemo applies the given memo changes, checking the write policy when one is set.+func (d *DB) UpdateMemo(ctx context.Context, update *store.UpdateMemo) error {+ b := newBatch()+ if update.Policy != nil {+ if err := validateMemoWritePolicy(ctx, d.db, update.ID, update.Policy, update); err != nil {+ return err+ }+ state, err := loadMemoState(ctx, d.db, update.ID)+ if err != nil {+ if errors.Is(err, sql.ErrNoRows) {+ return store.ErrMemoMutationConflict+ }+ return errors.Wrap(err, "failed to read memo")+ }+ b.guard(activeUserCondition, update.Policy.ActorUserID)+ condition, args := state.unchangedCondition()+ b.guard(condition, args...)+ memoGuardPolicyPlacement(b, state, update.Policy, update)+ }+ if err := memoAddUpdate(b, update); err != nil {+ return err+ }+ if _, err := b.commit(ctx, d); err != nil {+ return guardError(err, errors.Wrap(store.ErrMemoMutationConflict, "memo changed while updating"))+ }+ return nil+}++// memoGuardPolicyPlacement re-asserts the Space and share state that+// validateMemoWritePolicy accepted: the source Space still exists, the actor+// keeps the memberships the policy needs, and no active share blocks a move+// to the SPACE audience.+func memoGuardPolicyPlacement(b *batch, state *memoState, policy *store.MemoWritePolicy, update *store.UpdateMemo) {+ if state.spaceID != nil {+ b.guard("EXISTS (SELECT 1 FROM space WHERE id = ?)", *state.spaceID)+ if !policy.LifecycleOnly {+ b.guard(activeSpaceMemberCondition, *state.spaceID, policy.ActorUserID)+ }+ }+ if update == nil {+ return+ }+ if update.SpaceID != nil {+ b.guard(activeSpaceMemberCondition, *update.SpaceID, policy.ActorUserID)+ }+ if update.Visibility != nil && *update.Visibility == store.SpaceAudience {+ b.guard(`NOT EXISTS (SELECT 1 FROM memo_share+ WHERE memo_id = ? AND (expires_ts IS NULL OR expires_ts > CAST(strftime('%s', 'now') AS INTEGER)))`, state.id)+ }+}++// memoAddUpdate appends the memo UPDATE for the fields set on update, or+// nothing when there is no field to write.+func memoAddUpdate(b *batch, update *store.UpdateMemo) error {+ set, args := []string{}, []any{}+ if v := update.UID; v != nil {+ set, args = append(set, "uid = ?"), append(args, *v)+ }+ if v := update.CreatedTs; v != nil {+ set, args = append(set, "created_ts = ?"), append(args, *v)+ }+ if v := update.UpdatedTs; v != nil {+ set, args = append(set, "updated_ts = ?"), append(args, *v)+ }+ if v := update.RowStatus; v != nil {+ set, args = append(set, "row_status = ?"), append(args, *v)+ }+ if v := update.Content; v != nil {+ set, args = append(set, "content = ?"), append(args, *v)+ }+ if v := update.Visibility; v != nil {+ set, args = append(set, "visibility = ?"), append(args, *v)+ }+ if v := update.Pinned; v != nil {+ set, args = append(set, "pinned = ?"), append(args, *v)+ }+ if v := update.Payload; v != nil {+ payload, err := protojson.Marshal(v)+ if err != nil {+ return errors.Wrap(err, "failed to marshal memo payload")+ }+ set, args = append(set, "payload = ?"), append(args, string(payload))+ }+ if update.ClearSpace {+ set = append(set, "space_id = NULL")+ } else if v := update.SpaceID; v != nil {+ set, args = append(set, "space_id = ?"), append(args, *v)+ }+ if len(set) == 0 {+ return nil+ }+ args = append(args, update.ID)+ b.add("UPDATE memo SET "+strings.Join(set, ", ")+" WHERE id = ?", args...)+ return nil+}++// DeleteMemo removes a memo and its reactions.+func (d *DB) DeleteMemo(ctx context.Context, delete *store.DeleteMemo) error {+ b := newBatch()+ b.add("DELETE FROM memo WHERE id = ?", delete.ID)+ b.add("DELETE FROM reaction WHERE memo_id = ?", delete.ID)+ if _, err := b.commit(ctx, d); err != nil {+ return errors.Wrap(err, "failed to delete memo")+ }+ return nil+}diff --git a/store/db/d1/memo_delete_policy.go b/store/db/d1/memo_delete_policy.gonew file mode 100644index 000000000..88e52502e--- /dev/null+++ b/store/db/d1/memo_delete_policy.go@@ -0,0 +1,54 @@+package d1++import (+ "context"+ "database/sql"++ "github.com/pkg/errors"++ "github.com/usememos/memos/store"+)++// DeleteMemoWithPolicy removes a memo on behalf of an actor together with its shares, reactions, attachments, and relations.+func (d *DB) DeleteMemoWithPolicy(ctx context.Context, delete *store.DeleteMemoWithPolicy) (*store.DeleteMemoWithPolicyResult, error) {+ if err := requireActiveUser(ctx, d.db, delete.ActorUserID, store.ErrMemoPermissionDenied); err != nil {+ return nil, err+ }+ state, err := loadMemoState(ctx, d.db, delete.MemoID)+ if errors.Is(err, sql.ErrNoRows) {+ return nil, store.ErrMemoMutationConflict+ }+ if err != nil {+ return nil, errors.Wrap(err, "failed to read memo")+ }+ if state.creatorID != delete.ActorUserID {+ return nil, store.ErrMemoPermissionDenied+ }+ spaceExists, actorMember := false, false+ if state.spaceID != nil {+ spaceExists, actorMember, err = spaceState(ctx, d.db, *state.spaceID, delete.ActorUserID)+ if err != nil {+ return nil, errors.Wrap(err, "failed to read memo space")+ }+ }+ actorCanRead := store.MemoDeleteActorCanRead(state.rowStatus, state.visibility, state.spaceID, spaceExists, actorMember)++ memoIDs := []int32{delete.MemoID}+ attachments, err := listMemoSetAttachments(ctx, d.db, memoIDs)+ if err != nil {+ return nil, errors.Wrap(err, "failed to collect memo attachments")+ }++ b := newBatch()+ b.guard(activeUserCondition, delete.ActorUserID)+ b.guard("EXISTS (SELECT 1 FROM memo WHERE id = ? AND creator_id = ?)", delete.MemoID, delete.ActorUserID)+ // The attachments were listed before the batch and are returned for+ // storage cleanup; abort if the bound set changed so none is orphaned in+ // the database or missed by the caller.+ b.guard("(SELECT COUNT(*) FROM attachment WHERE memo_id = ?) = ?", delete.MemoID, len(attachments))+ addMemoSetDeletes(b, memoIDs, attachmentIDs(attachments))+ if _, err := b.commit(ctx, d); err != nil {+ return nil, guardError(err, errors.Wrap(store.ErrMemoMutationConflict, "memo changed while deleting"))+ }+ return &store.DeleteMemoWithPolicyResult{ActorCanRead: actorCanRead, Attachments: attachments}, nil+}diff --git a/store/db/d1/memo_mutation.go b/store/db/d1/memo_mutation.gonew file mode 100644index 000000000..6fc0d3ca1--- /dev/null+++ b/store/db/d1/memo_mutation.go@@ -0,0 +1,353 @@+package d1++import (+ "context"+ "database/sql"++ "github.com/pkg/errors"++ "github.com/usememos/memos/store"+)++// memoRef addresses the mutated memo inside batch statements. A memo created+// by the same batch has no id yet, so it is found through its unique uid.+type memoRef struct {+ expr string+ arg any+}++func memoRefByID(id int32) memoRef {+ return memoRef{expr: "?", arg: id}+}++func memoRefByUID(uid string) memoRef {+ return memoRef{expr: "(SELECT id FROM memo WHERE uid = ?)", arg: uid}+}++// memoMutationPlan is the state read by validation that the batch re-asserts+// with guards before applying the writes.+type memoMutationPlan struct {+ mutation *store.MemoMutation+ ref memoRef+ policy *store.MemoWritePolicy+ endpoints []*memoState+ // context is the comment context memo, set only with MemoCreate.+ context *memoState+ // memo is the existing memo, nil with MemoCreate.+ memo *memoState+ removed []*store.Attachment+}++// ApplyMemoMutation atomically updates a memo, attachment bindings, and+// reference relations. Every check the SQLite driver makes inside its+// transaction is made here as a read first and then re-asserted by a guard in+// the batch, so a concurrent change aborts the commit as a conflict.+func (d *DB) ApplyMemoMutation(ctx context.Context, mutation *store.MemoMutation) error {+ plan, err := d.planMemoMutation(ctx, mutation)+ if err != nil {+ return err+ }+ b := newBatch()+ if err := plan.addTo(b); err != nil {+ return err+ }+ if _, err := b.commit(ctx, d); err != nil {+ return guardError(err, errors.Wrap(store.ErrMemoMutationConflict, "memo state changed while applying mutation"))+ }+ return plan.finish(ctx, d.db)+}++// planMemoMutation performs the validation reads in the same order as the+// SQLite driver and collects the state the batch guards depend on.+func (d *DB) planMemoMutation(ctx context.Context, mutation *store.MemoMutation) (*memoMutationPlan, error) {+ plan := &memoMutationPlan{mutation: mutation}+ endpoints, err := memoValidateRelationEndpoints(ctx, d.db, mutation)+ if err != nil {+ return nil, err+ }+ plan.endpoints = endpoints++ if create := mutation.MemoCreate; create != nil {+ if err := plan.planCreate(ctx, d.db, create); err != nil {+ return nil, err+ }+ } else {+ plan.ref = memoRefByID(mutation.MemoID)+ }++ plan.policy = mutation.Policy+ if plan.policy == nil && mutation.MemoUpdate != nil {+ plan.policy = mutation.MemoUpdate.Policy+ }+ if plan.policy != nil {+ if err := validateMemoWritePolicy(ctx, d.db, mutation.MemoID, plan.policy, mutation.MemoUpdate); err != nil {+ return nil, err+ }+ }+ if mutation.MemoCreate == nil {+ if err := plan.planExistingMemo(ctx, d.db); err != nil {+ return nil, err+ }+ }+ if err := plan.planRemovedAttachments(ctx, d.db); err != nil {+ return nil, err+ }+ if err := plan.planBindings(ctx, d.db); err != nil {+ return nil, err+ }+ if err := plan.planRequiredAttachments(ctx, d.db); err != nil {+ return nil, err+ }+ if mutation.MemoUpdate != nil && mutation.MemoUpdate.ID != mutation.MemoID {+ return nil, errors.New("memo update target does not match attachment mutation")+ }+ if err := plan.planRelations(); err != nil {+ return nil, err+ }+ return plan, nil+}++// planCreate validates the memo creation and, for a comment, the context memo.+func (p *memoMutationPlan) planCreate(ctx context.Context, q querier, create *store.Memo) error {+ if err := memoValidateCreate(ctx, q, create); err != nil {+ return err+ }+ if contextID := p.mutation.CommentContextMemoID; contextID != nil {+ if *contextID <= 0 {+ return errors.New("invalid COMMENT relation")+ }+ if err := authorizeMemoComment(ctx, q, *contextID, create.CreatorID); err != nil {+ return err+ }+ state, err := loadMemoState(ctx, q, *contextID)+ if err != nil {+ if errors.Is(err, sql.ErrNoRows) {+ return store.ErrMemoSpaceNotWritable+ }+ return errors.Wrap(err, "failed to read comment context memo")+ }+ p.context = state+ }+ p.mutation.MemoCreatorID = create.CreatorID+ p.mutation.ExpectedMemoContent = create.Content+ p.ref = memoRefByUID(create.UID)+ return nil+}++// planExistingMemo checks the memo still matches what the caller computed+// the mutation from.+func (p *memoMutationPlan) planExistingMemo(ctx context.Context, q querier) error {+ state, err := loadMemoState(ctx, q, p.mutation.MemoID)+ if err != nil {+ if errors.Is(err, sql.ErrNoRows) {+ return errors.Wrap(store.ErrMemoMutationConflict, "memo no longer exists")+ }+ return errors.Wrap(err, "failed to read memo")+ }+ if state.creatorID != p.mutation.MemoCreatorID || state.content != p.mutation.ExpectedMemoContent {+ return errors.Wrap(store.ErrMemoMutationConflict, "memo changed while applying mutation")+ }+ p.memo = state+ return nil+}++// planRemovedAttachments checks that every removed attachment is still bound+// to the memo and owned by its creator.+func (p *memoMutationPlan) planRemovedAttachments(ctx context.Context, q querier) error {+ mutation := p.mutation+ removed, err := listAttachmentSnapshots(ctx, q, mutation.RemovedAttachmentIDs)+ if err != nil {+ return errors.Wrap(err, "failed to read removed attachments")+ }+ if len(removed) != len(mutation.RemovedAttachmentIDs) {+ return errors.Wrap(store.ErrMemoMutationConflict, "removed attachment no longer exists")+ }+ for _, attachment := range removed {+ if attachment.CreatorID != mutation.MemoCreatorID || attachment.MemoID == nil || *attachment.MemoID != mutation.MemoID {+ return errors.Wrap(store.ErrMemoMutationConflict, "attachment is no longer removable from the memo")+ }+ }+ p.removed = removed+ return nil+}++// planBindings checks each attachment is in the state the binding expects:+// still bound to this memo, or unbound and owned by the memo creator.+func (p *memoMutationPlan) planBindings(ctx context.Context, q querier) error {+ mutation := p.mutation+ for _, binding := range mutation.Bindings {+ var creatorID int32+ var memoID sql.NullInt32+ if err := q.QueryRowContext(ctx, "SELECT creator_id, memo_id FROM attachment WHERE id = ?", binding.ID).Scan(&creatorID, &memoID); err != nil {+ if errors.Is(err, sql.ErrNoRows) {+ return errors.Wrapf(store.ErrMemoMutationConflict, "attachment %s no longer exists", binding.UID)+ }+ return errors.Wrap(err, "failed to read attachment")+ }+ if binding.WasBoundToMemo {+ // A memo created by this mutation cannot have prior bindings.+ if mutation.MemoCreate != nil || !memoID.Valid || memoID.Int32 != mutation.MemoID {+ return errors.Wrapf(store.ErrMemoMutationConflict, "attachment %s is no longer bound to the memo", binding.UID)+ }+ } else if creatorID != mutation.MemoCreatorID || memoID.Valid {+ return errors.Wrapf(store.ErrMemoMutationConflict, "attachment %s is no longer available", binding.UID)+ }+ }+ return nil+}++// planRequiredAttachments checks each required attachment will be bound once+// the batch has applied the bindings. The batch guards this again after the+// binding updates, where the effect of the bindings is visible.+func (p *memoMutationPlan) planRequiredAttachments(ctx context.Context, q querier) error {+ mutation := p.mutation+ bound := make(map[int32]struct{}, len(mutation.Bindings))+ for _, binding := range mutation.Bindings {+ if binding != nil {+ bound[binding.ID] = struct{}{}+ }+ }+ for _, attachmentID := range mutation.RequiredAttachmentIDs {+ if _, ok := bound[attachmentID]; ok {+ continue+ }+ if mutation.MemoCreate != nil {+ return errors.Wrap(store.ErrMemoMutationConflict, "a referenced attachment is no longer bound to the memo")+ }+ var exists int+ err := q.QueryRowContext(ctx, "SELECT 1 FROM attachment WHERE id = ? AND memo_id = ?", attachmentID, mutation.MemoID).Scan(&exists)+ if errors.Is(err, sql.ErrNoRows) {+ return errors.Wrap(store.ErrMemoMutationConflict, "a referenced attachment is no longer bound to the memo")+ }+ if err != nil {+ return errors.Wrap(err, "failed to verify referenced attachment")+ }+ }+ return nil+}++// planRelations validates the reference relations. Relations of a memo+// created by this mutation receive their memo id after the commit.+func (p *memoMutationPlan) planRelations() error {+ mutation := p.mutation+ if !mutation.ReplaceReferenceRelations {+ return nil+ }+ for _, relation := range mutation.ReferenceRelations {+ if relation == nil || relation.Type != store.MemoRelationReference {+ return errors.New("invalid memo reference relation mutation")+ }+ if mutation.MemoCreate == nil && relation.MemoID != mutation.MemoID {+ return errors.New("invalid memo reference relation mutation")+ }+ }+ return nil+}++// addTo appends the guards and writes of the mutation to b.+func (p *memoMutationPlan) addTo(b *batch) error {+ mutation := p.mutation+ b.guard(activeUserCondition, mutation.MemoCreatorID)+ memoGuardRelationEndpoints(b, mutation.MemoCreatorID, p.endpoints)++ if create := mutation.MemoCreate; create != nil {+ memoGuardCreate(b, create)+ p.guardCommentContext(b, create.CreatorID)+ if err := memoAddInsert(b, create); err != nil {+ return err+ }+ } else {+ condition, args := p.memo.contentUnchangedCondition()+ b.guard(condition, args...)+ if p.policy != nil {+ memoGuardPolicyPlacement(b, p.memo, p.policy, mutation.MemoUpdate)+ }+ }++ p.addBindings(b)+ p.addRemovedAttachments(b)+ for _, attachmentID := range mutation.RequiredAttachmentIDs {+ b.guard("EXISTS (SELECT 1 FROM attachment WHERE id = ? AND memo_id = "+p.ref.expr+")", attachmentID, p.ref.arg)+ }+ if mutation.MemoUpdate != nil {+ if err := memoAddUpdate(b, mutation.MemoUpdate); err != nil {+ return err+ }+ }+ if mutation.ReplaceReferenceRelations {+ p.addReferenceRelations(b)+ }+ if mutation.MemoCreate != nil && mutation.CommentContextMemoID != nil {+ b.add("INSERT INTO memo_relation (memo_id, related_memo_id, type) VALUES ("+p.ref.expr+", ?, ?)",+ p.ref.arg, *mutation.CommentContextMemoID, store.MemoRelationComment)+ }+ return nil+}++// guardCommentContext re-asserts what authorizeMemoComment accepted: the+// context memo is unchanged and, when it sits in a Space, the actor is still+// an active member.+func (p *memoMutationPlan) guardCommentContext(b *batch, actorUserID int32) {+ if p.context == nil {+ return+ }+ condition, args := p.context.unchangedCondition()+ b.guard(condition, args...)+ if p.context.spaceID != nil {+ b.guard("EXISTS (SELECT 1 FROM space WHERE id = ?)", *p.context.spaceID)+ b.guard(activeSpaceMemberCondition, *p.context.spaceID, actorUserID)+ }+}++// addBindings guards each attachment's expected state and binds it.+func (p *memoMutationPlan) addBindings(b *batch) {+ mutation := p.mutation+ for _, binding := range mutation.Bindings {+ if binding.WasBoundToMemo {+ b.guard("EXISTS (SELECT 1 FROM attachment WHERE id = ? AND memo_id = "+p.ref.expr+")", binding.ID, p.ref.arg)+ } else {+ b.guard("EXISTS (SELECT 1 FROM attachment WHERE id = ? AND creator_id = ? AND memo_id IS NULL)", binding.ID, mutation.MemoCreatorID)+ }+ b.add("UPDATE attachment SET memo_id = "+p.ref.expr+", updated_ts = ? WHERE id = ?", p.ref.arg, binding.UpdatedTs, binding.ID)+ }+}++// addRemovedAttachments guards each removed attachment is still bound to the+// memo and deletes it.+func (p *memoMutationPlan) addRemovedAttachments(b *batch) {+ mutation := p.mutation+ for _, attachment := range p.removed {+ b.guard("EXISTS (SELECT 1 FROM attachment WHERE id = ? AND memo_id = ? AND creator_id = ?)", attachment.ID, mutation.MemoID, mutation.MemoCreatorID)+ b.add("DELETE FROM attachment WHERE id = ? AND memo_id = ?", attachment.ID, mutation.MemoID)+ }+}++// addReferenceRelations replaces the memo's REFERENCE relations.+func (p *memoMutationPlan) addReferenceRelations(b *batch) {+ b.add("DELETE FROM memo_relation WHERE memo_id = "+p.ref.expr+" AND type = ?", p.ref.arg, store.MemoRelationReference)+ for _, relation := range p.mutation.ReferenceRelations {+ b.add(`INSERT INTO memo_relation (memo_id, related_memo_id, type)+ VALUES (`+p.ref.expr+`, ?, ?)+ ON CONFLICT(memo_id, related_memo_id, type) DO UPDATE SET type = excluded.type`,+ p.ref.arg, relation.RelatedMemoID, relation.Type)+ }+}++// finish loads the generated columns of a created memo and propagates its id+// to the mutation, as the SQLite driver does before its writes.+func (p *memoMutationPlan) finish(ctx context.Context, q querier) error {+ create := p.mutation.MemoCreate+ if create == nil {+ return nil+ }+ if err := memoLoadCreated(ctx, q, create); err != nil {+ return err+ }+ p.mutation.MemoID = create.ID+ for _, relation := range p.mutation.ReferenceRelations {+ if relation != nil {+ relation.MemoID = create.ID+ }+ }+ return nil+}diff --git a/store/db/d1/memo_relation.go b/store/db/d1/memo_relation.gonew file mode 100644index 000000000..d40c9f76b--- /dev/null+++ b/store/db/d1/memo_relation.go@@ -0,0 +1,125 @@+package d1++import (+ "context"+ "fmt"+ "strings"++ "github.com/usememos/memos/filter"+ "github.com/usememos/memos/store"+)++// UpsertMemoRelation inserts or refreshes a relation between two memos.+func (d *DB) UpsertMemoRelation(ctx context.Context, create *store.MemoRelation) (*store.MemoRelation, error) {+ // One statement is already atomic on D1, so RETURNING can be read directly.+ stmt := `+ INSERT INTO memo_relation (memo_id, related_memo_id, type)+ VALUES (?, ?, ?)+ ON CONFLICT(memo_id, related_memo_id, type) DO UPDATE SET type = excluded.type+ RETURNING memo_id, related_memo_id, type+ `+ memoRelation := &store.MemoRelation{}+ if err := d.db.QueryRowContext(ctx, stmt, create.MemoID, create.RelatedMemoID, create.Type).Scan(+ &memoRelation.MemoID,+ &memoRelation.RelatedMemoID,+ &memoRelation.Type,+ ); err != nil {+ return nil, err+ }+ return memoRelation, nil+}++// ListMemoRelations returns the relations matching find.+func (d *DB) ListMemoRelations(ctx context.Context, find *store.FindMemoRelation) ([]*store.MemoRelation, error) {+ where, args, err := memoRelationConditions(ctx, find)+ if err != nil {+ return nil, err+ }+ query := `+ SELECT memo_id, related_memo_id, type+ FROM memo_relation+ WHERE ` + strings.Join(where, " AND ") + `+ ORDER BY memo_id DESC`+ query = appendLimit(query, find.Limit, find.Offset)++ rows, err := d.db.QueryContext(ctx, query, args...)+ if err != nil {+ return nil, err+ }+ defer rows.Close()++ list := []*store.MemoRelation{}+ for rows.Next() {+ memoRelation := &store.MemoRelation{}+ if err := rows.Scan(&memoRelation.MemoID, &memoRelation.RelatedMemoID, &memoRelation.Type); err != nil {+ return nil, err+ }+ list = append(list, memoRelation)+ }+ if err := rows.Err(); err != nil {+ return nil, err+ }+ return list, nil+}++// memoRelationConditions renders the WHERE clauses for find. Id lists are+// inlined as literals so they never count against the bind limit.+func memoRelationConditions(ctx context.Context, find *store.FindMemoRelation) ([]string, []any, error) {+ where, args := []string{"1 = 1"}, []any{}+ if find.MemoID != nil {+ where, args = append(where, "memo_id = ?"), append(args, *find.MemoID)+ }+ if find.RelatedMemoID != nil {+ where, args = append(where, "related_memo_id = ?"), append(args, *find.RelatedMemoID)+ }+ if find.Type != nil {+ where, args = append(where, "type = ?"), append(args, *find.Type)+ }+ if len(find.MemoIDList) > 0 {+ list := intList(find.MemoIDList)+ where = append(where, fmt.Sprintf("(memo_id IN %s OR related_memo_id IN %s)", list, list))+ }+ if len(find.SourceMemoIDList) > 0 {+ where = append(where, "memo_id IN "+intList(find.SourceMemoIDList))+ }+ if len(find.RelatedMemoIDList) > 0 {+ where = append(where, "related_memo_id IN "+intList(find.RelatedMemoIDList))+ }+ if find.SourceMemoRowStatus != nil {+ where, args = append(where, "memo_id IN (SELECT id FROM memo WHERE row_status = ?)"), append(args, *find.SourceMemoRowStatus)+ }+ if find.MemoFilter != nil {+ engine, err := filter.DefaultEngine()+ if err != nil {+ return nil, nil, err+ }+ stmt, err := engine.CompileToStatement(ctx, *find.MemoFilter, filter.RenderOptions{Dialect: filter.DialectD1})+ if err != nil {+ return nil, nil, err+ }+ if stmt.SQL != "" {+ where = append(where, fmt.Sprintf("memo_id IN (SELECT id FROM memo WHERE %s)", stmt.SQL))+ where = append(where, fmt.Sprintf("related_memo_id IN (SELECT id FROM memo WHERE %s)", stmt.SQL))+ args = append(args, stmt.Args...)+ args = append(args, stmt.Args...)+ }+ }+ return where, args, nil+}++// DeleteMemoRelation removes the relations matching delete.+func (d *DB) DeleteMemoRelation(ctx context.Context, delete *store.DeleteMemoRelation) error {+ if err := store.ValidateMemoRelationDelete(delete); err != nil {+ return err+ }+ where, args := []string{"1 = 1"}, []any{}+ if delete.MemoID != nil {+ where, args = append(where, "memo_id = ?"), append(args, *delete.MemoID)+ }+ if delete.RelatedMemoID != nil {+ where, args = append(where, "related_memo_id = ?"), append(args, *delete.RelatedMemoID)+ }+ where, args = append(where, "type = ?"), append(args, *delete.Type)+ _, err := d.execOne(ctx, "DELETE FROM memo_relation WHERE "+strings.Join(where, " AND "), args...)+ return err+}diff --git a/store/db/d1/memo_relation_policy.go b/store/db/d1/memo_relation_policy.gonew file mode 100644index 000000000..0f29b3275--- /dev/null+++ b/store/db/d1/memo_relation_policy.go@@ -0,0 +1,98 @@+package d1++import (+ "context"+ "database/sql"+ "slices"++ "github.com/pkg/errors"++ "github.com/usememos/memos/store"+)++// memoValidateRelationEndpoints authorizes every memo a mutation links to:+// the comment context or the mutated memo itself, plus each referenced memo.+// It returns the endpoint states so the batch can re-assert them.+func memoValidateRelationEndpoints(ctx context.Context, q querier, mutation *store.MemoMutation) ([]*memoState, error) {+ if !mutation.ReplaceReferenceRelations && mutation.CommentContextMemoID == nil {+ return nil, nil+ }++ var actorStatus store.RowStatus+ err := q.QueryRowContext(ctx, "SELECT row_status FROM user WHERE id = ?", mutation.MemoCreatorID).Scan(&actorStatus)+ if errors.Is(err, sql.ErrNoRows) {+ return nil, store.ErrMemoPermissionDenied+ }+ if err != nil {+ return nil, err+ }++ endpointIDs := memoRelationEndpointIDs(mutation)+ endpoints := make([]*memoState, 0, len(endpointIDs))+ for _, endpointID := range endpointIDs {+ state, err := loadMemoState(ctx, q, endpointID)+ if errors.Is(err, sql.ErrNoRows) {+ return nil, store.ErrMemoMutationConflict+ }+ if err != nil {+ return nil, err+ }+ snapshot := &store.MemoRelationEndpointSnapshot{+ ActorUserID: mutation.MemoCreatorID,+ ActorActive: actorStatus == store.Normal,+ EndpointID: state.id,+ EndpointCreatorID: state.creatorID,+ EndpointRowStatus: state.rowStatus,+ EndpointVisibility: state.visibility,+ EndpointSpaceID: state.spaceID,+ }+ if state.spaceID != nil {+ exists, member, err := spaceState(ctx, q, *state.spaceID, mutation.MemoCreatorID)+ if err != nil {+ return nil, err+ }+ snapshot.EndpointSpaceExists = exists+ snapshot.EndpointMemberActive = member+ }+ if err := store.ValidateMemoRelationEndpointRead(snapshot); err != nil {+ return nil, err+ }+ endpoints = append(endpoints, state)+ }+ return endpoints, nil+}++// memoGuardRelationEndpoints re-asserts inside the batch that each endpoint+// still has the state that passed memoValidateRelationEndpoints. Membership is+// only part of the read policy for the SPACE audience.+func memoGuardRelationEndpoints(b *batch, actorUserID int32, endpoints []*memoState) {+ for _, endpoint := range endpoints {+ condition, args := endpoint.unchangedCondition()+ b.guard(condition, args...)+ if endpoint.visibility == store.SpaceAudience && endpoint.spaceID != nil {+ b.guard(activeSpaceMemberCondition, *endpoint.spaceID, actorUserID)+ }+ }+}++// memoRelationEndpointIDs lists the distinct memos whose read authorization+// the mutation depends on, in ascending order.+func memoRelationEndpointIDs(mutation *store.MemoMutation) []int32 {+ unique := make(map[int32]struct{}, len(mutation.ReferenceRelations)+1)+ if mutation.MemoCreate == nil && mutation.MemoID > 0 {+ unique[mutation.MemoID] = struct{}{}+ } else if mutation.CommentContextMemoID != nil {+ unique[*mutation.CommentContextMemoID] = struct{}{}+ }+ for _, relation := range mutation.ReferenceRelations {+ if relation != nil {+ unique[relation.RelatedMemoID] = struct{}{}+ }+ }+ ids := make([]int32, 0, len(unique))+ for id := range unique {+ ids = append(ids, id)+ }+ slices.Sort(ids)+ return ids+}diff --git a/store/db/d1/memo_share.go b/store/db/d1/memo_share.gonew file mode 100644index 000000000..16d2ed18a--- /dev/null+++ b/store/db/d1/memo_share.go@@ -0,0 +1,149 @@+package d1++import (+ "context"+ "database/sql"+ "strings"++ "github.com/pkg/errors"++ "github.com/usememos/memos/store"+)++const shareColumns = "id, uid, memo_id, creator_id, created_ts, expires_ts"++// CreateMemoShare inserts a share grant, checking the memo write policy when one is set.+func (d *DB) CreateMemoShare(ctx context.Context, create *store.MemoShare) (*store.MemoShare, error) {+ columns, values, args := []string{"uid", "memo_id", "creator_id"}, []string{"?", "?", "?"}, []any{create.UID, create.MemoID, create.CreatorID}+ if create.ExpiresTs != nil {+ columns, values, args = append(columns, "expires_ts"), append(values, "?"), append(args, *create.ExpiresTs)+ }+ if create.Policy == nil {+ stmt := "INSERT INTO memo_share (" + strings.Join(columns, ", ") + ") VALUES (" + strings.Join(values, ", ") + ") RETURNING id, created_ts"+ if err := d.db.QueryRowContext(ctx, stmt, args...).Scan(&create.ID, &create.CreatedTs); err != nil {+ return nil, err+ }+ return create, nil+ }+ if err := validateMemoWritePolicy(ctx, d.db, create.MemoID, create.Policy, nil); err != nil {+ return nil, err+ }+ // The validated memo and actor are re-asserted by the insert itself:+ // selecting from the memo row and requiring the actor to still be active+ // makes a stale validation yield no row instead of a share.+ args = append(args, create.MemoID, create.Policy.ActorUserID)+ stmt := "INSERT INTO memo_share (" + strings.Join(columns, ", ") + ") SELECT " + strings.Join(values, ", ") ++ " FROM memo WHERE memo.id = ? AND " + activeUserCondition + " RETURNING id, created_ts"+ if err := d.db.QueryRowContext(ctx, stmt, args...).Scan(&create.ID, &create.CreatedTs); err != nil {+ if errors.Is(err, sql.ErrNoRows) {+ return nil, store.ErrMemoMutationConflict+ }+ return nil, err+ }+ return create, nil+}++// shareFindWhere renders the filter clauses of a share lookup.+func shareFindWhere(find *store.FindMemoShare) ([]string, []any) {+ where, args := []string{"1 = 1"}, []any{}+ if find.ID != nil {+ where, args = append(where, "id = ?"), append(args, *find.ID)+ }+ if find.UID != nil {+ where, args = append(where, "uid = ?"), append(args, *find.UID)+ }+ if find.MemoID != nil {+ where, args = append(where, "memo_id = ?"), append(args, *find.MemoID)+ }+ if find.CreatorID != nil {+ where, args = append(where, "creator_id = ?"), append(args, *find.CreatorID)+ }+ return where, args+}++// shareScanner is satisfied by *sql.Row and *sql.Rows.+type shareScanner interface {+ Scan(dest ...any) error+}++// shareScan reads one row selected with shareColumns.+func shareScan(scanner shareScanner) (*store.MemoShare, error) {+ share := &store.MemoShare{}+ if err := scanner.Scan(+ &share.ID,+ &share.UID,+ &share.MemoID,+ &share.CreatorID,+ &share.CreatedTs,+ &share.ExpiresTs,+ ); err != nil {+ return nil, err+ }+ return share, nil+}++// ListMemoShares returns the share grants matching find.+func (d *DB) ListMemoShares(ctx context.Context, find *store.FindMemoShare) ([]*store.MemoShare, error) {+ where, args := shareFindWhere(find)+ rows, err := d.db.QueryContext(ctx, "SELECT "+shareColumns+" FROM memo_share WHERE "+strings.Join(where, " AND ")+" ORDER BY id ASC", args...)+ if err != nil {+ return nil, err+ }+ defer rows.Close()++ list := []*store.MemoShare{}+ for rows.Next() {+ share, err := shareScan(rows)+ if err != nil {+ return nil, err+ }+ list = append(list, share)+ }+ if err := rows.Err(); err != nil {+ return nil, err+ }+ return list, nil+}++// GetMemoShare returns the first share grant matching find, or nil.+func (d *DB) GetMemoShare(ctx context.Context, find *store.FindMemoShare) (*store.MemoShare, error) {+ where, args := shareFindWhere(find)+ share, err := shareScan(d.db.QueryRowContext(ctx, "SELECT "+shareColumns+" FROM memo_share WHERE "+strings.Join(where, " AND ")+" LIMIT 1", args...))+ if err != nil {+ if errors.Is(err, sql.ErrNoRows) {+ return nil, nil+ }+ return nil, err+ }+ return share, nil+}++// DeleteMemoShare removes share grants, requiring exactly one revoked row when a policy is set.+func (d *DB) DeleteMemoShare(ctx context.Context, delete *store.DeleteMemoShare) error {+ where, args := []string{"1 = 1"}, []any{}+ if delete.ID != nil {+ where, args = append(where, "id = ?"), append(args, *delete.ID)+ }+ if delete.UID != nil {+ where, args = append(where, "uid = ?"), append(args, *delete.UID)+ }+ if delete.Policy == nil {+ _, err := d.execOne(ctx, "DELETE FROM memo_share WHERE "+strings.Join(where, " AND "), args...)+ return err+ }+ if err := validateMemoWritePolicy(ctx, d.db, *delete.MemoID, delete.Policy, nil); err != nil {+ return err+ }+ // One conditional statement: the share must still belong to the validated+ // memo, and exactly one row must go, or the revocation is reported as a+ // conflict.+ where, args = append(where, "memo_id = ?"), append(args, *delete.MemoID)+ result, err := d.execOne(ctx, "DELETE FROM memo_share WHERE "+strings.Join(where, " AND "), args...)+ if err != nil {+ return err+ }+ if result.Changes != 1 {+ return store.ErrMemoMutationConflict+ }+ return nil+}diff --git a/store/db/d1/reaction.go b/store/db/d1/reaction.gonew file mode 100644index 000000000..9272979be--- /dev/null+++ b/store/db/d1/reaction.go@@ -0,0 +1,216 @@+package d1++import (+ "context"+ "database/sql"+ "slices"+ "strings"++ "github.com/pkg/errors"++ "github.com/usememos/memos/store"+)++// reactionInsertStatement selects from the memo row so the insert is+// conditional on the memo still existing: a vanished memo yields no row+// instead of a reaction that outlives its memo.+const reactionInsertStatement = `+ INSERT INTO reaction (creator_id, memo_id, reaction_type)+ SELECT ?, memo.id, ?+ FROM memo+ WHERE memo.id = ?+ RETURNING id, created_ts`++// UpsertReaction records a reaction. With a policy, the participation state+// that authorized the write is re-asserted by guards in the same batch as+// the insert, so a memo or membership change between validation and write+// aborts it.+func (d *DB) UpsertReaction(ctx context.Context, upsert *store.Reaction) (*store.Reaction, error) {+ if upsert.Policy == nil {+ if err := d.db.QueryRowContext(ctx, reactionInsertStatement, upsert.CreatorID, upsert.ReactionType, upsert.MemoID).Scan(+ &upsert.ID,+ &upsert.CreatedTs,+ ); err != nil {+ if errors.Is(err, sql.ErrNoRows) {+ return nil, errors.Wrap(store.ErrReactionMemoNotFound, "failed to create reaction")+ }+ return nil, err+ }+ return upsert, nil+ }++ participation, err := reactionLoadParticipation(ctx, d.db, upsert)+ if err != nil {+ return nil, err+ }+ if err := store.ValidateReactionWriteParticipation(upsert, participation); err != nil {+ return nil, err+ }+ memo, err := loadMemoState(ctx, d.db, upsert.MemoID)+ if err != nil {+ if errors.Is(err, sql.ErrNoRows) {+ return nil, store.ErrReactionMemoNotFound+ }+ return nil, errors.Wrap(err, "failed to read reaction memo")+ }++ b := newBatch()+ b.guard(activeUserCondition, upsert.Policy.ActorUserID)+ condition, args := memo.unchangedCondition()+ b.guard(condition, args...)+ if memo.spaceID != nil {+ membership := activeSpaceMemberCondition+ if !participation.ContextMemberActive {+ membership = "NOT " + membership+ }+ b.guard(membership, *memo.spaceID, upsert.Policy.ActorUserID)+ }+ index := b.add(reactionInsertStatement, upsert.CreatorID, upsert.ReactionType, upsert.MemoID)+ results, err := b.commit(ctx, d)+ if err != nil {+ if isGuardFailure(err) {+ // Report the precondition that no longer holds, falling back to a+ // permission error when the state moved past what was validated.+ if err := reactionValidateWritePolicy(ctx, d.db, upsert); err != nil {+ return nil, err+ }+ return nil, errors.Wrap(store.ErrReactionPermissionDenied, "memo state changed while recording reaction")+ }+ return nil, err+ }+ rows := results[index].Rows+ if len(rows) == 0 || len(rows[0]) < 2 {+ return nil, errors.Wrap(store.ErrReactionMemoNotFound, "failed to create reaction")+ }+ id, idOK := rows[0][0].(int64)+ createdTs, tsOK := rows[0][1].(int64)+ if !idOK || !tsOK {+ return nil, errors.New("d1: unexpected reaction row shape")+ }+ upsert.ID = int32(id)+ upsert.CreatedTs = createdTs+ return upsert, nil+}++// ListReactions returns the reactions matching find.+func (d *DB) ListReactions(ctx context.Context, find *store.FindReaction) ([]*store.Reaction, error) {+ where, args := []string{"1 = 1"}, []any{}++ if find.ID != nil {+ where, args = append(where, "id = ?"), append(args, *find.ID)+ }+ if find.CreatorID != nil {+ where, args = append(where, "creator_id = ?"), append(args, *find.CreatorID)+ }+ if find.MemoID != nil {+ where, args = append(where, "memo_id = ?"), append(args, *find.MemoID)+ }+ if len(find.MemoIDList) == 0 {+ return reactionQuery(ctx, d.db, where, args)+ }++ // A memo id list can exceed D1's bind limit, so it is queried in chunks+ // and the union re-sorted to preserve the id order of a single query.+ list := []*store.Reaction{}+ for _, ids := range chunk(find.MemoIDList, inClauseBatchSize) {+ clause, memoArgs := inClause(ids)+ chunkWhere := append(append([]string{}, where...), "memo_id IN "+clause)+ chunkArgs := append(append([]any{}, args...), memoArgs...)+ reactions, err := reactionQuery(ctx, d.db, chunkWhere, chunkArgs)+ if err != nil {+ return nil, err+ }+ list = append(list, reactions...)+ }+ slices.SortFunc(list, func(a, b *store.Reaction) int { return int(a.ID) - int(b.ID) })+ return list, nil+}++// reactionQuery lists the reactions matching where, ordered by id.+func reactionQuery(ctx context.Context, q querier, where []string, args []any) ([]*store.Reaction, error) {+ rows, err := q.QueryContext(ctx, "SELECT id, created_ts, creator_id, memo_id, reaction_type FROM reaction WHERE "+strings.Join(where, " AND ")+" ORDER BY id ASC", args...)+ if err != nil {+ return nil, err+ }+ defer rows.Close()++ list := []*store.Reaction{}+ for rows.Next() {+ reaction := &store.Reaction{}+ if err := rows.Scan(+ &reaction.ID,+ &reaction.CreatedTs,+ &reaction.CreatorID,+ &reaction.MemoID,+ &reaction.ReactionType,+ ); err != nil {+ return nil, err+ }+ list = append(list, reaction)+ }+ if err := rows.Err(); err != nil {+ return nil, err+ }+ return list, nil+}++// GetReaction returns the first reaction matching find, or nil.+func (d *DB) GetReaction(ctx context.Context, find *store.FindReaction) (*store.Reaction, error) {+ list, err := d.ListReactions(ctx, find)+ if err != nil {+ return nil, err+ }+ if len(list) == 0 {+ return nil, nil+ }+ return list[0], nil+}++// DeleteReaction removes reactions, checking ownership when an actor is set.+func (d *DB) DeleteReaction(ctx context.Context, delete *store.DeleteReaction) error {+ if delete.ActorUserID != nil {+ return d.deleteReactionAsCreator(ctx, delete)+ }+ where, args := []string{}, []any{}+ if delete.ID != nil {+ where, args = append(where, "id = ?"), append(args, *delete.ID)+ }+ if delete.MemoID != nil {+ where, args = append(where, "memo_id = ?"), append(args, *delete.MemoID)+ }+ if len(where) == 0 {+ return nil+ }+ _, err := d.execOne(ctx, "DELETE FROM reaction WHERE "+strings.Join(where, " AND "), args...)+ return err+}++// deleteReactionAsCreator removes the reaction only when the actor owns it.+// Ownership is read first for the error mapping and then re-asserted in the+// DELETE itself, which stands in for the transaction a socket driver would use.+func (d *DB) deleteReactionAsCreator(ctx context.Context, delete *store.DeleteReaction) error {+ if delete.Policy != nil {+ if err := reactionValidateDeletePolicy(ctx, d.db, &store.Reaction{+ CreatorID: *delete.ActorUserID,+ MemoID: *delete.MemoID,+ Policy: delete.Policy,+ }); err != nil {+ return err+ }+ }++ var creatorID, memoID int32+ if err := d.db.QueryRowContext(ctx, "SELECT creator_id, memo_id FROM reaction WHERE id = ?", *delete.ID).Scan(&creatorID, &memoID); err != nil {+ if errors.Is(err, sql.ErrNoRows) {+ return nil+ }+ return errors.Wrap(err, "failed to read reaction for deletion")+ }+ if creatorID != *delete.ActorUserID || (delete.MemoID != nil && memoID != *delete.MemoID) {+ return store.ErrReactionPermissionDenied+ }+ if _, err := d.execOne(ctx, "DELETE FROM reaction WHERE id = ? AND creator_id = ?", *delete.ID, *delete.ActorUserID); err != nil {+ return errors.Wrap(err, "failed to delete reaction")+ }+ return nil+}diff --git a/store/db/d1/reaction_policy.go b/store/db/d1/reaction_policy.gonew file mode 100644index 000000000..674ed139e--- /dev/null+++ b/store/db/d1/reaction_policy.go@@ -0,0 +1,43 @@+package d1++import (+ "context"+ "database/sql"++ "github.com/pkg/errors"++ "github.com/usememos/memos/store"+)++// reactionValidateWritePolicy checks that the policy actor may react to the+// reaction's memo, reading the participation state through q.+func reactionValidateWritePolicy(ctx context.Context, q querier, reaction *store.Reaction) error {+ participation, err := reactionLoadParticipation(ctx, q, reaction)+ if err != nil {+ return err+ }+ return store.ValidateReactionWriteParticipation(reaction, participation)+}++// reactionValidateDeletePolicy checks that the policy actor may withdraw the+// reaction from its memo.+func reactionValidateDeletePolicy(ctx context.Context, q querier, reaction *store.Reaction) error {+ participation, err := reactionLoadParticipation(ctx, q, reaction)+ if err != nil {+ return err+ }+ return store.ValidateReactionWithdrawal(reaction, participation)+}++// reactionLoadParticipation resolves the actor's participation in the+// reaction's memo, mapping a missing memo to ErrReactionMemoNotFound.+func reactionLoadParticipation(ctx context.Context, q querier, reaction *store.Reaction) (*store.MemoCommentAuthorizationSnapshot, error) {+ participation, err := loadMemoParticipation(ctx, q, reaction.MemoID, reaction.Policy.ActorUserID)+ if err != nil {+ if errors.Is(err, sql.ErrNoRows) {+ return nil, store.ErrReactionMemoNotFound+ }+ return nil, errors.Wrap(err, "failed to read reaction participation")+ }+ return participation, nil+}diff --git a/store/db/d1/shared.go b/store/db/d1/shared.gonew file mode 100644index 000000000..056c933cc--- /dev/null+++ b/store/db/d1/shared.go@@ -0,0 +1,371 @@+package d1++import (+ "context"+ "database/sql"+ "fmt"+ "strings"++ "github.com/pkg/errors"++ storepb "github.com/usememos/memos/proto/gen/store"+ "github.com/usememos/memos/store"+)++// This file holds the validation reads and batch fragments shared by several+// driver methods. Reads run before a batch; guards re-check the invariants a+// read established so a concurrent change aborts the batch instead of+// producing a partial write.++// activeUserCondition is a guard condition asserting that a user exists and is+// not archived. It binds one argument: the user id.+const activeUserCondition = "EXISTS (SELECT 1 FROM user WHERE id = ? AND row_status = 'NORMAL')"++// activeSpaceMemberCondition asserts an active membership. It binds two+// arguments: the space id and the user id.+const activeSpaceMemberCondition = "EXISTS (SELECT 1 FROM space_member WHERE space_id = ? AND user_id = ? AND status = 'ACTIVE' AND role IN ('ADMIN', 'USER'))"++// requireActiveUser fails with notActive when the user is missing or archived.+func requireActiveUser(ctx context.Context, q querier, userID int32, notActive error) error {+ var rowStatus store.RowStatus+ err := q.QueryRowContext(ctx, "SELECT row_status FROM user WHERE id = ?", userID).Scan(&rowStatus)+ if errors.Is(err, sql.ErrNoRows) {+ return notActive+ }+ if err != nil {+ return err+ }+ if rowStatus != store.Normal {+ return notActive+ }+ return nil+}++// spaceExists reports whether the space row is present.+func spaceExists(ctx context.Context, q querier, spaceID int32) (bool, error) {+ var exists bool+ if err := q.QueryRowContext(ctx, "SELECT EXISTS(SELECT 1 FROM space WHERE id = ?)", spaceID).Scan(&exists); err != nil {+ return false, err+ }+ return exists, nil+}++// spaceMemberActive reports whether userID is an active member of spaceID.+func spaceMemberActive(ctx context.Context, q querier, spaceID, userID int32) (bool, error) {+ var role store.SpaceMemberRole+ err := q.QueryRowContext(ctx, `SELECT role FROM space_member+ WHERE space_id = ? AND user_id = ? AND status = 'ACTIVE' AND role IN ('ADMIN', 'USER')`, spaceID, userID).Scan(&role)+ if errors.Is(err, sql.ErrNoRows) {+ return false, nil+ }+ if err != nil {+ return false, err+ }+ return role.IsActiveMember(), nil+}++// spaceState resolves existence and the actor's active membership in one call.+func spaceState(ctx context.Context, q querier, spaceID, userID int32) (exists bool, member bool, err error) {+ exists, err = spaceExists(ctx, q, spaceID)+ if err != nil || !exists {+ return exists, false, err+ }+ member, err = spaceMemberActive(ctx, q, spaceID, userID)+ return true, member, err+}++// validateMemoWritePolicy checks that the actor may write memoID under policy,+// optionally for the pending update.+func validateMemoWritePolicy(ctx context.Context, q querier, memoID int32, policy *store.MemoWritePolicy, update *store.UpdateMemo) error {+ if err := requireActiveUser(ctx, q, policy.ActorUserID, store.ErrMemoSpaceMembershipRequired); err != nil {+ return err+ }++ snapshot := new(store.MemoWriteSnapshot)+ var spaceID sql.NullInt64+ if err := q.QueryRowContext(ctx, `SELECT creator_id, row_status, space_id, visibility FROM memo WHERE id = ?`, memoID).Scan(+ &snapshot.CreatorID, &snapshot.RowStatus, &spaceID, &snapshot.Visibility,+ ); err != nil {+ if errors.Is(err, sql.ErrNoRows) {+ return store.ErrMemoMutationConflict+ }+ return err+ }+ snapshot.SpaceID = store.NullInt32Pointer(spaceID)+ if snapshot.SpaceID != nil {+ exists, member, err := spaceState(ctx, q, *snapshot.SpaceID, policy.ActorUserID)+ if err != nil {+ return err+ }+ snapshot.SourceSpaceExists = exists+ snapshot.SourceMemberActive = member+ }+ if update != nil && update.SpaceID != nil {+ exists, member, err := spaceState(ctx, q, *update.SpaceID, policy.ActorUserID)+ if err != nil {+ return err+ }+ snapshot.TargetSpaceExists = exists+ snapshot.TargetMemberActive = member+ }+ if update != nil && update.Visibility != nil && *update.Visibility == store.SpaceAudience {+ var shareID int32+ err := q.QueryRowContext(ctx, `SELECT id FROM memo_share+ WHERE memo_id = ? AND (expires_ts IS NULL OR expires_ts > CAST(strftime('%s', 'now') AS INTEGER))+ LIMIT 1`, memoID).Scan(&shareID)+ snapshot.HasActiveShare = err == nil+ if err != nil && !errors.Is(err, sql.ErrNoRows) {+ return err+ }+ }+ return store.ValidateMemoWriteSnapshot(policy, update, snapshot)+}++// loadMemoParticipation resolves the actor, Space, membership, and memo state+// shared by comment and reaction participation checks. A missing memo yields+// sql.ErrNoRows.+func loadMemoParticipation(ctx context.Context, q querier, memoID, actorUserID int32) (*store.MemoCommentAuthorizationSnapshot, error) {+ snapshot := &store.MemoCommentAuthorizationSnapshot{ActorUserID: actorUserID, ContextID: memoID}+ var actorStatus store.RowStatus+ if err := q.QueryRowContext(ctx, "SELECT row_status FROM user WHERE id = ?", actorUserID).Scan(&actorStatus); err != nil {+ if errors.Is(err, sql.ErrNoRows) {+ return nil, store.ErrMemoPermissionDenied+ }+ return nil, err+ }+ snapshot.ActorActive = actorStatus == store.Normal++ var contextSpace sql.NullInt64+ if err := q.QueryRowContext(ctx, `SELECT creator_id, row_status, visibility, space_id FROM memo WHERE id = ?`, memoID).Scan(+ &snapshot.ContextCreatorID, &snapshot.ContextRowStatus, &snapshot.ContextVisibility, &contextSpace,+ ); err != nil {+ return nil, err+ }+ snapshot.ContextSpaceID = store.NullInt32Pointer(contextSpace)+ if snapshot.ContextSpaceID != nil {+ exists, member, err := spaceState(ctx, q, *snapshot.ContextSpaceID, actorUserID)+ if err != nil {+ return nil, err+ }+ snapshot.ContextSpaceExists = exists+ snapshot.ContextMemberActive = member+ }+ return snapshot, nil+}++// authorizeMemoComment checks that the actor may comment on contextMemoID.+func authorizeMemoComment(ctx context.Context, q querier, contextMemoID, actorUserID int32) error {+ snapshot, err := loadMemoParticipation(ctx, q, contextMemoID, actorUserID)+ if err != nil {+ if errors.Is(err, sql.ErrNoRows) {+ return store.ErrMemoSpaceNotWritable+ }+ return err+ }+ return store.ValidateMemoCommentAuthorization(snapshot)+}++// memoAccessPredicate builds the memo read predicate for memoAlias and appends+// its bind values to args.+func memoAccessPredicate(access *store.MemoAccessScope, memoAlias, memberAlias string, args *[]any) string {+ clauses := []string{}+ if access.AllowPublic {+ clauses = append(clauses, memoAlias+".visibility = 'PUBLIC'")+ }+ if access.UserID != nil {+ *args = append(*args, *access.UserID)+ authenticated := []string{}++ *args = append(*args, *access.UserID)+ authenticated = append(authenticated, "("+memoAlias+".visibility = 'PRIVATE' AND "+memoAlias+".creator_id = ?)")+ if access.AllowProtected {+ authenticated = append(authenticated, memoAlias+".visibility = 'PROTECTED'")+ }+ *args = append(*args, *access.UserID)+ authenticated = append(authenticated, "("+memoAlias+".visibility = 'SPACE' AND EXISTS (SELECT 1 FROM space_member AS "+memberAlias+" WHERE "+memberAlias+".space_id = "+memoAlias+".space_id AND "+memberAlias+".user_id = ? AND "+memberAlias+".status = 'ACTIVE' AND "+memberAlias+".role IN ('ADMIN', 'USER')))")++ clauses = append(clauses, "(EXISTS (SELECT 1 FROM user AS access_user WHERE access_user.id = ? AND access_user.row_status = 'NORMAL') AND ("+strings.Join(authenticated, " OR ")+"))")+ }+ if len(clauses) == 0 {+ return "1 = 0"+ }++ validMemo := "(" + memoAlias + ".visibility IN ('PUBLIC', 'PROTECTED', 'PRIVATE', 'SPACE')" ++ " AND EXISTS (SELECT 1 FROM user AS valid_creator WHERE valid_creator.id = " + memoAlias + ".creator_id AND valid_creator.row_status IN ('NORMAL', 'ARCHIVED'))" ++ " AND (" + memoAlias + ".visibility <> 'SPACE' OR (" + memoAlias + ".space_id IS NOT NULL" ++ " AND EXISTS (SELECT 1 FROM space AS valid_space WHERE valid_space.id = " + memoAlias + ".space_id))))"+ validState := memoAlias + ".row_status = 'NORMAL'"+ if access.UserID != nil {+ *args = append(*args, *access.UserID)+ validState = fmt.Sprintf("(%s.row_status = 'NORMAL' OR (%s.row_status = 'ARCHIVED' AND %s.creator_id = ?))", memoAlias, memoAlias, memoAlias)+ }+ return "(" + strings.Join(clauses, " OR ") + ") AND " + validMemo + " AND " + validState+}++// scanAttachmentSnapshots reads attachment rows selected by query into+// attachments, skipping ids already in seen.+func scanAttachmentSnapshots(ctx context.Context, q querier, query string, args []any, seen map[int32]struct{}, attachments *[]*store.Attachment) error {+ rows, err := q.QueryContext(ctx, query, args...)+ if err != nil {+ return err+ }+ defer rows.Close()+ for rows.Next() {+ attachment := &store.Attachment{}+ var memoID sql.NullInt32+ var storageType string+ var payloadBytes []byte+ if err := rows.Scan(&attachment.ID, &attachment.UID, &attachment.CreatorID, &memoID, &storageType, &attachment.Reference, &payloadBytes); err != nil {+ return err+ }+ if _, exists := seen[attachment.ID]; exists {+ continue+ }+ seen[attachment.ID] = struct{}{}+ if memoID.Valid {+ attachment.MemoID = &memoID.Int32+ }+ attachment.StorageType = storepb.AttachmentStorageType(storepb.AttachmentStorageType_value[storageType])+ payload := &storepb.AttachmentPayload{}+ if len(payloadBytes) > 0 {+ if err := protojsonUnmarshaler.Unmarshal(payloadBytes, payload); err != nil {+ return err+ }+ }+ attachment.Payload = payload+ *attachments = append(*attachments, attachment)+ }+ return rows.Err()+}++const attachmentSnapshotColumns = "id, uid, creator_id, memo_id, storage_type, reference, payload"++// listAttachmentSnapshots loads the attachments with the given ids, ordered by id.+func listAttachmentSnapshots(ctx context.Context, q querier, attachmentIDs []int32) ([]*store.Attachment, error) {+ attachments := make([]*store.Attachment, 0, len(attachmentIDs))+ seen := make(map[int32]struct{}, len(attachmentIDs))+ for _, ids := range chunk(attachmentIDs, inClauseBatchSize) {+ clause, args := inClause(ids)+ if err := scanAttachmentSnapshots(ctx, q, "SELECT "+attachmentSnapshotColumns+" FROM attachment WHERE id IN "+clause+" ORDER BY id", args, seen, &attachments); err != nil {+ return nil, err+ }+ }+ return attachments, nil+}++// listMemoSetAttachments loads every attachment bound to one of memoIDs.+func listMemoSetAttachments(ctx context.Context, q querier, memoIDs []int32) ([]*store.Attachment, error) {+ attachments := make([]*store.Attachment, 0)+ seen := make(map[int32]struct{})+ for _, ids := range chunk(memoIDs, inClauseBatchSize) {+ clause, args := inClause(ids)+ if err := scanAttachmentSnapshots(ctx, q, "SELECT "+attachmentSnapshotColumns+" FROM attachment WHERE memo_id IN "+clause+" ORDER BY id", args, seen, &attachments); err != nil {+ return nil, err+ }+ }+ return attachments, nil+}++// authorizeAttachmentMutation checks the actor may mutate attachments bound to+// memoIDs, optionally requiring the memos' current contents to match.+func authorizeAttachmentMutation(ctx context.Context, q querier, actorUserID int32, memoIDs []int32, expectedMemoContents map[int32]string) error {+ if err := requireActiveUser(ctx, q, actorUserID, store.ErrMemoPermissionDenied); err != nil {+ return err+ }+ for _, memoID := range memoIDs {+ snapshot := &store.MemoWriteSnapshot{}+ var currentSpace sql.NullInt64+ var content string+ if err := q.QueryRowContext(ctx, `SELECT creator_id, row_status, space_id, visibility, content FROM memo WHERE id = ?`, memoID).Scan(+ &snapshot.CreatorID, &snapshot.RowStatus, ¤tSpace, &snapshot.Visibility, &content,+ ); err != nil {+ if errors.Is(err, sql.ErrNoRows) {+ return store.ErrMemoMutationConflict+ }+ return errors.Wrap(err, "failed to read attachment memo")+ }+ snapshot.SpaceID = store.NullInt32Pointer(currentSpace)+ if expectedMemoContents != nil && content != expectedMemoContents[memoID] {+ return store.ErrMemoMutationConflict+ }+ if snapshot.SpaceID != nil {+ exists, member, err := spaceState(ctx, q, *snapshot.SpaceID, actorUserID)+ if err != nil {+ return errors.Wrap(err, "failed to read attachment memo space")+ }+ snapshot.SourceSpaceExists = exists+ snapshot.SourceMemberActive = member+ }+ if err := store.ValidateMemoWriteSnapshot(&store.MemoWritePolicy{ActorUserID: actorUserID}, nil, snapshot); err != nil {+ return err+ }+ }+ return nil+}++// attachmentIDs extracts the ids of attachments.+func attachmentIDs(attachments []*store.Attachment) []int32 {+ ids := make([]int32, 0, len(attachments))+ for _, attachment := range attachments {+ if attachment != nil {+ ids = append(ids, attachment.ID)+ }+ }+ return ids+}++// addMemoSetDeletes appends the statements that remove memoIDs together with+// their shares, reactions, bound attachments, and relations. Attachments are+// listed by the caller beforehand so it can report them for storage cleanup.+func addMemoSetDeletes(b *batch, memoIDs []int32, boundAttachmentIDs []int32) {+ for _, ids := range chunk(memoIDs, inClauseBatchSize) {+ clause, args := inClause(ids)+ b.add("DELETE FROM memo_share WHERE memo_id IN "+clause, args...)+ }+ for _, ids := range chunk(memoIDs, inClauseBatchSize) {+ clause, args := inClause(ids)+ b.add("DELETE FROM memo WHERE id IN "+clause, args...)+ }+ for _, ids := range chunk(memoIDs, inClauseBatchSize) {+ clause, args := inClause(ids)+ b.add("DELETE FROM reaction WHERE memo_id IN "+clause, args...)+ }+ for _, ids := range chunk(boundAttachmentIDs, inClauseBatchSize) {+ clause, args := inClause(ids)+ b.add("DELETE FROM attachment WHERE id IN "+clause, args...)+ }+ // Two IN lists share one statement, so halve the chunk to stay within the+ // bind limit.+ for _, ids := range chunk(memoIDs, inClauseBatchSize/2) {+ memoClause, args := inClause(ids)+ relatedClause, relatedArgs := inClause(ids)+ b.add("DELETE FROM memo_relation WHERE memo_id IN "+memoClause+" OR related_memo_id IN "+relatedClause, append(args, relatedArgs...)...)+ }+}++// listMemoIDs returns the ids selected by query, in query order.+func listMemoIDs(ctx context.Context, q querier, query string, args ...any) ([]int32, error) {+ rows, err := q.QueryContext(ctx, query, args...)+ if err != nil {+ return nil, err+ }+ defer rows.Close()+ ids := make([]int32, 0)+ for rows.Next() {+ var id int32+ if err := rows.Scan(&id); err != nil {+ return nil, err+ }+ ids = append(ids, id)+ }+ return ids, rows.Err()+}++// nullableEmail maps the store's "no address" value to SQL NULL so the unique+// index ignores users without an address.+func nullableEmail(email string) any {+ if email == "" {+ return nil+ }+ return email+}diff --git a/store/db/d1/space.go b/store/db/d1/space.gonew file mode 100644index 000000000..4110f4e4c--- /dev/null+++ b/store/db/d1/space.go@@ -0,0 +1,545 @@+package d1++import (+ "context"+ "database/sql"+ "strings"++ "github.com/pkg/errors"+ "google.golang.org/protobuf/encoding/protojson"++ storepb "github.com/usememos/memos/proto/gen/store"+ "github.com/usememos/memos/store"+)++// Space methods validate with plain reads and then write with either an+// atomic batch guarded by the same predicates or a single statement whose+// WHERE clause encodes them. When the write touches nothing, the validation+// is re-run so the caller receives the precise store error instead of a+// generic not-found.++// spaceAdminCondition asserts that a user is an active administrator whose+// account is not archived. It binds two arguments: the space id and the user+// id.+const spaceAdminCondition = "EXISTS (SELECT 1 FROM space_member admin_member JOIN user admin_user ON admin_user.id = admin_member.user_id" ++ " WHERE admin_member.space_id = ? AND admin_member.user_id = ? AND admin_member.status = 'ACTIVE' AND admin_member.role = 'ADMIN' AND admin_user.row_status = 'NORMAL')"++// spaceActiveAdminCount counts the active administrators whose accounts are+// not archived. It binds one argument: the space id.+const spaceActiveAdminCount = "(SELECT COUNT(*) FROM space_member counted_member JOIN user counted_user ON counted_user.id = counted_member.user_id" ++ " WHERE counted_member.space_id = ? AND counted_member.status = 'ACTIVE' AND counted_member.role = 'ADMIN' AND counted_user.row_status = 'NORMAL')"++const spaceColumns = "id, uid, title, description, payload"++// spaceRecheck runs validate again after a conditional write matched no row+// or a guard aborted a batch. The precondition that failed is reported when+// validation now fails; otherwise fallback explains the empty write. The+// user methods share it.+func spaceRecheck(validate func() error, fallback error) error {+ if err := validate(); err != nil {+ return err+ }+ return fallback+}++// spaceRequireAdmin fails with ErrSpacePermissionDenied unless the actor is an+// active, non-archived administrator of the space.+func spaceRequireAdmin(ctx context.Context, q querier, spaceID, actorUserID int32) error {+ var role store.SpaceMemberRole+ err := q.QueryRowContext(ctx, `SELECT sm.role FROM space_member sm JOIN user u ON u.id = sm.user_id+ WHERE sm.space_id = ? AND sm.user_id = ? AND sm.status = ? AND u.row_status = 'NORMAL'`, spaceID, actorUserID, store.SpaceMemberStatusActive).Scan(&role)+ if errors.Is(err, sql.ErrNoRows) {+ return store.ErrSpacePermissionDenied+ }+ if err != nil {+ return err+ }+ if role != store.SpaceMemberRoleAdmin {+ return store.ErrSpacePermissionDenied+ }+ return nil+}++// spaceAuthorizeAdmin checks that the space exists and the actor administers it.+func spaceAuthorizeAdmin(ctx context.Context, q querier, spaceID, actorUserID int32) error {+ exists, err := spaceExists(ctx, q, spaceID)+ if err != nil {+ return err+ }+ if !exists {+ return store.ErrSpaceNotFound+ }+ return spaceRequireAdmin(ctx, q, spaceID, actorUserID)+}++// spaceRequireAnotherAdmin fails with ErrLastSpaceAdmin when the space has at+// most one active administrator.+func spaceRequireAnotherAdmin(ctx context.Context, q querier, spaceID int32) error {+ var count int+ if err := q.QueryRowContext(ctx, "SELECT "+spaceActiveAdminCount, spaceID).Scan(&count); err != nil {+ return err+ }+ if count <= 1 {+ return store.ErrLastSpaceAdmin+ }+ return nil+}++func spaceMarshalPayload(payload *storepb.SpacePayload) (string, error) {+ if payload == nil {+ return "{}", nil+ }+ data, err := protojson.Marshal(payload)+ if err != nil {+ return "", errors.Wrap(err, "failed to marshal space payload")+ }+ return string(data), nil+}++type spaceRowScanner interface{ Scan(...any) error }++// spaceScan reads the spaceColumns of one row into a Space.+func spaceScan(row spaceRowScanner, extra ...any) (*store.Space, error) {+ space := &store.Space{}+ var payloadBytes []byte+ targets := append([]any{&space.ID, &space.UID, &space.Title, &space.Description, &payloadBytes}, extra...)+ if err := row.Scan(targets...); err != nil {+ return nil, err+ }+ space.Payload = &storepb.SpacePayload{}+ if err := protojsonUnmarshaler.Unmarshal(payloadBytes, space.Payload); err != nil {+ return nil, errors.Wrap(err, "failed to unmarshal space payload")+ }+ return space, nil+}++// spacePopulateSummary fills the viewer's role and the active member count.+func spacePopulateSummary(ctx context.Context, q querier, space *store.Space, userID int32) error {+ err := q.QueryRowContext(ctx, `SELECT viewer_member.role, COUNT(active_member.user_id)+ FROM space_member viewer_member+ JOIN user viewer_user ON viewer_user.id = viewer_member.user_id+ JOIN space_member active_member ON active_member.space_id = viewer_member.space_id AND active_member.status = 'ACTIVE' AND active_member.role IN ('ADMIN', 'USER')+ JOIN user active_user ON active_user.id = active_member.user_id AND active_user.row_status = 'NORMAL'+ WHERE viewer_member.space_id = ? AND viewer_member.user_id = ? AND viewer_member.status = 'ACTIVE'+ AND viewer_member.role IN ('ADMIN', 'USER') AND viewer_user.row_status = 'NORMAL'+ GROUP BY viewer_member.role`, space.ID, userID).Scan(&space.CurrentUserRole, &space.MemberCount)+ return errors.Wrap(err, "failed to populate space summary")+}++// CreateSpace inserts a Space with its creator as the first administrator.+func (d *DB) CreateSpace(ctx context.Context, create *store.Space, creatorID int32) (*store.Space, error) {+ if err := requireActiveUser(ctx, d.db, creatorID, store.ErrSpaceMemberNotActive); err != nil {+ return nil, err+ }+ payload, err := spaceMarshalPayload(create.Payload)+ if err != nil {+ return nil, err+ }+ // The membership row cannot use the id generated by the first insert+ // because a batch never reads its own writes; the unique uid resolves it.+ b := newBatch()+ b.guard(activeUserCondition, creatorID)+ b.add("INSERT INTO space (uid, title, description, payload) VALUES (?, ?, ?, ?)", create.UID, create.Title, create.Description, payload)+ b.add("INSERT INTO space_member (space_id, user_id, status, role) SELECT id, ?, ?, ? FROM space WHERE uid = ?",+ creatorID, store.SpaceMemberStatusActive, store.SpaceMemberRoleAdmin, create.UID)+ if _, err := b.commit(ctx, d); err != nil {+ if isUniqueViolation(err) {+ return nil, store.ErrSpaceAlreadyExists+ }+ if isGuardFailure(err) {+ return nil, store.ErrSpaceMemberNotActive+ }+ return nil, errors.Wrap(err, "failed to create space")+ }+ space, err := spaceScan(d.db.QueryRowContext(ctx, "SELECT "+spaceColumns+" FROM space WHERE uid = ?", create.UID))+ if err != nil {+ return nil, errors.Wrap(err, "failed to read created space")+ }+ space.CurrentUserRole = store.SpaceMemberRoleAdmin+ space.MemberCount = 1+ return space, nil+}++// ListSpaces returns the Spaces matching find.+func (d *DB) ListSpaces(ctx context.Context, find *store.FindSpace) ([]*store.Space, error) {+ where, args := []string{"1 = 1"}, []any{}+ selectFields := "space.id, space.uid, space.title, space.description, space.payload"+ joins := ""+ groupBy := ""+ if find.ID != nil {+ where, args = append(where, "space.id = ?"), append(args, *find.ID)+ }+ if len(find.IDList) > 0 {+ clause, ids := inClause(find.IDList)+ where, args = append(where, "space.id IN "+clause), append(args, ids...)+ }+ if find.UID != nil {+ where, args = append(where, "space.uid = ?"), append(args, *find.UID)+ }+ if find.MemberUserID != nil {+ selectFields += ", viewer_member.role, COUNT(active_member.user_id)"+ joins = ` JOIN space_member viewer_member ON viewer_member.space_id = space.id+ JOIN user viewer_user ON viewer_user.id = viewer_member.user_id+ JOIN space_member active_member ON active_member.space_id = space.id AND active_member.status = 'ACTIVE' AND active_member.role IN ('ADMIN', 'USER')+ JOIN user active_user ON active_user.id = active_member.user_id AND active_user.row_status = 'NORMAL'`+ where = append(where, "viewer_member.user_id = ?", "viewer_member.status = 'ACTIVE'", "viewer_member.role IN ('ADMIN', 'USER')", "viewer_user.row_status = 'NORMAL'")+ args = append(args, *find.MemberUserID)+ groupBy = " GROUP BY space.id, space.uid, space.title, space.description, space.payload, viewer_member.role"+ }+ query := "SELECT " + selectFields + " FROM space" + joins + " WHERE " + strings.Join(where, " AND ") + groupBy + " ORDER BY space.id DESC"+ query = appendLimit(query, find.Limit, find.Offset)+ rows, err := d.db.QueryContext(ctx, query, args...)+ if err != nil {+ return nil, err+ }+ defer rows.Close()+ spaces := []*store.Space{}+ for rows.Next() {+ var summary []any+ var role store.SpaceMemberRole+ var memberCount int32+ if find.MemberUserID != nil {+ summary = []any{&role, &memberCount}+ }+ space, err := spaceScan(rows, summary...)+ if err != nil {+ return nil, err+ }+ space.CurrentUserRole = role+ space.MemberCount = memberCount+ spaces = append(spaces, space)+ }+ return spaces, rows.Err()+}++// UpdateSpace applies the given Space changes on behalf of an administrator.+func (d *DB) UpdateSpace(ctx context.Context, update *store.UpdateSpace, actorUserID int32) (*store.Space, error) {+ validate := func() error { return spaceAuthorizeAdmin(ctx, d.db, update.ID, actorUserID) }+ if err := validate(); err != nil {+ return nil, err+ }+ sets, args := []string{}, []any{}+ if update.Title != nil {+ sets, args = append(sets, "title = ?"), append(args, *update.Title)+ }+ if update.Description != nil {+ sets, args = append(sets, "description = ?"), append(args, *update.Description)+ }+ if update.Payload != nil {+ payload, err := spaceMarshalPayload(update.Payload)+ if err != nil {+ return nil, err+ }+ sets, args = append(sets, "payload = ?"), append(args, payload)+ }+ args = append(args, update.ID, update.ID, actorUserID)+ query := "UPDATE space SET " + strings.Join(sets, ", ") + " WHERE id = ? AND " + spaceAdminCondition + " RETURNING " + spaceColumns+ space, err := spaceScan(d.db.QueryRowContext(ctx, query, args...))+ if errors.Is(err, sql.ErrNoRows) {+ return nil, spaceRecheck(validate, store.ErrSpacePermissionDenied)+ }+ if err != nil {+ return nil, err+ }+ if err := spacePopulateSummary(ctx, d.db, space, actorUserID); err != nil {+ return nil, err+ }+ return space, nil+}++// CreateSpaceInvitation invites a user to a Space on behalf of an administrator.+func (d *DB) CreateSpaceInvitation(ctx context.Context, create *store.SpaceInvitation, actorUserID int32) (*store.SpaceInvitation, error) {+ validate := func() error {+ if err := spaceAuthorizeAdmin(ctx, d.db, create.SpaceID, actorUserID); err != nil {+ return err+ }+ return requireActiveUser(ctx, d.db, create.UserID, store.ErrSpaceMemberNotActive)+ }+ if err := validate(); err != nil {+ return nil, err+ }+ b := newBatch()+ b.guard(spaceAdminCondition, create.SpaceID, actorUserID)+ b.guard(activeUserCondition, create.UserID)+ b.add("INSERT INTO space_member (space_id, user_id, status, role) VALUES (?, ?, ?, ?)", create.SpaceID, create.UserID, store.SpaceMemberStatusInvited, create.Role)+ if _, err := b.commit(ctx, d); err != nil {+ if isUniqueViolation(err) {+ return nil, store.ErrSpaceMemberAlreadyExists+ }+ if isGuardFailure(err) {+ return nil, spaceRecheck(validate, store.ErrSpacePermissionDenied)+ }+ return nil, err+ }+ return &store.SpaceInvitation{SpaceID: create.SpaceID, UserID: create.UserID, Role: create.Role}, nil+}++// ListSpaceMembers returns the active memberships matching find.+func (d *DB) ListSpaceMembers(ctx context.Context, find *store.FindSpaceMember) ([]*store.SpaceMember, error) {+ where, args := []string{+ "space_member.status = 'ACTIVE'",+ "space_member.role IN ('ADMIN', 'USER')",+ "EXISTS (SELECT 1 FROM space member_space WHERE member_space.id = space_member.space_id)",+ "EXISTS (SELECT 1 FROM user member_user WHERE member_user.id = space_member.user_id AND member_user.row_status = 'NORMAL')",+ }, []any{}+ if find.SpaceID != nil {+ where, args = append(where, "space_id = ?"), append(args, *find.SpaceID)+ }+ if find.UserID != nil {+ where, args = append(where, "user_id = ?"), append(args, *find.UserID)+ }+ if find.ViewerUserID != nil {+ where, args = append(where, "EXISTS (SELECT 1 FROM space_member viewer JOIN user viewer_user ON viewer_user.id = viewer.user_id WHERE viewer.space_id = space_member.space_id AND viewer.user_id = ? AND viewer.status = 'ACTIVE' AND viewer.role IN ('ADMIN', 'USER') AND viewer_user.row_status = 'NORMAL')"), append(args, *find.ViewerUserID)+ }+ query := "SELECT space_id, user_id, role FROM space_member WHERE " + strings.Join(where, " AND ") + " ORDER BY user_id ASC"+ query = appendLimit(query, find.Limit, find.Offset)+ rows, err := d.db.QueryContext(ctx, query, args...)+ if err != nil {+ return nil, err+ }+ defer rows.Close()+ members := []*store.SpaceMember{}+ for rows.Next() {+ member := &store.SpaceMember{}+ if err := rows.Scan(&member.SpaceID, &member.UserID, &member.Role); err != nil {+ return nil, err+ }+ members = append(members, member)+ }+ return members, rows.Err()+}++// ListSpaceInvitations returns the pending invitations matching find.+func (d *DB) ListSpaceInvitations(ctx context.Context, find *store.FindSpaceInvitation) ([]*store.SpaceInvitation, error) {+ where, args := []string{+ "space_member.status = 'INVITED'",+ "space_member.role IN ('ADMIN', 'USER')",+ "EXISTS (SELECT 1 FROM space invitation_space WHERE invitation_space.id = space_member.space_id)",+ "EXISTS (SELECT 1 FROM user invitee WHERE invitee.id = space_member.user_id)",+ }, []any{}+ if find.SpaceID != nil {+ where, args = append(where, "space_member.space_id = ?"), append(args, *find.SpaceID)+ }+ if find.UserID != nil {+ where, args = append(where, "space_member.user_id = ?"), append(args, *find.UserID)+ }+ if find.ViewerUserID != nil {+ where, args = append(where, `(+ (space_member.user_id = ? AND EXISTS (SELECT 1 FROM user viewer_user WHERE viewer_user.id = ? AND viewer_user.row_status = 'NORMAL'))+ OR EXISTS (+ SELECT 1 FROM space_member viewer+ JOIN user viewer_user ON viewer_user.id = viewer.user_id+ WHERE viewer.space_id = space_member.space_id+ AND viewer.user_id = ?+ AND viewer.status = 'ACTIVE'+ AND viewer.role = 'ADMIN'+ AND viewer_user.row_status = 'NORMAL'+ )+ )`), append(args, *find.ViewerUserID, *find.ViewerUserID, *find.ViewerUserID)+ }+ query := "SELECT space_id, user_id, role FROM space_member WHERE " + strings.Join(where, " AND ") + " ORDER BY space_id DESC, user_id ASC"+ query = appendLimit(query, find.Limit, find.Offset)+ rows, err := d.db.QueryContext(ctx, query, args...)+ if err != nil {+ return nil, err+ }+ defer rows.Close()+ invitations := []*store.SpaceInvitation{}+ for rows.Next() {+ invitation := &store.SpaceInvitation{}+ if err := rows.Scan(&invitation.SpaceID, &invitation.UserID, &invitation.Role); err != nil {+ return nil, err+ }+ invitations = append(invitations, invitation)+ }+ return invitations, rows.Err()+}++// AcceptSpaceInvitation turns the actor's own invitation into an active membership.+func (d *DB) AcceptSpaceInvitation(ctx context.Context, accept *store.AcceptSpaceInvitation, actorUserID int32) (*store.SpaceMember, error) {+ if accept.UserID != actorUserID {+ return nil, store.ErrSpacePermissionDenied+ }+ validate := func() error { return requireActiveUser(ctx, d.db, accept.UserID, store.ErrSpaceMemberNotActive) }+ if err := validate(); err != nil {+ return nil, err+ }+ member := &store.SpaceMember{}+ query := `UPDATE space_member SET status = ?+ WHERE space_id = ? AND user_id = ? AND status = ? AND role IN ('ADMIN', 'USER') AND ` + activeUserCondition + `+ RETURNING space_id, user_id, role`+ err := d.db.QueryRowContext(ctx, query, store.SpaceMemberStatusActive, accept.SpaceID, accept.UserID, store.SpaceMemberStatusInvited, accept.UserID).Scan(&member.SpaceID, &member.UserID, &member.Role)+ if errors.Is(err, sql.ErrNoRows) {+ return nil, spaceRecheck(validate, store.ErrSpaceInvitationNotFound)+ }+ if err != nil {+ return nil, err+ }+ return member, nil+}++// DeclineSpaceInvitation removes the actor's own pending invitation.+func (d *DB) DeclineSpaceInvitation(ctx context.Context, decline *store.DeclineSpaceInvitation, actorUserID int32) error {+ if decline.UserID != actorUserID {+ return store.ErrSpacePermissionDenied+ }+ validate := func() error { return requireActiveUser(ctx, d.db, decline.UserID, store.ErrSpaceMemberNotActive) }+ if err := validate(); err != nil {+ return err+ }+ query := "DELETE FROM space_member WHERE space_id = ? AND user_id = ? AND status = ? AND " + activeUserCondition+ return spaceDeleteInvitation(ctx, d, validate, query, decline.SpaceID, decline.UserID, store.SpaceMemberStatusInvited, decline.UserID)+}++// RevokeSpaceInvitation removes a pending invitation on behalf of an administrator.+func (d *DB) RevokeSpaceInvitation(ctx context.Context, revoke *store.RevokeSpaceInvitation, actorUserID int32) error {+ validate := func() error { return spaceAuthorizeAdmin(ctx, d.db, revoke.SpaceID, actorUserID) }+ if err := validate(); err != nil {+ return err+ }+ query := "DELETE FROM space_member WHERE space_id = ? AND user_id = ? AND status = ? AND " + spaceAdminCondition+ return spaceDeleteInvitation(ctx, d, validate, query, revoke.SpaceID, revoke.UserID, store.SpaceMemberStatusInvited, revoke.SpaceID, actorUserID)+}++// spaceDeleteInvitation runs a conditional invitation delete and explains an+// empty result through validate.+func spaceDeleteInvitation(ctx context.Context, d *DB, validate func() error, query string, args ...any) error {+ result, err := d.db.ExecContext(ctx, query, args...)+ if err != nil {+ return err+ }+ deleted, err := result.RowsAffected()+ if err != nil {+ return err+ }+ if deleted == 0 {+ return spaceRecheck(validate, store.ErrSpaceInvitationNotFound)+ }+ return nil+}++// spaceValidateMemberUpdate mirrors the predicates encoded in the+// UpdateSpaceMember statement so a no-op write can be explained.+func spaceValidateMemberUpdate(ctx context.Context, q querier, update *store.UpdateSpaceMember, actorUserID int32) error {+ if err := spaceAuthorizeAdmin(ctx, q, update.SpaceID, actorUserID); err != nil {+ return err+ }+ if err := requireActiveUser(ctx, q, update.UserID, store.ErrSpaceMemberNotActive); err != nil {+ return err+ }+ var currentRole store.SpaceMemberRole+ err := q.QueryRowContext(ctx, "SELECT role FROM space_member WHERE space_id = ? AND user_id = ? AND status = ?", update.SpaceID, update.UserID, store.SpaceMemberStatusActive).Scan(¤tRole)+ if errors.Is(err, sql.ErrNoRows) {+ return store.ErrSpaceMemberNotFound+ }+ if err != nil {+ return err+ }+ if currentRole == store.SpaceMemberRoleAdmin && update.Role != nil && *update.Role != store.SpaceMemberRoleAdmin {+ return spaceRequireAnotherAdmin(ctx, q, update.SpaceID)+ }+ return nil+}++// UpdateSpaceMember changes a membership on behalf of an administrator, keeping at least one active administrator.+func (d *DB) UpdateSpaceMember(ctx context.Context, update *store.UpdateSpaceMember, actorUserID int32) (*store.SpaceMember, error) {+ validate := func() error { return spaceValidateMemberUpdate(ctx, d.db, update, actorUserID) }+ if err := validate(); err != nil {+ return nil, err+ }+ sets, args := []string{}, []any{}+ if update.Role != nil {+ sets, args = append(sets, "role = ?"), append(args, *update.Role)+ }+ if len(sets) == 0 {+ // Nothing to write; report the current membership as an update would.+ member := &store.SpaceMember{}+ err := d.db.QueryRowContext(ctx, "SELECT space_id, user_id, role FROM space_member WHERE space_id = ? AND user_id = ? AND status = ?",+ update.SpaceID, update.UserID, store.SpaceMemberStatusActive).Scan(&member.SpaceID, &member.UserID, &member.Role)+ if errors.Is(err, sql.ErrNoRows) {+ return nil, store.ErrSpaceMemberNotFound+ }+ if err != nil {+ return nil, err+ }+ return member, nil+ }+ where := []string{"space_id = ?", "user_id = ?", "status = ?", spaceAdminCondition, activeUserCondition}+ args = append(args, update.SpaceID, update.UserID, store.SpaceMemberStatusActive, update.SpaceID, actorUserID, update.UserID)+ if update.Role != nil && *update.Role != store.SpaceMemberRoleAdmin {+ // Demoting the last active administrator would orphan the space.+ where = append(where, "(role <> 'ADMIN' OR "+spaceActiveAdminCount+" > 1)")+ args = append(args, update.SpaceID)+ }+ query := "UPDATE space_member SET " + strings.Join(sets, ", ") + " WHERE " + strings.Join(where, " AND ") + " RETURNING space_id, user_id, role"+ member := &store.SpaceMember{}+ err := d.db.QueryRowContext(ctx, query, args...).Scan(&member.SpaceID, &member.UserID, &member.Role)+ if errors.Is(err, sql.ErrNoRows) {+ return nil, spaceRecheck(validate, store.ErrSpaceMemberNotFound)+ }+ if err != nil {+ return nil, err+ }+ return member, nil+}++// spaceValidateMemberDelete mirrors the predicates encoded in the+// DeleteSpaceMember statement so a no-op write can be explained.+func spaceValidateMemberDelete(ctx context.Context, q querier, delete *store.DeleteSpaceMember, actorUserID int32) error {+ exists, err := spaceExists(ctx, q, delete.SpaceID)+ if err != nil {+ return err+ }+ if !exists {+ return store.ErrSpaceNotFound+ }+ if actorUserID != delete.UserID {+ if err := spaceRequireAdmin(ctx, q, delete.SpaceID, actorUserID); err != nil {+ return err+ }+ }+ var role store.SpaceMemberRole+ var targetStatus sql.NullString+ err = q.QueryRowContext(ctx, `SELECT sm.role, u.row_status FROM space_member sm LEFT JOIN user u ON u.id = sm.user_id+ WHERE sm.space_id = ? AND sm.user_id = ? AND sm.status = ?`, delete.SpaceID, delete.UserID, store.SpaceMemberStatusActive).Scan(&role, &targetStatus)+ if errors.Is(err, sql.ErrNoRows) {+ return store.ErrSpaceMemberNotFound+ }+ if err != nil {+ return err+ }+ if role == store.SpaceMemberRoleAdmin && targetStatus.Valid && store.RowStatus(targetStatus.String) == store.Normal {+ return spaceRequireAnotherAdmin(ctx, q, delete.SpaceID)+ }+ return nil+}++// DeleteSpaceMember removes a membership, keeping at least one active administrator.+func (d *DB) DeleteSpaceMember(ctx context.Context, delete *store.DeleteSpaceMember, actorUserID int32) error {+ validate := func() error { return spaceValidateMemberDelete(ctx, d.db, delete, actorUserID) }+ if err := validate(); err != nil {+ return err+ }+ where := []string{"space_id = ?", "user_id = ?", "status = ?", "EXISTS (SELECT 1 FROM space WHERE id = ?)"}+ args := []any{delete.SpaceID, delete.UserID, store.SpaceMemberStatusActive, delete.SpaceID}+ if actorUserID != delete.UserID {+ where = append(where, spaceAdminCondition)+ args = append(args, delete.SpaceID, actorUserID)+ }+ // An archived administrator may always be removed; an active one only+ // when another active administrator remains.+ where = append(where, "(role <> 'ADMIN' OR NOT "+activeUserCondition+" OR "+spaceActiveAdminCount+" > 1)")+ args = append(args, delete.UserID, delete.SpaceID)+ result, err := d.db.ExecContext(ctx, "DELETE FROM space_member WHERE "+strings.Join(where, " AND "), args...)+ if err != nil {+ return err+ }+ deleted, err := result.RowsAffected()+ if err != nil {+ return err+ }+ if deleted == 0 {+ return spaceRecheck(validate, store.ErrSpaceMemberNotFound)+ }+ return nil+}diff --git a/store/db/d1/space_delete.go b/store/db/d1/space_delete.gonew file mode 100644index 000000000..050c07a21--- /dev/null+++ b/store/db/d1/space_delete.go@@ -0,0 +1,78 @@+package d1++import (+ "context"+ "database/sql"++ "github.com/pkg/errors"++ "github.com/usememos/memos/store"+)++// errSpaceDeleteConflict reports that memos or attachments were added to or+// removed from the space between the validation reads and the batch. The+// caller can simply retry.+var errSpaceDeleteConflict = errors.New("space contents changed during deletion")++// spaceValidateDelete checks that the actor is an active administrator of an+// existing space.+func spaceValidateDelete(ctx context.Context, q querier, delete *store.DeleteSpace) error {+ if err := requireActiveUser(ctx, q, delete.ActorUserID, store.ErrSpacePermissionDenied); err != nil {+ return err+ }+ exists, err := spaceExists(ctx, q, delete.ID)+ if err != nil {+ return err+ }+ if !exists {+ return store.ErrSpaceNotFound+ }+ var role store.SpaceMemberRole+ err = q.QueryRowContext(ctx, "SELECT role FROM space_member WHERE space_id = ? AND user_id = ? AND status = 'ACTIVE'", delete.ID, delete.ActorUserID).Scan(&role)+ if errors.Is(err, sql.ErrNoRows) {+ return store.ErrSpacePermissionDenied+ }+ if err != nil {+ return errors.Wrap(err, "failed to read space administrator membership")+ }+ if role != store.SpaceMemberRoleAdmin {+ return store.ErrSpacePermissionDenied+ }+ return nil+}++// DeleteSpace hard-deletes only memos directly placed in the Space. Relations+// are removed when either endpoint is deleted, but are never traversed.+func (d *DB) DeleteSpace(ctx context.Context, delete *store.DeleteSpace) (*store.DeleteSpaceResult, error) {+ validate := func() error { return spaceValidateDelete(ctx, d.db, delete) }+ if err := validate(); err != nil {+ return nil, err+ }+ memoIDs, err := listMemoIDs(ctx, d.db, "SELECT id FROM memo WHERE space_id = ? ORDER BY id", delete.ID)+ if err != nil {+ return nil, errors.Wrap(err, "failed to collect assigned memos")+ }+ attachments, err := listMemoSetAttachments(ctx, d.db, memoIDs)+ if err != nil {+ return nil, errors.Wrap(err, "failed to collect memo attachments")+ }++ b := newBatch()+ b.guard(activeUserCondition, delete.ActorUserID)+ b.guard("EXISTS (SELECT 1 FROM space WHERE id = ?)", delete.ID)+ b.guard("EXISTS (SELECT 1 FROM space_member WHERE space_id = ? AND user_id = ? AND status = 'ACTIVE' AND role = 'ADMIN')", delete.ID, delete.ActorUserID)+ // The memo set was read outside the batch; abort when it changed so no+ // memo or attachment is left pointing at a deleted space.+ b.guard("(SELECT COUNT(*) FROM memo WHERE space_id = ?) = ?", delete.ID, len(memoIDs))+ b.guard("(SELECT COUNT(*) FROM attachment WHERE memo_id IN (SELECT id FROM memo WHERE space_id = ?)) = ?", delete.ID, len(attachments))+ addMemoSetDeletes(b, memoIDs, attachmentIDs(attachments))+ b.add("DELETE FROM space_member WHERE space_id = ?", delete.ID)+ b.add("DELETE FROM space WHERE id = ?", delete.ID)+ if _, err := b.commit(ctx, d); err != nil {+ if isGuardFailure(err) {+ return nil, spaceRecheck(validate, errSpaceDeleteConflict)+ }+ return nil, errors.Wrap(err, "failed to delete space")+ }+ return &store.DeleteSpaceResult{Attachments: attachments}, nil+}diff --git a/store/db/d1/sqldriver.go b/store/db/d1/sqldriver.gonew file mode 100644index 000000000..3266569fd--- /dev/null+++ b/store/db/d1/sqldriver.go@@ -0,0 +1,241 @@+package d1++import (+ "context"+ "database/sql"+ "database/sql/driver"+ "io"+ "strings"++ "github.com/pkg/errors"+)++// The types below adapt the REST client to database/sql so that the migrator+// and tests can use GetDB(). D1 has no interactive transactions: a Tx buffers+// its writes and commits them as one atomic batch, and refuses reads once a+// write has been buffered because the write is not yet visible.++var errUnsupportedQueryInTx = errors.New("d1: reads inside a transaction must precede writes; buffered writes are only visible after commit")++type sqlDriver struct{}++func (sqlDriver) Open(string) (driver.Conn, error) {+ return nil, errors.New("d1: open the database through NewDB")+}++type connector struct {+ client *client+}++func (c *connector) Connect(context.Context) (driver.Conn, error) {+ return &conn{client: c.client}, nil+}++func (*connector) Driver() driver.Driver {+ return sqlDriver{}+}++func openSQLDB(c *client) *sql.DB {+ db := sql.OpenDB(&connector{client: c})+ // Every connection is a stateless HTTP session; keep the pool small so+ // concurrent callers do not fan out into the REST rate limit.+ db.SetMaxOpenConns(8)+ return db+}++type conn struct {+ client *client+ tx *tx+}++func (c *conn) Prepare(query string) (driver.Stmt, error) {+ return &stmt{conn: c, query: query}, nil+}++func (*conn) Close() error {+ return nil+}++func (c *conn) Begin() (driver.Tx, error) {+ return c.BeginTx(context.Background(), driver.TxOptions{})+}++func (c *conn) BeginTx(context.Context, driver.TxOptions) (driver.Tx, error) {+ if c.tx != nil {+ return nil, errors.New("d1: transaction already in progress")+ }+ c.tx = &tx{conn: c}+ return c.tx, nil+}++func (c *conn) Ping(ctx context.Context) error {+ _, err := c.client.exec(ctx, statement{SQL: "SELECT 1"})+ return err+}++func (c *conn) ExecContext(ctx context.Context, query string, args []driver.NamedValue) (driver.Result, error) {+ stmt := statement{SQL: query, Args: namedValues(args)}+ if c.tx != nil {+ c.tx.pending = append(c.tx.pending, stmt)+ return deferredResult{}, nil+ }+ res, err := c.client.exec(ctx, stmt)+ if err != nil {+ return nil, err+ }+ return execResult{changes: res.Changes, lastRowID: res.LastRowID}, nil+}++func (c *conn) QueryContext(ctx context.Context, query string, args []driver.NamedValue) (driver.Rows, error) {+ if c.tx != nil && len(c.tx.pending) > 0 {+ return nil, errUnsupportedQueryInTx+ }+ res, err := c.client.exec(ctx, statement{SQL: query, Args: namedValues(args)})+ if err != nil {+ return nil, err+ }+ return &rows{result: res}, nil+}++// CheckNamedValue accepts every value encodeArgs understands so database/sql+// does not reject named string types before they reach the converter.+func (*conn) CheckNamedValue(value *driver.NamedValue) error {+ converted, err := driver.DefaultParameterConverter.ConvertValue(value.Value)+ if err != nil {+ return err+ }+ value.Value = converted+ return nil+}++func namedValues(args []driver.NamedValue) []any {+ values := make([]any, 0, len(args))+ for _, arg := range args {+ values = append(values, arg.Value)+ }+ return values+}++type stmt struct {+ conn *conn+ query string+}++func (*stmt) Close() error { return nil }++func (*stmt) NumInput() int { return -1 }++func (s *stmt) Exec(args []driver.Value) (driver.Result, error) {+ return s.ExecContext(context.Background(), valuesToNamed(args))+}++func (s *stmt) Query(args []driver.Value) (driver.Rows, error) {+ return s.QueryContext(context.Background(), valuesToNamed(args))+}++func (s *stmt) ExecContext(ctx context.Context, args []driver.NamedValue) (driver.Result, error) {+ return s.conn.ExecContext(ctx, s.query, args)+}++func (s *stmt) QueryContext(ctx context.Context, args []driver.NamedValue) (driver.Rows, error) {+ return s.conn.QueryContext(ctx, s.query, args)+}++func valuesToNamed(values []driver.Value) []driver.NamedValue {+ named := make([]driver.NamedValue, 0, len(values))+ for i, value := range values {+ named = append(named, driver.NamedValue{Ordinal: i + 1, Value: value})+ }+ return named+}++type tx struct {+ conn *conn+ pending []statement+}++func (t *tx) Commit() error {+ defer t.finish()+ if len(t.pending) == 0 {+ return nil+ }+ ctx := context.Background()+ if parameterFree(t.pending) {+ sqls := make([]string, 0, len(t.pending))+ for _, stmt := range t.pending {+ sqls = append(sqls, strings.TrimSpace(stmt.SQL))+ }+ return t.conn.client.script(ctx, strings.Join(sqls, "\n;\n"))+ }+ _, err := t.conn.client.batch(ctx, t.pending)+ return err+}++func (t *tx) Rollback() error {+ t.finish()+ return nil+}++func (t *tx) finish() {+ t.pending = nil+ if t.conn.tx == t {+ t.conn.tx = nil+ }+}++func parameterFree(stmts []statement) bool {+ for _, stmt := range stmts {+ if len(stmt.Args) > 0 {+ return false+ }+ }+ return true+}++type execResult struct {+ changes int64+ lastRowID int64+}++func (r execResult) LastInsertId() (int64, error) { return r.lastRowID, nil }++func (r execResult) RowsAffected() (int64, error) { return r.changes, nil }++// deferredResult is returned for writes buffered inside a transaction; their+// counters are unknown until the batch commits.+type deferredResult struct{}++func (deferredResult) LastInsertId() (int64, error) {+ return 0, errors.New("d1: last insert id is unavailable for writes buffered in a transaction")+}++func (deferredResult) RowsAffected() (int64, error) {+ return 0, errors.New("d1: rows affected is unavailable for writes buffered in a transaction")+}++type rows struct {+ result *result+ index int+}++func (r *rows) Columns() []string {+ return r.result.Columns+}++func (*rows) Close() error { return nil }++func (r *rows) Next(dest []driver.Value) error {+ if r.index >= len(r.result.Rows) {+ return io.EOF+ }+ row := r.result.Rows[r.index]+ r.index+++ for i := range dest {+ if i < len(row) {+ dest[i] = row[i]+ } else {+ dest[i] = nil+ }+ }+ return nil+}diff --git a/store/db/d1/user.go b/store/db/d1/user.gonew file mode 100644index 000000000..69ef0f669--- /dev/null+++ b/store/db/d1/user.go@@ -0,0 +1,225 @@+package d1++import (+ "context"+ "database/sql"+ "strings"++ "github.com/pkg/errors"++ "github.com/usememos/memos/store"+)++const userColumns = "id, username, role, email, nickname, password_hash, avatar_url, description, created_ts, updated_ts, row_status"++// userLastAdminCondition is true when the user is the only active,+// non-archived administrator of some space. It binds the user id twice.+const userLastAdminCondition = `EXISTS (+ SELECT 1 FROM space_member target+ WHERE target.user_id = ? AND target.status = 'ACTIVE' AND target.role = 'ADMIN'+ AND NOT EXISTS (+ SELECT 1 FROM space_member other JOIN user u ON u.id = other.user_id+ WHERE other.space_id = target.space_id AND other.user_id <> ?+ AND other.status = 'ACTIVE' AND other.role = 'ADMIN' AND u.row_status = 'NORMAL'+ ))`++type userRowScanner interface{ Scan(...any) error }++// userScan reads the userColumns of one row.+func userScan(row userRowScanner, user *store.User) error {+ var email sql.NullString+ if err := row.Scan(+ &user.ID,+ &user.Username,+ &user.Role,+ &email,+ &user.Nickname,+ &user.PasswordHash,+ &user.AvatarURL,+ &user.Description,+ &user.CreatedTs,+ &user.UpdatedTs,+ &user.RowStatus,+ ); err != nil {+ return err+ }+ user.Email = email.String+ return nil+}++// userInsert inserts create and fills the generated columns from the row.+func userInsert(ctx context.Context, q querier, create *store.User) error {+ query := "INSERT INTO user (username, role, email, nickname, password_hash, avatar_url) VALUES (?, ?, ?, ?, ?, ?) RETURNING id, description, created_ts, updated_ts, row_status"+ return q.QueryRowContext(ctx, query, create.Username, create.Role, nullableEmail(create.Email), create.Nickname, create.PasswordHash, create.AvatarURL).Scan(+ &create.ID,+ &create.Description,+ &create.CreatedTs,+ &create.UpdatedTs,+ &create.RowStatus,+ )+}++// CreateUser inserts a user.+func (d *DB) CreateUser(ctx context.Context, create *store.User) (*store.User, error) {+ if err := userInsert(ctx, d.db, create); err != nil {+ return nil, err+ }+ return create, nil+}++// userValidateArchive fails with ErrLastSpaceAdmin when archiving the user+// would leave a space without an active administrator. A missing user yields+// sql.ErrNoRows.+func userValidateArchive(ctx context.Context, q querier, userID int32) error {+ var current store.RowStatus+ if err := q.QueryRowContext(ctx, "SELECT row_status FROM user WHERE id = ?", userID).Scan(¤t); err != nil {+ return err+ }+ if current != store.Normal {+ return nil+ }+ var wouldLoseAdmin bool+ if err := q.QueryRowContext(ctx, "SELECT "+userLastAdminCondition, userID, userID).Scan(&wouldLoseAdmin); err != nil {+ return err+ }+ if wouldLoseAdmin {+ return store.ErrLastSpaceAdmin+ }+ return nil+}++// UpdateUser applies the given user changes, refusing to archive the last administrator of a Space.+func (d *DB) UpdateUser(ctx context.Context, update *store.UpdateUser) (*store.User, error) {+ archiving := update.RowStatus != nil && *update.RowStatus == store.Archived+ validate := func() error {+ if archiving {+ return userValidateArchive(ctx, d.db, update.ID)+ }+ return nil+ }+ if err := validate(); err != nil {+ return nil, err+ }+ set, args := []string{}, []any{}+ if v := update.UpdatedTs; v != nil {+ set, args = append(set, "updated_ts = ?"), append(args, *v)+ }+ if v := update.RowStatus; v != nil {+ set, args = append(set, "row_status = ?"), append(args, *v)+ }+ if v := update.Username; v != nil {+ set, args = append(set, "username = ?"), append(args, *v)+ }+ if v := update.Email; v != nil {+ set, args = append(set, "email = ?"), append(args, nullableEmail(*v))+ }+ if v := update.Nickname; v != nil {+ set, args = append(set, "nickname = ?"), append(args, *v)+ }+ if v := update.AvatarURL; v != nil {+ set, args = append(set, "avatar_url = ?"), append(args, *v)+ }+ if v := update.PasswordHash; v != nil {+ set, args = append(set, "password_hash = ?"), append(args, *v)+ }+ if v := update.Description; v != nil {+ set, args = append(set, "description = ?"), append(args, *v)+ }+ if v := update.Role; v != nil {+ set, args = append(set, "role = ?"), append(args, *v)+ }+ if len(set) == 0 {+ // Nothing to write; report the current row as an update would.+ user := &store.User{}+ if err := userScan(d.db.QueryRowContext(ctx, "SELECT "+userColumns+" FROM user WHERE id = ?", update.ID), user); err != nil {+ return nil, err+ }+ return user, nil+ }+ where := "id = ?"+ args = append(args, update.ID)+ if archiving {+ // The validation read is not transactional with this write, so the+ // last-admin invariant is re-asserted in the statement itself.+ where += " AND (row_status <> 'NORMAL' OR NOT " + userLastAdminCondition + ")"+ args = append(args, update.ID, update.ID)+ }+ query := "UPDATE user SET " + strings.Join(set, ", ") + " WHERE " + where + " RETURNING " + userColumns+ user := &store.User{}+ err := userScan(d.db.QueryRowContext(ctx, query, args...), user)+ if errors.Is(err, sql.ErrNoRows) {+ return nil, spaceRecheck(validate, err)+ }+ if err != nil {+ return nil, err+ }+ return user, nil+}++// ListUsers returns the users matching find.+func (d *DB) ListUsers(ctx context.Context, find *store.FindUser) ([]*store.User, error) {+ if len(find.Filters) > 0 {+ return nil, errors.Errorf("user filters are not supported")+ }+ where, args := []string{"1 = 1"}, []any{}+ orderBy := []string{"created_ts DESC", "row_status DESC", "id DESC"}++ if v := find.ID; v != nil {+ where, args = append(where, "id = ?"), append(args, *v)+ }+ if len(find.IDList) > 0 {+ clause, ids := inClause(find.IDList)+ where, args = append(where, "id IN "+clause), append(args, ids...)+ }+ if len(find.UsernameList) > 0 {+ clause, usernames := inClause(find.UsernameList)+ where, args = append(where, "username IN "+clause), append(args, usernames...)+ }+ if v := find.RowStatus; v != nil {+ where, args = append(where, "row_status = ?"), append(args, *v)+ }+ if v := find.Username; v != nil {+ where, args = append(where, "username = ?"), append(args, *v)+ }+ if v := find.Role; v != nil {+ where, args = append(where, "role = ?"), append(args, *v)+ }+ if v := find.Email; v != nil {+ where, args = append(where, "email = ?"), append(args, *v)+ }+ if v := find.Nickname; v != nil {+ where, args = append(where, "nickname = ?"), append(args, *v)+ }+ if v := find.Search; v != nil && strings.TrimSpace(*v) != "" {+ query := strings.ToLower(strings.TrimSpace(*v))+ where, args = append(where, "(LOWER(username) LIKE ? OR LOWER(nickname) LIKE ?)"), append(args, "%"+query+"%", "%"+query+"%")+ orderBy = []string{+ "CASE WHEN LOWER(username) = ? THEN 0 WHEN LOWER(username) LIKE ? THEN 1 WHEN LOWER(nickname) LIKE ? THEN 2 ELSE 3 END",+ "LENGTH(username) ASC",+ "created_ts DESC",+ "row_status DESC",+ }+ args = append(args, query, query+"%", query+"%")+ }+ query := "SELECT " + userColumns + " FROM user WHERE " + strings.Join(where, " AND ") + " ORDER BY " + strings.Join(orderBy, ", ")+ query = appendLimit(query, find.Limit, find.Offset)++ rows, err := d.db.QueryContext(ctx, query, args...)+ if err != nil {+ return nil, err+ }+ defer rows.Close()++ list := make([]*store.User, 0)+ for rows.Next() {+ user := &store.User{}+ if err := userScan(rows, user); err != nil {+ return nil, err+ }+ list = append(list, user)+ }+ if err := rows.Err(); err != nil {+ return nil, err+ }+ return list, nil+}diff --git a/store/db/d1/user_delete.go b/store/db/d1/user_delete.gonew file mode 100644index 000000000..fe2cf6423--- /dev/null+++ b/store/db/d1/user_delete.go@@ -0,0 +1,133 @@+package d1++import (+ "context"+ "database/sql"++ "github.com/pkg/errors"++ storepb "github.com/usememos/memos/proto/gen/store"+ "github.com/usememos/memos/store"+)++// errUserDeleteConflict reports that the user's memos or attachments changed+// between the validation reads and the batch. The caller can simply retry.+var errUserDeleteConflict = errors.New("user contents changed during deletion")++// userDeleteTargets holds the rows read before the delete batch: the memos+// and attachments to remove, and the setting keys reported to the caller.+type userDeleteTargets struct {+ memoIDs []int32+ attachments []*store.Attachment+ settingKeys []storepb.UserSetting_Key+}++// userRequireNoMembership fails with ErrUserHasSpaceMembership while the user+// still holds a membership that is not a pending invitation.+func userRequireNoMembership(ctx context.Context, q querier, userID int32) error {+ var count int+ if err := q.QueryRowContext(ctx, "SELECT COUNT(*) FROM space_member WHERE user_id = ? AND status <> 'INVITED'", userID).Scan(&count); err != nil {+ return errors.Wrap(err, "failed to check user space memberships")+ }+ if count != 0 {+ return store.ErrUserHasSpaceMembership+ }+ return nil+}++func userCollectDeleteTargets(ctx context.Context, q querier, userID int32) (*userDeleteTargets, error) {+ targets := &userDeleteTargets{}+ memoIDs, err := listMemoIDs(ctx, q, "SELECT id FROM memo WHERE creator_id = ? ORDER BY id", userID)+ if err != nil {+ return nil, errors.Wrap(err, "failed to list user memos")+ }+ targets.memoIDs = memoIDs++ // Attachments owned by the user and attachments bound to the user's memos+ // overlap; seen keeps each once.+ seen := make(map[int32]struct{})+ targets.attachments = make([]*store.Attachment, 0)+ if err := scanAttachmentSnapshots(ctx, q, "SELECT "+attachmentSnapshotColumns+" FROM attachment WHERE creator_id = ? ORDER BY id", []any{userID}, seen, &targets.attachments); err != nil {+ return nil, errors.Wrap(err, "failed to list user attachments")+ }+ for _, ids := range chunk(memoIDs, inClauseBatchSize) {+ clause, args := inClause(ids)+ if err := scanAttachmentSnapshots(ctx, q, "SELECT "+attachmentSnapshotColumns+" FROM attachment WHERE memo_id IN "+clause+" ORDER BY id", args, seen, &targets.attachments); err != nil {+ return nil, errors.Wrap(err, "failed to list user memo attachments")+ }+ }++ keys, err := userListSettingKeys(ctx, q, userID)+ if err != nil {+ return nil, errors.Wrap(err, "failed to list user setting keys")+ }+ targets.settingKeys = keys+ return targets, nil+}++func userListSettingKeys(ctx context.Context, q querier, userID int32) ([]storepb.UserSetting_Key, error) {+ rows, err := q.QueryContext(ctx, "SELECT key FROM user_setting WHERE user_id = ?", userID)+ if err != nil {+ return nil, err+ }+ defer rows.Close()+ keys := make([]storepb.UserSetting_Key, 0)+ for rows.Next() {+ var keyString string+ if err := rows.Scan(&keyString); err != nil {+ return nil, err+ }+ keys = append(keys, storepb.UserSetting_Key(storepb.UserSetting_Key_value[keyString]))+ }+ return keys, rows.Err()+}++// DeleteUser removes a user together with the memos, attachments, reactions, shares, inbox messages, identities, and settings they own.+func (d *DB) DeleteUser(ctx context.Context, delete *store.DeleteUser) (*store.DeleteUserResult, error) {+ var userID int32+ err := d.db.QueryRowContext(ctx, "SELECT id FROM user WHERE id = ?", delete.ID).Scan(&userID)+ if errors.Is(err, sql.ErrNoRows) {+ return &store.DeleteUserResult{}, nil+ }+ if err != nil {+ return nil, errors.Wrap(err, "failed to read user")+ }+ validate := func() error { return userRequireNoMembership(ctx, d.db, delete.ID) }+ if err := validate(); err != nil {+ return nil, err+ }+ targets, err := userCollectDeleteTargets(ctx, d.db, delete.ID)+ if err != nil {+ return nil, errors.Wrap(err, "failed to collect delete user targets")+ }+ if store.GetDeleteUserFailpoint(ctx) == store.DeleteUserFailpointBeforeCommit {+ return nil, errors.New("delete user failpoint before commit")+ }++ b := newBatch()+ b.guard("NOT EXISTS (SELECT 1 FROM space_member WHERE user_id = ? AND status <> 'INVITED')", delete.ID)+ // The memo and attachment sets were read outside the batch; abort when+ // they changed so nothing authored by the user survives its deletion.+ b.guard("(SELECT COUNT(*) FROM memo WHERE creator_id = ?) = ?", delete.ID, len(targets.memoIDs))+ b.guard("(SELECT COUNT(*) FROM attachment WHERE creator_id = ? OR memo_id IN (SELECT id FROM memo WHERE creator_id = ?)) = ?", delete.ID, delete.ID, len(targets.attachments))+ b.add("DELETE FROM space_member WHERE user_id = ? AND status = 'INVITED'", delete.ID)+ addMemoSetDeletes(b, targets.memoIDs, attachmentIDs(targets.attachments))+ b.add("DELETE FROM reaction WHERE creator_id = ?", delete.ID)+ b.add("DELETE FROM memo_share WHERE creator_id = ?", delete.ID)+ // Inbox rows are removed by their participant columns directly; there is+ // no reason to pre-read ids the caller never sees.+ b.add("DELETE FROM inbox WHERE sender_id = ? OR receiver_id = ?", delete.ID, delete.ID)+ b.add("DELETE FROM user_identity WHERE user_id = ?", delete.ID)+ b.add("DELETE FROM user_setting WHERE user_id = ?", delete.ID)+ b.add("DELETE FROM user WHERE id = ?", delete.ID)+ if _, err := b.commit(ctx, d); err != nil {+ if isGuardFailure(err) {+ return nil, spaceRecheck(validate, errUserDeleteConflict)+ }+ return nil, errors.Wrap(err, "failed to delete user")+ }+ return &store.DeleteUserResult{+ Attachments: targets.attachments,+ UserSettingKeys: targets.settingKeys,+ }, nil+}diff --git a/store/db/d1/user_identity.go b/store/db/d1/user_identity.gonew file mode 100644index 000000000..98d5301ee--- /dev/null+++ b/store/db/d1/user_identity.go@@ -0,0 +1,98 @@+package d1++import (+ "context"+ "strings"++ "github.com/pkg/errors"++ "github.com/usememos/memos/store"+)++const userIdentityColumns = "id, user_id, provider, extern_uid, created_ts, updated_ts"++// CreateUserIdentity links an external identity to a user.+func (d *DB) CreateUserIdentity(ctx context.Context, create *store.UserIdentity) (*store.UserIdentity, error) {+ query := "INSERT INTO user_identity (user_id, provider, extern_uid) VALUES (?, ?, ?) RETURNING id, created_ts, updated_ts"+ if err := d.db.QueryRowContext(ctx, query, create.UserID, create.Provider, create.ExternUID).Scan(&create.ID, &create.CreatedTs, &create.UpdatedTs); err != nil {+ return nil, errors.Wrap(err, "failed to create user identity")+ }+ return create, nil+}++// CreateUserWithIdentity inserts both rows in one batch. The identity cannot+// use the id generated by the user insert, because a batch never reads its+// own writes, so it resolves the user through the unique username and both+// rows are read back after the commit.+func (d *DB) CreateUserWithIdentity(ctx context.Context, createUser *store.User, createIdentity *store.UserIdentity) (*store.User, error) {+ b := newBatch()+ b.add("INSERT INTO user (username, role, email, nickname, password_hash, avatar_url) VALUES (?, ?, ?, ?, ?, ?)",+ createUser.Username, createUser.Role, nullableEmail(createUser.Email), createUser.Nickname, createUser.PasswordHash, createUser.AvatarURL)+ b.add("INSERT INTO user_identity (user_id, provider, extern_uid) SELECT id, ?, ? FROM user WHERE username = ?",+ createIdentity.Provider, createIdentity.ExternUID, createUser.Username)+ if _, err := b.commit(ctx, d); err != nil {+ return nil, errors.Wrap(err, "failed to create user with identity")+ }+ if err := d.db.QueryRowContext(ctx, "SELECT id, description, created_ts, updated_ts, row_status FROM user WHERE username = ?", createUser.Username).Scan(+ &createUser.ID, &createUser.Description, &createUser.CreatedTs, &createUser.UpdatedTs, &createUser.RowStatus,+ ); err != nil {+ return nil, errors.Wrap(err, "failed to read created user")+ }+ if err := d.db.QueryRowContext(ctx, "SELECT id, user_id, created_ts, updated_ts FROM user_identity WHERE provider = ? AND extern_uid = ?", createIdentity.Provider, createIdentity.ExternUID).Scan(+ &createIdentity.ID, &createIdentity.UserID, &createIdentity.CreatedTs, &createIdentity.UpdatedTs,+ ); err != nil {+ return nil, errors.Wrap(err, "failed to read created user identity")+ }+ return createUser, nil+}++// ListUserIdentities returns the identities matching find.+func (d *DB) ListUserIdentities(ctx context.Context, find *store.FindUserIdentity) ([]*store.UserIdentity, error) {+ where, args := []string{"1 = 1"}, []any{}+ if find.ID != nil {+ where, args = append(where, "id = ?"), append(args, *find.ID)+ }+ if find.UserID != nil {+ where, args = append(where, "user_id = ?"), append(args, *find.UserID)+ }+ if find.Provider != nil {+ where, args = append(where, "provider = ?"), append(args, *find.Provider)+ }+ if find.ExternUID != nil {+ where, args = append(where, "extern_uid = ?"), append(args, *find.ExternUID)+ }+ rows, err := d.db.QueryContext(ctx, "SELECT "+userIdentityColumns+" FROM user_identity WHERE "+strings.Join(where, " AND ")+" ORDER BY id ASC", args...)+ if err != nil {+ return nil, err+ }+ defer rows.Close()++ list := []*store.UserIdentity{}+ for rows.Next() {+ identity := &store.UserIdentity{}+ if err := rows.Scan(&identity.ID, &identity.UserID, &identity.Provider, &identity.ExternUID, &identity.CreatedTs, &identity.UpdatedTs); err != nil {+ return nil, err+ }+ list = append(list, identity)+ }+ if err := rows.Err(); err != nil {+ return nil, err+ }+ return list, nil+}++// DeleteUserIdentities removes the identities matching delete.+func (d *DB) DeleteUserIdentities(ctx context.Context, delete *store.DeleteUserIdentity) error {+ where, args := []string{"1 = 1"}, []any{}+ if delete.ID != nil {+ where, args = append(where, "id = ?"), append(args, *delete.ID)+ }+ if delete.UserID != nil {+ where, args = append(where, "user_id = ?"), append(args, *delete.UserID)+ }+ if delete.Provider != nil {+ where, args = append(where, "provider = ?"), append(args, *delete.Provider)+ }+ _, err := d.execOne(ctx, "DELETE FROM user_identity WHERE "+strings.Join(where, " AND "), args...)+ return err+}diff --git a/store/db/d1/user_setting.go b/store/db/d1/user_setting.gonew file mode 100644index 000000000..0a4083937--- /dev/null+++ b/store/db/d1/user_setting.go@@ -0,0 +1,93 @@+package d1++import (+ "context"+ "strings"++ "github.com/pkg/errors"++ storepb "github.com/usememos/memos/proto/gen/store"+ "github.com/usememos/memos/store"+)++// UpsertUserSetting inserts or replaces a user setting.+func (d *DB) UpsertUserSetting(ctx context.Context, upsert *store.UserSetting) (*store.UserSetting, error) {+ query := `INSERT INTO user_setting (user_id, key, value) VALUES (?, ?, ?)+ ON CONFLICT(user_id, key) DO UPDATE SET value = EXCLUDED.value`+ if _, err := d.execOne(ctx, query, upsert.UserID, upsert.Key.String(), upsert.Value); err != nil {+ return nil, err+ }+ return upsert, nil+}++// ListUserSettings returns the user settings matching find.+func (d *DB) ListUserSettings(ctx context.Context, find *store.FindUserSetting) ([]*store.UserSetting, error) {+ where, args := []string{"1 = 1"}, []any{}+ if v := find.Key; v != storepb.UserSetting_KEY_UNSPECIFIED {+ where, args = append(where, "key = ?"), append(args, v.String())+ }+ if v := find.UserID; v != nil {+ where, args = append(where, "user_id = ?"), append(args, *v)+ }+ rows, err := d.db.QueryContext(ctx, "SELECT user_id, key, value FROM user_setting WHERE "+strings.Join(where, " AND "), args...)+ if err != nil {+ return nil, err+ }+ defer rows.Close()++ list := make([]*store.UserSetting, 0)+ for rows.Next() {+ setting := &store.UserSetting{}+ var keyString string+ if err := rows.Scan(&setting.UserID, &keyString, &setting.Value); err != nil {+ return nil, err+ }+ setting.Key = storepb.UserSetting_Key(storepb.UserSetting_Key_value[keyString])+ list = append(list, setting)+ }+ if err := rows.Err(); err != nil {+ return nil, err+ }+ return list, nil+}++// DeleteUserSettings removes the user settings matching delete.+func (d *DB) DeleteUserSettings(ctx context.Context, delete *store.DeleteUserSetting) error {+ where, args := []string{"1 = 1"}, []any{}+ if v := delete.Key; v != storepb.UserSetting_KEY_UNSPECIFIED {+ where, args = append(where, "key = ?"), append(args, v.String())+ }+ if v := delete.UserID; v != nil {+ where, args = append(where, "user_id = ?"), append(args, *v)+ }+ _, err := d.execOne(ctx, "DELETE FROM user_setting WHERE "+strings.Join(where, " AND "), args...)+ return err+}++// GetUserByPATHash locates the setting row holding the token through D1's+// JSON1 functions and then matches the token in Go.+func (d *DB) GetUserByPATHash(ctx context.Context, tokenHash string) (*store.PATQueryResult, error) {+ query := `SELECT user_setting.user_id, user_setting.value+ FROM user_setting+ WHERE user_setting.key = 'PERSONAL_ACCESS_TOKENS'+ AND EXISTS (+ SELECT 1+ FROM json_each(json_extract(user_setting.value, '$.tokens')) AS token+ WHERE json_extract(token.value, '$.tokenHash') = ?+ )`+ var userID int32+ var tokensJSON string+ if err := d.db.QueryRowContext(ctx, query, tokenHash).Scan(&userID, &tokensJSON); err != nil {+ return nil, err+ }+ setting := &storepb.PersonalAccessTokensUserSetting{}+ if err := protojsonUnmarshaler.Unmarshal([]byte(tokensJSON), setting); err != nil {+ return nil, err+ }+ for _, pat := range setting.Tokens {+ if pat.TokenHash == tokenHash {+ return &store.PATQueryResult{UserID: userID, PAT: pat}, nil+ }+ }+ return nil, errors.New("PAT not found")+}diff --git a/store/db/db.go b/store/db/db.goindex 586a434e6..ea3573654 100644--- a/store/db/db.go+++ b/store/db/db.go@@ -5,6 +5,7 @@ import ("github.com/usememos/memos/internal/profile""github.com/usememos/memos/store"+ "github.com/usememos/memos/store/db/d1""github.com/usememos/memos/store/db/mysql""github.com/usememos/memos/store/db/postgres""github.com/usememos/memos/store/db/sqlite"@@ -22,6 +23,8 @@ func NewDBDriver(profile *profile.Profile) (store.Driver, error) {driver, err = mysql.NewDB(profile)case "postgres":driver, err = postgres.NewDB(profile)+ case "d1":+ driver, err = d1.NewDB(profile)default:return nil, errors.New("unknown db driver")}diff --git a/store/doc.go b/store/doc.goindex 616e17898..c377486a1 100644--- a/store/doc.go+++ b/store/doc.go@@ -1,5 +1,5 @@// Package store is the persistence layer: the Store facade, the Driver-// interface implemented by store/db/{sqlite,mysql,postgres}, migrations, seed+// interface implemented by store/db/{sqlite,mysql,postgres,d1}, migrations, seed// data, and the in-memory cache.//// Layering: store may import provider, markdown, filter, proto/gen, anddiff --git a/store/migration/d1/LATEST.sql b/store/migration/d1/LATEST.sqlnew file mode 100644index 000000000..6c8b718fa--- /dev/null+++ b/store/migration/d1/LATEST.sql@@ -0,0 +1,164 @@+-- Schema for Cloudflare D1. D1 runs SQLite, enforces foreign keys by default,+-- and has no interactive transactions; see store/db/d1/doc comments.++-- d1_guard backs conditional writes inside atomic batches: inserting 0+-- violates the CHECK and aborts the whole batch. It never holds rows.+CREATE TABLE d1_guard (+ ok INTEGER NOT NULL CONSTRAINT d1_guard_ok CHECK (ok = 1)+);++-- system_setting+CREATE TABLE system_setting (+ name TEXT NOT NULL,+ value TEXT NOT NULL,+ description TEXT NOT NULL DEFAULT '',+ UNIQUE(name)+);++-- user+CREATE TABLE user (+ id INTEGER PRIMARY KEY AUTOINCREMENT,+ created_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),+ updated_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),+ row_status TEXT NOT NULL CHECK (row_status IN ('NORMAL', 'ARCHIVED')) DEFAULT 'NORMAL',+ username TEXT COLLATE BINARY NOT NULL UNIQUE,+ role TEXT NOT NULL DEFAULT 'USER',+ email TEXT COLLATE BINARY DEFAULT NULL,+ nickname TEXT NOT NULL DEFAULT '',+ password_hash TEXT NOT NULL,+ avatar_url TEXT NOT NULL DEFAULT '',+ description TEXT NOT NULL DEFAULT ''+);++CREATE UNIQUE INDEX idx_user_email ON user(email);++-- user_setting+CREATE TABLE user_setting (+ user_id INTEGER NOT NULL,+ key TEXT NOT NULL,+ value TEXT NOT NULL,+ UNIQUE(user_id, key)+);++-- space+CREATE TABLE space (+ id INTEGER PRIMARY KEY AUTOINCREMENT,+ uid TEXT NOT NULL UNIQUE,+ title TEXT NOT NULL,+ description TEXT NOT NULL DEFAULT '',+ payload TEXT NOT NULL DEFAULT '{}'+);++-- space membership+CREATE TABLE space_member (+ space_id INTEGER NOT NULL,+ user_id INTEGER NOT NULL,+ status TEXT NOT NULL,+ role TEXT NOT NULL CHECK (role IN ('ADMIN', 'USER')),+ PRIMARY KEY (space_id, user_id)+);++CREATE INDEX idx_space_member_user_id ON space_member(user_id, space_id);++-- memo+CREATE TABLE memo (+ id INTEGER PRIMARY KEY AUTOINCREMENT,+ uid TEXT NOT NULL UNIQUE,+ creator_id INTEGER NOT NULL,+ created_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),+ updated_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),+ row_status TEXT NOT NULL CHECK (row_status IN ('NORMAL', 'ARCHIVED')) DEFAULT 'NORMAL',+ content TEXT NOT NULL DEFAULT '',+ visibility TEXT NOT NULL CHECK (visibility IN ('PUBLIC', 'PROTECTED', 'PRIVATE', 'SPACE')) DEFAULT 'PRIVATE',+ pinned INTEGER NOT NULL CHECK (pinned IN (0, 1)) DEFAULT 0,+ payload TEXT NOT NULL DEFAULT '{}',+ space_id INTEGER DEFAULT NULL+);++CREATE INDEX idx_memo_creator_id ON memo(creator_id);+CREATE INDEX idx_memo_space_id ON memo(space_id, row_status, created_ts DESC, id DESC);++-- memo_relation+CREATE TABLE memo_relation (+ memo_id INTEGER NOT NULL,+ related_memo_id INTEGER NOT NULL,+ type TEXT NOT NULL,+ UNIQUE(memo_id, related_memo_id, type)+);++CREATE INDEX idx_memo_relation_related_type_memo+ ON memo_relation(related_memo_id, type, memo_id);++-- attachment+CREATE TABLE attachment (+ id INTEGER PRIMARY KEY AUTOINCREMENT,+ uid TEXT NOT NULL UNIQUE,+ creator_id INTEGER NOT NULL,+ created_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),+ updated_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),+ filename TEXT NOT NULL DEFAULT '',+ blob BLOB DEFAULT NULL,+ type TEXT NOT NULL DEFAULT '',+ size INTEGER NOT NULL DEFAULT 0,+ memo_id INTEGER,+ storage_type TEXT NOT NULL DEFAULT '',+ reference TEXT NOT NULL DEFAULT '',+ payload TEXT NOT NULL DEFAULT '{}'+);++-- idp+CREATE TABLE idp (+ id INTEGER PRIMARY KEY AUTOINCREMENT,+ uid TEXT NOT NULL UNIQUE,+ name TEXT NOT NULL,+ type TEXT NOT NULL,+ identifier_filter TEXT NOT NULL DEFAULT '',+ config TEXT NOT NULL DEFAULT '{}'+);++-- inbox+CREATE TABLE inbox (+ id INTEGER PRIMARY KEY AUTOINCREMENT,+ created_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),+ sender_id INTEGER NOT NULL,+ receiver_id INTEGER NOT NULL,+ status TEXT NOT NULL,+ message TEXT NOT NULL DEFAULT '{}'+);++-- memo reaction+CREATE TABLE reaction (+ id INTEGER PRIMARY KEY AUTOINCREMENT,+ created_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),+ creator_id INTEGER NOT NULL,+ memo_id INTEGER NOT NULL,+ reaction_type TEXT NOT NULL,+ UNIQUE(creator_id, memo_id, reaction_type)+);++-- memo_share+CREATE TABLE memo_share (+ id INTEGER PRIMARY KEY AUTOINCREMENT,+ uid TEXT NOT NULL UNIQUE,+ memo_id INTEGER NOT NULL,+ creator_id INTEGER NOT NULL,+ created_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),+ expires_ts BIGINT DEFAULT NULL,+ FOREIGN KEY (memo_id) REFERENCES memo(id) ON DELETE CASCADE+);++CREATE INDEX idx_memo_share_memo_id ON memo_share(memo_id);++-- user_identity+CREATE TABLE user_identity (+ id INTEGER PRIMARY KEY AUTOINCREMENT,+ user_id INTEGER NOT NULL,+ provider TEXT NOT NULL,+ extern_uid TEXT NOT NULL,+ created_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),+ updated_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),+ UNIQUE (provider, extern_uid),+ UNIQUE (user_id, provider)+);++CREATE INDEX idx_user_identity_user_id ON user_identity(user_id);diff --git a/store/migrator.go b/store/migrator.goindex 786d02b38..9cb3b7141 100644--- a/store/migrator.go+++ b/store/migrator.go@@ -323,11 +323,22 @@ func (s *Store) seed(ctx context.Context) error {}// GetCurrentSchemaVersion returns the latest schema version available for the configured database driver.+//+// A driver introduced after the last schema change ships only LATEST.sql and+// no versioned migration files. LATEST.sql describes the same schema on every+// driver, so such a driver reports the highest version any driver's migration+// directory defines and picks up future migrations from there.func (s *Store) GetCurrentSchemaVersion() (string, error) {filePaths, err := fs.Glob(migrationFS, fmt.Sprintf("%s*/*.sql", s.getMigrationBasePath()))if err != nil {return "", errors.Wrap(err, "failed to read migration files")}+ if len(filePaths) == 0 {+ filePaths, err = fs.Glob(migrationFS, "migration/*/*/*.sql")+ if err != nil {+ return "", errors.Wrap(err, "failed to read migration files")+ }+ }if len(filePaths) == 0 {return defaultSchemaVersion, nil}diff --git a/store/test/README.md b/store/test/README.mdindex 988c63177..48f16825f 100644--- a/store/test/README.md+++ b/store/test/README.md@@ -11,3 +11,17 @@ DRIVER=mysql DSN=root@/memos_test go test -v ./test/store/...- `DRIVER` should be set to `mysql`.- `DSN` should be set to the DSN of your MySQL server.++## How to test store with Cloudflare D1?++`DRIVER=d1` runs the suite against an in-process emulation of the D1 REST API+(`store/db/d1/d1test`), so no Cloudflare account is needed:++```go+DRIVER=d1 go test -v ./store/test/...+```++To run against a real database instead, set `D1_DSN` to+`d1://<account_id>/<database_id>?token=<api_token>`. That database is shared+by every test and is not reset between them, so run one test at a time with+`-run` and expect to clean it up afterwards.diff --git a/store/test/main_test.go b/store/test/main_test.goindex 1a1139f19..444c8c59d 100644--- a/store/test/main_test.go+++ b/store/test/main_test.go@@ -6,6 +6,8 @@ import ("os/exec""path/filepath""runtime"+ "slices"+ "strings""testing")@@ -22,7 +24,7 @@ func TestMain(m *testing.M) {}func runAllDrivers() {- drivers := []string{"sqlite", "mysql", "postgres"}+ drivers := []string{"sqlite", "mysql", "postgres", "d1"}_, currentFile, _, _ := runtime.Caller(0)projectRoot := filepath.Dir(filepath.Dir(filepath.Dir(currentFile)))@@ -32,7 +34,11 @@ func runAllDrivers() {cmd := exec.Command("go", "test", "-v", "-count=1", "./store/test/...")cmd.Dir = projectRoot- cmd.Env = append(os.Environ(), "DRIVER="+driver)+ // A D1_DSN names one shared database; the loop's parallel tests need+ // the per-test emulator instead, so it is dropped from the child.+ env := slices.DeleteFunc(os.Environ(), func(entry string) bool { return strings.HasPrefix(entry, "D1_DSN=") })+ env = append(env, "DRIVER="+driver)+ cmd.Env = envcmd.Stdout = os.Stdoutcmd.Stderr = os.Stderrdiff --git a/store/test/memo_filter_test.go b/store/test/memo_filter_test.goindex e159b98ae..0c99525ef 100644--- a/store/test/memo_filter_test.go+++ b/store/test/memo_filter_test.go@@ -72,6 +72,7 @@ func TestMemoFilterContentUnicode(t *testing.T) {func TestMemoFilterContentUnicodeCaseFold(t *testing.T) {t.Parallel()+ skipUnicodeFoldOnD1(t)tc := NewMemoFilterTestContext(t)defer tc.Close()@@ -179,6 +180,7 @@ func TestMemoFilterStartsWithCombinedAndNegated(t *testing.T) {func TestMemoFilterContentMatchesAdvanced(t *testing.T) {t.Parallel()+ skipRegexFiltersOnD1(t)tc := NewMemoFilterTestContext(t)defer tc.Close()@@ -276,6 +278,7 @@ func TestMemoFilterContentEndsWith(t *testing.T) {func TestMemoFilterContentMatches(t *testing.T) {t.Parallel()+ skipRegexFiltersOnD1(t)tc := NewMemoFilterTestContext(t)defer tc.Close()diff --git a/store/test/migrator_test.go b/store/test/migrator_test.goindex 8525ae111..15c185d47 100644--- a/store/test/migrator_test.go+++ b/store/test/migrator_test.go@@ -153,6 +153,7 @@ func TestMigrationWithData(t *testing.T) {func TestMigrationMultiSpacesPreservesMemosAndRelations(t *testing.T) {ctx := context.Background()driver := getDriverFromEnv()+ skipWithoutLegacySchema(t, driver)dsn := getTestingProfileForDriver(t, driver).DSNdb, err := sql.Open(driver, dsn)require.NoError(t, err)@@ -294,6 +295,7 @@ func TestMigrationMultiSpacesPreservesMemosAndRelations(t *testing.T) {func TestMigrationSpaceMemberStatusBackfillsActive(t *testing.T) {ctx := context.Background()+ skipWithoutLegacySchema(t, getDriverFromEnv())ts := NewTestingStore(ctx, t)owner, err := createTestingHostUser(ctx, ts)@@ -430,6 +432,7 @@ func TestConcurrentInstanceAccessInitializationKeepsFirstInsert(t *testing.T) {func TestMigrationStorageSetting(t *testing.T) {ctx := context.Background()driver := getDriverFromEnv()+ skipWithoutLegacySchema(t, driver)legacyDatabase, err := protojson.Marshal(&storepb.InstanceStorageSetting{StorageType: storepb.InstanceStorageSetting_DATABASE,@@ -662,6 +665,7 @@ func assertStorageMigrationAttachments(ctx context.Context, t *testing.T, ts *stfunc TestMigrationReactionMemoID(t *testing.T) {ctx := context.Background()driver := getDriverFromEnv()+ skipWithoutLegacySchema(t, driver)dsn := getTestingProfileForDriver(t, driver).DSNdb, err := sql.Open(driver, dsn)@@ -735,6 +739,7 @@ func TestMigrationReactionMemoID(t *testing.T) {func TestMigrationLegacyS3Attachment(t *testing.T) {ctx := context.Background()driver := getDriverFromEnv()+ skipWithoutLegacySchema(t, driver)server := fakes3.New(t, "legacy-attachments")config := server.Config("legacy-attachments")content := []byte("attachment uploaded before named storage migration")@@ -824,6 +829,7 @@ func TestMigrationLegacyS3Attachment(t *testing.T) {func TestMigrationMemoViewSetting(t *testing.T) {ctx := context.Background()driver := getDriverFromEnv()+ skipWithoutLegacySchema(t, driver)var dsn stringswitch driver {case "sqlite":@@ -1008,6 +1014,7 @@ func TestMigrationCopiesInstanceTagsToUserSettings(t *testing.T) {func TestMigrationCaseSensitiveUsername(t *testing.T) {ctx := context.Background()driver := getDriverFromEnv()+ skipWithoutLegacySchema(t, driver)var dsn stringswitch driver {case "sqlite":@@ -1076,6 +1083,15 @@ func TestMigrationCaseSensitiveUsername(t *testing.T) {require.Equal(t, "Alice", upper.Username)}+// skipWithoutLegacySchema skips migration tests that start from a pre-0.31+// schema on drivers introduced after it, which have no legacy layout to migrate.+func skipWithoutLegacySchema(t *testing.T, driver string) {+ t.Helper()+ if driver == "d1" {+ t.Skip("D1 databases start at the current schema; there is no legacy schema to migrate")+ }+}+func legacySchemaFixture(driver string) string {switch driver {case "mysql":diff --git a/store/test/migrator_unique_email_test.go b/store/test/migrator_unique_email_test.goindex 7b16a87a0..163587bad 100644--- a/store/test/migrator_unique_email_test.go+++ b/store/test/migrator_unique_email_test.go@@ -21,6 +21,7 @@ import (func TestMigrationUniqueEmail(t *testing.T) {ctx := context.Background()driver := getDriverFromEnv()+ skipWithoutLegacySchema(t, driver)var dsn stringswitch driver {case "sqlite":diff --git a/store/test/migrator_upgrade_test.go b/store/test/migrator_upgrade_test.goindex bba268342..9bdd96c6d 100644--- a/store/test/migrator_upgrade_test.go+++ b/store/test/migrator_upgrade_test.go@@ -153,6 +153,9 @@ func prepareUpgradeFixture(t *testing.T, driver, version string) (MemosContainerDriver: driver,DSN: containerDSN,}, hostDSN+ case "d1":+ t.Skip("previous stable releases cannot run on D1; there is no schema to upgrade from")+ return MemosContainerConfig{}, ""default:t.Fatalf("unsupported driver: %s", driver)return MemosContainerConfig{}, ""diff --git a/store/test/reaction_test.go b/store/test/reaction_test.goindex adb25e8a1..2877d3047 100644--- a/store/test/reaction_test.go+++ b/store/test/reaction_test.go@@ -353,6 +353,9 @@ func TestReactionInsertedAfterMemoCleanupStartsIsRemoved(t *testing.T) {}func TestReactionUpsertBlocksWhileMemoDeleteIsUncommitted(t *testing.T) {+ if getDriverFromEnv() == "d1" {+ t.Skip("D1 has no interactive transactions, so an uncommitted delete cannot hold a lock")+ }setupContext := context.Background()ts := NewTestingStore(setupContext, t)t.Cleanup(func() {diff --git a/store/test/space_parent_lock_test.go b/store/test/space_parent_lock_test.goindex 7a8e61358..2791ec21f 100644--- a/store/test/space_parent_lock_test.go+++ b/store/test/space_parent_lock_test.go@@ -53,8 +53,8 @@ func TestSQLiteStaleRelationshipWriteCannotOutliveDeletedParent(t *testing.T) {func TestSpaceInvitationWaitsForConcurrentUserDelete(t *testing.T) {driver := getDriverFromEnv()- if driver == "sqlite" {- t.Skip("SQLite serializes the competing writes and rejects stale transaction upgrades")+ if driver == "sqlite" || driver == "d1" {+ t.Skip("SQLite serializes the competing writes and rejects stale transaction upgrades; D1 has no row locks")}setupCtx := context.Background()@@ -116,8 +116,8 @@ func TestSpaceInvitationWaitsForConcurrentUserDelete(t *testing.T) {func TestSpaceInvitationWaitsForConcurrentSpaceDelete(t *testing.T) {driver := getDriverFromEnv()- if driver == "sqlite" {- t.Skip("SQLite serializes the competing writes and rejects stale transaction upgrades")+ if driver == "sqlite" || driver == "d1" {+ t.Skip("SQLite serializes the competing writes and rejects stale transaction upgrades; D1 has no row locks")}setupCtx := context.Background()diff --git a/store/test/space_payload_test.go b/store/test/space_payload_test.goindex 1444dfdbe..21c67c205 100644--- a/store/test/space_payload_test.go+++ b/store/test/space_payload_test.go@@ -55,6 +55,7 @@ func TestSpacePayloadRoundTrip(t *testing.T) {func TestMigrationSpacePayloadBackfillsDefault(t *testing.T) {ctx := context.Background()+ skipWithoutLegacySchema(t, getDriverFromEnv())ts := NewTestingStore(ctx, t)defer ts.Close()owner, err := createTestingHostUser(ctx, ts)diff --git a/store/test/store.go b/store/test/store.goindex 54c3c202d..5b9a973a1 100644--- a/store/test/store.go+++ b/store/test/store.go@@ -16,12 +16,16 @@ import ("github.com/usememos/memos/internal/version""github.com/usememos/memos/store""github.com/usememos/memos/store/db"+ "github.com/usememos/memos/store/db/d1/d1test")// NewTestingStore creates a new testing store with a fresh database.// Each test gets its own isolated database:// - SQLite: new temp file per test// - MySQL/PostgreSQL: new database per test in shared container+// - D1: new emulated database per test (see store/db/d1/d1test), or the+// database named by D1_DSN when set; that database is shared, so run one+// test at a time against itfunc NewTestingStore(ctx context.Context, t *testing.T) *store.Store {driver := getDriverFromEnv()profile := getTestingProfileForDriver(t, driver)@@ -105,6 +109,11 @@ func getTestingProfileForDriver(t *testing.T, driver string) *profile.Profile {dsn = GetMySQLDSN(t)case "postgres":dsn = GetPostgresDSN(t)+ case "d1":+ dsn = os.Getenv("D1_DSN")+ if dsn == "" {+ dsn = d1test.New(t).DSN()+ }default:t.Fatalf("unsupported driver: %s", driver)}@@ -125,3 +134,22 @@ func getDriverFromEnv() string {}return driver}++// skipRegexFiltersOnD1 skips tests whose matches() patterns go beyond+// literal text with anchors: D1 has no REGEXP operator, so the filter engine+// only expresses those and rejects everything else at compile time.+func skipRegexFiltersOnD1(t *testing.T) {+ t.Helper()+ if getDriverFromEnv() == "d1" {+ t.Skip("only literal regular expressions are supported on Cloudflare D1")+ }+}++// skipUnicodeFoldOnD1 skips tests that need Unicode case folding: D1 cannot+// register custom functions, so it folds with the ASCII-only LOWER().+func skipUnicodeFoldOnD1(t *testing.T) {+ t.Helper()+ if getDriverFromEnv() == "d1" {+ t.Skip("Cloudflare D1 folds case with ASCII-only LOWER()")+ }+}