Privacy-first password manager with built-in email aliasing. Fully encrypted and self-hostable.
Add technical architecture docs for reference (#1404)
Leendert de Borst4d48d2fbc120
diff --git a/docs/technical-architecture/OFFLINE_MODE.md b/docs/technical-architecture/OFFLINE_MODE.mdnew file mode 100644index 000000000..607dad3d7--- /dev/null+++ b/docs/technical-architecture/OFFLINE_MODE.md@@ -0,0 +1,858 @@+# Offline Sync Implementation Specification++## Overview++This document defines the offline sync architecture for AliasVault clients. All clients (browser extension, mobile apps, web app) should follow these patterns to ensure consistent behavior and prevent data loss.++**Reference Implementation**: Browser extension (`apps/browser-extension`)++---++## Core Principles++1. **Local-first**: Always work with local vault, sync in background+2. **Never lose user data**: Local changes must survive network failures, app crashes, restarts+3. **LWW (Last-Write-Wins)**: Conflict resolution based on `UpdatedAt` timestamps+4. **Atomic metadata**: Vault blob and sync state stored together+5. **Server is source of truth for revision**: Server assigns revision numbers+6. **Race detection**: Mutation sequence counter prevents concurrent overwrites++---++## Storage Architecture++### Three Storage Layers++```+1. Persistent Storage (survives app restart):+ - Encrypted vault blob+ - Sync state (isDirty, mutationSequence, serverRevision)+ - Vault metadata (email domains)+ - Offline mode flag+ - Auth tokens & encryption params++2. Session Storage (cleared on app close):+ - Derived encryption key (SECURITY: never persist!)+ - Temporary form values++3. Memory (runtime only):+ - sqliteClient (in-memory database)+ - Cached vault blob (reduces decryption)+ - React/UI state mirrors+```++### Persistent Storage Keys++```+local:encryptedVault - Full encrypted SQLite blob+local:serverRevision - Last synced server revision number+local:isDirty - Boolean: true if local has unsynced changes+local:mutationSequence - Counter: increments on each local mutation+local:isOfflineMode - Boolean: true if currently offline+local:encryptionKeyDerivationParams - Salt/algorithm for key derivation+local:username - For display purposes+local:publicEmailDomains - Email domain lists+local:privateEmailDomains+local:hiddenPrivateEmailDomains+```++### Session Storage Keys++```+session:encryptionKey - Derived encryption key (SECURITY: never persist!)+```++---++## Critical: Atomic Vault Storage Pattern++**All vault storage operations MUST include sync flags atomically.**++This prevents a race condition where:+1. Vault saved locally+2. App crashes before `isDirty` is set+3. On restart, `isDirty=false` → sync overwrites local changes → **DATA LOSS**++### Two Storage Modes++The vault storage handler supports two distinct modes:++#### Mode 1: Local Mutation (markDirty=true)+```typescript+// After user creates/edits/deletes an item:+storeEncryptedVault({+ vaultBlob: encryptedVault,+ markDirty: true // Always succeeds, increments mutationSequence+});++// Atomically stores:+// - local:encryptedVault = vaultBlob+// - local:isDirty = true+// - local:mutationSequence++ (incremented)+```++#### Mode 2: Sync Operation (expectedMutationSeq provided)+```typescript+// After downloading/merging from server:+storeEncryptedVault({+ vaultBlob: encryptedVault,+ serverRevision: newRevision,+ expectedMutationSeq: capturedSequence // Race detection!+});++// Only succeeds if: currentMutationSeq === expectedMutationSeq+// Returns { success: false } if mismatch (concurrent mutation detected)+```++### Why Mutation Sequence?++The `mutationSequence` counter solves a critical race condition:++```+Problem:+ Time 1: Sync starts, captures mutationSequence=5+ Time 2: User edits item → mutationSequence=6+ Time 3: Sync tries to save server vault+ If we overwrite, user's edit is LOST!++Solution:+ Time 3: Sync tries to save with expectedMutationSeq=5+ But current is 6 → REJECTED+ Sync restarts with fresh data+```++### Sync State Truth Table++| Operation | isDirty | mutationSequence | serverRevision |+|------------------------------|---------|------------------|----------------|+| Fetch vault from server | `false` | unchanged | server's rev |+| Local mutation (any mode) | `true` | incremented | unchanged |+| Successful upload to server | `false`*| unchanged | new server rev |+| Failed upload (network) | `true` | unchanged | unchanged |+| Race detected during sync | `true` | unchanged | unchanged |+| Vault cleared (logout) | removed | removed | removed |++*Only cleared if no mutations occurred during upload (mutationSeq unchanged)++---++## Unlock Flow++**Key principle**: Always unlock from local vault first, then sync in background.++### Pseudocode++```+UNLOCK(password_or_pin):+ // 1. Derive encryption key+ if ONLINE:+ params = GET_ENCRYPTION_PARAMS_FROM_SERVER()+ STORE_ENCRYPTION_PARAMS(params)+ else:+ params = GET_STORED_ENCRYPTION_PARAMS()+ if params is null:+ ERROR("Cannot unlock offline without previous login")++ encryptionKey = DERIVE_KEY(password_or_pin, params)+ STORE_ENCRYPTION_KEY_IN_SESSION(encryptionKey)++ // 2. ALWAYS initialize from local vault first+ localVault = GET_ENCRYPTED_VAULT_FROM_LOCAL_STORAGE()+ decryptedVault = DECRYPT(localVault, encryptionKey)+ INITIALIZE_DATABASE(decryptedVault)++ // 3. Navigate to main app, sync in background+ NAVIGATE_TO_MAIN_VIEW()+ SYNC_VAULT_IN_BACKGROUND() // Non-blocking!+```++**Why unlock from local first?**+- User gets immediate access to their data+- Local changes are never lost+- Sync happens after unlock (can fail safely)+- UI doesn't block on network++---++## Sync Flow++**Key principles**:+1. Check `isDirty` before overwriting local vault+2. **Upload pending changes** - sync is responsible for uploading, not just downloading+3. **Race detection** - use mutation sequence to detect concurrent edits+4. **Recursive retry** - if race detected, restart sync++### Pseudocode++```+SYNC_VAULT():+ if not ONLINE:+ SET_OFFLINE_MODE(true)+ return SUCCESS // Continue with local vault++ SET_OFFLINE_MODE(false)++ // Capture state at start for race detection+ syncState = GET_SYNC_STATE()+ mutationSeqAtStart = syncState.mutationSequence++ serverRevision = GET_SERVER_REVISION()+ localRevision = syncState.serverRevision+ isDirty = syncState.isDirty++ if serverRevision > localRevision:+ // Server has newer vault - download it+ serverVault = FETCH_SERVER_VAULT()++ if isDirty:+ // CRITICAL: We have local changes - must merge!+ localVault = GET_LOCAL_ENCRYPTED_VAULT()+ mergedVault = LWW_MERGE(localVault, serverVault)++ // Store merged vault with race detection+ result = STORE_ENCRYPTED_VAULT({+ vaultBlob: mergedVault,+ expectedMutationSeq: mutationSeqAtStart+ })++ if not result.success:+ // Race detected - concurrent mutation happened+ return SYNC_VAULT() // Recursive retry++ // Upload merged vault to server+ uploadResponse = UPLOAD_VAULT(mergedVault)+ if uploadResponse.success:+ MARK_VAULT_CLEAN(uploadResponse.newRevision, mutationSeqAtStart)++ INITIALIZE_DATABASE(mergedVault)+ else:+ // No local changes - safe to overwrite+ result = STORE_ENCRYPTED_VAULT({+ vaultBlob: serverVault,+ serverRevision: serverRevision,+ expectedMutationSeq: mutationSeqAtStart+ })++ if not result.success:+ return SYNC_VAULT() // Race detected++ INITIALIZE_DATABASE(serverVault)++ else if serverRevision == localRevision:+ if isDirty:+ // Local changes at same revision - upload them!+ localVault = GET_LOCAL_ENCRYPTED_VAULT()+ uploadResponse = UPLOAD_VAULT(localVault, mutationSeqAtStart)++ if uploadResponse.success:+ MARK_VAULT_CLEAN(uploadResponse.newRevision, mutationSeqAtStart)+ else if uploadResponse.status == OUTDATED:+ // Another device uploaded - recurse to merge+ return SYNC_VAULT()+ // else: keep isDirty true, retry later+ else:+ // Already in sync - nothing to do+ pass++ return SUCCESS+```++### Mark Vault Clean Logic++```+MARK_VAULT_CLEAN(newServerRevision, mutationSeqAtStart):+ currentMutationSeq = GET_CURRENT_MUTATION_SEQUENCE()++ // Always update server revision+ SET_SERVER_REVISION(newServerRevision)++ // Only clear dirty flag if no mutations happened during sync+ if currentMutationSeq == mutationSeqAtStart:+ SET_IS_DIRTY(false)+ // else: keep dirty, another mutation needs syncing+```++### Critical: Sync Must Upload++A common mistake is to only handle downloads during sync. The sync flow **must also upload** pending local changes:++1. **After merge**: Upload merged vault to server immediately+2. **Same revision with pending**: Upload local vault to server+3. **Only clear `isDirty`** after successful upload AND no concurrent mutations++---++## Mutation Flow++**Key principle**: Save locally first, trigger sync in background. LWW merge handles conflicts.++### Two Mutation Patterns++#### Pattern 1: Blocking Mutation (with loading state)+```+MUTATE_VAULT_BLOCKING(operation):+ SET_IS_LOADING(true)++ // 1. Apply mutation to local database+ operation() // e.g., create/update/delete item++ // 2. Export and encrypt+ vaultBlob = EXPORT_DATABASE_TO_BASE64()+ encryptedVault = ENCRYPT(vaultBlob, encryptionKey)++ // 3. Store locally with dirty flag (ATOMIC!)+ STORE_ENCRYPTED_VAULT({+ vaultBlob: encryptedVault,+ markDirty: true+ })++ // 4. Sync and wait for completion+ await SYNC_VAULT()++ SET_IS_LOADING(false)+```++#### Pattern 2: Non-Blocking Mutation (async sync)+```+MUTATE_VAULT_ASYNC(operation):+ // 1. Apply mutation to local database+ operation()++ // 2. Export and encrypt+ vaultBlob = EXPORT_DATABASE_TO_BASE64()+ encryptedVault = ENCRYPT(vaultBlob, encryptionKey)++ // 3. Store locally (ATOMIC!)+ STORE_ENCRYPTED_VAULT({+ vaultBlob: encryptedVault,+ markDirty: true+ })++ // 4. Trigger sync in background (fire-and-forget)+ TRIGGER_BACKGROUND_SYNC()+ // UI unblocks immediately, user can continue+```++### Background Sync Queue++```+TRIGGER_BACKGROUND_SYNC():+ if SYNC_IN_PROGRESS:+ return // Let current sync pick up changes++ SET_SYNC_IN_PROGRESS(true)++ SYNC_VAULT()++ // After sync, check if more mutations happened+ if IS_DIRTY:+ TRIGGER_BACKGROUND_SYNC() // Recursive: sync again++ SET_SYNC_IN_PROGRESS(false)+```++### Why No Pre-Sync?++Previous versions would sync before mutations to ensure working with the latest vault. This is unnecessary because:++1. **LWW merge resolves conflicts** - Even if mutating a stale vault, the merge will pick the record with the latest `UpdatedAt`+2. **Popup open triggers sync** - Vault is reasonably fresh when user starts interacting+3. **Simpler mental model** - Mutation = save + sync, that's it+4. **Fewer edge cases** - No pre-sync failure handling needed++---++## In-Memory Cache Strategy++To avoid repeated decryption, the implementation uses an in-memory cache:++```+CACHE:+ cachedSqliteClient: SqliteClient | null+ cachedVaultBlob: string | null++ON_LOCAL_MUTATION:+ // Don't clear cache - local mutations work directly on cached client+ cachedSqliteClient.execute(mutation)+ EXPORT_AND_STORE_ENCRYPTED()++ON_EXTERNAL_UPDATE (login, sync download, logout):+ // Clear cache - external data needs re-initialization+ cachedSqliteClient = null+ cachedVaultBlob = null+ INITIALIZE_FROM_STORAGE()+```++**Why this matters:**+- Local mutations are fast (no decryption)+- External updates ensure consistency (fresh decrypt)+- Reduces CPU usage significantly++---++## LWW Merge Algorithm++### Core Rule++For each record, compare `UpdatedAt` timestamps. The record with the **later** timestamp wins.++### Merge Pseudocode++```+LWW_MERGE(localVault, serverVault):+ mergedVault = CLONE(serverVault) // Start with server as base++ for each TABLE in SYNCABLE_TABLES:+ localRecords = GET_RECORDS(localVault, TABLE)+ serverRecords = GET_RECORDS(serverVault, TABLE)++ // Build lookup for server records+ serverById = MAP(serverRecords, record => record.Id)++ for each localRecord in localRecords:+ serverRecord = serverById[localRecord.Id]++ if serverRecord is null:+ // Local-only record - keep it+ ADD_RECORD(mergedVault, TABLE, localRecord)++ else if localRecord.UpdatedAt > serverRecord.UpdatedAt:+ // Local is newer - use local+ UPDATE_RECORD(mergedVault, TABLE, localRecord)++ // else: server is newer or equal - already in merged (from clone)++ return mergedVault+```++### Syncable Tables++All entities extending `SyncableEntity`:+- `Item` - Main credential/login entity+- `FieldValue` - Field data per item+- `Folder` - Hierarchical folders+- `Tag` - User tags+- `ItemTag` - Item-tag relationships+- `Attachment` - File attachments+- `TotpCode` - TOTP secrets+- `Passkey` - WebAuthn credentials+- `FieldDefinition` - Custom field definitions+- `FieldHistory` - Field change history+- `Logo` - Deduplicated logos++### Special Cases++1. **Soft deletes are sticky**: `IsDeleted=true` should only be overwritten by a record with later `UpdatedAt` that explicitly sets `IsDeleted=false`++2. **DeletedAt handling**: For items moved to trash, `DeletedAt` timestamp determines trash state++3. **Child entities**: Merge independently - a newer parent doesn't automatically make children newer++---++## UI Indicators++### States to Display (Priority Order)++| Priority | State | Condition | UI |+|----------|-------|-----------|-----|+| 1 | **Offline** | `isOffline=true` | Amber badge: "Offline" |+| 2 | **Syncing** | `isSyncing=true` | Green badge with spinner: "Syncing vault..." |+| 3 | **Pending** | `isDirty=true` | Blue badge with spinner: "Pending sync" |+| 4 | **Synced** | all false | No indicator (hidden) |++### Important: Only show indicators when vault is unlocked++```typescript+// Don't show sync indicators when vault is locked+if (!isLoggedIn || !dbAvailable) {+ return null; // No indicator+}+```++### State Transitions++```+User makes edit:+ Synced → Pending → (sync starts) → Syncing → Synced++Network goes down:+ Synced → Offline++Network restored + pending changes:+ Offline → Pending → Syncing → Synced+```++---++## Offline Mode Behaviors++### Entry Conditions++- Server returns connection error (detected as version `0.0.0`)+- User must have local vault (`dbAvailable=true`)+- Otherwise throws error (cannot use app without vault)++### Offline Capabilities++- Create/edit/delete credentials locally+- All mutations increment `mutationSequence` and set `isDirty=true`+- UI shows "Offline" badge+- Can lock/unlock vault using stored encryption params+- Form values persist encrypted in session storage++### Return to Online++```+Network restored:+1. Next syncVault() call succeeds+2. Server revision > local?+ └─ Merge local with server+3. Upload merged/local vault+4. setIsOffline(false)+5. UI shows normal state+```++---++## Vault Pruning (Trash Auto-Cleanup)++Items moved to trash (`DeletedAt` set) are automatically pruned after 30 days. This happens as part of the sync flow, before uploading the vault to the server.++### Pruning Flow++```+BEFORE_UPLOAD():+ 1. Read Items, FieldValues, Attachments, TotpCodes, Passkeys tables+ 2. Find items where:+ - DeletedAt is set (in trash)+ - DeletedAt < (now - 30 days)+ - IsDeleted = false (not already permanently deleted)+ 3. For each expired item:+ - Set IsDeleted = true, UpdatedAt = now+ - Set IsDeleted = true for all related entities+ 4. Execute SQL statements returned by Rust+ 5. Continue with vault upload+```++### Implementation Details++- **Rust module**: `vault_pruner` in `core/rust/src/vault_pruner/mod.rs`+- **Browser extension**: Called in `VaultMessageHandler.ts` → `uploadNewVaultToServer()`+- **AliasVault.Client**: Called in `DbService.cs` → `PruneExpiredTrashItemsAsync()`+- **Retention period**: 30 days (configurable in Rust API)+- **Graceful failure**: If pruning fails, continues with upload (logs warning)++### Key Points++1. **Pruning is automatic**: Happens transparently during sync+2. **User can restore within 30 days**: Items in "Recently Deleted" can be restored+3. **Permanent after 30 days**: Items are marked with `IsDeleted = true` (tombstone)+4. **Cross-platform consistency**: Rust WASM handles logic for all clients++---++## Key Files (Browser Extension)++| File | Purpose |+|------|---------|+| `VaultMessageHandler.ts` | Background storage handlers with race detection, prune integration |+| `DbContext.tsx` | React context for vault state management |+| `useVaultSync.ts` | **Primary sync logic** - handles download, upload, merge, offline, race detection |+| `useVaultMutate.ts` | Mutation execution with blocking/async patterns |+| `VaultMergeService.ts` | LWW merge and prune implementation |+| `ServerSyncIndicator.tsx` | UI status indicator component |+| `Unlock.tsx` | Unlock flow (local-first) |+| `Reinitialize.tsx` | Post-unlock initialization and background sync trigger |+| `background.ts` | Message routing between popup and background |++---++## Testing Scenarios++### Basic Offline Flow+1. [ ] Go offline (disable network)+2. [ ] Make mutation (create/edit item)+3. [ ] Verify saved locally (indicator shows "Offline")+4. [ ] Lock vault+5. [ ] Go online+6. [ ] Unlock vault+7. [ ] Verify local changes preserved and synced++### Merge Conflict+1. [ ] Device A: Create item, sync+2. [ ] Device B: Sync to get item+3. [ ] Device A: Go offline, edit item+4. [ ] Device B: Edit same item, sync+5. [ ] Device A: Go online+6. [ ] Verify LWW merge (later edit wins)++### Crash Recovery+1. [ ] Make offline mutation+2. [ ] Force-kill app (simulate crash)+3. [ ] Reopen app+4. [ ] Verify `isDirty=true` preserved+5. [ ] Verify local changes still exist++### Unlock with Pending Changes+1. [ ] Go offline+2. [ ] Make mutation+3. [ ] Lock vault+4. [ ] Go online+5. [ ] Unlock vault+6. [ ] Verify merge happens before using server vault++### Race Condition (Concurrent Mutations)+1. [ ] Start sync operation (e.g., downloading large vault)+2. [ ] During sync, make local mutation+3. [ ] Verify sync detects race (mutationSeq changed)+4. [ ] Verify sync restarts and includes new mutation+5. [ ] Verify no data loss++---++## FieldValue Update Strategy (SqliteClient)++When updating an Item with FieldValues, the `updateItem()` method uses a **preserve-and-track** strategy to maintain stable FieldValue IDs for proper merge behavior.++### Why Stable IDs Matter++The merge service matches FieldValues by composite key (`ItemId + FieldKey`), but having stable IDs improves efficiency and reduces merge complexity. If we deleted all FieldValues and recreated them with new IDs on every save, the merge would work but create unnecessary churn.++### The Strategy++```+UPDATE_ITEM(item):+ 1. QUERY existing FieldValues for this item+ existingFields = SELECT * FROM FieldValues WHERE ItemId = ? AND IsDeleted = 0++ 2. BUILD lookup map by composite key+ existingByKey = MAP each field to "{FieldKey}:{index}"++ 3. TRACK which existing IDs we process+ processedIds = SET()++ 4. FOR EACH field in item.Fields:+ compositeKey = "{field.FieldKey}:{index}"+ existing = existingByKey[compositeKey]++ IF existing:+ // UPDATE in place - preserves the original FieldValue ID+ processedIds.add(existing.Id)+ IF value changed:+ UPDATE FieldValues SET Value = ?, UpdatedAt = ? WHERE Id = ?+ ELSE:+ // INSERT new FieldValue with random UUID+ INSERT INTO FieldValues (Id, ItemId, FieldKey, Value, ...)+ VALUES (randomUUID(), ...)++ 5. SOFT-DELETE removed fields+ FOR EACH existing in existingFields:+ IF existing.Id NOT IN processedIds:+ // Field was removed - soft delete it+ UPDATE FieldValues SET IsDeleted = 1, UpdatedAt = ? WHERE Id = ?+```++### Benefits++1. **Stable IDs for unchanged fields**: Fields that aren't modified keep their original UUID+2. **Efficient updates**: Only changed fields get new `UpdatedAt` timestamps+3. **Clean removal**: Removed fields are properly soft-deleted+4. **Merge-friendly**: Composite key matching in merge handles cases where IDs differ++### Example Scenario++```+Initial state: Item has FieldValues:+ - {Id: "A", FieldKey: "login.username", Value: "[email protected]"}+ - {Id: "B", FieldKey: "login.password", Value: "secret123"}++User edits only the password:++After updateItem():+ - {Id: "A", FieldKey: "login.username", Value: "[email protected]"} // UNCHANGED - same ID+ - {Id: "B", FieldKey: "login.password", Value: "newsecret456", UpdatedAt: NOW} // UPDATED in place+```++### Key Files++- `SqliteClient.ts` → `updateItem()` - Implements the preserve-and-track strategy+- `VaultMergeService.ts` → `mergeTableByCompositeKey()` - Handles merge by composite key++---++## Security Considerations++1. **Encryption key NEVER persisted** - Only in session storage+2. **Vault blob is encrypted** - Safe to persist locally+3. **User must re-authenticate after app close** - Session key is lost+4. **No plaintext in local storage** - All vault data encrypted+5. **Derivation params safe to store** - Salt alone cannot derive key++---++## Common Pitfalls to Avoid++### 1. Overwriting local changes+**Wrong:**+```typescript+if (serverRevision > localRevision) {+ initializeDatabase(serverVault); // Overwrites local changes!+}+```++**Correct:**+```typescript+if (serverRevision > localRevision) {+ if (isDirty) {+ mergedVault = merge(localVault, serverVault);+ initializeDatabase(mergedVault);+ } else {+ initializeDatabase(serverVault);+ }+}+```++### 2. Separate storage calls (non-atomic)+**Wrong:**+```typescript+await storage.setItem('local:encryptedVault', vault);+await storage.setItem('local:isDirty', true); // Crash here = lost flag!+```++**Correct:**+```typescript+await storage.setItems([+ { key: 'local:encryptedVault', value: vault },+ { key: 'local:isDirty', value: true },+ { key: 'local:mutationSequence', value: sequence + 1 }+]); // Atomic+```++### 3. Fetching server vault during unlock+**Wrong:**+```typescript+// Unlock.tsx+serverVault = await fetchFromServer();+initializeDatabase(serverVault); // Skips local!+```++**Correct:**+```typescript+// Unlock.tsx - always local first+localVault = await getLocalVault();+initializeDatabaseFromLocal(localVault);+// Then trigger background sync+triggerBackgroundSync();+```++### 4. Ignoring race conditions+**Wrong:**+```typescript+// Sync downloads server vault+const serverVault = await downloadVault();+// User edits during download (undetected!)+await storeVault(serverVault); // Overwrites user's edit!+```++**Correct:**+```typescript+const mutationSeqAtStart = await getMutationSequence();+const serverVault = await downloadVault();++const result = await storeVault({+ vault: serverVault,+ expectedMutationSeq: mutationSeqAtStart // Race detection!+});++if (!result.success) {+ return syncVault(); // Retry - will merge user's edit+}+```++### 5. Clearing dirty flag prematurely+**Wrong:**+```typescript+await uploadVault(localVault);+await setIsDirty(false); // What if user edited during upload?+```++**Correct:**+```typescript+const mutationSeqAtStart = getMutationSequence();+await uploadVault(localVault);++// Only clear if no mutations happened during upload+if (getCurrentMutationSequence() === mutationSeqAtStart) {+ await setIsDirty(false);+}+```++---++## Disaster Recovery & Data Loss Prevention++### Server RPO Recovery++AliasVault implements two disaster recovery mechanisms to prevent data loss:++1. **Vault Preservation on Forced Logout** - Prevents data loss when session is forcibly terminated+2. **Server Rollback Handling** - Enables recovery when server has lower revision than client++**For detailed implementation, see:** [SERVER_RPO_RECOVERY.md](./SERVER_RPO_RECOVERY.md)++### Quick Summary++#### Vault Preservation on Forced Logout++**Problem:** When forced logout occurs (401 Unauthorized), if local vault is deleted, unsynced changes are lost.++**Solution:**+- **User-initiated logout**: Check `isDirty`, show warning dialog if unsynced changes, clear all data+- **Forced logout**: Preserve vault data in place, only clear session data (tokens, encryption key)+- **On login**: Check for existing vault, try to decrypt with new key, compare revisions with server++Vault data is preserved in regular storage keys (no separate "orphaned" keys). On re-login, the sync flow handles recovery automatically.++#### Server Rollback Handling++**Problem:** When server rolls back to earlier revision (e.g., 95 < client 100), sync needs to recover server state.++**Solution:** Explicit handling in sync logic:++```typescript+if (serverRevision < clientRevision) {+ // Server data loss detected - upload client vault to recover+ uploadVault();+ // Server assigns newRevision = clientRevision + 1 (creates gap)+}+```++**Why gap is acceptable:**+- Provides audit trail of disaster recovery event+- Server history is admin-only+- Clients only care about latest revision++---++## Revision History++- **2025-12**: Initial implementation with atomic `hasPendingSync` flag+- **2025-12**: Centralized merge logic in `useVaultSync.ts`+- **2025-12**: Simplified unlock flow to always use local vault first+- **2025-12**: `OfflineIndicator` with proper locked state handling+- **2025-12**: Preserve-and-track strategy for FieldValue updates in `updateItem()`+- **2025-12**: Composite key matching for FieldValues merge in `VaultMergeService`+- **2025-12**: **Major update**: Renamed `hasPendingSync` to `isDirty` to match implementation+- **2025-12**: **`mutationSequence` counter** for race condition detection+- **2025-12**: In-memory cache strategy documentation+- **2025-12**: UI indicators section with priority order and `ServerSyncIndicator` component+- **2025-12**: Blocking vs non-blocking mutation patterns+- **2025-12**: Background sync queue logic+- **2025-12**: Expanded race condition documentation and pitfalls+- **2025-12**: Concurrent mutation test scenario+- **2025-12**: **`vault_pruner` module** for 30-day trash auto-cleanup+- **2025-12**: Renamed `merge` module to `vault_merge` for consistency+- **2025-12**: Removed old 7-day `VaultCleanupSoftDeletedRecords` in favor of 30-day prune+- **2025-12**: "Recently Deleted" page in AliasVault.Client with restore functionality+- **2026-01**: **Disaster recovery section** - Server RPO recovery and vault preservation on forced logoutdiff --git a/docs/technical-architecture/SERVER_RPO_RECOVERY.md b/docs/technical-architecture/SERVER_RPO_RECOVERY.mdnew file mode 100644index 000000000..9a1775a1c--- /dev/null+++ b/docs/technical-architecture/SERVER_RPO_RECOVERY.md@@ -0,0 +1,347 @@+# Server RPO Recovery & Vault Preservation++## Overview++This document describes two mechanisms to handle edge cases in vault synchronization:++1. **Vault Preservation on Forced Logout** - Prevents data loss when session is forcibly terminated+2. **Server Rollback Handling** - Enables recovery when server has lower revision than client++Both mechanisms address disaster recovery scenarios where server data loss (RPO - Recovery Point Objective) could result in client data being lost.++---++## Vault Preservation on Forced Logout++### Problem++When forced logout occurs (401 Unauthorized due to token revocation), if the client deletes the local vault immediately, any unsynced changes would be permanently lost.++### Solution++Two distinct logout flows handle this:++#### Flow A: User-Initiated Logout (Explicit button press)++**Behavior:**+1. Check if vault has unsynced changes (`isDirty = true`)+2. If dirty, show warning dialog asking user to confirm data loss+3. If user confirms, proceed with full vault deletion+4. If user cancels, abort logout++**Why:** User has agency to prevent data loss by syncing first or canceling logout.++#### Flow B: Forced Logout (401, token revocation, password change)++**Behavior:**+1. **Always** preserve encrypted vault + metadata (regardless of `isDirty` state)+2. Clear active session (tokens, current vault references)+3. Prefill username on login page as subtle hint++**Why:**+- User had no control over logout (forced by server)+- Vault might be more advanced than server (server rollback scenario)+- No security risk (vault is encrypted)+- Enables silent recovery on next login++### Storage Design++Forced logout **preserves the existing vault data in place** while clearing only session data. This allows recovery on re-login through normal sync flow.++#### Browser Extension Storage++**Vault Keys (preserved on forced logout):**+```typescript+'local:encryptedVault' // Encrypted vault blob - PRESERVED+'local:serverRevision' // Server revision number - PRESERVED+'local:isDirty' // Has unsynced changes - PRESERVED+'local:mutationSequence' // Mutation counter - PRESERVED+'local:username' // Username for login prefill - PRESERVED+'local:encryptionKeyDerivationParams' // Key derivation params - PRESERVED+```++**Session Keys (cleared on forced logout):**+```typescript+'local:accessToken' // JWT access token - CLEARED+'local:refreshToken' // JWT refresh token - CLEARED+'session:encryptionKey' // In-memory encryption key - CLEARED+```++#### Mobile App Storage (iOS/Android)++The mobile apps use native storage through `VaultStore` (Swift/Kotlin) which maintains:++**Persisted Storage (preserved on forced logout via `clearSession()`):**+- Encrypted vault database file+- Server revision number+- Dirty flag and mutation sequence+- Username (for login prefill)+- Encryption key derivation params++**Session Data (cleared on forced logout via `clearSession()`):**+- In-memory decryption key (keychain/keystore)+- Auth tokens+- In-memory decrypted vault reference++The native `clearSession()` method clears session data while preserving the persisted vault, whereas `clearVault()` clears everything including the persisted database.++### Login Flow with Silent Recovery++#### Browser Extension++When user logs in after a forced logout:++1. **Check for existing vault** in storage (`persistAndLoadVault()` in [Login.tsx](../../apps/browser-extension/src/entrypoints/popup/pages/auth/Login.tsx))+ - Reads `local:encryptedVault` and `local:serverRevision`+ - If no existing vault → Normal fresh login flow++2. **Attempt to decrypt existing vault** with derived encryption key+ - Uses `EncryptionUtility.symmetricDecrypt()` to verify vault can be decrypted+ - If decryption fails → Password was changed → Use server vault instead+ - If decryption succeeds → Continue to revision comparison++3. **Compare local revision with server revision**+ - If `localRev >= serverRev` → Keep local vault (don't overwrite), load existing decrypted vault+ - If `localRev < serverRev` → Server is more advanced → Download and use server vault++4. **Continue with normal sync flow**+ - The sync logic in `useVaultSync.ts` handles server rollback detection automatically+ - If `serverRevision < clientRevision`, sync uploads to recover server state++#### Mobile App (iOS/Android)++When user logs in after a forced logout:++1. **Check for existing vault** via `NativeVaultManager.hasEncryptedDatabase()` ([login.tsx](../../apps/mobile-app/app/login.tsx))+ - If no existing vault → Normal fresh login flow++2. **Attempt to unlock existing vault** via `NativeVaultManager.unlockVault()`+ - Native code attempts to decrypt vault using the newly derived encryption key+ - If decryption fails → Password was changed → Call `NativeVaultManager.clearVault()` to clear local vault+ - If decryption succeeds → Vault is valid, proceed to sync++3. **Sync handles revision comparison**+ - `syncVault()` is called which internally compares revisions+ - Native `VaultSync` (Swift/Kotlin) handles all cases:+ - `serverRev > localRev` → Download server vault+ - `serverRev == localRev && isDirty` → Upload local changes+ - `serverRev < localRev` → Server rollback detected → Upload to recover server+ - `serverRev == localRev && !isDirty` → Already in sync++### Recovery Decision Matrix++| Scenario | Vault Decrypt | Revision Comparison | Action |+|----------|---------------|---------------------|--------|+| Password unchanged, server rolled back | ✅ Success | `localRev >= serverRev` | Keep local vault → Sync uploads → **Data recovered!** |+| Password unchanged, server advanced | ✅ Success | `localRev < serverRev` | Use server vault (server is authoritative) |+| Password changed | ❌ Fails | N/A | Clear local vault, use server vault |+| No existing vault | N/A | N/A | Normal fresh login flow |++### Implementation Summary++#### Browser Extension++**Key files:**+- [AuthContext.tsx](../../apps/browser-extension/src/entrypoints/popup/context/AuthContext.tsx) - `clearAuthUserInitiated()` vs `clearAuthForced()`+- [AppContext.tsx](../../apps/browser-extension/src/entrypoints/popup/context/AppContext.tsx) - `logout()` calls `clearAuthForced()` to preserve vault+- [Settings.tsx](../../apps/browser-extension/src/entrypoints/popup/pages/settings/Settings.tsx) - User-initiated logout with dirty vault warning+- [Login.tsx](../../apps/browser-extension/src/entrypoints/popup/pages/auth/Login.tsx) - `persistAndLoadVault()` handles existing vault recovery+- [useVaultSync.ts](../../apps/browser-extension/src/entrypoints/popup/hooks/useVaultSync.ts) - Server rollback detection (lines 288-329)++#### Mobile App (Native iOS)++**Key files:**+- [VaultStore+Cache.swift](../../apps/mobile-app/ios/VaultStoreKit/VaultStore+Cache.swift) - `clearSession()` preserves vault, `clearVault()` clears all+- [VaultManager.swift](../../apps/mobile-app/ios/NativeVaultManager/VaultManager.swift) - React Native bridge for `clearSession()`+- [VaultStore+Sync.swift](../../apps/mobile-app/ios/VaultStoreKit/VaultStore+Sync.swift) - Server rollback detection (lines 229-241)++#### Mobile App (Native Android)++**Key files:**+- [VaultCache.kt](../../apps/mobile-app/android/app/src/main/java/net/aliasvault/app/vaultstore/VaultCache.kt) - `clearSession()` preserves vault+- [VaultStore.kt](../../apps/mobile-app/android/app/src/main/java/net/aliasvault/app/vaultstore/VaultStore.kt) - Exposes `clearSession()`+- [NativeVaultManager.kt](../../apps/mobile-app/android/app/src/main/java/net/aliasvault/app/nativevaultmanager/NativeVaultManager.kt) - React Native bridge+- [VaultSync.kt](../../apps/mobile-app/android/app/src/main/java/net/aliasvault/app/vaultstore/VaultSync.kt) - Server rollback detection (lines 117-131)++#### Mobile App (React Native)++**Key files:**+- [NativeVaultManager.ts](../../apps/mobile-app/specs/NativeVaultManager.ts) - `clearSession()` spec interface+- [AuthContext.tsx](../../apps/mobile-app/context/AuthContext.tsx) - `clearAuthUserInitiated()` vs `clearAuthForced()`+- [AppContext.tsx](../../apps/mobile-app/context/AppContext.tsx) - `logout()` calls `clearAuthForced()`+- [settings/index.tsx](../../apps/mobile-app/app/(tabs)/settings/index.tsx) - User-initiated logout with dirty vault warning+- [login.tsx](../../apps/mobile-app/app/login.tsx) - Existing vault recovery check (lines 196-210)++---++## Server Rollback Handling++### Problem++When server experiences data loss and recovers to an earlier revision (e.g., server at rev 95, client at rev 100), the sync logic needs to handle the `serverRevision < clientRevision` case to recover server state.++### Solution++Explicit handling for `serverRevision < clientRevision` case:++**Behavior:**+1. Detect server revision rollback (server is behind client)+2. Force upload client vault to recover server state+3. Server assigns new revision number (client's rev + 1, creating gap)+4. Client updates local `serverRevision` to new value+5. Log recovery event for audit trail++**Why this works:**+- Server already accepts advanced client revisions+- Server calculates `newRevisionNumber = clientRevision + 1`+- Creates revision gap (e.g., 95 → 101) which serves as audit trail+- Gap is visible to admins, indicates disaster recovery event++### Server Revision Assignment Logic++Looking at `apps/server/AliasVault.Api/Controllers/VaultController.cs:162-170`:++```csharp+// Server calculates new revision number+var newRevisionNumber = model.CurrentRevisionNumber + 1;++// Checks if server is already ahead+if (latestVault.RevisionNumber >= newRevisionNumber)+{+ return Ok(new VaultUpdateResponse {+ Status = VaultStatus.Outdated,+ NewRevisionNumber = latestVault.RevisionNumber+ });+}++// Otherwise accepts and saves with calculated revision+```++**Example:**+- Client sends `currentRevisionNumber = 100`+- Server calculates `newRevisionNumber = 100 + 1 = 101`+- Server checks: `latestVault.RevisionNumber (95) >= 101`? **NO**+- Server accepts and saves as **revision 101** ✅+- Creates gap: 95 → 101 (revisions 96-100 missing from server history)++**Why gap is acceptable:**+- Provides audit trail of disaster recovery event+- Server history is admin-only, not shared with clients+- Clients only care about latest revision+- Gap timestamp shows when recovery occurred++### Implementation++All platforms detect server rollback (`serverRevision < clientRevision`) and automatically upload to recover server state:++- **Browser Extension**: [useVaultSync.ts](../../apps/browser-extension/src/entrypoints/popup/hooks/useVaultSync.ts) (lines 288-329)+- **Android**: [VaultSync.kt](../../apps/mobile-app/android/app/src/main/java/net/aliasvault/app/vaultstore/VaultSync.kt) (lines 117-131)+- **iOS**: [VaultStore+Sync.swift](../../apps/mobile-app/ios/VaultStoreKit/VaultStore+Sync.swift) (lines 229-241)++---++## Testing Checklist++### Vault Preservation on Forced Logout++**Test Cases:**++- [ ] **User logout (clean vault)**+ - Vault has no unsynced changes+ - Click logout button in Settings+ - Expected: Normal confirmation dialog, vault cleared after logout++- [ ] **User logout (dirty vault, cancel)**+ - Make local change (vault becomes dirty - indicator shows)+ - Click logout button in Settings+ - Expected: Warning dialog appears about unsynced changes+ - Click "Cancel"+ - Expected: Logout aborted, still logged in, vault preserved++- [ ] **User logout (dirty vault, confirm)**+ - Make local change (vault becomes dirty)+ - Click logout button in Settings+ - Expected: Warning dialog appears+ - Click "Log out anyway"+ - Expected: Vault cleared, logged out++- [ ] **Forced logout (password unchanged)**+ - Simulate 401 response (e.g., revoke tokens on server)+ - Expected: Vault data preserved on device+ - Login with same username and password+ - Expected: Existing vault loaded, sync resumes normally++- [ ] **Forced logout (password changed)**+ - Change password on another device+ - First device receives 401+ - Expected: Vault data preserved (encrypted)+ - Login with new password+ - Expected: Old vault can't decrypt → Use server vault instead++- [ ] **Forced logout + server rollback**+ - Client at rev 100, server recovers to rev 95+ - Forced logout due to token expiry+ - Expected: Vault preserved with rev 100+ - Login+ - Expected: Local vault (rev 100) detected as more advanced, sync uploads → server recovers to rev 101++### Server Rollback Handling++**Test Cases:**++- [ ] **Server rollback (client clean, rev > server)**+ - Client at rev 100, isDirty=false+ - Server recovers to rev 95+ - Sync+ - Expected: Client uploads vault, server saves as rev 101++- [ ] **Server rollback (client dirty, rev > server)**+ - Client at rev 100, isDirty=true+ - Server recovers to rev 95+ - Sync+ - Expected: Client uploads vault, server saves as rev 101++- [ ] **Concurrent rollback recovery**+ - Client A at rev 100, Client B at rev 100+ - Server recovers to rev 95+ - Client A syncs first → Server saves as rev 101+ - Client B syncs → Server returns Outdated+ - Expected: Client B re-syncs and downloads rev 101++- [ ] **Normal sync after rollback recovery**+ - Server recovered to rev 101 via client upload+ - Make local change → Client at rev 101, isDirty=true+ - Sync+ - Expected: Normal upload, server saves as rev 102++---++## Design Decisions++### Vault Preservation Strategy++- Forced logout only clears session data (tokens, in-memory key)+- Vault data remains in regular storage keys+- On re-login, existing vault is detected and recovered through normal sync flow++This approach:+- Reduces storage complexity (no separate "orphaned" keys)+- Leverages existing sync logic for recovery+- Maintains single source of truth for vault data++### Recovery is Transparent++- No UI feedback when vault recovery happens on login+- Recovery happens silently through normal sync process+- Less confusing for users than explicit "recovery" messages++---++## Related Documentation++- [DATAMODEL_REFACTOR.md](./DATAMODEL_REFACTOR.md) - Vault storage architecture+- [OFFLINE_MODE.md](./OFFLINE_MODE.md) - Offline sync behavior+- Vault sync implementation files:+ - Browser: `apps/browser-extension/src/entrypoints/popup/hooks/useVaultSync.ts`+ - Android: `apps/mobile-app/android/app/src/main/java/net/aliasvault/app/vaultstore/VaultSync.kt`+ - iOS: `apps/mobile-app/ios/VaultStoreKit/VaultStore+Sync.swift`