Skip to content

armgit / papra

Public Imported from GitHub
0

The minimalistic document archiving platform.

feat(search-parser): add search query parser package (#728)

Corentin Thomasset417d21556020
diff --git a/packages/search-parser/LICENSE b/packages/search-parser/LICENSE
new file mode 100644
index 00000000..820dafbb
--- /dev/null
+++ b/packages/search-parser/LICENSE
@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Corentin THOMASSET
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
diff --git a/packages/search-parser/README.md b/packages/search-parser/README.md
new file mode 100644
index 00000000..157e3fc0
--- /dev/null
+++ b/packages/search-parser/README.md
@@ -0,0 +1,255 @@
+# @papra/search-parser
+
+A search query parser library for building GitHub-style search syntax with filters, logical operators, and full-text search.
+
+## Features
+
+- **Full-text search**: Extract search terms from queries
+- **Filters**: Support for field:value syntax with multiple operators (=, >, <, >=, <=)
+- **Logical operators**: AND, OR, NOT with proper precedence
+- **Grouping**: Parentheses for controlling evaluation order
+- **Negation**: Both `-filter:value` and `NOT filter:value` syntax
+- **Quoted values**: Support for spaces in filter values and search terms
+- **Escape sequences**: Escape quotes and colons with backslash
+- **Configurable limits**: Max depth and token count for safety
+- **Graceful error handling**: Best-effort parsing with issue reporting
+- **Zero dependencies**: No runtime dependencies
+- **Universal**: Works in Node.js, browsers, Deno, Cloudflare Workers, etc.
+- **Type-safe**: Fully typed with TypeScript
+
+## Installation
+
+```bash
+npm install @papra/search-parser
+# or
+pnpm add @papra/search-parser
+# or
+yarn add @papra/search-parser
+```
+
+## Usage
+
+```typescript
+import { parseSearchQuery } from '@papra/search-parser';
+
+// Simple full-text search
+const result1 = parseSearchQuery({ query: 'my invoice' });
+// {
+// expression: { type: 'and', operands: [] },
+// search: 'my invoice',
+// issues: []
+// }
+
+// Filter with equality
+const result2 = parseSearchQuery({ query: 'tag:invoice' });
+// {
+// expression: {
+// type: 'filter',
+// field: 'tag',
+// operator: '=',
+// value: 'invoice'
+// },
+// search: undefined,
+// issues: []
+// }
+
+// Complex query with operators and grouping
+const result3 = parseSearchQuery({
+ query: '(tag:invoice OR tag:receipt) AND createdAt:>2024-01-01'
+});
+// {
+// expression: {
+// type: 'and',
+// operands: [
+// {
+// type: 'or',
+// operands: [
+// { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },
+// { type: 'filter', field: 'tag', operator: '=', value: 'receipt' }
+// ]
+// },
+// { type: 'filter', field: 'createdAt', operator: '>', value: '2024-01-01' }
+// ]
+// },
+// search: undefined,
+// issues: []
+// }
+```
+
+## Query Syntax
+
+### Full-text Search
+
+```
+my invoice
+"my special invoice" # Quoted for multi-word terms
+"my \"special\" invoice" # Escaped quotes
+```
+
+### Filters
+
+```
+tag:invoice # Equality (implicit)
+tag:=invoice # Equality (explicit)
+createdAt:>2024-01-01 # Greater than
+createdAt:<2024-12-31 # Less than
+createdAt:>=2024-01-01 # Greater than or equal
+createdAt:<=2024-12-31 # Less than or equal
+```
+
+### Quoted Filter Values
+
+```
+tag:"my invoices" # Spaces in value
+tag:"my \"special\" invoices" # Escaped quotes in value
+tag:my\:\:special\:\:tag # Escaped colons in value
+```
+
+### Logical Operators
+
+```
+tag:invoice AND status:active # Explicit AND
+tag:invoice status:active # Implicit AND
+tag:invoice OR tag:receipt # OR
+NOT tag:personal # NOT
+tag:invoice OR tag:receipt AND status:active # Precedence: AND > OR
+```
+
+### Negation
+
+```
+-tag:personal # Minus prefix
+NOT tag:personal # NOT keyword
+NOT (tag:personal OR tag:private) # Negated group
+```
+
+### Grouping
+
+```
+(tag:invoice OR tag:receipt)
+(tag:invoice OR tag:receipt) AND status:active
+```
+
+### Combining Filters and Search
+
+```
+tag:invoice my document # Filter + search
+foo tag:invoice bar # Search terms can be anywhere
+```
+
+### Escaping
+
+```
+tag\:invoice # Escape colon to prevent filter parsing
+ # Results in search text "tag:invoice"
+```
+
+## API
+
+### parseSearchQuery
+
+```typescript
+function parseSearchQuery(options: {
+ query: string;
+ maxDepth?: number; // Default: 10
+ maxTokens?: number; // Default: 200
+}): ParsedQuery;
+```
+
+## Operator Precedence
+
+From highest to lowest:
+
+1. **NOT** (highest)
+2. **AND**
+3. **OR** (lowest)
+
+Use parentheses to override precedence:
+
+```
+tag:invoice OR tag:receipt AND status:active
+# Parsed as: tag:invoice OR (tag:receipt AND status:active)
+
+(tag:invoice OR tag:receipt) AND status:active
+# Parsed as: (tag:invoice OR tag:receipt) AND status:active
+```
+
+## Error Handling
+
+### Issue Structure
+
+All parsing issues are returned with both a machine-readable error code and a human-readable message:
+
+```typescript
+type Issue = {
+ code: string; // Machine-readable error code
+ message: string; // Human-readable error message
+};
+```
+
+### Error Codes
+
+The library exports an `ERROR_CODES` constant with all available error codes:
+
+```typescript
+import { ERROR_CODES } from '@papra/search-parser';
+
+const result = parseSearchQuery({ query: '(tag:invoice' });
+
+// Check for specific error by code
+const hasUnmatchedParen = result.issues.some(
+ issue => issue.code === ERROR_CODES.UNMATCHED_OPENING_PARENTHESIS
+);
+```
+
+Available error codes are defined in [`src/errors.ts`](src/errors.ts).
+
+## Safety Features
+
+### Maximum Depth
+
+Prevents deeply nested expressions (e.g., excessive parentheses):
+
+```typescript
+parseSearchQuery({
+ query: '((((((((((tag:invoice))))))))))',
+ maxDepth: 5 // Will report issue if exceeded
+});
+```
+
+### Maximum Tokens
+
+Prevents excessively long queries:
+
+```typescript
+parseSearchQuery({
+ query: 'tag1:val1 tag2:val2 ... tag100:val100',
+ maxTokens: 50 // Will report issue if exceeded
+});
+```
+
+### Graceful Error Handling
+
+Malformed queries are handled gracefully with best-effort parsing:
+
+```typescript
+parseSearchQuery({ query: '(tag:invoice' });
+// {
+// expression: { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },
+// search: undefined,
+// issues: [
+// {
+// code: 'unmatched-opening-parenthesis',
+// message: 'Unmatched opening parenthesis'
+// }
+// ]
+// }
+```
+
+## License
+
+MIT
+
+## Contributing
+
+Contributions are welcome! Please open an issue or pull request on GitHub.
diff --git a/packages/search-parser/eslint.config.js b/packages/search-parser/eslint.config.js
new file mode 100644
index 00000000..2aae9a63
--- /dev/null
+++ b/packages/search-parser/eslint.config.js
@@ -0,0 +1,26 @@
+import antfu from '@antfu/eslint-config';
+
+export default antfu({
+ stylistic: {
+ semi: true,
+ },
+
+ ignores: [
+ 'example.ts',
+ 'README.md',
+ ],
+
+ rules: {
+ // To allow export on top of files
+ 'ts/no-use-before-define': ['error', { allowNamedExports: true, functions: false }],
+ 'curly': ['error', 'all'],
+ 'vitest/consistent-test-it': ['error', { fn: 'test' }],
+ 'ts/consistent-type-definitions': ['error', 'type'],
+ 'style/brace-style': ['error', '1tbs', { allowSingleLine: false }],
+ 'unused-imports/no-unused-vars': ['error', {
+ argsIgnorePattern: '^_',
+ varsIgnorePattern: '^_',
+ caughtErrorsIgnorePattern: '^_',
+ }],
+ },
+});
diff --git a/packages/search-parser/package.json b/packages/search-parser/package.json
new file mode 100644
index 00000000..4ed65cc5
--- /dev/null
+++ b/packages/search-parser/package.json
@@ -0,0 +1,52 @@
+{
+ "name": "@papra/search-parser",
+ "type": "module",
+ "version": "0.0.0",
+ "description": "Search query parser library",
+ "author": "Corentin Thomasset <[email protected]> (https://corentin.tech)",
+ "license": "MIT",
+ "repository": {
+ "type": "git",
+ "url": "https://github.com/papra-hq/papra",
+ "directory": "packages/search-parser"
+ },
+ "bugs": {
+ "url": "https://github.com/papra-hq/papra/issues"
+ },
+ "keywords": [
+ "papra",
+ "search",
+ "parser",
+ "query"
+ ],
+ "exports": {
+ ".": "./dist/index.js",
+ "./package.json": "./package.json"
+ },
+ "main": "./dist/index.js",
+ "module": "./dist/index.js",
+ "types": "./dist/index.d.ts",
+ "files": [
+ "dist"
+ ],
+ "scripts": {
+ "lint": "eslint .",
+ "lint:fix": "eslint --fix .",
+ "test": "vitest run",
+ "test:watch": "vitest watch",
+ "typecheck": "tsc --noEmit",
+ "build": "tsdown",
+ "build:watch": "tsdown --watch",
+ "dev": "pnpm build:watch",
+ "prepublishOnly": "pnpm run build"
+ },
+ "dependencies": {},
+ "devDependencies": {
+ "@antfu/eslint-config": "catalog:",
+ "eslint": "catalog:",
+ "standardwebhooks": "^1.0.0",
+ "tsdown": "catalog:",
+ "typescript": "catalog:",
+ "vitest": "catalog:"
+ }
+}
diff --git a/packages/search-parser/src/errors.ts b/packages/search-parser/src/errors.ts
new file mode 100644
index 00000000..9f4758de
--- /dev/null
+++ b/packages/search-parser/src/errors.ts
@@ -0,0 +1,8 @@
+export const ERROR_CODES = {
+ MAX_TOKENS_EXCEEDED: 'max-tokens-exceeded',
+ MAX_NESTING_DEPTH_EXCEEDED: 'max-nesting-depth-exceeded',
+ UNMATCHED_OPENING_PARENTHESIS: 'unmatched-opening-parenthesis',
+ UNMATCHED_CLOSING_PARENTHESIS: 'unmatched-closing-parenthesis',
+ UNCLOSED_QUOTED_STRING: 'unclosed-quoted-string',
+ MISSING_OPERAND_FOR_NOT: 'missing-operand-for-not',
+};
diff --git a/packages/search-parser/src/index.ts b/packages/search-parser/src/index.ts
new file mode 100644
index 00000000..680e0b56
--- /dev/null
+++ b/packages/search-parser/src/index.ts
@@ -0,0 +1,18 @@
+export {
+ ERROR_CODES,
+} from './errors';
+
+export {
+ parseSearchQuery,
+} from './parser';
+
+export type {
+ AndExpression,
+ Expression,
+ FilterExpression,
+ Issue,
+ NotExpression,
+ Operator,
+ OrExpression,
+ ParsedQuery,
+} from './parser.types';
diff --git a/packages/search-parser/src/parser.test.ts b/packages/search-parser/src/parser.test.ts
new file mode 100644
index 00000000..f6c182d0
--- /dev/null
+++ b/packages/search-parser/src/parser.test.ts
@@ -0,0 +1,569 @@
+import { describe, expect, test } from 'vitest';
+import { parseSearchQuery } from './parser';
+
+describe('parseSearchQuery', () => {
+ describe('when parsing simple queries', () => {
+ test('returns empty expression and no search terms for empty query', () => {
+ expect(parseSearchQuery({ query: '' })).toEqual({
+ expression: { type: 'and', operands: [] },
+ search: undefined,
+ issues: [],
+ });
+ });
+
+ test('extracts full-text search terms when no filters present', () => {
+ expect(parseSearchQuery({ query: 'my invoice' })).toEqual({
+ expression: { type: 'and', operands: [] },
+ search: 'my invoice',
+ issues: [],
+ });
+ });
+
+ test('handles quoted search terms', () => {
+ expect(parseSearchQuery({ query: '"my special invoice"' })).toEqual({
+ expression: { type: 'and', operands: [] },
+ search: 'my special invoice',
+ issues: [],
+ });
+ });
+
+ test('handles escaped quotes in search terms', () => {
+ expect(parseSearchQuery({ query: '"my \\"special\\" invoice"' })).toEqual({
+ expression: { type: 'and', operands: [] },
+ search: 'my "special" invoice',
+ issues: [],
+ });
+ });
+
+ test('handles escaped colons to prevent filter parsing', () => {
+ expect(parseSearchQuery({ query: 'tag\\:invoice' })).toEqual({
+ expression: { type: 'and', operands: [] },
+ search: 'tag:invoice',
+ issues: [],
+ });
+ });
+ });
+
+ describe('when parsing filter expressions', () => {
+ test('parses simple filter with equality operator', () => {
+ expect(parseSearchQuery({ query: 'tag:invoice' })).toEqual({
+ expression: {
+ type: 'filter',
+ field: 'tag',
+ operator: '=',
+ value: 'invoice',
+ },
+ search: undefined,
+ issues: [],
+ });
+ });
+
+ test('parses filter with explicit equality operator', () => {
+ expect(parseSearchQuery({ query: 'tag:=invoice' })).toEqual({
+ expression: {
+ type: 'filter',
+ field: 'tag',
+ operator: '=',
+ value: 'invoice',
+ },
+ search: undefined,
+ issues: [],
+ });
+ });
+
+ test('parses filter with quoted value containing spaces', () => {
+ expect(parseSearchQuery({ query: 'tag:"my invoices"' })).toEqual({
+ expression: {
+ type: 'filter',
+ field: 'tag',
+ operator: '=',
+ value: 'my invoices',
+ },
+ search: undefined,
+ issues: [],
+ });
+ });
+
+ test('parses filter with escaped quotes in value', () => {
+ expect(parseSearchQuery({ query: 'tag:"my \\"special\\" invoices"' })).toEqual({
+ expression: {
+ type: 'filter',
+ field: 'tag',
+ operator: '=',
+ value: 'my "special" invoices',
+ },
+ search: undefined,
+ issues: [],
+ });
+ });
+
+ test('parses filter with escaped colons in value', () => {
+ expect(parseSearchQuery({ query: 'tag:my\\:\\:special\\:\\:tag' })).toEqual({
+ expression: {
+ type: 'filter',
+ field: 'tag',
+ operator: '=',
+ value: 'my::special::tag',
+ },
+ search: undefined,
+ issues: [],
+ });
+ });
+
+ test('parses filter with escaped colons in field name', () => {
+ expect(parseSearchQuery({ query: 'my\\:field:value' })).toEqual({
+ expression: {
+ type: 'filter',
+ field: 'my:field',
+ operator: '=',
+ value: 'value',
+ },
+ search: undefined,
+ issues: [],
+ });
+ });
+
+ test('parses filter with comparison operators', () => {
+ expect(parseSearchQuery({ query: 'createdAt:>2024-01-01' })).toEqual({
+ expression: {
+ type: 'filter',
+ field: 'createdAt',
+ operator: '>',
+ value: '2024-01-01',
+ },
+ search: undefined,
+ issues: [],
+ });
+
+ expect(parseSearchQuery({ query: 'createdAt:<2024-01-01' })).toEqual({
+ expression: {
+ type: 'filter',
+ field: 'createdAt',
+ operator: '<',
+ value: '2024-01-01',
+ },
+ search: undefined,
+ issues: [],
+ });
+
+ expect(parseSearchQuery({ query: 'createdAt:>=2024-01-01' })).toEqual({
+ expression: {
+ type: 'filter',
+ field: 'createdAt',
+ operator: '>=',
+ value: '2024-01-01',
+ },
+ search: undefined,
+ issues: [],
+ });
+
+ expect(parseSearchQuery({ query: 'createdAt:<=2024-01-01' })).toEqual({
+ expression: {
+ type: 'filter',
+ field: 'createdAt',
+ operator: '<=',
+ value: '2024-01-01',
+ },
+ search: undefined,
+ issues: [],
+ });
+ });
+
+ test('parses negated filter with minus prefix', () => {
+ expect(parseSearchQuery({ query: '-tag:personal' })).toEqual({
+ expression: {
+ type: 'not',
+ operand: {
+ type: 'filter',
+ field: 'tag',
+ operator: '=',
+ value: 'personal',
+ },
+ },
+ search: undefined,
+ issues: [],
+ });
+ });
+ });
+
+ describe('when combining filters and search terms', () => {
+ test('combines filter with search terms', () => {
+ expect(parseSearchQuery({ query: 'tag:invoice my invoice' })).toEqual({
+ expression: {
+ type: 'filter',
+ field: 'tag',
+ operator: '=',
+ value: 'invoice',
+ },
+ search: 'my invoice',
+ issues: [],
+ });
+ });
+
+ test('extracts search terms before and after filters', () => {
+ expect(parseSearchQuery({ query: 'foo tag:invoice bar' })).toEqual({
+ expression: {
+ type: 'filter',
+ field: 'tag',
+ operator: '=',
+ value: 'invoice',
+ },
+ search: 'foo bar',
+ issues: [],
+ });
+ });
+ });
+
+ describe('when using AND operator', () => {
+ test('combines filters with explicit AND', () => {
+ expect(parseSearchQuery({ query: 'tag:invoice AND tag:receipt' })).toEqual({
+ expression: {
+ type: 'and',
+ operands: [
+ { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },
+ { type: 'filter', field: 'tag', operator: '=', value: 'receipt' },
+ ],
+ },
+ search: undefined,
+ issues: [],
+ });
+ });
+
+ test('combines filters with implicit AND', () => {
+ expect(parseSearchQuery({ query: 'tag:invoice tag:receipt' })).toEqual({
+ expression: {
+ type: 'and',
+ operands: [
+ { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },
+ { type: 'filter', field: 'tag', operator: '=', value: 'receipt' },
+ ],
+ },
+ search: undefined,
+ issues: [],
+ });
+ });
+
+ test('combines multiple filters with AND', () => {
+ expect(parseSearchQuery({ query: 'tag:invoice createdAt:>2024-01-01 status:active' })).toEqual({
+ expression: {
+ type: 'and',
+ operands: [
+ { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },
+ { type: 'filter', field: 'createdAt', operator: '>', value: '2024-01-01' },
+ { type: 'filter', field: 'status', operator: '=', value: 'active' },
+ ],
+ },
+ search: undefined,
+ issues: [],
+ });
+ });
+ });
+
+ describe('when using OR operator', () => {
+ test('combines filters with OR', () => {
+ expect(parseSearchQuery({ query: 'tag:invoice OR tag:receipt' })).toEqual({
+ expression: {
+ type: 'or',
+ operands: [
+ { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },
+ { type: 'filter', field: 'tag', operator: '=', value: 'receipt' },
+ ],
+ },
+ search: undefined,
+ issues: [],
+ });
+ });
+
+ test('combines multiple filters with OR', () => {
+ expect(parseSearchQuery({ query: 'tag:invoice OR tag:receipt OR tag:bill' })).toEqual({
+ expression: {
+ type: 'or',
+ operands: [
+ { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },
+ { type: 'filter', field: 'tag', operator: '=', value: 'receipt' },
+ { type: 'filter', field: 'tag', operator: '=', value: 'bill' },
+ ],
+ },
+ search: undefined,
+ issues: [],
+ });
+ });
+ });
+
+ describe('when using NOT operator', () => {
+ test('negates filter with NOT keyword', () => {
+ expect(parseSearchQuery({ query: 'NOT tag:personal' })).toEqual({
+ expression: {
+ type: 'not',
+ operand: {
+ type: 'filter',
+ field: 'tag',
+ operator: '=',
+ value: 'personal',
+ },
+ },
+ search: undefined,
+ issues: [],
+ });
+ });
+
+ test('negates grouped expression', () => {
+ expect(parseSearchQuery({ query: 'NOT (tag:personal OR tag:private)' })).toEqual({
+ expression: {
+ type: 'not',
+ operand: {
+ type: 'or',
+ operands: [
+ { type: 'filter', field: 'tag', operator: '=', value: 'personal' },
+ { type: 'filter', field: 'tag', operator: '=', value: 'private' },
+ ],
+ },
+ },
+ search: undefined,
+ issues: [],
+ });
+ });
+ });
+
+ describe('when using grouping with parentheses', () => {
+ test('groups expressions with parentheses', () => {
+ expect(parseSearchQuery({ query: '(tag:invoice OR tag:receipt)' })).toEqual({
+ expression: {
+ type: 'or',
+ operands: [
+ { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },
+ { type: 'filter', field: 'tag', operator: '=', value: 'receipt' },
+ ],
+ },
+ search: undefined,
+ issues: [],
+ });
+ });
+
+ test('combines grouped expressions with AND', () => {
+ expect(parseSearchQuery({ query: '(tag:invoice OR tag:receipt) AND createdAt:>2024-01-01' })).toEqual({
+ expression: {
+ type: 'and',
+ operands: [
+ {
+ type: 'or',
+ operands: [
+ { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },
+ { type: 'filter', field: 'tag', operator: '=', value: 'receipt' },
+ ],
+ },
+ { type: 'filter', field: 'createdAt', operator: '>', value: '2024-01-01' },
+ ],
+ },
+ search: undefined,
+ issues: [],
+ });
+ });
+
+ test('handles nested grouping', () => {
+ expect(parseSearchQuery({ query: '((tag:invoice OR tag:receipt) AND status:active)' })).toEqual({
+ expression: {
+ type: 'and',
+ operands: [
+ {
+ type: 'or',
+ operands: [
+ { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },
+ { type: 'filter', field: 'tag', operator: '=', value: 'receipt' },
+ ],
+ },
+ { type: 'filter', field: 'status', operator: '=', value: 'active' },
+ ],
+ },
+ search: undefined,
+ issues: [],
+ });
+ });
+ });
+
+ describe('when respecting operator precedence', () => {
+ test('aND binds tighter than OR', () => {
+ expect(parseSearchQuery({ query: 'tag:invoice OR tag:receipt AND tag:company' })).toEqual({
+ expression: {
+ type: 'or',
+ operands: [
+ { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },
+ {
+ type: 'and',
+ operands: [
+ { type: 'filter', field: 'tag', operator: '=', value: 'receipt' },
+ { type: 'filter', field: 'tag', operator: '=', value: 'company' },
+ ],
+ },
+ ],
+ },
+ search: undefined,
+ issues: [],
+ });
+ });
+
+ test('nOT binds tighter than AND', () => {
+ expect(parseSearchQuery({ query: 'NOT tag:invoice AND tag:receipt' })).toEqual({
+ expression: {
+ type: 'and',
+ operands: [
+ {
+ type: 'not',
+ operand: { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },
+ },
+ { type: 'filter', field: 'tag', operator: '=', value: 'receipt' },
+ ],
+ },
+ search: undefined,
+ issues: [],
+ });
+ });
+
+ test('parentheses override precedence', () => {
+ expect(parseSearchQuery({ query: '(tag:invoice OR tag:receipt) AND tag:company' })).toEqual({
+ expression: {
+ type: 'and',
+ operands: [
+ {
+ type: 'or',
+ operands: [
+ { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },
+ { type: 'filter', field: 'tag', operator: '=', value: 'receipt' },
+ ],
+ },
+ { type: 'filter', field: 'tag', operator: '=', value: 'company' },
+ ],
+ },
+ search: undefined,
+ issues: [],
+ });
+ });
+ });
+
+ describe('when handling malformed queries', () => {
+ test('handles unclosed quoted string gracefully', () => {
+ const result = parseSearchQuery({ query: '"unclosed string' });
+ expect(result.search).toBe('unclosed string');
+ expect(result.issues).toContainEqual({
+ code: 'unclosed-quoted-string',
+ message: 'Unclosed quoted string',
+ });
+ });
+
+ test('handles unmatched opening parenthesis gracefully', () => {
+ const result = parseSearchQuery({ query: '(tag:invoice' });
+ expect(result.expression).toEqual({
+ type: 'filter',
+ field: 'tag',
+ operator: '=',
+ value: 'invoice',
+ });
+ expect(result.issues).toContainEqual({
+ code: 'unmatched-opening-parenthesis',
+ message: 'Unmatched opening parenthesis',
+ });
+ });
+
+ test('handles unmatched closing parenthesis gracefully', () => {
+ const result = parseSearchQuery({ query: 'tag:invoice)' });
+ expect(result.expression).toEqual({
+ type: 'filter',
+ field: 'tag',
+ operator: '=',
+ value: 'invoice',
+ });
+ expect(result.issues).toContainEqual({
+ code: 'unmatched-closing-parenthesis',
+ message: 'Unmatched closing parenthesis',
+ });
+ });
+
+ test('handles NOT without operand gracefully', () => {
+ const result = parseSearchQuery({ query: 'tag:invoice AND NOT' });
+ expect(result.expression).toEqual({
+ type: 'filter',
+ field: 'tag',
+ operator: '=',
+ value: 'invoice',
+ });
+ expect(result.issues).toContainEqual({
+ code: 'missing-operand-for-not',
+ message: 'NOT operator requires an operand',
+ });
+ });
+ });
+
+ describe('when enforcing limits', () => {
+ test('respects maximum token limit', () => {
+ const longQuery = Array.from({ length: 50 }).map((_, i) => `tag${i}:value${i}`).join(' ');
+ const result = parseSearchQuery({ query: longQuery, maxTokens: 10 });
+ expect(result.issues).toContainEqual({
+ code: 'max-tokens-exceeded',
+ message: 'Maximum token limit of 10 exceeded',
+ });
+ });
+
+ test('respects maximum depth limit', () => {
+ const deepQuery = '((((((((((tag:invoice))))))))))';
+ const result = parseSearchQuery({ query: deepQuery, maxDepth: 5 });
+ expect(result.issues).toContainEqual({
+ code: 'max-nesting-depth-exceeded',
+ message: 'Maximum nesting depth of 5 exceeded',
+ });
+ });
+ });
+
+ describe('when handling complex real-world queries', () => {
+ test('parses complex query with multiple operators and grouping', () => {
+ expect(
+ parseSearchQuery({
+ query: '(tag:invoice OR tag:receipt) AND createdAt:>2024-01-01 AND NOT tag:personal my document',
+ }),
+ ).toEqual({
+ expression: {
+ type: 'and',
+ operands: [
+ {
+ type: 'or',
+ operands: [
+ { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },
+ { type: 'filter', field: 'tag', operator: '=', value: 'receipt' },
+ ],
+ },
+ { type: 'filter', field: 'createdAt', operator: '>', value: '2024-01-01' },
+ {
+ type: 'not',
+ operand: { type: 'filter', field: 'tag', operator: '=', value: 'personal' },
+ },
+ ],
+ },
+ search: 'my document',
+ issues: [],
+ });
+ });
+
+ test('parses query with multiple negations', () => {
+ expect(parseSearchQuery({ query: '-tag:personal -tag:private -tag:confidential' })).toEqual({
+ expression: {
+ type: 'and',
+ operands: [
+ {
+ type: 'not',
+ operand: { type: 'filter', field: 'tag', operator: '=', value: 'personal' },
+ },
+ {
+ type: 'not',
+ operand: { type: 'filter', field: 'tag', operator: '=', value: 'private' },
+ },
+ {
+ type: 'not',
+ operand: { type: 'filter', field: 'tag', operator: '=', value: 'confidential' },
+ },
+ ],
+ },
+ search: undefined,
+ issues: [],
+ });
+ });
+ });
+});
diff --git a/packages/search-parser/src/parser.ts b/packages/search-parser/src/parser.ts
new file mode 100644
index 00000000..d3979062
--- /dev/null
+++ b/packages/search-parser/src/parser.ts
@@ -0,0 +1,208 @@
+import type { Expression, Issue, ParsedQuery } from './parser.types';
+import type { Token } from './tokenizer';
+import { ERROR_CODES } from './errors';
+import { tokenize } from './tokenizer';
+
+export function parseSearchQuery(
+ {
+ query,
+ maxDepth = 10,
+ maxTokens = 200,
+ }: {
+ query: string;
+ maxDepth?: number;
+ maxTokens?: number;
+ },
+): ParsedQuery {
+ const { tokens, issues: tokenizerIssues } = tokenize({ query, maxTokens });
+
+ const { expression, issues: parserIssues, search } = parseExpression({ tokens, maxDepth });
+
+ return {
+ expression,
+ search,
+ issues: [...tokenizerIssues, ...parserIssues],
+ };
+}
+
+function parseExpression({ tokens, maxDepth }: { tokens: Token[]; maxDepth: number }): ParsedQuery {
+ const searchTerms: string[] = [];
+ const parserIssues: Issue[] = [];
+
+ let currentTokenIndex = 0;
+ let currentDepth = 0;
+
+ const peek = (): Token => tokens[currentTokenIndex] ?? { type: 'EOF' };
+ const advance = (): Token => tokens[currentTokenIndex++] ?? { type: 'EOF' };
+
+ const checkDepth = (): boolean => {
+ if (currentDepth >= maxDepth) {
+ parserIssues.push({
+ code: ERROR_CODES.MAX_NESTING_DEPTH_EXCEEDED,
+ message: `Maximum nesting depth of ${maxDepth} exceeded`,
+ });
+ return false;
+ }
+ return true;
+ };
+
+ // Parse primary expression (filter, parentheses, text)
+ function parsePrimaryExpression(): Expression | undefined {
+ const token = peek();
+
+ if (token.type === 'LPAREN') {
+ advance(); // Consume (
+
+ if (!checkDepth()) {
+ return undefined;
+ }
+
+ currentDepth++;
+ const expr = parseOrExpression();
+ currentDepth--;
+
+ if (peek().type === 'RPAREN') {
+ advance(); // Consume )
+ } else {
+ parserIssues.push({
+ code: ERROR_CODES.UNMATCHED_OPENING_PARENTHESIS,
+ message: 'Unmatched opening parenthesis',
+ });
+ }
+
+ return expr;
+ }
+
+ if (token.type === 'FILTER') {
+ advance();
+ const filterExpr: Expression = {
+ type: 'filter',
+ field: token.field,
+ operator: token.operator,
+ value: token.value,
+ };
+
+ if (token.negated) {
+ return { type: 'not', operand: filterExpr };
+ }
+
+ return filterExpr;
+ }
+
+ if (token.type === 'TEXT') {
+ advance();
+ searchTerms.push(token.value);
+ return undefined; // Text doesn't produce an expression
+ }
+
+ return undefined;
+ }
+
+ // Parse NOT expression (highest precedence)
+ function parseUnaryExpression(): Expression | undefined {
+ if (peek().type === 'NOT') {
+ advance(); // Consume NOT
+
+ if (!checkDepth()) {
+ return undefined;
+ }
+
+ currentDepth++;
+ const operand = parseUnaryExpression();
+ currentDepth--;
+
+ if (!operand) {
+ parserIssues.push({
+ code: ERROR_CODES.MISSING_OPERAND_FOR_NOT,
+ message: 'NOT operator requires an operand',
+ });
+ return undefined;
+ }
+
+ return { type: 'not', operand };
+ }
+
+ return parsePrimaryExpression();
+ }
+
+ // Parse AND expression (higher precedence than OR)
+ function parseAndExpression(): Expression | undefined {
+ const operands: Expression[] = [];
+
+ while (true) {
+ const next = peek();
+
+ // Stop if we hit EOF, OR operator, or closing paren
+ if (next.type === 'EOF' || next.type === 'OR' || next.type === 'RPAREN') {
+ break;
+ }
+
+ // Consume explicit AND operator
+ if (next.type === 'AND') {
+ advance();
+ continue;
+ }
+
+ const expr = parseUnaryExpression();
+ if (expr) {
+ operands.push(expr);
+ }
+ }
+
+ if (operands.length === 0) {
+ return undefined;
+ }
+
+ if (operands.length === 1) {
+ return operands[0];
+ }
+
+ return { type: 'and', operands };
+ };
+
+ // Parse OR expression (lowest precedence)
+ function parseOrExpression(): Expression | undefined {
+ const left = parseAndExpression();
+ if (!left) {
+ return undefined;
+ }
+
+ const operands: Expression[] = [left];
+
+ while (peek().type === 'OR') {
+ advance(); // Consume OR
+ const right = parseAndExpression();
+ if (right) {
+ operands.push(right);
+ }
+ }
+
+ if (operands.length === 1) {
+ return operands[0];
+ }
+
+ return { type: 'or', operands };
+ };
+
+ // Start parsing
+ const expression = parseOrExpression();
+
+ // Check for unmatched closing parentheses
+ while (peek().type === 'RPAREN') {
+ parserIssues.push({
+ message: 'Unmatched closing parenthesis',
+ code: ERROR_CODES.UNMATCHED_CLOSING_PARENTHESIS,
+ });
+ advance();
+ }
+
+ // Build final result
+ const finalExpression: Expression = expression ?? { type: 'and', operands: [] };
+ const search = searchTerms.length > 0 ? searchTerms.join(' ') : undefined;
+
+ return {
+ expression: finalExpression,
+ search,
+ issues: parserIssues,
+ };
+}
diff --git a/packages/search-parser/src/parser.types.ts b/packages/search-parser/src/parser.types.ts
new file mode 100644
index 00000000..caba3469
--- /dev/null
+++ b/packages/search-parser/src/parser.types.ts
@@ -0,0 +1,40 @@
+export type Issue = {
+ message: string;
+ code: string;
+};
+
+export type ParsedQuery = {
+ expression: Expression;
+ search: string | undefined;
+ issues: Issue[];
+};
+
+export type Expression
+ = | AndExpression
+ | OrExpression
+ | NotExpression
+ | FilterExpression;
+
+export type AndExpression = {
+ type: 'and';
+ operands: Expression[];
+};
+
+export type OrExpression = {
+ type: 'or';
+ operands: Expression[];
+};
+
+export type NotExpression = {
+ type: 'not';
+ operand: Expression;
+};
+
+export type Operator = '>' | '<' | '>=' | '<=' | '=';
+
+export type FilterExpression = {
+ type: 'filter';
+ field: string;
+ operator: Operator;
+ value: string;
+};
diff --git a/packages/search-parser/src/tokenizer.ts b/packages/search-parser/src/tokenizer.ts
new file mode 100644
index 00000000..9904b0e6
--- /dev/null
+++ b/packages/search-parser/src/tokenizer.ts
@@ -0,0 +1,288 @@
+import type { Issue, Operator } from './parser.types';
+import { ERROR_CODES } from './errors';
+
+export type Token
+ = | { type: 'LPAREN' }
+ | { type: 'RPAREN' }
+ | { type: 'AND' }
+ | { type: 'OR' }
+ | { type: 'NOT' }
+ | { type: 'FILTER'; field: string; operator: Operator; value: string; negated: boolean }
+ | { type: 'TEXT'; value: string }
+ | { type: 'EOF' };
+
+export type TokenizeResult = {
+ tokens: Token[];
+ issues: Issue[];
+};
+
+export function tokenize({ query, maxTokens }: { query: string; maxTokens: number }): TokenizeResult {
+ const tokens: Token[] = [];
+ const issues: Issue[] = [];
+ let pos = 0;
+
+ const peek = (): string | undefined => query[pos];
+ const advance = (): string => query[pos++] || '';
+
+ const skipWhitespace = (): void => {
+ while (pos < query.length) {
+ const char = query[pos];
+ if (!char || !/\s/.test(char)) {
+ break;
+ }
+ pos++;
+ }
+ };
+
+ const readQuotedString = (): string | undefined => {
+ if (peek() !== '"') {
+ return undefined;
+ }
+
+ advance(); // Skip opening quote
+ let value = '';
+ let escaped = false;
+
+ while (pos < query.length) {
+ const char = peek();
+
+ if (!char) {
+ break;
+ }
+
+ if (escaped) {
+ value += char;
+ escaped = false;
+ advance();
+ } else if (char === '\\') {
+ escaped = true;
+ advance();
+ } else if (char === '"') {
+ advance(); // Skip closing quote
+ return value;
+ } else {
+ value += char;
+ advance();
+ }
+ }
+
+ issues.push({
+ code: ERROR_CODES.UNCLOSED_QUOTED_STRING,
+ message: 'Unclosed quoted string',
+ });
+ return value;
+ };
+
+ const readUnquotedToken = (stopAtQuote = false): string => {
+ let value = '';
+
+ while (pos < query.length) {
+ const char = peek();
+
+ if (!char || /\s/.test(char) || char === '(' || char === ')') {
+ break;
+ }
+
+ // Stop at quote if requested (for filter field names before quoted values)
+ if (stopAtQuote && char === '"') {
+ break;
+ }
+
+ // Keep backslashes in unquoted tokens so parseFilter can detect escaped colons
+ value += advance();
+ }
+
+ return value;
+ };
+
+ const readFilterValue = (): string => {
+ skipWhitespace();
+
+ // Check if value is quoted
+ const quotedValue = readQuotedString();
+ if (quotedValue !== undefined) {
+ return quotedValue;
+ }
+
+ // Read unquoted value (up to whitespace or special chars)
+ let value = '';
+ while (pos < query.length) {
+ const char = peek();
+
+ if (!char || /\s/.test(char) || char === '(' || char === ')') {
+ break;
+ }
+
+ if (char === '\\') {
+ advance();
+ if (pos < query.length) {
+ value += advance();
+ }
+ } else {
+ value += advance();
+ }
+ }
+
+ return value.replace(/\\:/g, ':');
+ };
+
+ const hasUnescapedColon = (str: string): boolean => {
+ for (let i = 0; i < str.length; i++) {
+ if (str[i] === ':' && (i === 0 || str[i - 1] !== '\\')) {
+ return true;
+ }
+ }
+ return false;
+ };
+
+ const parseFilter = (token: string, negated: boolean): Token | undefined => {
+ // Check for unescaped colons
+ if (!hasUnescapedColon(token)) {
+ return undefined;
+ }
+
+ // Find first unescaped colon
+ let firstColonIndex = -1;
+ for (let i = 0; i < token.length; i++) {
+ if (token[i] === ':' && (i === 0 || token[i - 1] !== '\\')) {
+ firstColonIndex = i;
+ break;
+ }
+ }
+
+ if (firstColonIndex === -1) {
+ return undefined;
+ }
+
+ const field = token.slice(0, firstColonIndex).replace(/\\:/g, ':');
+ const afterColon = token.slice(firstColonIndex + 1);
+
+ // Check for operator at the start
+ let operator: Operator = '=';
+ let operatorLength = 0;
+
+ if (afterColon.startsWith('>=')) {
+ operator = '>=';
+ operatorLength = 2;
+ } else if (afterColon.startsWith('<=')) {
+ operator = '<=';
+ operatorLength = 2;
+ } else if (afterColon.startsWith('>')) {
+ operator = '>';
+ operatorLength = 1;
+ } else if (afterColon.startsWith('<')) {
+ operator = '<';
+ operatorLength = 1;
+ } else if (afterColon.startsWith('=')) {
+ operator = '=';
+ operatorLength = 1;
+ }
+
+ // If there's nothing after the operator in the token, read the value from input
+ let value = afterColon.slice(operatorLength).replace(/\\:/g, ':');
+
+ // If the value is empty and we still have input, this might be a case like "tag:" followed by a quoted string
+ if (!value) {
+ value = readFilterValue();
+ }
+
+ return { type: 'FILTER', field, operator, value, negated };
+ };
+
+ while (pos < query.length) {
+ if (tokens.length >= maxTokens) {
+ issues.push({
+ code: ERROR_CODES.MAX_TOKENS_EXCEEDED,
+ message: `Maximum token limit of ${maxTokens} exceeded`,
+ });
+ break;
+ }
+
+ skipWhitespace();
+
+ if (pos >= query.length) {
+ break;
+ }
+
+ const char = peek();
+
+ // Handle parentheses
+ if (char === '(') {
+ advance();
+ tokens.push({ type: 'LPAREN' });
+ continue;
+ }
+
+ if (char === ')') {
+ advance();
+ tokens.push({ type: 'RPAREN' });
+ continue;
+ }
+
+ // Handle negation prefix
+ const nextChar = query[pos + 1];
+ if (char === '-' && nextChar && !/\s/.test(nextChar)) {
+ advance();
+ skipWhitespace();
+
+ // Read the next token (could be quoted or unquoted)
+ const quotedValue = readQuotedString();
+ const token = quotedValue !== undefined ? quotedValue : readUnquotedToken();
+
+ // Try to parse as filter
+ const filter = parseFilter(token, true);
+ if (filter) {
+ tokens.push(filter);
+ } else {
+ // If not a filter, treat as negated text search (which we'll handle as NOT operator)
+ const unescapedText = token.replace(/\\(.)/g, '$1');
+ tokens.push({ type: 'NOT' });
+ tokens.push({ type: 'TEXT', value: unescapedText });
+ }
+ continue;
+ }
+
+ // Read quoted or unquoted token
+ const quotedValue = readQuotedString();
+ const token = quotedValue !== undefined ? quotedValue : readUnquotedToken(true);
+
+ if (!token) {
+ advance(); // Skip invalid character
+ continue;
+ }
+
+ // Check for operators
+ const upperToken = token.toUpperCase();
+ if (upperToken === 'AND') {
+ tokens.push({ type: 'AND' });
+ continue;
+ }
+
+ if (upperToken === 'OR') {
+ tokens.push({ type: 'OR' });
+ continue;
+ }
+
+ if (upperToken === 'NOT') {
+ tokens.push({ type: 'NOT' });
+ continue;
+ }
+
+ // Try to parse as filter (only if not quoted)
+ if (quotedValue === undefined) {
+ const filter = parseFilter(token, false);
+ if (filter) {
+ tokens.push(filter);
+ continue;
+ }
+ }
+
+ // Otherwise, treat as text (unescape backslashes)
+ const unescapedText = token.replace(/\\(.)/g, '$1');
+ tokens.push({ type: 'TEXT', value: unescapedText });
+ }
+
+ tokens.push({ type: 'EOF' });
+
+ return { tokens, issues };
+}
diff --git a/packages/search-parser/tsconfig.json b/packages/search-parser/tsconfig.json
new file mode 100644
index 00000000..43dd7d01
--- /dev/null
+++ b/packages/search-parser/tsconfig.json
@@ -0,0 +1,19 @@
+{
+ "compilerOptions": {
+ "target": "ESNext",
+ "moduleDetection": "force",
+ "module": "preserve",
+ "resolveJsonModule": true,
+ "allowJs": true,
+ "strict": true,
+ "noImplicitOverride": true,
+ "noUncheckedIndexedAccess": true,
+ "noEmit": true,
+ "allowSyntheticDefaultImports": true,
+ "esModuleInterop": true,
+ "forceConsistentCasingInFileNames": true,
+ "isolatedModules": true,
+ "verbatimModuleSyntax": true,
+ "skipLibCheck": true
+ }
+}
diff --git a/packages/search-parser/tsdown.config.ts b/packages/search-parser/tsdown.config.ts
new file mode 100644
index 00000000..bcf4ac58
--- /dev/null
+++ b/packages/search-parser/tsdown.config.ts
@@ -0,0 +1,9 @@
+import { defineConfig } from 'tsdown';
+
+export default defineConfig({
+ entry: ['./src/index.ts'],
+ dts: { sourcemap: true },
+ sourcemap: true,
+ clean: true,
+ exports: true, // auto update the package.json exports, main, module and types fields
+});
diff --git a/packages/search-parser/vitest.config.ts b/packages/search-parser/vitest.config.ts
new file mode 100644
index 00000000..fca640ba
--- /dev/null
+++ b/packages/search-parser/vitest.config.ts
@@ -0,0 +1,9 @@
+import { defineConfig } from 'vitest/config';
+
+export default defineConfig({
+ test: {
+ env: {
+ TZ: 'UTC',
+ },
+ },
+});
diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml
index 61e15024..bd735dec 100644
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -134,7 +134,7 @@ importers:
dependencies:
'@better-auth/expo':
specifier: 'catalog:'
'@corentinth/chisels':
specifier: 'catalog:'
version: 2.1.0
@@ -738,6 +738,27 @@ importers:
specifier: 'catalog:'
+ packages/search-parser:
+ devDependencies:
+ '@antfu/eslint-config':
+ specifier: 'catalog:'
+ eslint:
+ specifier: 'catalog:'
+ version: 9.39.1([email protected])
+ standardwebhooks:
+ specifier: ^1.0.0
+ version: 1.0.0
+ tsdown:
+ specifier: 'catalog:'
+ version: 0.13.4([email protected])
+ typescript:
+ specifier: 'catalog:'
+ version: 5.9.3
+ vitest:
+ specifier: 'catalog:'
+
packages/webhooks:
dependencies:
'@corentinth/chisels':
@@ -13736,23 +13757,12 @@ snapshots:
nanostores: 1.0.1
zod: 4.1.12
- dependencies:
- '@better-auth/utils': 0.3.0
- '@better-fetch/fetch': 1.1.18
- '@standard-schema/spec': 1.0.0
- better-call: 1.1.5([email protected])
- jose: 6.1.0
- kysely: 0.28.8
- nanostores: 1.0.1
- zod: 4.1.12
-
dependencies:
'@better-auth/utils': 0.3.0
'@better-fetch/fetch': 1.1.18
'@standard-schema/spec': 1.0.0
- better-call: 1.1.5([email protected])
+ better-call: 1.1.5([email protected])
jose: 6.1.0
kysely: 0.28.8
nanostores: 1.0.1
@@ -13771,9 +13781,9 @@ snapshots:
expo-network: 8.0.8([email protected])([email protected])
dependencies:
'@better-fetch/fetch': 1.1.18
better-call: 1.1.5([email protected])
@@ -18672,7 +18682,7 @@ snapshots:
'@better-fetch/fetch': 1.1.18
'@noble/ciphers': 2.0.1
'@noble/hashes': 2.0.1
- better-call: 1.1.5([email protected])
+ better-call: 1.1.5([email protected])
defu: 6.1.4
jose: 6.1.0
kysely: 0.28.8