The minimalistic document archiving platform.
feat(search-parser): add search query parser package (#728)
Corentin Thomasset417d21556020
diff --git a/packages/search-parser/LICENSE b/packages/search-parser/LICENSEnew file mode 100644index 00000000..820dafbb--- /dev/null+++ b/packages/search-parser/LICENSE@@ -0,0 +1,21 @@+MIT License++Copyright (c) 2026 Corentin THOMASSET++Permission is hereby granted, free of charge, to any person obtaining a copy+of this software and associated documentation files (the "Software"), to deal+in the Software without restriction, including without limitation the rights+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell+copies of the Software, and to permit persons to whom the Software is+furnished to do so, subject to the following conditions:++The above copyright notice and this permission notice shall be included in all+copies or substantial portions of the Software.++THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE+SOFTWARE.diff --git a/packages/search-parser/README.md b/packages/search-parser/README.mdnew file mode 100644index 00000000..157e3fc0--- /dev/null+++ b/packages/search-parser/README.md@@ -0,0 +1,255 @@+# @papra/search-parser++A search query parser library for building GitHub-style search syntax with filters, logical operators, and full-text search.++## Features++- **Full-text search**: Extract search terms from queries+- **Filters**: Support for field:value syntax with multiple operators (=, >, <, >=, <=)+- **Logical operators**: AND, OR, NOT with proper precedence+- **Grouping**: Parentheses for controlling evaluation order+- **Negation**: Both `-filter:value` and `NOT filter:value` syntax+- **Quoted values**: Support for spaces in filter values and search terms+- **Escape sequences**: Escape quotes and colons with backslash+- **Configurable limits**: Max depth and token count for safety+- **Graceful error handling**: Best-effort parsing with issue reporting+- **Zero dependencies**: No runtime dependencies+- **Universal**: Works in Node.js, browsers, Deno, Cloudflare Workers, etc.+- **Type-safe**: Fully typed with TypeScript++## Installation++```bash+npm install @papra/search-parser+# or+pnpm add @papra/search-parser+# or+yarn add @papra/search-parser+```++## Usage++```typescript+import { parseSearchQuery } from '@papra/search-parser';++// Simple full-text search+const result1 = parseSearchQuery({ query: 'my invoice' });+// {+// expression: { type: 'and', operands: [] },+// search: 'my invoice',+// issues: []+// }++// Filter with equality+const result2 = parseSearchQuery({ query: 'tag:invoice' });+// {+// expression: {+// type: 'filter',+// field: 'tag',+// operator: '=',+// value: 'invoice'+// },+// search: undefined,+// issues: []+// }++// Complex query with operators and grouping+const result3 = parseSearchQuery({+ query: '(tag:invoice OR tag:receipt) AND createdAt:>2024-01-01'+});+// {+// expression: {+// type: 'and',+// operands: [+// {+// type: 'or',+// operands: [+// { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },+// { type: 'filter', field: 'tag', operator: '=', value: 'receipt' }+// ]+// },+// { type: 'filter', field: 'createdAt', operator: '>', value: '2024-01-01' }+// ]+// },+// search: undefined,+// issues: []+// }+```++## Query Syntax++### Full-text Search++```+my invoice+"my special invoice" # Quoted for multi-word terms+"my \"special\" invoice" # Escaped quotes+```++### Filters++```+tag:invoice # Equality (implicit)+tag:=invoice # Equality (explicit)+createdAt:>2024-01-01 # Greater than+createdAt:<2024-12-31 # Less than+createdAt:>=2024-01-01 # Greater than or equal+createdAt:<=2024-12-31 # Less than or equal+```++### Quoted Filter Values++```+tag:"my invoices" # Spaces in value+tag:"my \"special\" invoices" # Escaped quotes in value+tag:my\:\:special\:\:tag # Escaped colons in value+```++### Logical Operators++```+tag:invoice AND status:active # Explicit AND+tag:invoice status:active # Implicit AND+tag:invoice OR tag:receipt # OR+NOT tag:personal # NOT+tag:invoice OR tag:receipt AND status:active # Precedence: AND > OR+```++### Negation++```+-tag:personal # Minus prefix+NOT tag:personal # NOT keyword+NOT (tag:personal OR tag:private) # Negated group+```++### Grouping++```+(tag:invoice OR tag:receipt)+(tag:invoice OR tag:receipt) AND status:active+```++### Combining Filters and Search++```+tag:invoice my document # Filter + search+foo tag:invoice bar # Search terms can be anywhere+```++### Escaping++```+tag\:invoice # Escape colon to prevent filter parsing+ # Results in search text "tag:invoice"+```++## API++### parseSearchQuery++```typescript+function parseSearchQuery(options: {+ query: string;+ maxDepth?: number; // Default: 10+ maxTokens?: number; // Default: 200+}): ParsedQuery;+```++## Operator Precedence++From highest to lowest:++1. **NOT** (highest)+2. **AND**+3. **OR** (lowest)++Use parentheses to override precedence:++```+tag:invoice OR tag:receipt AND status:active+# Parsed as: tag:invoice OR (tag:receipt AND status:active)++(tag:invoice OR tag:receipt) AND status:active+# Parsed as: (tag:invoice OR tag:receipt) AND status:active+```++## Error Handling++### Issue Structure++All parsing issues are returned with both a machine-readable error code and a human-readable message:++```typescript+type Issue = {+ code: string; // Machine-readable error code+ message: string; // Human-readable error message+};+```++### Error Codes++The library exports an `ERROR_CODES` constant with all available error codes:++```typescript+import { ERROR_CODES } from '@papra/search-parser';++const result = parseSearchQuery({ query: '(tag:invoice' });++// Check for specific error by code+const hasUnmatchedParen = result.issues.some(+ issue => issue.code === ERROR_CODES.UNMATCHED_OPENING_PARENTHESIS+);+```++Available error codes are defined in [`src/errors.ts`](src/errors.ts).++## Safety Features++### Maximum Depth++Prevents deeply nested expressions (e.g., excessive parentheses):++```typescript+parseSearchQuery({+ query: '((((((((((tag:invoice))))))))))',+ maxDepth: 5 // Will report issue if exceeded+});+```++### Maximum Tokens++Prevents excessively long queries:++```typescript+parseSearchQuery({+ query: 'tag1:val1 tag2:val2 ... tag100:val100',+ maxTokens: 50 // Will report issue if exceeded+});+```++### Graceful Error Handling++Malformed queries are handled gracefully with best-effort parsing:++```typescript+parseSearchQuery({ query: '(tag:invoice' });+// {+// expression: { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },+// search: undefined,+// issues: [+// {+// code: 'unmatched-opening-parenthesis',+// message: 'Unmatched opening parenthesis'+// }+// ]+// }+```++## License++MIT++## Contributing++Contributions are welcome! Please open an issue or pull request on GitHub.diff --git a/packages/search-parser/eslint.config.js b/packages/search-parser/eslint.config.jsnew file mode 100644index 00000000..2aae9a63--- /dev/null+++ b/packages/search-parser/eslint.config.js@@ -0,0 +1,26 @@+import antfu from '@antfu/eslint-config';++export default antfu({+ stylistic: {+ semi: true,+ },++ ignores: [+ 'example.ts',+ 'README.md',+ ],++ rules: {+ // To allow export on top of files+ 'ts/no-use-before-define': ['error', { allowNamedExports: true, functions: false }],+ 'curly': ['error', 'all'],+ 'vitest/consistent-test-it': ['error', { fn: 'test' }],+ 'ts/consistent-type-definitions': ['error', 'type'],+ 'style/brace-style': ['error', '1tbs', { allowSingleLine: false }],+ 'unused-imports/no-unused-vars': ['error', {+ argsIgnorePattern: '^_',+ varsIgnorePattern: '^_',+ caughtErrorsIgnorePattern: '^_',+ }],+ },+});diff --git a/packages/search-parser/package.json b/packages/search-parser/package.jsonnew file mode 100644index 00000000..4ed65cc5--- /dev/null+++ b/packages/search-parser/package.json@@ -0,0 +1,52 @@+{+ "name": "@papra/search-parser",+ "type": "module",+ "version": "0.0.0",+ "description": "Search query parser library",+ "author": "Corentin Thomasset <[email protected]> (https://corentin.tech)",+ "license": "MIT",+ "repository": {+ "type": "git",+ "url": "https://github.com/papra-hq/papra",+ "directory": "packages/search-parser"+ },+ "bugs": {+ "url": "https://github.com/papra-hq/papra/issues"+ },+ "keywords": [+ "papra",+ "search",+ "parser",+ "query"+ ],+ "exports": {+ ".": "./dist/index.js",+ "./package.json": "./package.json"+ },+ "main": "./dist/index.js",+ "module": "./dist/index.js",+ "types": "./dist/index.d.ts",+ "files": [+ "dist"+ ],+ "scripts": {+ "lint": "eslint .",+ "lint:fix": "eslint --fix .",+ "test": "vitest run",+ "test:watch": "vitest watch",+ "typecheck": "tsc --noEmit",+ "build": "tsdown",+ "build:watch": "tsdown --watch",+ "dev": "pnpm build:watch",+ "prepublishOnly": "pnpm run build"+ },+ "dependencies": {},+ "devDependencies": {+ "@antfu/eslint-config": "catalog:",+ "eslint": "catalog:",+ "standardwebhooks": "^1.0.0",+ "tsdown": "catalog:",+ "typescript": "catalog:",+ "vitest": "catalog:"+ }+}diff --git a/packages/search-parser/src/errors.ts b/packages/search-parser/src/errors.tsnew file mode 100644index 00000000..9f4758de--- /dev/null+++ b/packages/search-parser/src/errors.ts@@ -0,0 +1,8 @@+export const ERROR_CODES = {+ MAX_TOKENS_EXCEEDED: 'max-tokens-exceeded',+ MAX_NESTING_DEPTH_EXCEEDED: 'max-nesting-depth-exceeded',+ UNMATCHED_OPENING_PARENTHESIS: 'unmatched-opening-parenthesis',+ UNMATCHED_CLOSING_PARENTHESIS: 'unmatched-closing-parenthesis',+ UNCLOSED_QUOTED_STRING: 'unclosed-quoted-string',+ MISSING_OPERAND_FOR_NOT: 'missing-operand-for-not',+};diff --git a/packages/search-parser/src/index.ts b/packages/search-parser/src/index.tsnew file mode 100644index 00000000..680e0b56--- /dev/null+++ b/packages/search-parser/src/index.ts@@ -0,0 +1,18 @@+export {+ ERROR_CODES,+} from './errors';++export {+ parseSearchQuery,+} from './parser';++export type {+ AndExpression,+ Expression,+ FilterExpression,+ Issue,+ NotExpression,+ Operator,+ OrExpression,+ ParsedQuery,+} from './parser.types';diff --git a/packages/search-parser/src/parser.test.ts b/packages/search-parser/src/parser.test.tsnew file mode 100644index 00000000..f6c182d0--- /dev/null+++ b/packages/search-parser/src/parser.test.ts@@ -0,0 +1,569 @@+import { describe, expect, test } from 'vitest';+import { parseSearchQuery } from './parser';++describe('parseSearchQuery', () => {+ describe('when parsing simple queries', () => {+ test('returns empty expression and no search terms for empty query', () => {+ expect(parseSearchQuery({ query: '' })).toEqual({+ expression: { type: 'and', operands: [] },+ search: undefined,+ issues: [],+ });+ });++ test('extracts full-text search terms when no filters present', () => {+ expect(parseSearchQuery({ query: 'my invoice' })).toEqual({+ expression: { type: 'and', operands: [] },+ search: 'my invoice',+ issues: [],+ });+ });++ test('handles quoted search terms', () => {+ expect(parseSearchQuery({ query: '"my special invoice"' })).toEqual({+ expression: { type: 'and', operands: [] },+ search: 'my special invoice',+ issues: [],+ });+ });++ test('handles escaped quotes in search terms', () => {+ expect(parseSearchQuery({ query: '"my \\"special\\" invoice"' })).toEqual({+ expression: { type: 'and', operands: [] },+ search: 'my "special" invoice',+ issues: [],+ });+ });++ test('handles escaped colons to prevent filter parsing', () => {+ expect(parseSearchQuery({ query: 'tag\\:invoice' })).toEqual({+ expression: { type: 'and', operands: [] },+ search: 'tag:invoice',+ issues: [],+ });+ });+ });++ describe('when parsing filter expressions', () => {+ test('parses simple filter with equality operator', () => {+ expect(parseSearchQuery({ query: 'tag:invoice' })).toEqual({+ expression: {+ type: 'filter',+ field: 'tag',+ operator: '=',+ value: 'invoice',+ },+ search: undefined,+ issues: [],+ });+ });++ test('parses filter with explicit equality operator', () => {+ expect(parseSearchQuery({ query: 'tag:=invoice' })).toEqual({+ expression: {+ type: 'filter',+ field: 'tag',+ operator: '=',+ value: 'invoice',+ },+ search: undefined,+ issues: [],+ });+ });++ test('parses filter with quoted value containing spaces', () => {+ expect(parseSearchQuery({ query: 'tag:"my invoices"' })).toEqual({+ expression: {+ type: 'filter',+ field: 'tag',+ operator: '=',+ value: 'my invoices',+ },+ search: undefined,+ issues: [],+ });+ });++ test('parses filter with escaped quotes in value', () => {+ expect(parseSearchQuery({ query: 'tag:"my \\"special\\" invoices"' })).toEqual({+ expression: {+ type: 'filter',+ field: 'tag',+ operator: '=',+ value: 'my "special" invoices',+ },+ search: undefined,+ issues: [],+ });+ });++ test('parses filter with escaped colons in value', () => {+ expect(parseSearchQuery({ query: 'tag:my\\:\\:special\\:\\:tag' })).toEqual({+ expression: {+ type: 'filter',+ field: 'tag',+ operator: '=',+ value: 'my::special::tag',+ },+ search: undefined,+ issues: [],+ });+ });++ test('parses filter with escaped colons in field name', () => {+ expect(parseSearchQuery({ query: 'my\\:field:value' })).toEqual({+ expression: {+ type: 'filter',+ field: 'my:field',+ operator: '=',+ value: 'value',+ },+ search: undefined,+ issues: [],+ });+ });++ test('parses filter with comparison operators', () => {+ expect(parseSearchQuery({ query: 'createdAt:>2024-01-01' })).toEqual({+ expression: {+ type: 'filter',+ field: 'createdAt',+ operator: '>',+ value: '2024-01-01',+ },+ search: undefined,+ issues: [],+ });++ expect(parseSearchQuery({ query: 'createdAt:<2024-01-01' })).toEqual({+ expression: {+ type: 'filter',+ field: 'createdAt',+ operator: '<',+ value: '2024-01-01',+ },+ search: undefined,+ issues: [],+ });++ expect(parseSearchQuery({ query: 'createdAt:>=2024-01-01' })).toEqual({+ expression: {+ type: 'filter',+ field: 'createdAt',+ operator: '>=',+ value: '2024-01-01',+ },+ search: undefined,+ issues: [],+ });++ expect(parseSearchQuery({ query: 'createdAt:<=2024-01-01' })).toEqual({+ expression: {+ type: 'filter',+ field: 'createdAt',+ operator: '<=',+ value: '2024-01-01',+ },+ search: undefined,+ issues: [],+ });+ });++ test('parses negated filter with minus prefix', () => {+ expect(parseSearchQuery({ query: '-tag:personal' })).toEqual({+ expression: {+ type: 'not',+ operand: {+ type: 'filter',+ field: 'tag',+ operator: '=',+ value: 'personal',+ },+ },+ search: undefined,+ issues: [],+ });+ });+ });++ describe('when combining filters and search terms', () => {+ test('combines filter with search terms', () => {+ expect(parseSearchQuery({ query: 'tag:invoice my invoice' })).toEqual({+ expression: {+ type: 'filter',+ field: 'tag',+ operator: '=',+ value: 'invoice',+ },+ search: 'my invoice',+ issues: [],+ });+ });++ test('extracts search terms before and after filters', () => {+ expect(parseSearchQuery({ query: 'foo tag:invoice bar' })).toEqual({+ expression: {+ type: 'filter',+ field: 'tag',+ operator: '=',+ value: 'invoice',+ },+ search: 'foo bar',+ issues: [],+ });+ });+ });++ describe('when using AND operator', () => {+ test('combines filters with explicit AND', () => {+ expect(parseSearchQuery({ query: 'tag:invoice AND tag:receipt' })).toEqual({+ expression: {+ type: 'and',+ operands: [+ { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },+ { type: 'filter', field: 'tag', operator: '=', value: 'receipt' },+ ],+ },+ search: undefined,+ issues: [],+ });+ });++ test('combines filters with implicit AND', () => {+ expect(parseSearchQuery({ query: 'tag:invoice tag:receipt' })).toEqual({+ expression: {+ type: 'and',+ operands: [+ { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },+ { type: 'filter', field: 'tag', operator: '=', value: 'receipt' },+ ],+ },+ search: undefined,+ issues: [],+ });+ });++ test('combines multiple filters with AND', () => {+ expect(parseSearchQuery({ query: 'tag:invoice createdAt:>2024-01-01 status:active' })).toEqual({+ expression: {+ type: 'and',+ operands: [+ { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },+ { type: 'filter', field: 'createdAt', operator: '>', value: '2024-01-01' },+ { type: 'filter', field: 'status', operator: '=', value: 'active' },+ ],+ },+ search: undefined,+ issues: [],+ });+ });+ });++ describe('when using OR operator', () => {+ test('combines filters with OR', () => {+ expect(parseSearchQuery({ query: 'tag:invoice OR tag:receipt' })).toEqual({+ expression: {+ type: 'or',+ operands: [+ { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },+ { type: 'filter', field: 'tag', operator: '=', value: 'receipt' },+ ],+ },+ search: undefined,+ issues: [],+ });+ });++ test('combines multiple filters with OR', () => {+ expect(parseSearchQuery({ query: 'tag:invoice OR tag:receipt OR tag:bill' })).toEqual({+ expression: {+ type: 'or',+ operands: [+ { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },+ { type: 'filter', field: 'tag', operator: '=', value: 'receipt' },+ { type: 'filter', field: 'tag', operator: '=', value: 'bill' },+ ],+ },+ search: undefined,+ issues: [],+ });+ });+ });++ describe('when using NOT operator', () => {+ test('negates filter with NOT keyword', () => {+ expect(parseSearchQuery({ query: 'NOT tag:personal' })).toEqual({+ expression: {+ type: 'not',+ operand: {+ type: 'filter',+ field: 'tag',+ operator: '=',+ value: 'personal',+ },+ },+ search: undefined,+ issues: [],+ });+ });++ test('negates grouped expression', () => {+ expect(parseSearchQuery({ query: 'NOT (tag:personal OR tag:private)' })).toEqual({+ expression: {+ type: 'not',+ operand: {+ type: 'or',+ operands: [+ { type: 'filter', field: 'tag', operator: '=', value: 'personal' },+ { type: 'filter', field: 'tag', operator: '=', value: 'private' },+ ],+ },+ },+ search: undefined,+ issues: [],+ });+ });+ });++ describe('when using grouping with parentheses', () => {+ test('groups expressions with parentheses', () => {+ expect(parseSearchQuery({ query: '(tag:invoice OR tag:receipt)' })).toEqual({+ expression: {+ type: 'or',+ operands: [+ { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },+ { type: 'filter', field: 'tag', operator: '=', value: 'receipt' },+ ],+ },+ search: undefined,+ issues: [],+ });+ });++ test('combines grouped expressions with AND', () => {+ expect(parseSearchQuery({ query: '(tag:invoice OR tag:receipt) AND createdAt:>2024-01-01' })).toEqual({+ expression: {+ type: 'and',+ operands: [+ {+ type: 'or',+ operands: [+ { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },+ { type: 'filter', field: 'tag', operator: '=', value: 'receipt' },+ ],+ },+ { type: 'filter', field: 'createdAt', operator: '>', value: '2024-01-01' },+ ],+ },+ search: undefined,+ issues: [],+ });+ });++ test('handles nested grouping', () => {+ expect(parseSearchQuery({ query: '((tag:invoice OR tag:receipt) AND status:active)' })).toEqual({+ expression: {+ type: 'and',+ operands: [+ {+ type: 'or',+ operands: [+ { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },+ { type: 'filter', field: 'tag', operator: '=', value: 'receipt' },+ ],+ },+ { type: 'filter', field: 'status', operator: '=', value: 'active' },+ ],+ },+ search: undefined,+ issues: [],+ });+ });+ });++ describe('when respecting operator precedence', () => {+ test('aND binds tighter than OR', () => {+ expect(parseSearchQuery({ query: 'tag:invoice OR tag:receipt AND tag:company' })).toEqual({+ expression: {+ type: 'or',+ operands: [+ { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },+ {+ type: 'and',+ operands: [+ { type: 'filter', field: 'tag', operator: '=', value: 'receipt' },+ { type: 'filter', field: 'tag', operator: '=', value: 'company' },+ ],+ },+ ],+ },+ search: undefined,+ issues: [],+ });+ });++ test('nOT binds tighter than AND', () => {+ expect(parseSearchQuery({ query: 'NOT tag:invoice AND tag:receipt' })).toEqual({+ expression: {+ type: 'and',+ operands: [+ {+ type: 'not',+ operand: { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },+ },+ { type: 'filter', field: 'tag', operator: '=', value: 'receipt' },+ ],+ },+ search: undefined,+ issues: [],+ });+ });++ test('parentheses override precedence', () => {+ expect(parseSearchQuery({ query: '(tag:invoice OR tag:receipt) AND tag:company' })).toEqual({+ expression: {+ type: 'and',+ operands: [+ {+ type: 'or',+ operands: [+ { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },+ { type: 'filter', field: 'tag', operator: '=', value: 'receipt' },+ ],+ },+ { type: 'filter', field: 'tag', operator: '=', value: 'company' },+ ],+ },+ search: undefined,+ issues: [],+ });+ });+ });++ describe('when handling malformed queries', () => {+ test('handles unclosed quoted string gracefully', () => {+ const result = parseSearchQuery({ query: '"unclosed string' });+ expect(result.search).toBe('unclosed string');+ expect(result.issues).toContainEqual({+ code: 'unclosed-quoted-string',+ message: 'Unclosed quoted string',+ });+ });++ test('handles unmatched opening parenthesis gracefully', () => {+ const result = parseSearchQuery({ query: '(tag:invoice' });+ expect(result.expression).toEqual({+ type: 'filter',+ field: 'tag',+ operator: '=',+ value: 'invoice',+ });+ expect(result.issues).toContainEqual({+ code: 'unmatched-opening-parenthesis',+ message: 'Unmatched opening parenthesis',+ });+ });++ test('handles unmatched closing parenthesis gracefully', () => {+ const result = parseSearchQuery({ query: 'tag:invoice)' });+ expect(result.expression).toEqual({+ type: 'filter',+ field: 'tag',+ operator: '=',+ value: 'invoice',+ });+ expect(result.issues).toContainEqual({+ code: 'unmatched-closing-parenthesis',+ message: 'Unmatched closing parenthesis',+ });+ });++ test('handles NOT without operand gracefully', () => {+ const result = parseSearchQuery({ query: 'tag:invoice AND NOT' });+ expect(result.expression).toEqual({+ type: 'filter',+ field: 'tag',+ operator: '=',+ value: 'invoice',+ });+ expect(result.issues).toContainEqual({+ code: 'missing-operand-for-not',+ message: 'NOT operator requires an operand',+ });+ });+ });++ describe('when enforcing limits', () => {+ test('respects maximum token limit', () => {+ const longQuery = Array.from({ length: 50 }).map((_, i) => `tag${i}:value${i}`).join(' ');+ const result = parseSearchQuery({ query: longQuery, maxTokens: 10 });+ expect(result.issues).toContainEqual({+ code: 'max-tokens-exceeded',+ message: 'Maximum token limit of 10 exceeded',+ });+ });++ test('respects maximum depth limit', () => {+ const deepQuery = '((((((((((tag:invoice))))))))))';+ const result = parseSearchQuery({ query: deepQuery, maxDepth: 5 });+ expect(result.issues).toContainEqual({+ code: 'max-nesting-depth-exceeded',+ message: 'Maximum nesting depth of 5 exceeded',+ });+ });+ });++ describe('when handling complex real-world queries', () => {+ test('parses complex query with multiple operators and grouping', () => {+ expect(+ parseSearchQuery({+ query: '(tag:invoice OR tag:receipt) AND createdAt:>2024-01-01 AND NOT tag:personal my document',+ }),+ ).toEqual({+ expression: {+ type: 'and',+ operands: [+ {+ type: 'or',+ operands: [+ { type: 'filter', field: 'tag', operator: '=', value: 'invoice' },+ { type: 'filter', field: 'tag', operator: '=', value: 'receipt' },+ ],+ },+ { type: 'filter', field: 'createdAt', operator: '>', value: '2024-01-01' },+ {+ type: 'not',+ operand: { type: 'filter', field: 'tag', operator: '=', value: 'personal' },+ },+ ],+ },+ search: 'my document',+ issues: [],+ });+ });++ test('parses query with multiple negations', () => {+ expect(parseSearchQuery({ query: '-tag:personal -tag:private -tag:confidential' })).toEqual({+ expression: {+ type: 'and',+ operands: [+ {+ type: 'not',+ operand: { type: 'filter', field: 'tag', operator: '=', value: 'personal' },+ },+ {+ type: 'not',+ operand: { type: 'filter', field: 'tag', operator: '=', value: 'private' },+ },+ {+ type: 'not',+ operand: { type: 'filter', field: 'tag', operator: '=', value: 'confidential' },+ },+ ],+ },+ search: undefined,+ issues: [],+ });+ });+ });+});diff --git a/packages/search-parser/src/parser.ts b/packages/search-parser/src/parser.tsnew file mode 100644index 00000000..d3979062--- /dev/null+++ b/packages/search-parser/src/parser.ts@@ -0,0 +1,208 @@+import type { Expression, Issue, ParsedQuery } from './parser.types';+import type { Token } from './tokenizer';+import { ERROR_CODES } from './errors';+import { tokenize } from './tokenizer';++export function parseSearchQuery(+ {+ query,+ maxDepth = 10,+ maxTokens = 200,+ }: {+ query: string;+ maxDepth?: number;+ maxTokens?: number;+ },+): ParsedQuery {+ const { tokens, issues: tokenizerIssues } = tokenize({ query, maxTokens });++ const { expression, issues: parserIssues, search } = parseExpression({ tokens, maxDepth });++ return {+ expression,+ search,+ issues: [...tokenizerIssues, ...parserIssues],+ };+}++function parseExpression({ tokens, maxDepth }: { tokens: Token[]; maxDepth: number }): ParsedQuery {+ const searchTerms: string[] = [];+ const parserIssues: Issue[] = [];++ let currentTokenIndex = 0;+ let currentDepth = 0;++ const peek = (): Token => tokens[currentTokenIndex] ?? { type: 'EOF' };+ const advance = (): Token => tokens[currentTokenIndex++] ?? { type: 'EOF' };++ const checkDepth = (): boolean => {+ if (currentDepth >= maxDepth) {+ parserIssues.push({+ code: ERROR_CODES.MAX_NESTING_DEPTH_EXCEEDED,+ message: `Maximum nesting depth of ${maxDepth} exceeded`,+ });+ return false;+ }+ return true;+ };++ // Parse primary expression (filter, parentheses, text)+ function parsePrimaryExpression(): Expression | undefined {+ const token = peek();++ if (token.type === 'LPAREN') {+ advance(); // Consume (++ if (!checkDepth()) {+ return undefined;+ }++ currentDepth++;+ const expr = parseOrExpression();+ currentDepth--;++ if (peek().type === 'RPAREN') {+ advance(); // Consume )+ } else {+ parserIssues.push({+ code: ERROR_CODES.UNMATCHED_OPENING_PARENTHESIS,+ message: 'Unmatched opening parenthesis',+ });+ }++ return expr;+ }++ if (token.type === 'FILTER') {+ advance();+ const filterExpr: Expression = {+ type: 'filter',+ field: token.field,+ operator: token.operator,+ value: token.value,+ };++ if (token.negated) {+ return { type: 'not', operand: filterExpr };+ }++ return filterExpr;+ }++ if (token.type === 'TEXT') {+ advance();+ searchTerms.push(token.value);+ return undefined; // Text doesn't produce an expression+ }++ return undefined;+ }++ // Parse NOT expression (highest precedence)+ function parseUnaryExpression(): Expression | undefined {+ if (peek().type === 'NOT') {+ advance(); // Consume NOT++ if (!checkDepth()) {+ return undefined;+ }++ currentDepth++;+ const operand = parseUnaryExpression();+ currentDepth--;++ if (!operand) {+ parserIssues.push({+ code: ERROR_CODES.MISSING_OPERAND_FOR_NOT,+ message: 'NOT operator requires an operand',+ });+ return undefined;+ }++ return { type: 'not', operand };+ }++ return parsePrimaryExpression();+ }++ // Parse AND expression (higher precedence than OR)+ function parseAndExpression(): Expression | undefined {+ const operands: Expression[] = [];++ while (true) {+ const next = peek();++ // Stop if we hit EOF, OR operator, or closing paren+ if (next.type === 'EOF' || next.type === 'OR' || next.type === 'RPAREN') {+ break;+ }++ // Consume explicit AND operator+ if (next.type === 'AND') {+ advance();+ continue;+ }++ const expr = parseUnaryExpression();+ if (expr) {+ operands.push(expr);+ }+ }++ if (operands.length === 0) {+ return undefined;+ }++ if (operands.length === 1) {+ return operands[0];+ }++ return { type: 'and', operands };+ };++ // Parse OR expression (lowest precedence)+ function parseOrExpression(): Expression | undefined {+ const left = parseAndExpression();+ if (!left) {+ return undefined;+ }++ const operands: Expression[] = [left];++ while (peek().type === 'OR') {+ advance(); // Consume OR+ const right = parseAndExpression();+ if (right) {+ operands.push(right);+ }+ }++ if (operands.length === 1) {+ return operands[0];+ }++ return { type: 'or', operands };+ };++ // Start parsing+ const expression = parseOrExpression();++ // Check for unmatched closing parentheses+ while (peek().type === 'RPAREN') {+ parserIssues.push({+ message: 'Unmatched closing parenthesis',+ code: ERROR_CODES.UNMATCHED_CLOSING_PARENTHESIS,+ });+ advance();+ }++ // Build final result+ const finalExpression: Expression = expression ?? { type: 'and', operands: [] };+ const search = searchTerms.length > 0 ? searchTerms.join(' ') : undefined;++ return {+ expression: finalExpression,+ search,+ issues: parserIssues,+ };+}diff --git a/packages/search-parser/src/parser.types.ts b/packages/search-parser/src/parser.types.tsnew file mode 100644index 00000000..caba3469--- /dev/null+++ b/packages/search-parser/src/parser.types.ts@@ -0,0 +1,40 @@+export type Issue = {+ message: string;+ code: string;+};++export type ParsedQuery = {+ expression: Expression;+ search: string | undefined;+ issues: Issue[];+};++export type Expression+ = | AndExpression+ | OrExpression+ | NotExpression+ | FilterExpression;++export type AndExpression = {+ type: 'and';+ operands: Expression[];+};++export type OrExpression = {+ type: 'or';+ operands: Expression[];+};++export type NotExpression = {+ type: 'not';+ operand: Expression;+};++export type Operator = '>' | '<' | '>=' | '<=' | '=';++export type FilterExpression = {+ type: 'filter';+ field: string;+ operator: Operator;+ value: string;+};diff --git a/packages/search-parser/src/tokenizer.ts b/packages/search-parser/src/tokenizer.tsnew file mode 100644index 00000000..9904b0e6--- /dev/null+++ b/packages/search-parser/src/tokenizer.ts@@ -0,0 +1,288 @@+import type { Issue, Operator } from './parser.types';+import { ERROR_CODES } from './errors';++export type Token+ = | { type: 'LPAREN' }+ | { type: 'RPAREN' }+ | { type: 'AND' }+ | { type: 'OR' }+ | { type: 'NOT' }+ | { type: 'FILTER'; field: string; operator: Operator; value: string; negated: boolean }+ | { type: 'TEXT'; value: string }+ | { type: 'EOF' };++export type TokenizeResult = {+ tokens: Token[];+ issues: Issue[];+};++export function tokenize({ query, maxTokens }: { query: string; maxTokens: number }): TokenizeResult {+ const tokens: Token[] = [];+ const issues: Issue[] = [];+ let pos = 0;++ const peek = (): string | undefined => query[pos];+ const advance = (): string => query[pos++] || '';++ const skipWhitespace = (): void => {+ while (pos < query.length) {+ const char = query[pos];+ if (!char || !/\s/.test(char)) {+ break;+ }+ pos++;+ }+ };++ const readQuotedString = (): string | undefined => {+ if (peek() !== '"') {+ return undefined;+ }++ advance(); // Skip opening quote+ let value = '';+ let escaped = false;++ while (pos < query.length) {+ const char = peek();++ if (!char) {+ break;+ }++ if (escaped) {+ value += char;+ escaped = false;+ advance();+ } else if (char === '\\') {+ escaped = true;+ advance();+ } else if (char === '"') {+ advance(); // Skip closing quote+ return value;+ } else {+ value += char;+ advance();+ }+ }++ issues.push({+ code: ERROR_CODES.UNCLOSED_QUOTED_STRING,+ message: 'Unclosed quoted string',+ });+ return value;+ };++ const readUnquotedToken = (stopAtQuote = false): string => {+ let value = '';++ while (pos < query.length) {+ const char = peek();++ if (!char || /\s/.test(char) || char === '(' || char === ')') {+ break;+ }++ // Stop at quote if requested (for filter field names before quoted values)+ if (stopAtQuote && char === '"') {+ break;+ }++ // Keep backslashes in unquoted tokens so parseFilter can detect escaped colons+ value += advance();+ }++ return value;+ };++ const readFilterValue = (): string => {+ skipWhitespace();++ // Check if value is quoted+ const quotedValue = readQuotedString();+ if (quotedValue !== undefined) {+ return quotedValue;+ }++ // Read unquoted value (up to whitespace or special chars)+ let value = '';+ while (pos < query.length) {+ const char = peek();++ if (!char || /\s/.test(char) || char === '(' || char === ')') {+ break;+ }++ if (char === '\\') {+ advance();+ if (pos < query.length) {+ value += advance();+ }+ } else {+ value += advance();+ }+ }++ return value.replace(/\\:/g, ':');+ };++ const hasUnescapedColon = (str: string): boolean => {+ for (let i = 0; i < str.length; i++) {+ if (str[i] === ':' && (i === 0 || str[i - 1] !== '\\')) {+ return true;+ }+ }+ return false;+ };++ const parseFilter = (token: string, negated: boolean): Token | undefined => {+ // Check for unescaped colons+ if (!hasUnescapedColon(token)) {+ return undefined;+ }++ // Find first unescaped colon+ let firstColonIndex = -1;+ for (let i = 0; i < token.length; i++) {+ if (token[i] === ':' && (i === 0 || token[i - 1] !== '\\')) {+ firstColonIndex = i;+ break;+ }+ }++ if (firstColonIndex === -1) {+ return undefined;+ }++ const field = token.slice(0, firstColonIndex).replace(/\\:/g, ':');+ const afterColon = token.slice(firstColonIndex + 1);++ // Check for operator at the start+ let operator: Operator = '=';+ let operatorLength = 0;++ if (afterColon.startsWith('>=')) {+ operator = '>=';+ operatorLength = 2;+ } else if (afterColon.startsWith('<=')) {+ operator = '<=';+ operatorLength = 2;+ } else if (afterColon.startsWith('>')) {+ operator = '>';+ operatorLength = 1;+ } else if (afterColon.startsWith('<')) {+ operator = '<';+ operatorLength = 1;+ } else if (afterColon.startsWith('=')) {+ operator = '=';+ operatorLength = 1;+ }++ // If there's nothing after the operator in the token, read the value from input+ let value = afterColon.slice(operatorLength).replace(/\\:/g, ':');++ // If the value is empty and we still have input, this might be a case like "tag:" followed by a quoted string+ if (!value) {+ value = readFilterValue();+ }++ return { type: 'FILTER', field, operator, value, negated };+ };++ while (pos < query.length) {+ if (tokens.length >= maxTokens) {+ issues.push({+ code: ERROR_CODES.MAX_TOKENS_EXCEEDED,+ message: `Maximum token limit of ${maxTokens} exceeded`,+ });+ break;+ }++ skipWhitespace();++ if (pos >= query.length) {+ break;+ }++ const char = peek();++ // Handle parentheses+ if (char === '(') {+ advance();+ tokens.push({ type: 'LPAREN' });+ continue;+ }++ if (char === ')') {+ advance();+ tokens.push({ type: 'RPAREN' });+ continue;+ }++ // Handle negation prefix+ const nextChar = query[pos + 1];+ if (char === '-' && nextChar && !/\s/.test(nextChar)) {+ advance();+ skipWhitespace();++ // Read the next token (could be quoted or unquoted)+ const quotedValue = readQuotedString();+ const token = quotedValue !== undefined ? quotedValue : readUnquotedToken();++ // Try to parse as filter+ const filter = parseFilter(token, true);+ if (filter) {+ tokens.push(filter);+ } else {+ // If not a filter, treat as negated text search (which we'll handle as NOT operator)+ const unescapedText = token.replace(/\\(.)/g, '$1');+ tokens.push({ type: 'NOT' });+ tokens.push({ type: 'TEXT', value: unescapedText });+ }+ continue;+ }++ // Read quoted or unquoted token+ const quotedValue = readQuotedString();+ const token = quotedValue !== undefined ? quotedValue : readUnquotedToken(true);++ if (!token) {+ advance(); // Skip invalid character+ continue;+ }++ // Check for operators+ const upperToken = token.toUpperCase();+ if (upperToken === 'AND') {+ tokens.push({ type: 'AND' });+ continue;+ }++ if (upperToken === 'OR') {+ tokens.push({ type: 'OR' });+ continue;+ }++ if (upperToken === 'NOT') {+ tokens.push({ type: 'NOT' });+ continue;+ }++ // Try to parse as filter (only if not quoted)+ if (quotedValue === undefined) {+ const filter = parseFilter(token, false);+ if (filter) {+ tokens.push(filter);+ continue;+ }+ }++ // Otherwise, treat as text (unescape backslashes)+ const unescapedText = token.replace(/\\(.)/g, '$1');+ tokens.push({ type: 'TEXT', value: unescapedText });+ }++ tokens.push({ type: 'EOF' });++ return { tokens, issues };+}diff --git a/packages/search-parser/tsconfig.json b/packages/search-parser/tsconfig.jsonnew file mode 100644index 00000000..43dd7d01--- /dev/null+++ b/packages/search-parser/tsconfig.json@@ -0,0 +1,19 @@+{+ "compilerOptions": {+ "target": "ESNext",+ "moduleDetection": "force",+ "module": "preserve",+ "resolveJsonModule": true,+ "allowJs": true,+ "strict": true,+ "noImplicitOverride": true,+ "noUncheckedIndexedAccess": true,+ "noEmit": true,+ "allowSyntheticDefaultImports": true,+ "esModuleInterop": true,+ "forceConsistentCasingInFileNames": true,+ "isolatedModules": true,+ "verbatimModuleSyntax": true,+ "skipLibCheck": true+ }+}diff --git a/packages/search-parser/tsdown.config.ts b/packages/search-parser/tsdown.config.tsnew file mode 100644index 00000000..bcf4ac58--- /dev/null+++ b/packages/search-parser/tsdown.config.ts@@ -0,0 +1,9 @@+import { defineConfig } from 'tsdown';++export default defineConfig({+ entry: ['./src/index.ts'],+ dts: { sourcemap: true },+ sourcemap: true,+ clean: true,+ exports: true, // auto update the package.json exports, main, module and types fields+});diff --git a/packages/search-parser/vitest.config.ts b/packages/search-parser/vitest.config.tsnew file mode 100644index 00000000..fca640ba--- /dev/null+++ b/packages/search-parser/vitest.config.ts@@ -0,0 +1,9 @@+import { defineConfig } from 'vitest/config';++export default defineConfig({+ test: {+ env: {+ TZ: 'UTC',+ },+ },+});diff --git a/pnpm-lock.yaml b/pnpm-lock.yamlindex 61e15024..bd735dec 100644--- a/pnpm-lock.yaml+++ b/pnpm-lock.yaml@@ -134,7 +134,7 @@ importers:dependencies:'@better-auth/expo':specifier: 'catalog:'- version: 1.4.6(@better-auth/[email protected](@better-auth/[email protected])(@better-fetch/[email protected])([email protected]([email protected]))([email protected])([email protected])([email protected]))([email protected]([email protected]([email protected]))([email protected])([email protected])([email protected]([email protected])))([email protected])([email protected])([email protected]([email protected])([email protected]))([email protected]([email protected])([email protected](@babel/[email protected])(@react-native-community/[email protected]([email protected]))(@types/[email protected])([email protected])))+ version: 1.4.6(@better-auth/[email protected](@better-auth/[email protected])(@better-fetch/[email protected])([email protected]([email protected]))([email protected])([email protected])([email protected]))([email protected]([email protected]([email protected]))([email protected])([email protected])([email protected]([email protected])))([email protected])([email protected])([email protected]([email protected])([email protected]))([email protected]([email protected])([email protected](@babel/[email protected])(@react-native-community/[email protected]([email protected]))(@types/[email protected])([email protected])))'@corentinth/chisels':specifier: 'catalog:'version: 2.1.0@@ -738,6 +738,27 @@ importers:specifier: 'catalog:'version: 4.0.3(@types/[email protected])(@types/[email protected])([email protected])([email protected])([email protected])([email protected])([email protected])([email protected])+ packages/search-parser:+ devDependencies:+ '@antfu/eslint-config':+ specifier: 'catalog:'+ version: 6.2.0(@vue/[email protected])([email protected]([email protected]))([email protected]([email protected]([email protected]))([email protected]))([email protected]([email protected]([email protected]))([email protected]))([email protected]([email protected]))([email protected])([email protected](@types/[email protected])(@types/[email protected])([email protected])([email protected])([email protected])([email protected])([email protected])([email protected]))+ eslint:+ specifier: 'catalog:'+ version: 9.39.1([email protected])+ standardwebhooks:+ specifier: ^1.0.0+ version: 1.0.0+ tsdown:+ specifier: 'catalog:'+ version: 0.13.4([email protected])+ typescript:+ specifier: 'catalog:'+ version: 5.9.3+ vitest:+ specifier: 'catalog:'+ version: 4.0.3(@types/[email protected])(@types/[email protected])([email protected])([email protected])([email protected])([email protected])([email protected])([email protected])+packages/webhooks:dependencies:'@corentinth/chisels':@@ -13736,23 +13757,12 @@ snapshots:nanostores: 1.0.1zod: 4.1.12- '@better-auth/[email protected](@better-auth/[email protected])(@better-fetch/[email protected])([email protected]([email protected]))([email protected])([email protected])([email protected])':- dependencies:- '@better-auth/utils': 0.3.0- '@better-fetch/fetch': 1.1.18- '@standard-schema/spec': 1.0.0- better-call: 1.1.5([email protected])- jose: 6.1.0- kysely: 0.28.8- nanostores: 1.0.1- zod: 4.1.12-'@better-auth/[email protected](@better-auth/[email protected])(@better-fetch/[email protected])([email protected]([email protected]))([email protected])([email protected])([email protected])':dependencies:'@better-auth/utils': 0.3.0'@better-fetch/fetch': 1.1.18'@standard-schema/spec': 1.0.0- better-call: 1.1.5([email protected])+ better-call: 1.1.5([email protected])jose: 6.1.0kysely: 0.28.8nanostores: 1.0.1@@ -13771,9 +13781,9 @@ snapshots:expo-network: 8.0.8([email protected])([email protected])expo-web-browser: 15.0.9([email protected])([email protected](@babel/[email protected])(@react-native-community/[email protected]([email protected]))(@types/[email protected])([email protected]))- '@better-auth/[email protected](@better-auth/[email protected](@better-auth/[email protected])(@better-fetch/[email protected])([email protected]([email protected]))([email protected])([email protected])([email protected]))([email protected]([email protected]([email protected]))([email protected])([email protected])([email protected]([email protected])))([email protected])([email protected])([email protected]([email protected])([email protected]))([email protected]([email protected])([email protected](@babel/[email protected])(@react-native-community/[email protected]([email protected]))(@types/[email protected])([email protected])))':+ '@better-auth/[email protected](@better-auth/[email protected](@better-auth/[email protected])(@better-fetch/[email protected])([email protected]([email protected]))([email protected])([email protected])([email protected]))([email protected]([email protected]([email protected]))([email protected])([email protected])([email protected]([email protected])))([email protected])([email protected])([email protected]([email protected])([email protected]))([email protected]([email protected])([email protected](@babel/[email protected])(@react-native-community/[email protected]([email protected]))(@types/[email protected])([email protected])))':dependencies:- '@better-auth/core': 1.4.6(@better-auth/[email protected])(@better-fetch/[email protected])([email protected]([email protected]))([email protected])([email protected])([email protected])+ '@better-auth/core': 1.4.6(@better-auth/[email protected])(@better-fetch/[email protected])([email protected]([email protected]))([email protected])([email protected])([email protected])'@better-fetch/fetch': 1.1.18better-auth: 1.4.6([email protected]([email protected]))([email protected])([email protected])([email protected]([email protected]))better-call: 1.1.5([email protected])@@ -18672,7 +18682,7 @@ snapshots:'@better-fetch/fetch': 1.1.18'@noble/ciphers': 2.0.1'@noble/hashes': 2.0.1- better-call: 1.1.5([email protected])+ better-call: 1.1.5([email protected])defu: 6.1.4jose: 6.1.0kysely: 0.28.8