Skip to content

Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

docs: reapply Algolia docs search (#3454)

Rahul Tarak18c3b98de690
diff --git a/.github/workflows/docs-search-sync.yml b/.github/workflows/docs-search-sync.yml
new file mode 100644
index 000000000..4b36ce82c
--- /dev/null
+++ b/.github/workflows/docs-search-sync.yml
@@ -0,0 +1,46 @@
+name: Docs - Sync Algolia Search
+
+on:
+ push:
+ branches:
+ - next
+ paths:
+ - 'docs/**'
+ workflow_dispatch:
+
+permissions:
+ contents: read
+
+jobs:
+ sync-search:
+ runs-on: ubuntu-latest
+ defaults:
+ run:
+ working-directory: ./docs
+ env:
+ ALGOLIA_APP_ID: ${{ vars.ALGOLIA_APP_ID || '62HI9PQZ1L' }}
+ ALGOLIA_ADMIN_API_KEY: ${{ secrets.ALGOLIA_ADMIN_API_KEY }}
+ ALGOLIA_INDEX_NAME: ${{ vars.ALGOLIA_INDEX_NAME || 'docs_composio_dev_62hi9pqz1l_pages' }}
+
+ steps:
+ - name: Checkout repository
+ uses: actions/checkout@v6
+
+ - name: Setup Bun
+ uses: oven-sh/setup-bun@v2
+ with:
+ bun-version-file: '.bun-version'
+
+ - name: Install dependencies
+ run: bun install
+
+ - name: Validate search index generation
+ run: bun run sync:search --dry-run
+
+ - name: Sync Algolia search index
+ if: env.ALGOLIA_APP_ID != '' && env.ALGOLIA_ADMIN_API_KEY != ''
+ run: bun run sync:search
+
+ - name: Skip sync without Algolia secrets
+ if: env.ALGOLIA_APP_ID == '' || env.ALGOLIA_ADMIN_API_KEY == ''
+ run: echo "Skipping Algolia sync because ALGOLIA_APP_ID or ALGOLIA_ADMIN_API_KEY is not configured."
diff --git a/docs/CLAUDE.md b/docs/CLAUDE.md
index 4862683d2..52f3df02f 100644
--- a/docs/CLAUDE.md
+++ b/docs/CLAUDE.md
@@ -9,6 +9,8 @@ bun install # Install dependencies
bun run dev # Dev server (http://localhost:3000)
bun run build # Production build (validates TS code blocks)
bun run types:check # Type check
+bun run sync:search # Build and sync Algolia docs search index (requires Algolia env vars)
+bun run test:search # Test live Algolia search relevance from the terminal
```
## Project Structure
@@ -83,6 +85,12 @@ Detailed documentation for Claude is organized in `.claude/`:
`content/docs/glossary.mdx` defines key Composio terms (auth config, session, toolkit, etc.) using `<Glossary>` and `<GlossaryTerm name="...">` components (`components/glossary.tsx`). The component renders a filterable two-column table. The markdown converter in `lib/source.ts` converts `<GlossaryTerm>` tags to `### Term` headings for LLM-friendly output. When adding new Composio concepts, add a `<GlossaryTerm>` entry and update the `keywords` frontmatter array.
+## Search
+
+The docs search dialog uses Algolia in production when `NEXT_PUBLIC_ALGOLIA_SEARCH_API_KEY` is configured. `NEXT_PUBLIC_ALGOLIA_APP_ID` defaults to `62HI9PQZ1L`, and `NEXT_PUBLIC_ALGOLIA_INDEX_NAME` defaults to `docs_composio_dev_62hi9pqz1l_pages`. Without a search API key it falls back to `/api/search` for local development and tests. Algolia searches request `clickAnalytics`, and the client sends search result view/click events with `search-insights`.
+
+Run `ALGOLIA_APP_ID=62HI9PQZ1L ALGOLIA_ADMIN_API_KEY=... ALGOLIA_INDEX_NAME=docs_composio_dev_62hi9pqz1l_pages bun run sync:search` from `docs/` to rebuild the Algolia index. The sync script does not use Algolia Crawler: it creates section-sized records from MDX/OpenAPI/toolkit data, configures searchable attributes, `customRanking`, and `attributeForDistinct`, then replaces the index objects. Use `bun run sync:search --dry-run --samples` to inspect generated records and `ALGOLIA_SEARCH_API_KEY=... bun run test:search "oauth auth config"` to test live relevance.
+
## API Versioning
The API reference supports v3.1 (latest, default) and v3.0 side-by-side:
diff --git a/docs/README.md b/docs/README.md
index 6da6874e2..182775229 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -74,6 +74,31 @@ All TypeScript code blocks in MDX files are **type-checked at build time** using
See `CLAUDE.md` for detailed patterns and troubleshooting.
+## Search
+
+Docs search uses Algolia when `NEXT_PUBLIC_ALGOLIA_SEARCH_API_KEY` is set; otherwise it falls back to the local Fumadocs `/api/search` endpoint for development and tests. The sync script builds a first-party index from MDX/OpenAPI/toolkit data (no crawler required), splits long pages into section-sized records, configures searchable attributes/custom ranking/distinct, requests `clickAnalytics`, and sends search result view/click events with `search-insights`.
+
+```bash
+NEXT_PUBLIC_ALGOLIA_APP_ID=62HI9PQZ1L # optional; default shown
+NEXT_PUBLIC_ALGOLIA_SEARCH_API_KEY=...
+NEXT_PUBLIC_ALGOLIA_INDEX_NAME=docs_composio_dev_62hi9pqz1l_pages # optional; default shown
+```
+
+Sync the Algolia index with an admin key:
+
+```bash
+ALGOLIA_APP_ID=62HI9PQZ1L
+ALGOLIA_ADMIN_API_KEY=...
+ALGOLIA_INDEX_NAME=docs_composio_dev_62hi9pqz1l_pages bun run sync:search
+```
+
+Preview the generated records or test live relevance:
+
+```bash
+bun run sync:search --dry-run --samples
+ALGOLIA_SEARCH_API_KEY=... bun run test:search "oauth auth config" "gmail send email"
+```
+
## Commands
| Command | Description |
@@ -81,4 +106,6 @@ See `CLAUDE.md` for detailed patterns and troubleshooting.
| `bun run dev` | Dev server |
| `bun run build` | Production build (validates TS code blocks) |
| `bun run types:check` | Type check |
+| `bun run sync:search` | Sync docs search records to Algolia |
+| `bun run test:search` | Query the configured Algolia index from the terminal |
diff --git a/docs/app/api/search/route.ts b/docs/app/api/search/route.ts
index ba05af37b..be0355baa 100644
--- a/docs/app/api/search/route.ts
+++ b/docs/app/api/search/route.ts
@@ -1,89 +1,8 @@
-// Use direct imports from collections to avoid top-level await in lib/source.ts
-import { docs, reference, cookbooks, toolkits, changelog } from 'fumadocs-mdx:collections/server';
import { createSearchAPI } from 'fumadocs-core/search/server';
-import { loader, multiple } from 'fumadocs-core/source';
-import { lucideIconsPlugin } from 'fumadocs-core/source/lucide-icons';
-import { openapiSource, openapiPlugin } from 'fumadocs-openapi/server';
-import { openapi } from '@/lib/openapi';
-import { getAllToolkitsSync } from '@/lib/toolkit-data';
+import { getDocsSearchIndexes } from '@/lib/search-index';
-// Create loaders directly here to avoid the problematic lib/source.ts import
-const docsSource = loader({
- baseUrl: '/docs',
- source: docs.toFumadocsSource(),
- plugins: [lucideIconsPlugin()],
-});
-
-const cookbooksSource = loader({
- baseUrl: '/cookbooks',
- source: cookbooks.toFumadocsSource(),
- plugins: [lucideIconsPlugin()],
-});
-
-const toolkitsSource = loader({
- baseUrl: '/toolkits',
- source: toolkits.toFumadocsSource(),
- plugins: [lucideIconsPlugin()],
-});
-
-// Dynamic toolkit entries from toolkits.json
-const mdxToolkitSlugs = new Set(
- toolkitsSource.getPages().map((page) => page.slugs.join('/')),
-);
-
-const dynamicToolkitIndexes = getAllToolkitsSync()
- .filter((toolkit) => !mdxToolkitSlugs.has(toolkit.slug))
- .map((toolkit) => ({
- id: `/toolkits/${toolkit.slug}`,
- title: toolkit.name,
- description: toolkit.description,
- url: `/toolkits/${toolkit.slug}`,
- structuredData: { headings: [] as never[], contents: [] as never[] },
- keywords: [toolkit.slug, toolkit.category].filter(Boolean) as string[],
- }));
-
-// Changelog entries
-const changelogIndexes = changelog.map((entry) => ({
- id: `/docs/changelog/${entry.date.replace(/-/g, '/')}#${entry.title}`,
- title: entry.title,
- description: entry.description ?? '',
- url: `/docs/changelog/${entry.date.replace(/-/g, '/')}`,
- structuredData: { headings: [] as never[], contents: [] as never[] },
- keywords: ['changelog'],
-}));
-
-// Use dynamic indexes to support async OpenAPI page loading
+// Fallback/local search endpoint. Production search uses Algolia from the client
+// when NEXT_PUBLIC_ALGOLIA_* variables are configured.
export const { GET } = createSearchAPI('advanced', {
- indexes: async () => {
- // Load OpenAPI pages and build full reference source
- const openapiPages = await openapiSource(openapi, {
- groupBy: 'tag',
- baseDir: 'api-reference',
- });
-
- const fullReferenceSource = loader({
- baseUrl: '/reference',
- source: multiple({
- mdx: reference.toFumadocsSource(),
- openapi: openapiPages,
- }),
- plugins: [lucideIconsPlugin(), openapiPlugin()],
- });
-
- const mdxIndexes = [
- ...docsSource.getPages(),
- ...cookbooksSource.getPages(),
- ...toolkitsSource.getPages(),
- ...fullReferenceSource.getPages(),
- ].map((page) => ({
- id: page.url,
- title: page.data.title ?? 'Untitled',
- description: page.data.description,
- url: page.url,
- structuredData: page.data.structuredData,
- keywords: 'keywords' in page.data ? (page.data.keywords as string[]) : undefined,
- }));
-
- return [...mdxIndexes, ...dynamicToolkitIndexes, ...changelogIndexes];
- },
+ indexes: getDocsSearchIndexes,
});
diff --git a/docs/bun.lock b/docs/bun.lock
index 96f1e5674..ba8797694 100644
--- a/docs/bun.lock
+++ b/docs/bun.lock
@@ -1,5 +1,6 @@
{
"lockfileVersion": 1,
+ "configVersion": 0,
"workspaces": {
"": {
"name": "fumadocs",
@@ -7,6 +8,7 @@
"@radix-ui/react-tabs": "^1.1.13",
"@shikijs/twoslash": "^3.21.0",
"@vercel/analytics": "^1.6.1",
+ "algoliasearch": "^5",
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.1",
"fumadocs-core": "^16.4.2",
@@ -19,6 +21,7 @@
"posthog-js": "^1.319.0",
"react": "^19.2.3",
"react-dom": "^19.2.3",
+ "search-insights": "^2",
"shiki": "^3.20.0",
"tailwind-merge": "^3.4.0",
},
@@ -95,6 +98,34 @@
"@ai-sdk/ui-utils-v5": ["@ai-sdk/[email protected]", "", { "dependencies": { "@ai-sdk/provider": "1.1.3", "@ai-sdk/provider-utils": "2.2.8", "zod-to-json-schema": "^3.24.1" }, "peerDependencies": { "zod": "^3.23.8" } }, "sha512-3zcwCc8ezzFlwp3ZD15wAPjf2Au4s3vAbKsXQVyhxODHcmu0iyPO2Eua6D/vicq/AUm/BAo60r97O6HU+EI0+w=="],
+ "@algolia/abtesting": ["@algolia/[email protected]", "", { "dependencies": { "@algolia/client-common": "5.52.1", "@algolia/requester-browser-xhr": "5.52.1", "@algolia/requester-fetch": "5.52.1", "@algolia/requester-node-http": "5.52.1" } }, "sha512-aehCadlWOGvrT91KUIZpC0MbB8KBW9yUuvTJFd2xesR7le/IsT4nJUnjCCZ4ZqZCeTcPHPV5mo//fZ5oxcSVYw=="],
+
+ "@algolia/client-abtesting": ["@algolia/[email protected]", "", { "dependencies": { "@algolia/client-common": "5.52.1", "@algolia/requester-browser-xhr": "5.52.1", "@algolia/requester-fetch": "5.52.1", "@algolia/requester-node-http": "5.52.1" } }, "sha512-HmXOGBOAOJPounpBzBpuY0zDYeiCpxgHnQmuA7JO6ScukcBdGp3/XM9zJk5pJx/xNGD68mbPGXWpDxGtl6BwDQ=="],
+
+ "@algolia/client-analytics": ["@algolia/[email protected]", "", { "dependencies": { "@algolia/client-common": "5.52.1", "@algolia/requester-browser-xhr": "5.52.1", "@algolia/requester-fetch": "5.52.1", "@algolia/requester-node-http": "5.52.1" } }, "sha512-5oo4+I8iixie9vXhCyNFCzeIr8pqA3FQ//VsLHTDvZAV4ttYOPGvYHGQq5NSalrLx5Jc3dRro/5uDOlnUMcBJg=="],
+
+ "@algolia/client-common": ["@algolia/[email protected]", "", {}, "sha512-qCDoZfx5MpX7XQzvQ3bC4tSEMkQWQMaF/ABtLuoze03Y/flR563CCSws02qIJ23oX7lxl92LsilZjINVyTdtLw=="],
+
+ "@algolia/client-insights": ["@algolia/[email protected]", "", { "dependencies": { "@algolia/client-common": "5.52.1", "@algolia/requester-browser-xhr": "5.52.1", "@algolia/requester-fetch": "5.52.1", "@algolia/requester-node-http": "5.52.1" } }, "sha512-hnGs0/lsFJ2PWDxNBz7pxreXo/Xz7gxYRcfePBUjsH26ad0kU/sgnVZd9LwWBpsQv65z2jlb5dkyaB9WE9M9FQ=="],
+
+ "@algolia/client-personalization": ["@algolia/[email protected]", "", { "dependencies": { "@algolia/client-common": "5.52.1", "@algolia/requester-browser-xhr": "5.52.1", "@algolia/requester-fetch": "5.52.1", "@algolia/requester-node-http": "5.52.1" } }, "sha512-2VxxNc/uBysyKvGeBdSM5n9eIDKH8kWD7wd9/yqbJAiVwU4Yv6tU1LSJusHKrXV/aCu1KW7t9Gug9QyeEmtn/Q=="],
+
+ "@algolia/client-query-suggestions": ["@algolia/[email protected]", "", { "dependencies": { "@algolia/client-common": "5.52.1", "@algolia/requester-browser-xhr": "5.52.1", "@algolia/requester-fetch": "5.52.1", "@algolia/requester-node-http": "5.52.1" } }, "sha512-O6mPtsw3xEfNOe6gWFpYLeAZAIljNa4Hgna3bq15PwyN7nbjTY0wXJFRbzs/0YVf75Br+SbOQUmjKxXYjDiSiQ=="],
+
+ "@algolia/client-search": ["@algolia/[email protected]", "", { "dependencies": { "@algolia/client-common": "5.52.1", "@algolia/requester-browser-xhr": "5.52.1", "@algolia/requester-fetch": "5.52.1", "@algolia/requester-node-http": "5.52.1" } }, "sha512-gA8oJOV1LnQQkDf91iebNnFInHuW0gRPEgLSOQ7EfipCEjYTHm5swm1DlH9H5RaRw4RrHuzHBegnlzc0MAstcg=="],
+
+ "@algolia/ingestion": ["@algolia/[email protected]", "", { "dependencies": { "@algolia/client-common": "5.52.1", "@algolia/requester-browser-xhr": "5.52.1", "@algolia/requester-fetch": "5.52.1", "@algolia/requester-node-http": "5.52.1" } }, "sha512-U9zZfc5xIu9wRxZkt+HceJUAD4VKHKbAyLSloJdEyMRmphXeibfrY9cxqIXBcmPeZzGhn3Imb35Dq8l19PkJhw=="],
+
+ "@algolia/monitoring": ["@algolia/[email protected]", "", { "dependencies": { "@algolia/client-common": "5.52.1", "@algolia/requester-browser-xhr": "5.52.1", "@algolia/requester-fetch": "5.52.1", "@algolia/requester-node-http": "5.52.1" } }, "sha512-a3SGNceHmkQfq77iG8Ka+w1pvwfZa/0lzEIgse30fL0kD+yKnd/dg0dQvSfFPAEt2f21DMcGkDSSeJlO3KdQjQ=="],
+
+ "@algolia/recommend": ["@algolia/[email protected]", "", { "dependencies": { "@algolia/client-common": "5.52.1", "@algolia/requester-browser-xhr": "5.52.1", "@algolia/requester-fetch": "5.52.1", "@algolia/requester-node-http": "5.52.1" } }, "sha512-z98QEguCFDpxb4S/PyrUK1igqF8tPsdbqOUUO6ON91vJ58w+Gwa6ncrI0oNXSFcrkxA5EqPKPQ2A1PBCn08TYQ=="],
+
+ "@algolia/requester-browser-xhr": ["@algolia/[email protected]", "", { "dependencies": { "@algolia/client-common": "5.52.1" } }, "sha512-CI7+/0I11QeZM59Uc8whd2or0kqzFVjpaPn9Qpwll/krHcBAxk24WkAQ6WX+IwDVMfpont4YGbKwAmCre3vE8Q=="],
+
+ "@algolia/requester-fetch": ["@algolia/[email protected]", "", { "dependencies": { "@algolia/client-common": "5.52.1" } }, "sha512-S6bDuw9byfOvm3T71cgdoZgrgnZq6hpdMLkx52Louh57nUAmvGQESz2aojOynQHjbTiV55smvAFbgn0qT4tJrg=="],
+
+ "@algolia/requester-node-http": ["@algolia/[email protected]", "", { "dependencies": { "@algolia/client-common": "5.52.1" } }, "sha512-tqZXM+54rWo4mk5jL5Z/flE11nPmNEdXwFBM5py9DkOmbjeCNemfVd45FyM97XdzfZ0dl9uOJC6PYn1FpkeyQg=="],
+
"@alloc/quick-lru": ["@alloc/[email protected]", "", {}, "sha512-UrcABB+4bUrFABwbluTIBErXwvbsU/V7TZWfmbgJfbkwiBuziS9gxdODUyuiecfdGQ85jglMW6juS3+z5TsKLw=="],
"@antfu/install-pkg": ["@antfu/[email protected]", "", { "dependencies": { "package-manager-detector": "^1.3.0", "tinyexec": "^1.0.1" } }, "sha512-MGQsmw10ZyI+EJo45CdSER4zEb+p31LpDAFp2Z3gkSd1yqVZGi0Ebx++YTEMonJy4oChEMLsxZ64j8FH6sSqtQ=="],
@@ -865,6 +896,8 @@
"ajv-formats": ["[email protected]", "", { "dependencies": { "ajv": "^8.0.0" } }, "sha512-8iUql50EUR+uUcdRQ3HDqa6EVyo3docL8g5WJ3FNcWmu62IbkGUue/pEyLBW8VGKKucTPgqeks4fIU1DA4yowQ=="],
+ "algoliasearch": ["[email protected]", "", { "dependencies": { "@algolia/abtesting": "1.18.1", "@algolia/client-abtesting": "5.52.1", "@algolia/client-analytics": "5.52.1", "@algolia/client-common": "5.52.1", "@algolia/client-insights": "5.52.1", "@algolia/client-personalization": "5.52.1", "@algolia/client-query-suggestions": "5.52.1", "@algolia/client-search": "5.52.1", "@algolia/ingestion": "1.52.1", "@algolia/monitoring": "1.52.1", "@algolia/recommend": "5.52.1", "@algolia/requester-browser-xhr": "5.52.1", "@algolia/requester-fetch": "5.52.1", "@algolia/requester-node-http": "5.52.1" } }, "sha512-fHA8+kXTbjagw3jkLiaS7KKrH8qe2DyOsiUhGlN4cdT77PEsfqXZl7ewDk1hsg+pJnPlnE50XtLxjR91iJOpmg=="],
+
"alien-signals": ["[email protected]", "", {}, "sha512-d9dYqZTS90WLiU0I5c6DHj/HcKkF8ZyGN3G5x8wSbslulz70KOxaqCT0hQCo9KOyhVqzqGojvNdJXoTumZOtcw=="],
"ansi-regex": ["[email protected]", "", {}, "sha512-Bq3SmSpyFHaWjPk8If9yc6svM8c56dB5BAtW4Qbw5jHTwwXXcTLoRMkpDJp6VL0XzlWaCHTXrkFURMYmD0sLqg=="],
@@ -1999,6 +2032,8 @@
"scroll-into-view-if-needed": ["[email protected]", "", { "dependencies": { "compute-scroll-into-view": "^3.0.2" } }, "sha512-49oNpRjWRvnU8NyGVmUaYG4jtTkNonFZI86MmGRDqBphEK2EXT9gdEUoQPZhuBM8yWHxCWbobltqYO5M4XrUvQ=="],
+ "search-insights": ["[email protected]", "", {}, "sha512-RQPdCYTa8A68uM2jwxoY842xDhvx3E5LFL1LxvxCNMev4o5mLuokczhzjAgGwUZBAmOKZknArSxLKmXtIi2AxQ=="],
+
"section-matter": ["[email protected]", "", { "dependencies": { "extend-shallow": "^2.0.1", "kind-of": "^6.0.0" } }, "sha512-vfD3pmTzGpufjScBh50YHKzEu2lxBWhVEHsNGoEXmCmn2hKGfeNLYMzCJpe8cD7gqX7TJluOVpBkAequ6dgMmA=="],
"secure-json-parse": ["[email protected]", "", {}, "sha512-6aU+Rwsezw7VR8/nyvKTx8QpWH9FrcYiXXlqC4z5d5XQBDRqtbfsRjnwGyqbi3gddNtWHuEk9OANUotL26qKUw=="],
diff --git a/docs/components/custom-search-dialog.tsx b/docs/components/custom-search-dialog.tsx
index 388f2c389..9f1108ea7 100644
--- a/docs/components/custom-search-dialog.tsx
+++ b/docs/components/custom-search-dialog.tsx
@@ -1,7 +1,9 @@
'use client';
-import { useEffect, useState } from 'react';
+import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import Link from 'next/link';
+import { liteClient } from 'algoliasearch/lite';
+import aa from 'search-insights';
import {
SearchDialog,
SearchDialogClose,
@@ -15,14 +17,20 @@ import {
} from 'fumadocs-ui/components/dialog/search';
import { useI18n } from 'fumadocs-ui/contexts/i18n';
import { useDocsSearch } from 'fumadocs-core/search/client';
+import type { BaseIndex } from 'fumadocs-core/search/algolia';
import { BotMessageSquare } from 'lucide-react';
import { toggleDecimalWidget, detectMac } from './ask-ai-button';
function MetaKey() {
const [key, setKey] = useState('⌘');
+
useEffect(() => {
- if (!detectMac()) setKey('Ctrl');
+ if (detectMac()) return;
+
+ const id = window.setTimeout(() => setKey('Ctrl'), 0);
+ return () => window.clearTimeout(id);
}, []);
+
return key;
}
@@ -37,17 +45,182 @@ interface CustomSearchDialogProps extends SharedProps {
api?: string;
}
+type AlgoliaHit = {
+ objectID: string;
+ url?: string;
+};
+
+type AlgoliaSearchResponse = {
+ hits: AlgoliaHit[];
+ queryID?: string;
+};
+
+type AlgoliaHitMeta = {
+ objectID: string;
+ position: number;
+ queryID?: string;
+};
+
+type SearchResultItem = {
+ content: unknown;
+ breadcrumbs?: unknown[];
+};
+
+const SEARCH_BREADCRUMB_LABELS: Record<string, string> = {
+ toolkits: 'Toolkit',
+ toolkit: 'Toolkit',
+ cookbooks: 'Cookbook',
+ cookbook: 'Cookbook',
+ docs: 'Docs',
+ reference: 'Reference',
+ 'api-reference': 'API Reference',
+ changelog: 'Changelog',
+};
+
+function normalizeSearchBreadcrumb(value: string): string {
+ return SEARCH_BREADCRUMB_LABELS[value.toLowerCase()] ?? value;
+}
+
+function normalizeSearchResult<T extends SearchResultItem>(result: T): T {
+ if (!result.breadcrumbs) return result;
+
+ const breadcrumbs = result.breadcrumbs.map((breadcrumb) =>
+ typeof breadcrumb === 'string' ? normalizeSearchBreadcrumb(breadcrumb) : breadcrumb,
+ );
+
+ const content = typeof result.content === 'string' ? result.content.toLowerCase() : null;
+ const dedupedBreadcrumbs = breadcrumbs.filter((breadcrumb, index) => {
+ if (index !== breadcrumbs.length - 1 || content === null || typeof breadcrumb !== 'string') {
+ return true;
+ }
+
+ return breadcrumb.toLowerCase() !== content;
+ });
+
+ return {
+ ...result,
+ breadcrumbs: dedupedBreadcrumbs,
+ };
+}
+
export default function CustomSearchDialog({
defaultLinks = [],
api = '/api/search',
...props
}: CustomSearchDialogProps) {
const { locale } = useI18n();
- const { search, setSearch, query } = useDocsSearch({
- type: 'fetch',
- locale,
- api,
- });
+ const algoliaHitMetaRef = useRef(new Map<string, AlgoliaHitMeta>());
+ const insightsInitializedRef = useRef(false);
+
+ const ensureAlgoliaInsights = useCallback((appId: string, searchApiKey: string) => {
+ if (insightsInitializedRef.current) return;
+
+ aa('init', {
+ appId,
+ apiKey: searchApiKey,
+ useCookie: true,
+ });
+ insightsInitializedRef.current = true;
+ }, []);
+
+ const clientOptions = useMemo(() => {
+ const appId = process.env.NEXT_PUBLIC_ALGOLIA_APP_ID ?? '62HI9PQZ1L';
+ const searchApiKey = process.env.NEXT_PUBLIC_ALGOLIA_SEARCH_API_KEY;
+ const indexName = process.env.NEXT_PUBLIC_ALGOLIA_INDEX_NAME ?? 'docs_composio_dev_62hi9pqz1l_pages';
+
+ if (appId && searchApiKey) {
+ const client = liteClient(appId, searchApiKey);
+
+ return {
+ type: 'algolia' as const,
+ client,
+ indexName,
+ locale,
+ onSearch: async (query: string, tag?: string) => {
+ ensureAlgoliaInsights(appId, searchApiKey);
+
+ const result = await client.searchForHits<BaseIndex>({
+ requests: [
+ {
+ type: 'default',
+ indexName,
+ query,
+ distinct: true,
+ hitsPerPage: 10,
+ filters: tag ? `tag:${tag}` : undefined,
+ clickAnalytics: true,
+ },
+ ],
+ });
+ const response = result.results[0] as AlgoliaSearchResponse;
+ const metadata = new Map<string, AlgoliaHitMeta>();
+
+ response.hits.forEach((hit, index) => {
+ if (!hit.url || metadata.has(hit.url)) return;
+ metadata.set(hit.url, {
+ objectID: hit.objectID,
+ position: index + 1,
+ queryID: response.queryID,
+ });
+ });
+
+ algoliaHitMetaRef.current = metadata;
+
+ const viewedObjectIDs = Array.from(metadata.values())
+ .slice(0, 20)
+ .map((hit) => hit.objectID);
+
+ if (viewedObjectIDs.length > 0) {
+ aa('viewedObjectIDs', {
+ index: indexName,
+ eventName: 'Docs Search Results Viewed',
+ objectIDs: viewedObjectIDs,
+ });
+ }
+
+ return result;
+ },
+ };
+ }
+
+ return {
+ type: 'fetch' as const,
+ locale,
+ api,
+ };
+ }, [api, ensureAlgoliaInsights, locale]);
+
+ const { search, setSearch, query } = useDocsSearch(clientOptions);
+ const searchResults = useMemo(() => {
+ if (query.data === 'empty' || !query.data) return query.data;
+ return query.data.map(normalizeSearchResult);
+ }, [query.data]);
+
+ const trackAlgoliaClick = useCallback((href: string) => {
+ const url = new URL(href, window.location.origin);
+ const path = `${url.pathname}${url.hash}`;
+ const hit = algoliaHitMetaRef.current.get(path) ?? algoliaHitMetaRef.current.get(url.pathname);
+ const indexName = process.env.NEXT_PUBLIC_ALGOLIA_INDEX_NAME ?? 'docs_composio_dev_62hi9pqz1l_pages';
+
+ if (!hit) return;
+
+ if (hit.queryID) {
+ aa('clickedObjectIDsAfterSearch', {
+ index: indexName,
+ eventName: 'Docs Search Result Clicked',
+ queryID: hit.queryID,
+ objectIDs: [hit.objectID],
+ positions: [hit.position],
+ });
+ return;
+ }
+
+ aa('clickedObjectIDs', {
+ index: indexName,
+ eventName: 'Docs Search Result Clicked',
+ objectIDs: [hit.objectID],
+ });
+ }, []);
return (
<SearchDialog
@@ -63,7 +236,19 @@ export default function CustomSearchDialog({
<SearchDialogInput />
<SearchDialogClose />
</SearchDialogHeader>
- <div className="search-scroll-container max-md:min-h-0 max-md:flex-1 max-md:overflow-y-auto">
+ <div
+ className="search-scroll-container max-md:min-h-0 max-md:flex-1 max-md:overflow-y-auto"
+ onClickCapture={(event) => {
+ const target = event.target;
+ if (!(target instanceof Element)) return;
+
+ const anchor = target.closest('a[href]');
+ const href = anchor?.getAttribute('href');
+ if (!href) return;
+
+ trackAlgoliaClick(href);
+ }}
+ >
{query.data === 'empty' && defaultLinks.length > 0 ? (
<div className="flex flex-col p-2">
{defaultLinks.map((link) => (
@@ -81,7 +266,7 @@ export default function CustomSearchDialog({
))}
</div>
) : (
- <SearchDialogList items={query.data === 'empty' ? null : query.data} />
+ <SearchDialogList items={searchResults === 'empty' ? null : searchResults} />
)}
</div>
<div className="flex items-center justify-between border-t px-3 py-2">
diff --git a/docs/lib/search-index.ts b/docs/lib/search-index.ts
new file mode 100644
index 000000000..7fa0c4935
--- /dev/null
+++ b/docs/lib/search-index.ts
@@ -0,0 +1,618 @@
+// Use direct imports from collections to avoid top-level await in lib/source.ts.
+import { createHash } from 'node:crypto';
+import { existsSync, readdirSync, readFileSync } from 'node:fs';
+import { join, relative } from 'node:path';
+import { docs, reference, cookbooks, toolkits } from 'fumadocs-mdx:collections/server';
+import { loader, multiple } from 'fumadocs-core/source';
+import { lucideIconsPlugin } from 'fumadocs-core/source/lucide-icons';
+import { openapiSource, openapiPlugin } from 'fumadocs-openapi/server';
+import type { AdvancedIndex } from 'fumadocs-core/search/server';
+import type { BaseIndex } from 'fumadocs-core/search/algolia';
+import { openapi } from '@/lib/openapi';
+import { mdxToCleanMarkdown } from '@/lib/source';
+import { getAllToolkitsSync } from '@/lib/toolkit-data';
+
+export const ALGOLIA_DEFAULT_APP_ID = '62HI9PQZ1L';
+export const ALGOLIA_DEFAULT_INDEX_NAME = 'docs_composio_dev_62hi9pqz1l_pages';
+
+const MAX_CHUNK_CHARS = 3_800;
+const MAX_CHUNK_BYTES = 9_000;
+const MAX_TOOL_ALIAS_ITEMS = 80;
+const MAX_TOOL_ALIAS_BYTES = 2_500;
+
+// Create loaders directly here to avoid the problematic lib/source.ts import in the
+// fallback route. This route is intentionally still Fumadocs/Orama-backed for local
+// development when Algolia env vars aren't configured.
+const docsSource = loader({
+ baseUrl: '/docs',
+ source: docs.toFumadocsSource(),
+ plugins: [lucideIconsPlugin()],
+});
+
+const cookbooksSource = loader({
+ baseUrl: '/cookbooks',
+ source: cookbooks.toFumadocsSource(),
+ plugins: [lucideIconsPlugin()],
+});
+
+const toolkitsSource = loader({
+ baseUrl: '/toolkits',
+ source: toolkits.toFumadocsSource(),
+ plugins: [lucideIconsPlugin()],
+});
+
+type SearchIndex = AdvancedIndex & {
+ keywords?: string[];
+};
+
+export type AlgoliaDocsRecord = BaseIndex & {
+ objectID: string;
+ description?: string;
+ keywords?: string[];
+ slug?: string;
+ headings?: string[];
+ tool_names?: string[];
+ tool_slugs?: string[];
+ type: string;
+ lang: string;
+ page_rank: number;
+ toolkit_popularity: number;
+ section_rank: number;
+ position: number;
+ depth: number;
+ tags?: string[];
+};
+
+function getFrontmatter(source: string): string {
+ return source.match(/^---\n([\s\S]*?)\n---/)?.[1] ?? '';
+}
+
+function getFrontmatterValue(frontmatter: string, key: string): string | undefined {
+ const match = frontmatter.match(new RegExp(`^${key}:\\s*(?:"([^"]*)"|'([^']*)'|(.+))$`, 'm'));
+ const value = match?.[1] ?? match?.[2] ?? match?.[3];
+ return value?.trim();
+}
+
+function getFrontmatterList(frontmatter: string, key: string): string[] {
+ const raw = getFrontmatterValue(frontmatter, key);
+ if (!raw) return [];
+
+ if (raw.startsWith('[') && raw.endsWith(']')) {
+ return raw
+ .slice(1, -1)
+ .split(',')
+ .map((value) => value.trim().replace(/^['"]|['"]$/g, ''))
+ .filter(Boolean);
+ }
+
+ return [raw];
+}
+
+function listContentFiles(dir: string): string[] {
+ if (!existsSync(dir)) return [];
+
+ return readdirSync(dir, { withFileTypes: true }).flatMap((entry) => {
+ const path = join(dir, entry.name);
+ if (entry.isDirectory()) return listContentFiles(path);
+ if (entry.isFile() && /\.mdx?$/.test(entry.name)) return [path];
+ return [];
+ });
+}
+
+function slugify(value: string): string {
+ return value
+ .toLowerCase()
+ .trim()
+ .replace(/[<>]/g, '')
+ .replace(/[`*_~\[\]()]/g, '')
+ .replace(/&[a-z]+;/g, '')
+ .replace(/[^\w\s-]/g, '')
+ .replace(/\s+/g, '-')
+ .replace(/-+/g, '-')
+ .replace(/^-|-$/g, '');
+}
+
+function uniqueSlug(value: string, seen: Map<string, number>): string {
+ const base = slugify(value) || 'section';
+ const count = seen.get(base) ?? 0;
+ seen.set(base, count + 1);
+ return count === 0 ? base : `${base}-${count + 1}`;
+}
+
+function splitIntoChunks(text: string): string[] {
+ const paragraphs = text
+ .split(/\n{2,}/)
+ .map((paragraph) => paragraph.trim())
+ .filter(Boolean);
+
+ const chunks: string[] = [];
+ let current = '';
+
+ const pushCurrent = () => {
+ const trimmed = current.trim();
+ if (trimmed) chunks.push(trimmed);
+ current = '';
+ };
+
+ for (const paragraph of paragraphs) {
+ const next = current ? `${current}\n\n${paragraph}` : paragraph;
+ if (next.length <= MAX_CHUNK_CHARS && Buffer.byteLength(next, 'utf8') <= MAX_CHUNK_BYTES) {
+ current = next;
+ continue;
+ }
+
+ pushCurrent();
+
+ if (paragraph.length <= MAX_CHUNK_CHARS && Buffer.byteLength(paragraph, 'utf8') <= MAX_CHUNK_BYTES) {
+ current = paragraph;
+ continue;
+ }
+
+ for (let i = 0; i < paragraph.length; i += MAX_CHUNK_CHARS) {
+ chunks.push(paragraph.slice(i, i + MAX_CHUNK_CHARS).trim());
+ }
+ }
+
+ pushCurrent();
+ return chunks;
+}
+
+function contentHash(value: string): string {
+ return createHash('sha1').update(value).digest('hex').slice(0, 10);
+}
+
+function urlFromContentPath(path: string): { url: string; type: string } | undefined {
+ const rel = relative(join(process.cwd(), 'content'), path).replace(/\\/g, '/');
+ const withoutExt = rel.replace(/\.mdx?$/, '');
+ const parts = withoutExt.split('/');
+ const collection = parts.shift();
+ if (!collection) return undefined;
+
+ if (collection === 'docs') {
+ return { url: `/docs/${parts.join('/')}`.replace(/\/index$/, ''), type: 'docs' };
+ }
+ if (collection === 'cookbooks') {
+ return { url: `/cookbooks/${parts.join('/')}`.replace(/\/index$/, ''), type: 'cookbooks' };
+ }
+ if (collection === 'reference') {
+ const url = `/reference/${parts.join('/')}`.replace(/\/index$/, '');
+ return { url, type: url.startsWith('/reference/v3') ? 'v3-reference' : 'reference' };
+ }
+ if (collection === 'toolkits') {
+ if (parts[0] === 'faq') return undefined;
+ return { url: `/toolkits/${parts.join('/')}`.replace(/\/index$/, ''), type: 'toolkits' };
+ }
+ if (collection === 'changelog') return undefined;
+
+ return undefined;
+}
+
+function slugTokens(url: string): string {
+ return url
+ .split(/[/-]/)
+ .map((part) => part.replace(/[_#?=&.]/g, ' '))
+ .join(' ')
+ .replace(/\s+/g, ' ')
+ .trim();
+}
+
+const typeLabels: Record<string, string> = {
+ docs: 'Docs',
+ cookbooks: 'Cookbook',
+ reference: 'Reference',
+ 'v3-reference': 'Legacy v3 Reference',
+ toolkits: 'Toolkit',
+ changelog: 'Changelog',
+ 'api-reference': 'API Reference',
+};
+
+function titleizeSlug(value: string): string {
+ return value
+ .split('-')
+ .filter(Boolean)
+ .map((part) => part.charAt(0).toUpperCase() + part.slice(1))
+ .join(' ')
+ .replace(/\bApi\b/g, 'API')
+ .replace(/\bMcp\b/g, 'MCP')
+ .replace(/\bSdk\b/g, 'SDK')
+ .replace(/\bOauth\b/g, 'OAuth');
+}
+
+function breadcrumbsForUrl(url: string, type: string): string[] {
+ const label = typeLabels[type] ?? titleizeSlug(type);
+
+ if (type === 'toolkits' || type === 'cookbooks' || type === 'changelog') {
+ return [label];
+ }
+
+ const parts = url.split('/').filter(Boolean);
+ const parentParts = parts.slice(1, -1);
+ return [label, ...parentParts.map(titleizeSlug)].filter(Boolean);
+}
+
+const TOOLKIT_POPULARITY_OVERRIDES: Record<string, number> = {
+ gmail: 1_000,
+ github: 980,
+ slack: 960,
+ googledrive: 940,
+ googlecalendar: 930,
+ notion: 920,
+ linear: 900,
+ jira: 890,
+ hubspot: 860,
+ salesforce: 850,
+ resend: 650,
+ zoho_mail: 520,
+ mailchimp: 500,
+ sendgrid: 480,
+ mailsoftly: 260,
+};
+
+function getToolkitSlugFromUrl(url: string): string | null {
+ return url.match(/^\/toolkits\/([^/#?]+)/)?.[1] ?? null;
+}
+
+function toolkitPopularity(url: string, type: string): number {
+ if (type !== 'toolkits') return 0;
+
+ const slug = getToolkitSlugFromUrl(url);
+ if (!slug) return 0;
+
+ const toolkit = toolkitBySlug.get(slug);
+ const override = TOOLKIT_POPULARITY_OVERRIDES[slug] ?? 0;
+ const managedAuthBoost = toolkit?.composioManagedAuthSchemes?.length ? 120 : 0;
+ const authBoost = toolkit?.authSchemes?.includes('OAUTH2') ? 40 : 0;
+ const triggerBoost = Math.min((toolkit?.triggerCount ?? 0) * 10, 80);
+ const toolCountBoost = Math.min((toolkit?.toolCount ?? 0), 100);
+
+ return override + managedAuthBoost + authBoost + triggerBoost + toolCountBoost;
+}
+
+function pageRank(url: string, type: string): number {
+ // Prefer conceptual docs over generated/reference material when textual
+ // relevance is otherwise close. Toolkit aliases can still win earlier via
+ // searchableAttributes when the query matches a tool name/slug exactly.
+ if (type === 'docs') {
+ if (url === '/docs' || url === '/docs/') return 2_400;
+ if (url.includes('/quickstart')) return 2_300;
+ if (url.includes('/authentication')) return 2_220;
+ if (url.includes('/tools-and-toolkits')) return 2_180;
+ if (url.includes('/configuring-sessions')) return 2_120;
+ return 2_000;
+ }
+
+ if (type === 'cookbooks') return 1_500;
+ if (type === 'toolkits') return 1_250;
+ // Current v3.1 reference should be available, but conceptual docs should
+ // win whenever both match. Legacy v3 reference is only a last-resort result.
+ if (type === 'reference') return 650;
+ if (type === 'api-reference') return 560;
+ if (type === 'v3-reference') return 25;
+ if (type === 'changelog') return 300;
+ return 400;
+}
+
+function recordsFromMarkdownPage(input: {
+ url: string;
+ type: string;
+ title: string;
+ description?: string;
+ keywords?: string[];
+ markdown: string;
+ breadcrumbs?: string[];
+ tags?: string[];
+ toolNames?: string[];
+ toolSlugs?: string[];
+}): AlgoliaDocsRecord[] {
+ const clean = mdxToCleanMarkdown(input.markdown);
+ const lines = clean.split('\n');
+ const headingSlugs = new Map<string, number>();
+ const headings: string[] = [];
+ const sections: Array<{ heading?: string; section_id?: string; depth: number; text: string; position: number }> = [];
+ let currentHeading: string | undefined;
+ let currentSectionId: string | undefined;
+ let currentDepth = 0;
+ let currentLines: string[] = [];
+ let sectionPosition = 0;
+
+ const flush = () => {
+ const text = currentLines.join('\n').trim();
+ if (!text) return;
+ sections.push({
+ heading: currentHeading,
+ section_id: currentSectionId,
+ depth: currentDepth,
+ text,
+ position: sectionPosition++,
+ });
+ currentLines = [];
+ };
+
+ for (const line of lines) {
+ const match = line.match(/^(#{1,6})\s+(.+)$/);
+ if (match) {
+ flush();
+ const depth = match[1].length;
+ const heading = match[2].trim();
+ currentHeading = heading;
+ currentSectionId = uniqueSlug(heading, headingSlugs);
+ currentDepth = depth;
+ headings.push(heading);
+ currentLines.push(line);
+ continue;
+ }
+
+ currentLines.push(line);
+ }
+ flush();
+
+ const fallbackText = clean.trim() || [input.title, input.description, ...(input.keywords ?? [])].filter(Boolean).join('\n');
+ if (sections.length === 0 && fallbackText) {
+ sections.push({ text: fallbackText, position: 0, depth: 0 });
+ }
+
+ let recordPosition = 0;
+ return sections.flatMap((section) => {
+ const chunks = splitIntoChunks(section.text);
+ return chunks.map((chunk, chunkIndex) => {
+ const sectionPart = section.section_id ?? 'overview';
+ const objectID = `${input.url}__${sectionPart}__${chunkIndex}__${contentHash(chunk)}`;
+ const sectionRank = Math.max(10, 120 - section.depth * 12 - chunkIndex * 2);
+
+ const position = recordPosition++;
+ const includeToolkitAliases = input.type === 'toolkits' && position === 0;
+
+ return {
+ objectID,
+ title: input.title,
+ description: input.description,
+ breadcrumbs: input.breadcrumbs,
+ url: input.url,
+ page_id: input.url,
+ section: section.heading,
+ section_id: section.section_id,
+ content: chunk,
+ keywords: input.keywords,
+ slug: slugTokens(input.url),
+ headings,
+ tool_names: includeToolkitAliases ? input.toolNames : undefined,
+ tool_slugs: includeToolkitAliases ? input.toolSlugs : undefined,
+ type: input.type,
+ lang: 'en',
+ tags: input.tags,
+ page_rank: pageRank(input.url, input.type),
+ toolkit_popularity: toolkitPopularity(input.url, input.type),
+ section_rank: sectionRank,
+ position,
+ depth: section.depth,
+ } satisfies AlgoliaDocsRecord;
+ });
+ });
+}
+
+const toolkitBySlug = new Map(getAllToolkitsSync().map((toolkit) => [toolkit.slug, toolkit]));
+
+function limitToolkitAliases(values: string[]): string[] {
+ const aliases: string[] = [];
+ let bytes = 0;
+
+ for (const value of values) {
+ const trimmed = value.trim();
+ if (!trimmed) continue;
+
+ const nextBytes = Buffer.byteLength(JSON.stringify(trimmed), 'utf8');
+ if (aliases.length >= MAX_TOOL_ALIAS_ITEMS || bytes + nextBytes > MAX_TOOL_ALIAS_BYTES) break;
+
+ aliases.push(trimmed);
+ bytes += nextBytes;
+ }
+
+ return aliases;
+}
+
+function getToolkitSearchFields(slug: string): Pick<Parameters<typeof recordsFromMarkdownPage>[0], 'toolNames' | 'toolSlugs' | 'tags'> {
+ const toolkit = toolkitBySlug.get(slug);
+ if (!toolkit) return {};
+
+ return {
+ toolNames: limitToolkitAliases(toolkit.tools.map((tool) => tool.name).filter(Boolean)),
+ toolSlugs: limitToolkitAliases(toolkit.tools.map((tool) => tool.slug).filter(Boolean)),
+ tags: [toolkit.category].filter(Boolean) as string[],
+ };
+}
+
+function getFilesystemRecords(): AlgoliaDocsRecord[] {
+ const contentDir = join(process.cwd(), 'content');
+ return listContentFiles(contentDir).flatMap((file) => {
+ const route = urlFromContentPath(file);
+ if (!route) return [];
+
+ const source = readFileSync(file, 'utf8');
+ const frontmatter = getFrontmatter(source);
+ const title = getFrontmatterValue(frontmatter, 'title');
+ if (!title) return [];
+
+ const toolkitFields = route.type === 'toolkits'
+ ? getToolkitSearchFields(route.url.replace(/^\/toolkits\//, ''))
+ : {};
+
+ return recordsFromMarkdownPage({
+ url: route.url,
+ type: route.type,
+ title,
+ description: getFrontmatterValue(frontmatter, 'description'),
+ keywords: getFrontmatterList(frontmatter, 'keywords'),
+ markdown: source,
+ breadcrumbs: breadcrumbsForUrl(route.url, route.type),
+ ...toolkitFields,
+ });
+ });
+}
+
+// Dynamic toolkit entries from toolkits.json. We include a compact sample of tool
+// names/slugs so queries like "gmail send email" can still find the Gmail toolkit
+// page without indexing the full tool catalog as noisy giant records.
+const mdxToolkitSlugs = new Set(
+ toolkitsSource.getPages().map((page) => page.slugs.join('/')),
+);
+
+const dynamicToolkitIndexes = getAllToolkitsSync()
+ .filter((toolkit) => !mdxToolkitSlugs.has(toolkit.slug))
+ .map((toolkit) => ({
+ id: `/toolkits/${toolkit.slug}`,
+ title: toolkit.name,
+ description: toolkit.description,
+ url: `/toolkits/${toolkit.slug}`,
+ structuredData: { headings: [], contents: [] },
+ keywords: [toolkit.slug, toolkit.category].filter(Boolean) as string[],
+ } satisfies SearchIndex));
+
+function getDynamicToolkitRecords(): AlgoliaDocsRecord[] {
+ return getAllToolkitsSync()
+ .filter((toolkit) => !mdxToolkitSlugs.has(toolkit.slug))
+ .flatMap((toolkit) => {
+ const toolsText = (toolkit.tools ?? [])
+ .slice(0, 60)
+ .map((tool) => `${tool.slug ?? ''} ${tool.name ?? ''}`)
+ .join('\n');
+
+ return recordsFromMarkdownPage({
+ url: `/toolkits/${toolkit.slug}`,
+ type: 'toolkits',
+ title: toolkit.name,
+ description: toolkit.description,
+ keywords: [toolkit.slug, toolkit.category].filter(Boolean) as string[],
+ markdown: `# ${toolkit.name}\n\n${toolkit.description ?? ''}\n\n## Available tools\n\n${toolsText}`,
+ breadcrumbs: breadcrumbsForUrl(`/toolkits/${toolkit.slug}`, 'toolkits'),
+ ...getToolkitSearchFields(toolkit.slug),
+ });
+ });
+}
+
+function getChangelogIndexes(): SearchIndex[] {
+ const changelogDir = join(process.cwd(), 'content/changelog');
+ if (!existsSync(changelogDir)) return [];
+
+ return readdirSync(changelogDir)
+ .filter((file) => file.endsWith('.mdx'))
+ .flatMap((file) => {
+ const source = readFileSync(join(changelogDir, file), 'utf8');
+ const frontmatter = getFrontmatter(source);
+ const date = getFrontmatterValue(frontmatter, 'date');
+ const title = getFrontmatterValue(frontmatter, 'title');
+ if (!date || !title) return [];
+
+ const url = `/docs/changelog/${date.replace(/-/g, '/')}`;
+
+ return [{
+ id: `${url}#${title}`,
+ title,
+ description: getFrontmatterValue(frontmatter, 'description') ?? '',
+ url,
+ structuredData: { headings: [], contents: [] },
+ keywords: ['changelog'],
+ } satisfies SearchIndex];
+ });
+}
+
+function getChangelogRecords(): AlgoliaDocsRecord[] {
+ const changelogDir = join(process.cwd(), 'content/changelog');
+ if (!existsSync(changelogDir)) return [];
+
+ return readdirSync(changelogDir)
+ .filter((file) => file.endsWith('.mdx'))
+ .flatMap((file) => {
+ const source = readFileSync(join(changelogDir, file), 'utf8');
+ const frontmatter = getFrontmatter(source);
+ const date = getFrontmatterValue(frontmatter, 'date');
+ const title = getFrontmatterValue(frontmatter, 'title');
+ if (!date || !title) return [];
+
+ return recordsFromMarkdownPage({
+ url: `/docs/changelog/${date.replace(/-/g, '/')}`,
+ type: 'changelog',
+ title,
+ description: getFrontmatterValue(frontmatter, 'description'),
+ keywords: ['changelog', date],
+ markdown: source,
+ breadcrumbs: breadcrumbsForUrl(`/docs/changelog/${date.replace(/-/g, '/')}`, 'changelog'),
+ });
+ });
+}
+
+export async function getDocsSearchIndexes(): Promise<SearchIndex[]> {
+ // Load OpenAPI pages and build full reference source.
+ const openapiPages = await openapiSource(openapi, {
+ groupBy: 'tag',
+ baseDir: 'api-reference',
+ });
+
+ const fullReferenceSource = loader({
+ baseUrl: '/reference',
+ source: multiple({
+ mdx: reference.toFumadocsSource(),
+ openapi: openapiPages,
+ }),
+ plugins: [lucideIconsPlugin(), openapiPlugin()],
+ });
+
+ const mdxIndexes = [
+ ...docsSource.getPages(),
+ ...cookbooksSource.getPages(),
+ ...toolkitsSource.getPages(),
+ ...fullReferenceSource.getPages(),
+ ].map((page) => ({
+ id: page.url,
+ title: page.data.title ?? 'Untitled',
+ description: page.data.description,
+ url: page.url,
+ structuredData: page.data.structuredData,
+ keywords: 'keywords' in page.data ? (page.data.keywords as string[]) : undefined,
+ } satisfies SearchIndex));
+
+ return [...mdxIndexes, ...dynamicToolkitIndexes, ...getChangelogIndexes()];
+}
+
+async function getOpenApiRecords(): Promise<AlgoliaDocsRecord[]> {
+ const openapiPages = await openapiSource(openapi, {
+ groupBy: 'tag',
+ baseDir: 'api-reference',
+ });
+
+ const openapiOnlySource = loader({
+ baseUrl: '/reference',
+ source: openapiPages,
+ plugins: [lucideIconsPlugin(), openapiPlugin()],
+ });
+
+ return openapiOnlySource.getPages().flatMap((page) => {
+ const contents = [
+ page.data.description,
+ ...(page.data.structuredData?.headings ?? []).map((heading) => heading.content),
+ ...(page.data.structuredData?.contents ?? []).map((content) => content.content),
+ ].filter(Boolean).join('\n\n');
+
+ return recordsFromMarkdownPage({
+ url: page.url,
+ type: 'api-reference',
+ title: page.data.title ?? 'Untitled',
+ description: page.data.description,
+ markdown: `# ${page.data.title ?? 'Untitled'}\n\n${contents}`,
+ breadcrumbs: breadcrumbsForUrl(page.url, 'api-reference'),
+ });
+ });
+}
+
+export async function getAlgoliaSearchDocuments(): Promise<AlgoliaDocsRecord[]> {
+ const records = [
+ ...getFilesystemRecords(),
+ ...getDynamicToolkitRecords(),
+ ...getChangelogRecords(),
+ ...await getOpenApiRecords(),
+ ];
+
+ const deduped = new Map<string, AlgoliaDocsRecord>();
+ for (const record of records) deduped.set(record.objectID, record);
+
+ return Array.from(deduped.values());
+}
diff --git a/docs/package.json b/docs/package.json
index 3585453aa..a7b1f231b 100644
--- a/docs/package.json
+++ b/docs/package.json
@@ -10,6 +10,8 @@
"postinstall": "fumadocs-mdx",
"lint": "eslint",
"lint:links": "bun scripts/validate-links.ts",
+ "sync:search": "bun scripts/sync-algolia-search.ts",
+ "test:search": "bun scripts/test-algolia-search.ts",
"generate:toolkits": "bun scripts/generate-toolkits.ts",
"generate:meta-tools": "bun scripts/generate-meta-tools.ts",
"generate:api-index": "bun scripts/generate-api-index.ts",
@@ -20,6 +22,7 @@
"@radix-ui/react-tabs": "^1.1.13",
"@shikijs/twoslash": "^3.21.0",
"@vercel/analytics": "^1.6.1",
+ "algoliasearch": "^5",
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.1",
"fumadocs-core": "^16.4.2",
@@ -32,6 +35,7 @@
"posthog-js": "^1.319.0",
"react": "^19.2.3",
"react-dom": "^19.2.3",
+ "search-insights": "^2",
"shiki": "^3.20.0",
"tailwind-merge": "^3.4.0"
},
diff --git a/docs/scripts/sync-algolia-search.ts b/docs/scripts/sync-algolia-search.ts
new file mode 100644
index 000000000..1ddc278a4
--- /dev/null
+++ b/docs/scripts/sync-algolia-search.ts
@@ -0,0 +1,102 @@
+import { algoliasearch } from 'algoliasearch';
+import {
+ ALGOLIA_DEFAULT_APP_ID,
+ ALGOLIA_DEFAULT_INDEX_NAME,
+ getAlgoliaSearchDocuments,
+} from '@/lib/search-index';
+
+function requireEnv(name: string, fallback?: string): string {
+ const value = process.env[name] ?? fallback;
+ if (!value) {
+ throw new Error(`Missing required environment variable: ${name}`);
+ }
+ return value;
+}
+
+const dryRun = process.argv.includes('--dry-run');
+const showSamples = process.argv.includes('--samples');
+const indexName =
+ process.env.ALGOLIA_INDEX_NAME ?? process.env.NEXT_PUBLIC_ALGOLIA_INDEX_NAME ?? ALGOLIA_DEFAULT_INDEX_NAME;
+
+const records = await getAlgoliaSearchDocuments();
+
+if (records.length === 0) {
+ throw new Error('Refusing to sync an empty Algolia docs index.');
+}
+
+if (dryRun) {
+ const uniquePages = new Set(records.map((record) => record.page_id)).size;
+ console.log(`Prepared ${records.length} Algolia records across ${uniquePages} docs pages for index "${indexName}".`);
+
+ if (showSamples) {
+ console.log(JSON.stringify(records.slice(0, 5), null, 2));
+ }
+
+ process.exit(0);
+}
+
+const appId = requireEnv(
+ 'ALGOLIA_APP_ID',
+ process.env.NEXT_PUBLIC_ALGOLIA_APP_ID ?? ALGOLIA_DEFAULT_APP_ID,
+);
+const adminApiKey = requireEnv('ALGOLIA_ADMIN_API_KEY');
+const client = algoliasearch(appId, adminApiKey);
+
+console.log(`Configuring Algolia index "${indexName}"...`);
+await client.setSettings({
+ indexName,
+ indexSettings: {
+ attributeForDistinct: 'page_id',
+ attributesToRetrieve: [
+ 'objectID',
+ 'title',
+ 'description',
+ 'section',
+ 'content',
+ 'url',
+ 'section_id',
+ 'breadcrumbs',
+ 'page_id',
+ 'type',
+ ],
+ searchableAttributes: [
+ 'unordered(title)',
+ 'unordered(keywords)',
+ 'unordered(slug)',
+ 'unordered(section)',
+ 'unordered(headings)',
+ 'unordered(tool_names)',
+ 'unordered(tool_slugs)',
+ 'unordered(description)',
+ 'unordered(content)',
+ ],
+ customRanking: [
+ 'desc(page_rank)',
+ 'desc(toolkit_popularity)',
+ 'desc(section_rank)',
+ 'asc(position)',
+ ],
+ attributesForFaceting: [
+ 'filterOnly(type)',
+ 'filterOnly(lang)',
+ 'searchable(tags)',
+ ],
+ attributesToHighlight: ['title', 'section', 'content'],
+ attributesToSnippet: ['content:30'],
+ highlightPreTag: '<mark>',
+ highlightPostTag: '</mark>',
+ ignorePlurals: true,
+ minProximity: 1,
+ removeStopWords: false,
+ removeWordsIfNoResults: 'lastWords',
+ typoTolerance: true,
+ advancedSyntax: true,
+ },
+});
+
+console.log(`Replacing ${records.length} records in Algolia index "${indexName}"...`);
+await client.replaceAllObjects({
+ indexName,
+ objects: records.map((record) => ({ ...record })),
+});
+console.log(`Synced Algolia docs search index "${indexName}".`);
diff --git a/docs/scripts/test-algolia-search.ts b/docs/scripts/test-algolia-search.ts
new file mode 100644
index 000000000..3e583356a
--- /dev/null
+++ b/docs/scripts/test-algolia-search.ts
@@ -0,0 +1,56 @@
+import { liteClient } from 'algoliasearch/lite';
+import type { AlgoliaDocsRecord } from '@/lib/search-index';
+import { ALGOLIA_DEFAULT_APP_ID, ALGOLIA_DEFAULT_INDEX_NAME } from '@/lib/search-index';
+
+function requireEnv(name: string, fallback?: string): string {
+ const value = process.env[name] ?? fallback;
+ if (!value) throw new Error(`Missing required environment variable: ${name}`);
+ return value;
+}
+
+const appId = requireEnv('ALGOLIA_APP_ID', process.env.NEXT_PUBLIC_ALGOLIA_APP_ID ?? ALGOLIA_DEFAULT_APP_ID);
+const searchApiKey = requireEnv(
+ 'ALGOLIA_SEARCH_API_KEY',
+ process.env.NEXT_PUBLIC_ALGOLIA_SEARCH_API_KEY ?? process.env.ALGOLIA_ADMIN_API_KEY,
+);
+const indexName = process.env.ALGOLIA_INDEX_NAME ?? process.env.NEXT_PUBLIC_ALGOLIA_INDEX_NAME ?? ALGOLIA_DEFAULT_INDEX_NAME;
+const queries = process.argv.slice(2);
+
+if (queries.length === 0) {
+ queries.push(
+ 'quickstart',
+ 'oauth auth config',
+ 'create connected account',
+ 'gmail send email',
+ 'tool router session search',
+ 'webhook verification',
+ );
+}
+
+const client = liteClient(appId, searchApiKey);
+
+for (const query of queries) {
+ const result = await client.searchForHits<AlgoliaDocsRecord>({
+ requests: [
+ {
+ type: 'default',
+ indexName,
+ query,
+ hitsPerPage: 5,
+ distinct: true,
+ getRankingInfo: true,
+ clickAnalytics: true,
+ },
+ ],
+ });
+
+ const response = result.results[0];
+ console.log(`\n## ${query}`);
+ response.hits.forEach((hit, index) => {
+ const section = hit.section ? ` — ${hit.section}` : '';
+ const snippet = hit.content.replace(/\s+/g, ' ').slice(0, 180);
+ console.log(`${index + 1}. ${hit.title}${section}`);
+ console.log(` ${hit.url}`);
+ console.log(` ${snippet}${snippet.length === 180 ? '…' : ''}`);
+ });
+}