Skip to content

Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

feat(llms): add llms-txt generation and deployment scripts

[email protected]8926accf1218
diff --git a/.github/workflows/publish_docs.yml b/.github/workflows/publish_docs.yml
index 12d08dcfb..e3e11b176 100644
--- a/.github/workflows/publish_docs.yml
+++ b/.github/workflows/publish_docs.yml
@@ -35,4 +35,6 @@ jobs:
POSTHOG_API_KEY: ${{ secrets.POSTHOG_API_KEY }}
FERN_TOKEN: ${{ secrets.FERN_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+ CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
+ CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
run: pnpm turbo run publish
diff --git a/fern/llms-txt/package.json b/fern/llms-txt/package.json
new file mode 100644
index 000000000..6897ddae1
--- /dev/null
+++ b/fern/llms-txt/package.json
@@ -0,0 +1,15 @@
+{
+ "name": "composio-docs-proxy",
+ "version": "0.0.0",
+ "private": true,
+ "scripts": {
+ "build": "echo 'llms.txt generated by parent turbo task'",
+ "deploy": "wrangler deploy",
+ "publish": "wrangler deploy",
+ "dev": "wrangler dev",
+ "start": "wrangler dev"
+ },
+ "devDependencies": {
+ "wrangler": "^4.26.1"
+ }
+}
\ No newline at end of file
diff --git a/fern/llms-txt/public/llms.txt b/fern/llms-txt/public/llms.txt
new file mode 100644
index 000000000..7c3f0b1af
--- /dev/null
+++ b/fern/llms-txt/public/llms.txt
@@ -0,0 +1,57 @@
+# Composio Docs
+
+## Docs
+
+- [Welcome](https://docs.composio.dev/docs/welcome)
+- [Quickstart](https://docs.composio.dev/docs/quickstart): Add authenticated tool-calling to any LLM agent in three steps.
+- [IDE and Agent Setup](https://docs.composio.dev/docs/dev-setup): Cursor is an AI-powered code editor that can significantly accelerate your development with Composio. Here's how to set it up optimally.
+- [Providers](https://docs.composio.dev/docs/providers): Think of providers as translators. Different AI frameworks (like OpenAI, Anthropic Claude, or LangChain) expect tools to be formatted in their spec...
+- [Executing Tools](https://docs.composio.dev/docs/executing-tools): For non-agent providers—such as OpenAI, Anthropic, and Google AI—you can process tool calls using the tool call handlers provided by the Composio SDK.
+- [Authenticating Tools](https://docs.composio.dev/docs/authenticating-tools): Each toolkit comes with its own _auth config_.
+- [Fetch and Filter Tools](https://docs.composio.dev/docs/fetching-tools): Toolkits are collections of tools from a specific app!
+- [Schema Modifiers](https://docs.composio.dev/docs/schema-modifiers): Schema modifiers transform a tool's schema before the tool is seen by an agent.
+- [Before Tool Execution](https://docs.composio.dev/docs/before-execution): These modifiers are called before the tool is executed by the LLM. This allows you to modify the _arguments_ called by the LLM before they are exec...
+- [After Tool Execution](https://docs.composio.dev/docs/after-execution): These modifiers are called after the tool is executed by the LLM. This allows you to modify the _result_ of the tool before it is returned to the a...
+- [Custom tools](https://docs.composio.dev/docs/custom-tools): A standalone tool is the simplest form of custom tool. It only requires input parameters and an execute function:
+- [Custom Auth Configs](https://docs.composio.dev/docs/custom-auth-configs): Some apps, like PostHog, Hubspot, Linear, etc. allow customizing the auth config for your usage.
+- [Programmatic Auth Configs](https://docs.composio.dev/docs/programmatic-auth-configs): Since OAuth2 is the most common authentication type for applications, Composio provides managed auth for most OAuth2 based applications.
+- [Custom Auth Parameters](https://docs.composio.dev/docs/custom-auth-params): First, initialize the Composio SDK with your API key:
+- [Using Triggers](https://docs.composio.dev/docs/using-triggers): Head to any app that supports triggers, such as Slack in the dashboard and enable the trigger of choice.
+- [Composio MCP Servers](https://docs.composio.dev/docs/mcp-overview): Our MCP SDK features an intuitive, powerful, and developer-friendly API:
+- [Developers](https://docs.composio.dev/docs/mcp-developers): - Create and Configure Toolkit permissions:
+- [Partner Program](https://docs.composio.dev/docs/mcp-partner-api): As a partner, you receive:
+- [Providers](https://docs.composio.dev/docs/mcp-providers): The Mastra Provider is tailored for the Mastra TypeScript-native agent framework. This client provides a getServer method to obtain the Composio MC...
+- [Migration Guide](https://docs.composio.dev/docs/migration): A lot of the changes are on the infra side, but from the SDK point of view, here is what you can expect:
+- [Basic Hono Server](https://docs.composio.dev/examples/sample-webserver): A simple example of a web server that uses the Composio SDK to call tools.
+- [Full Stack Chat App](https://docs.composio.dev/examples/chat-app): Ensure you've followed the README.md in the composiohq/chat repository to set up the project locally.
+- [AI Gmail Labeller](https://docs.composio.dev/examples/gmail-reaper): - Clone the filter-gmails-example repository and set the project up locally.
+- [Tool Generator](https://docs.composio.dev/examples/tool-generator): An example that show you how to showcase auth scheme and action params for tools
+- [OpenAI Provider](https://docs.composio.dev/providers/openai.mdx): By default, the OpenAI Provider is installed when you install the Composio SDK. You can also install it manually:
+- [Anthropic Provider](https://docs.composio.dev/providers/anthropic.mdx): The Anthropic provider can be installed for both SDKs.
+- [Google Provider](https://docs.composio.dev/providers/google.mdx): The Google Gen AI SDK and theprovider can only be installed for the Python SDK.
+- [LangChain Provider](https://docs.composio.dev/providers/langchain.mdx): pip install composio_langchain==0.8.0 langchain
+- [CrewAI Provider](https://docs.composio.dev/providers/crewai.mdx): The CrewAI Provider formats the Composio tools into an object compatible with CrewAI's tool calling capabilities. Agents and LLM workflows built wi...
+- [Vercel AI SDK Provider](https://docs.composio.dev/providers/vercel.mdx): Vercel AI SDK and the provider are only available for the TypeScript SDK.
+- [OpenAI Agents Provider](https://docs.composio.dev/providers/openai-agents.mdx): The OpenAI Agents Provider is a provider that formats the Composio tools into an object compatible with OpenAI's Agents API.
+- [Mastra Provider](https://docs.composio.dev/providers/mastra.mdx): npm install @composio/mastra
+- [TypeScript](https://docs.composio.dev/providers/custom/typescript): The Composio SDK uses a provider architecture to adapt tools for different AI frameworks. The provider handles:
+- [Python](https://docs.composio.dev/providers/custom/python): The Composio SDK uses a provider architecture to adapt tools for different AI frameworks. The provider handles:
+- [Introduction](https://docs.composio.dev/tools/introduction.mdx)
+- [Premium Tools](https://docs.composio.dev/tools/src/premium-tools.mdx): Premium tools are essential tool calls that cost us more to run, normally paying for more premium services or longer running tasks.
+- [Github](https://docs.composio.dev/tools/github): GitHub is a code hosting platform for version control and collaboration, offering Git-based repository management, issue tracking, and continuous i...
+- [Gmail](https://docs.composio.dev/tools/gmail): Gmail is Google’s email service, featuring spam protection, search functions, and seamless integration with other G Suite apps for productivity
+- [Notion](https://docs.composio.dev/tools/notion): Notion centralizes notes, docs, wikis, and tasks in a unified workspace, letting teams build custom workflows for collaboration and knowledge manag...
+- [Googlecalendar](https://docs.composio.dev/tools/googlecalendar): SLUG: GOOGLECALENDAR
+- [Composio](https://docs.composio.dev/tools/composio): Composio enables AI Agents and LLMs to authenticate and integrate with various tools via function calling.
+- [Python SDK Reference](https://docs.composio.dev/sdk-reference/python/sdk-reference.md): This is the comprehensive SDK reference for the Composio Python SDK.
+- [Composio](https://docs.composio.dev/sdk-reference/type-script/composio/classes/Composio.mdx): Defined in: ts/packages/core/src/composio.ts:82
+- [ComposioProvider](https://docs.composio.dev/sdk-reference/type-script/provider/ComposioProvider/classes/ComposioProvider.mdx): Defined in: ts/packages/core/src/provider/ComposioProvider.ts:14
+- [OpenAIProvider](https://docs.composio.dev/sdk-reference/type-script/provider/OpenAIProvider/classes/OpenAIProvider.mdx): Defined in: ts/packages/core/src/provider/OpenAIProvider.ts:22
+- [BaseAgenticProvider](https://docs.composio.dev/sdk-reference/type-script/provider/BaseProvider/classes/BaseAgenticProvider.mdx): Defined in: ts/packages/core/src/provider/BaseProvider.ts:106
+- [BaseNonAgenticProvider](https://docs.composio.dev/sdk-reference/type-script/provider/BaseProvider/classes/BaseNonAgenticProvider.mdx): Defined in: ts/packages/core/src/provider/BaseProvider.ts:80
+- [Tools](https://docs.composio.dev/sdk-reference/type-script/models/Tools/classes/Tools.mdx): Defined in: ts/packages/core/src/models/Tools.ts:53
+- [Toolkits](https://docs.composio.dev/sdk-reference/type-script/models/Toolkits/classes/Toolkits.mdx): Defined in: ts/packages/core/src/models/Toolkits.ts:31
+- [Triggers](https://docs.composio.dev/sdk-reference/type-script/models/Triggers/classes/Triggers.mdx): Defined in: ts/packages/core/src/models/Triggers.ts:39
+- [AuthConfigs](https://docs.composio.dev/sdk-reference/type-script/AuthConfigs/classes/AuthConfigs.mdx): Defined in: ts/packages/core/src/models/AuthConfigs.ts:43
+- [ConnectedAccounts](https://docs.composio.dev/sdk-reference/type-script/ConnectedAccounts/classes/ConnectedAccounts.mdx): Defined in: ts/packages/core/src/models/ConnectedAccounts.ts:42
+- [CustomTools](https://docs.composio.dev/sdk-reference/type-script/CustomTools/classes/CustomTools.mdx): Defined in: ts/packages/core/src/models/CustomTools.ts:37
diff --git a/fern/llms-txt/src/index.js b/fern/llms-txt/src/index.js
new file mode 100644
index 000000000..ea1a579ee
--- /dev/null
+++ b/fern/llms-txt/src/index.js
@@ -0,0 +1,13 @@
+export default {
+ async fetch(request, env, ctx) {
+ const url = new URL(request.url);
+
+ // Serve static llms.txt
+ if (url.pathname === '/llms.txt') {
+ return env.ASSETS.fetch(request);
+ }
+
+ // Everything else — proxy to origin that docs.composio.dev points to
+ return fetch(request);
+ }
+}
\ No newline at end of file
diff --git a/fern/llms-txt/turbo.json b/fern/llms-txt/turbo.json
new file mode 100644
index 000000000..c663f8c4b
--- /dev/null
+++ b/fern/llms-txt/turbo.json
@@ -0,0 +1,14 @@
+{
+ "$schema": "https://turbo.build/schema.json",
+ "extends": ["//"],
+ "tasks": {
+ "build": {
+ "cache": false
+ },
+ "publish": {
+ "dependsOn": ["^build"],
+ "cache": false,
+ "env": ["CLOUDFLARE_API_TOKEN", "CLOUDFLARE_ACCOUNT_ID"]
+ }
+ }
+}
\ No newline at end of file
diff --git a/fern/llms-txt/wrangler.jsonc b/fern/llms-txt/wrangler.jsonc
new file mode 100644
index 000000000..11cfdca64
--- /dev/null
+++ b/fern/llms-txt/wrangler.jsonc
@@ -0,0 +1,13 @@
+{
+ "$schema": "node_modules/wrangler/config-schema.json",
+ "name": "composio-docs-proxy",
+ "main": "src/index.js",
+ "compatibility_date": "2025-07-26",
+ "assets": {
+ "directory": "./public/",
+ "binding": "ASSETS"
+ },
+ "routes": [
+ { "pattern": "docs.composio.dev/*", "zone_name": "composio.dev" }
+ ]
+ }
\ No newline at end of file
diff --git a/fern/package.json b/fern/package.json
index cc746404e..f2c72e9ab 100644
--- a/fern/package.json
+++ b/fern/package.json
@@ -28,7 +28,10 @@
"publish": "fern generate --docs",
"generate": "pnpm tools:generate && pnpm sdkdocs",
"snippet:new": "bun run scripts/snippet-new.ts",
- "doc:new": "bun run scripts/doc-new.ts"
+ "doc:new": "bun run scripts/doc-new.ts",
+ "llms:build": "bun run scripts/build-llms-txt.ts",
+ "llms:dev": "bun run scripts/build-llms-txt.ts --watch",
+ "llms:deploy": "cd llms-txt && npx wrangler deploy"
},
"dependencies": {
"fern-api": "^0.51.39",
diff --git a/fern/scripts/build-llms-txt.ts b/fern/scripts/build-llms-txt.ts
new file mode 100644
index 000000000..3a35c86e2
--- /dev/null
+++ b/fern/scripts/build-llms-txt.ts
@@ -0,0 +1,335 @@
+#!/usr/bin/env bun
+/* eslint-disable no-console */
+
+import fs from 'fs';
+import path from 'path';
+import yaml from 'yaml';
+import { watch } from 'fs';
+
+// Define directories
+const FERN_DIR = path.join(import.meta.dir, '..');
+const DOCS_YML_PATH = path.join(FERN_DIR, 'docs.yml');
+const LLMS_TXT_PATH = path.join(FERN_DIR, 'llms-txt', 'public', 'llms.txt');
+const PAGES_DIR = path.join(FERN_DIR, 'pages');
+const TOOLS_DIR = path.join(FERN_DIR, 'tools');
+
+interface DocsConfig {
+ title: string;
+ navigation: Array<{
+ tab: string;
+ layout?: LayoutItem[];
+ }>;
+ tabs?: {
+ [key: string]: {
+ 'display-name': string;
+ 'skip-slug'?: boolean;
+ };
+ };
+}
+
+interface LayoutItem {
+ section?: string;
+ contents?: ContentItem[];
+ page?: string;
+ path?: string;
+ slug?: string;
+}
+
+interface ContentItem {
+ page?: string;
+ path?: string;
+ slug?: string;
+ section?: string;
+ contents?: ContentItem[];
+}
+
+interface LlmsEntry {
+ title: string;
+ url: string;
+ description?: string;
+}
+
+// Extract description from MDX content
+function extractDescription(mdxPath: string): string | undefined {
+ try {
+ const fullPath = path.join(FERN_DIR, mdxPath);
+ if (!fs.existsSync(fullPath)) {
+ console.warn(`⚠️ File not found: ${fullPath}`);
+ return undefined;
+ }
+
+ const content = fs.readFileSync(fullPath, 'utf8');
+
+ // Try to extract from frontmatter description
+ const frontmatterMatch = content.match(/^---\s*\n([\s\S]*?)\n---/);
+ if (frontmatterMatch) {
+ const frontmatter = frontmatterMatch[1];
+ const descMatch = frontmatter.match(/description:\s*["']?(.+?)["']?\s*$/m);
+ if (descMatch) {
+ return descMatch[1].trim();
+ }
+ }
+
+ // Extract first meaningful paragraph after the title
+ const lines = content.split('\n');
+ let inFrontmatter = false;
+ let foundTitle = false;
+ let description = '';
+
+ for (const line of lines) {
+ const trimmedLine = line.trim();
+
+ // Handle frontmatter
+ if (trimmedLine === '---') {
+ inFrontmatter = !inFrontmatter;
+ continue;
+ }
+
+ if (inFrontmatter) continue;
+
+ // Skip titles
+ if (trimmedLine.startsWith('#')) {
+ foundTitle = true;
+ continue;
+ }
+
+ // Skip empty lines, code blocks, and JSX
+ if (!trimmedLine ||
+ trimmedLine.startsWith('```') ||
+ trimmedLine.startsWith('<') ||
+ trimmedLine.startsWith('import ') ||
+ trimmedLine.startsWith('export ')) {
+ continue;
+ }
+
+ // Found a paragraph
+ if (foundTitle && trimmedLine.length > 20) {
+ description = trimmedLine;
+ break;
+ }
+ }
+
+ // Clean up the description
+ if (description) {
+ // Remove MDX components and markdown syntax
+ description = description
+ .replace(/<[^>]+>/g, '') // Remove HTML/JSX tags
+ .replace(/\[([^\]]+)\]\([^)]+\)/g, '$1') // Convert [text](url) to text
+ .replace(/`([^`]+)`/g, '$1') // Remove inline code backticks
+ .replace(/\*\*([^*]+)\*\*/g, '$1') // Remove bold
+ .replace(/\*([^*]+)\*/g, '$1') // Remove italic
+ .trim();
+
+ // Truncate if too long
+ if (description.length > 150) {
+ description = description.substring(0, 147) + '...';
+ }
+
+ return description;
+ }
+
+ return undefined;
+ } catch (error) {
+ console.warn(`⚠️ Error reading ${mdxPath}:`, error);
+ return undefined;
+ }
+}
+
+// Process content items recursively
+function processContentItems(items: ContentItem[], tabSlug: string, entries: LlmsEntry[], parentSlug?: string): void {
+ for (const item of items) {
+ if (item.page && item.path) {
+ let url: string;
+
+ if (item.slug) {
+ // Use explicit slug if provided
+ const slugParts = [tabSlug, item.slug].filter(Boolean);
+ url = `https://docs.composio.dev/${slugParts.join('/')}`;
+ } else if (item.path) {
+ // For SDK reference pages without explicit slugs, construct from path
+ const pathMatch = item.path.match(/pages\/dist\/(.+)$/);
+ if (pathMatch) {
+ let constructedPath = pathMatch[1];
+ // Handle sdk-reference special case
+ if (tabSlug === 'sdk-reference' && constructedPath.startsWith('sdk/')) {
+ constructedPath = constructedPath.replace('sdk/', 'sdk-reference/');
+ // Convert typescript to type-script
+ constructedPath = constructedPath.replace('typescript/', 'type-script/');
+ }
+ url = `https://docs.composio.dev/${constructedPath}`;
+ } else if (item.path.includes('tools/')) {
+ // Handle tools pages
+ const toolName = item.path.match(/tools\/(.+)\.mdx$/)?.[1] || item.page.toLowerCase();
+ url = `https://docs.composio.dev/tools/${toolName}.mdx`;
+ } else {
+ // Fallback
+ const slugParts = [tabSlug, parentSlug].filter(Boolean);
+ url = `https://docs.composio.dev/${slugParts.join('/')}/${item.page.toLowerCase().replace(/\s+/g, '-')}.mdx`;
+ }
+ } else {
+ // Fallback
+ const slugParts = [tabSlug, parentSlug].filter(Boolean);
+ url = `https://docs.composio.dev/${slugParts.join('/')}/${item.page.toLowerCase().replace(/\s+/g, '-')}.mdx`;
+ }
+
+ const description = extractDescription(item.path);
+
+ // Override specific page titles
+ let title = item.page;
+ if (title === 'Overview' && url.includes('mcp-overview')) {
+ title = 'Composio MCP Servers';
+ }
+
+ entries.push({
+ title: title,
+ url: url,
+ description
+ });
+ }
+
+ // Process nested sections
+ if (item.section && item.contents) {
+ const newParentSlug = item.slug || parentSlug;
+ processContentItems(item.contents, tabSlug, entries, newParentSlug);
+ }
+ }
+}
+
+// Process navigation items
+function processNavigation(config: DocsConfig): LlmsEntry[] {
+ const entries: LlmsEntry[] = [];
+
+ for (const navItem of config.navigation) {
+ if (!navItem.layout) continue;
+
+ const tab = navItem.tab;
+ const tabConfig = config.tabs?.[tab];
+ const skipSlug = tabConfig?.['skip-slug'] || false;
+ const tabSlug = skipSlug ? '' : tab;
+
+ for (const layoutItem of navItem.layout) {
+ // Process direct pages
+ if (layoutItem.page && layoutItem.path && layoutItem.slug) {
+ const slugParts = [tabSlug, layoutItem.slug].filter(Boolean);
+ const url = `https://docs.composio.dev/${slugParts.join('/')}`;
+ const description = extractDescription(layoutItem.path);
+
+ entries.push({
+ title: layoutItem.page,
+ url: url.endsWith('.mdx') || url.endsWith('.md') ? url : url + '.mdx',
+ description
+ });
+ }
+
+ // Process section contents
+ if (layoutItem.section && layoutItem.contents) {
+ processContentItems(layoutItem.contents, tabSlug, entries, layoutItem.slug);
+ }
+ }
+ }
+
+ return entries;
+}
+
+// Generate llms.txt content
+function generateLlmsTxt(entries: LlmsEntry[]): string {
+ let content = '# Composio Docs\n\n## Docs\n\n';
+
+ for (const entry of entries) {
+ if (entry.description) {
+ content += `- [${entry.title}](${entry.url}): ${entry.description}\n`;
+ } else {
+ content += `- [${entry.title}](${entry.url})\n`;
+ }
+ }
+
+ return content;
+}
+
+// Main build function
+async function buildLlmsTxt(): Promise<void> {
+ console.log('🔨 Building llms.txt from docs.yml...');
+
+ try {
+ // Read and parse docs.yml
+ const docsYmlContent = fs.readFileSync(DOCS_YML_PATH, 'utf8');
+ const config = yaml.parse(docsYmlContent) as DocsConfig;
+
+ // Process navigation to extract all entries
+ const entries = processNavigation(config);
+ console.log(`📄 Found ${entries.length} documentation pages`);
+
+ // Generate llms.txt content
+ const llmsTxtContent = generateLlmsTxt(entries);
+
+ // Ensure output directory exists
+ const outputDir = path.dirname(LLMS_TXT_PATH);
+ if (!fs.existsSync(outputDir)) {
+ fs.mkdirSync(outputDir, { recursive: true });
+ }
+
+ // Write llms.txt
+ fs.writeFileSync(LLMS_TXT_PATH, llmsTxtContent);
+ console.log(`✅ Generated llms.txt at ${path.relative(FERN_DIR, LLMS_TXT_PATH)}`);
+
+ } catch (error) {
+ console.error('❌ Error building llms.txt:', error);
+ process.exit(1);
+ }
+}
+
+// Watch mode
+function startWatchMode(): void {
+ console.log('👀 Watching for changes...');
+
+ // Watch docs.yml
+ const docsWatcher = watch(DOCS_YML_PATH, async (eventType) => {
+ if (eventType === 'change') {
+ console.log('🔄 docs.yml changed, rebuilding...');
+ await buildLlmsTxt();
+ }
+ });
+
+ // Watch MDX files in pages directory
+ const pagesWatcher = watch(PAGES_DIR, { recursive: true }, async (eventType, filename) => {
+ if (filename && (filename.endsWith('.mdx') || filename.endsWith('.md'))) {
+ console.log(`🔄 Page changed: ${filename}, rebuilding...`);
+ await buildLlmsTxt();
+ }
+ });
+
+ // Watch MDX files in tools directory
+ const toolsWatcher = watch(TOOLS_DIR, { recursive: true }, async (eventType, filename) => {
+ if (filename && (filename.endsWith('.mdx') || filename.endsWith('.md'))) {
+ console.log(`🔄 Tool doc changed: ${filename}, rebuilding...`);
+ await buildLlmsTxt();
+ }
+ });
+
+ // Cleanup on exit
+ process.on('SIGINT', () => {
+ console.log('\n👋 Stopping watchers...');
+ docsWatcher.close();
+ pagesWatcher.close();
+ toolsWatcher.close();
+ process.exit(0);
+ });
+}
+
+// Main function
+async function main(): Promise<void> {
+ await buildLlmsTxt();
+
+ // Check if we should run in watch mode
+ const isWatchMode = process.argv.includes('--watch') || process.argv.includes('-w');
+
+ if (isWatchMode) {
+ startWatchMode();
+ }
+}
+
+// Run the script
+main().catch(error => {
+ console.error('❌ Fatal error:', error);
+ process.exit(1);
+});
\ No newline at end of file
diff --git a/fern/turbo.json b/fern/turbo.json
index e24f8a6b2..3af115bcd 100644
--- a/fern/turbo.json
+++ b/fern/turbo.json
@@ -3,8 +3,8 @@
"extends": ["//"],
"tasks": {
"build": {
- "dependsOn": ["api:pull", "add-og", "snippets:build", "tools:generate", "sdkdocs"],
- "outputs": ["apis/openapi.yml", "apis/openapi-mcp.json", "pages/dist/**", "tools/**"],
+ "dependsOn": ["api:pull", "add-og", "snippets:build", "tools:generate", "sdkdocs", "llms:build"],
+ "outputs": ["apis/openapi.yml", "apis/openapi-mcp.json", "pages/dist/**", "tools/**", "llms-txt/public/llms.txt"],
"cache": true
},
@@ -79,6 +79,23 @@
"cache": true
},
+ "llms:build": {
+ "outputs": ["llms-txt/public/llms.txt"],
+ "inputs": ["docs.yml", "pages/**/*.mdx", "pages/**/*.md", "tools/**/*.mdx", "scripts/build-llms-txt.ts"],
+ "cache": true
+ },
+
+ "llms:dev": {
+ "persistent": true,
+ "cache": false
+ },
+
+ "llms:deploy": {
+ "dependsOn": ["llms:build"],
+ "cache": false,
+ "env": ["CLOUDFLARE_API_TOKEN", "CLOUDFLARE_ACCOUNT_ID"]
+ },
+
"check": {
"dependsOn": ["snippets:build"],
"cache": false
@@ -96,9 +113,9 @@
},
"publish": {
- "dependsOn": ["build"],
+ "dependsOn": ["build", "llms:deploy"],
"cache": false,
- "env": ["FERN_TOKEN", "POSTHOG_API_KEY"]
+ "env": ["FERN_TOKEN", "POSTHOG_API_KEY", "CLOUDFLARE_API_TOKEN", "CLOUDFLARE_ACCOUNT_ID"]
},
"clean": {
diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml
index f843f2a1f..c52a04d21 100644
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -160,6 +160,12 @@ importers:
specifier: ^2.8.0
version: 2.8.0
+ fern/llms-txt:
+ devDependencies:
+ wrangler:
+ specifier: ^4.26.1
+ version: 4.26.1
+
ts/examples/anthropic:
dependencies:
'@anthropic-ai/sdk':
@@ -1119,6 +1125,15 @@ packages:
workerd:
optional: true
+ '@cloudflare/[email protected]':
+ resolution: {integrity: sha512-CZe9B2VbjIQjBTyc+KoZcN1oUcm4T6GgCXoel9O7647djHuSRAa6sM6G+NdxWArATZgeMMbsvn9C50GCcnIatA==}
+ peerDependencies:
+ unenv: 2.0.0-rc.19
+ workerd: ^1.20250722.0
+ peerDependenciesMeta:
+ workerd:
+ optional: true
+
'@cloudflare/[email protected]':
resolution: {integrity: sha512-axyXWKYZP5XZulJ4MfN4Su6RDx0AM8rfubqTr2jAoeFK5KZG/W396SgIpd59ZSoDQKx+LLGXP75TFEgBIS3Pog==}
peerDependencies:
@@ -1132,30 +1147,60 @@ packages:
cpu: [x64]
os: [darwin]
+ '@cloudflare/[email protected]':
+ resolution: {integrity: sha512-SOpQqQ2blLY0io/vErve44vJC1M5i7RHuMBdrdEPIEtxiLBTdOOVp4nqZ3KchocxZjskgTc2N4N3b5hNYuKDGw==}
+ engines: {node: '>=16'}
+ cpu: [x64]
+ os: [darwin]
+
'@cloudflare/[email protected]':
resolution: {integrity: sha512-JTX0exbC9/ZtMmQQA8tDZEZFMXZrxOpTUj2hHnsUkErWYkr5SSZH04RBhPg6dU4VL8bXuB5/eJAh7+P9cZAp7g==}
engines: {node: '>=16'}
cpu: [arm64]
os: [darwin]
+ '@cloudflare/[email protected]':
+ resolution: {integrity: sha512-I+TOQ+YQahxL/K7eS2GJzv5CZzSVaZoyqfB15Q71MT/+wyzPCaFDTt+fg3uXdwpaIQEMUfqFNpTQSqbKHAYNgA==}
+ engines: {node: '>=16'}
+ cpu: [arm64]
+ os: [darwin]
+
'@cloudflare/[email protected]':
resolution: {integrity: sha512-8jkSoVRJ+1bOx3tuWlZCGaGCV2ew7/jFMl6V3CPXOoEtERUHsZBQLVkQIGKcmC/LKSj7f/mpyBUeu2EPTo2HEg==}
engines: {node: '>=16'}
cpu: [x64]
os: [linux]
+ '@cloudflare/[email protected]':
+ resolution: {integrity: sha512-WSCv4o2uOW6b++ROVazrEW+jjZdBqCmXmmt7uVVfvjVxlzoYVwK9IvV2IXe4gsJ99HG9I0YCa7AT743cZ7TNNg==}
+ engines: {node: '>=16'}
+ cpu: [x64]
+ os: [linux]
+
'@cloudflare/[email protected]':
resolution: {integrity: sha512-YAzcOyu897z5dQKFzme1oujGWMGEJCR7/Wrrm1nSP6dqutxFPTubRADM8BHn2CV3ij//vaPnAeLmZE3jVwOwig==}
engines: {node: '>=16'}
cpu: [arm64]
os: [linux]
+ '@cloudflare/[email protected]':
+ resolution: {integrity: sha512-jNokAGL3EQqH+31b0dX8+tlbKdjt/0UtTLvgD1e+7bOD92lzjYMa/CixHyMIY/FVvhsN4TNqfiz4cqroABTlhg==}
+ engines: {node: '>=16'}
+ cpu: [arm64]
+ os: [linux]
+
'@cloudflare/[email protected]':
resolution: {integrity: sha512-XWM/6sagDrO0CYDKhXhPjM23qusvIN1ju9ZEml6gOQs8tNOFnq6Cn6X9FAmnyapRFCGUSEC3HZYJAm7zwVKaMA==}
engines: {node: '>=16'}
cpu: [x64]
os: [win32]
+ '@cloudflare/[email protected]':
+ resolution: {integrity: sha512-DiPTY63TNh6/ylvfutNQzYZi688x6NJDjQoqf5uiCp7xHweWx+GpVs42sZPeeXqCNvhm4dYjHjuigXJNh7t8Uw==}
+ engines: {node: '>=16'}
+ cpu: [x64]
+ os: [win32]
+
'@cloudflare/[email protected]':
resolution: {integrity: sha512-OYmVTA45rEEsOmchtltDnrG6BBaOPB7LUGKoyE8AwkfanUOc3/wwuMZ50i1jq6NKrV1TZLsKUKzrwe1NtVGOzQ==}
@@ -2760,6 +2805,15 @@ packages:
resolution: {integrity: sha512-wwQAWhWSuHaag8c4q/KN/vCoeOJYshAIvMQwD4GpSb3OiZklFfvAgmj0VCBBImRpuF/aFgIRzllXlVX93Jevww==}
+ '@poppinss/[email protected]':
+ resolution: {integrity: sha512-FvdDqtcRCtz6hThExcFOgW0cWX+xwSMWcRuQe5ZEb2m7cVQOAVZOIMt+/v9RxGiD9/OY16qJBXK4CVKWAPalBw==}
+
+ '@poppinss/[email protected]':
+ resolution: {integrity: sha512-iG0TIdqv8xJ3Lt9O8DrPRxw1MRLjNpoqiSGU03P/wNLP/s0ra0udPJ1J2Tx5M0J3H/cVyEgpbn8xUKRY9j59kQ==}
+
+ '@poppinss/[email protected]':
+ resolution: {integrity: sha512-m7bpKCD4QMlFCjA/nKTs23fuvoVFoA83brRKmObCUNmi/9tVu8Ve3w4YQAnJu4q3Tjf5fr685HYIC/IA2zHRSg==}
+
'@protobufjs/[email protected]':
resolution: {integrity: sha512-j+gKExEuLmKwvz3OgROXtrJ2UG2x8Ch2YZUxahh+s1F2HZ+wAceUNLkvy6zKCPVRkU++ZWQrdxsUeQXmcg4uoQ==}
@@ -2905,6 +2959,10 @@ packages:
'@shikijs/[email protected]':
resolution: {integrity: sha512-83yeghZ2xxin3Nj8z1NMd/NCuca+gsYXswywDy5bHvwlWL8tpTQmzGeUuHd9FC3E/SBEMvzJRwWEOz5gGes9Qg==}
+ '@sindresorhus/[email protected]':
+ resolution: {integrity: sha512-d9xRovfKNz1SKieM0qJdO+PQonjnnIfSNWfHYnBSJ9hkjm0ZPw6HlxscDXYstp3z+7V2GOFHc+J0CYrYTjqCJw==}
+ engines: {node: '>=18'}
+
'@sindresorhus/[email protected]':
resolution: {integrity: sha512-MkngSCRZ8JdSOCHRaYd+D01XhvU3Hjy6MGl06zhOk614hp9EOAp5gIkBeQg7wtmxpitU6eAL4kdiRMcJa2dlrw==}
engines: {node: '>=12'}
@@ -3124,6 +3182,9 @@ packages:
resolution: {integrity: sha512-4QvC49HTteI1gfemu0I1syWovJgPvGn7CVUoN9ZFkdvr/cCFkrEL7qNCdx/2eICqDWEGnnr68oMdSIPCLAriSQ==}
engines: {node: '>=18.0.0'}
+ '@speed-highlight/[email protected]':
+ resolution: {integrity: sha512-0dxmVj4gxg3Jg879kvFS/msl4s9F3T9UXC1InxgOf7t5NvcPD97u/WTA5vL/IxWHMn7qSxBozqrnnE2wvl1m8g==}
+
'@standard-schema/[email protected]':
resolution: {integrity: sha512-m2bOd0f2RT9k8QJx1JN85cZYyH1RqFBdlwtkSlf4tBDYLCiiZnv1fIIwacK6cqwXavOydf0NPToMQgpKq+dVlA==}
@@ -3767,6 +3828,10 @@ packages:
resolution: {integrity: sha512-yki5XnKuf750l50uGTllt6kKILY4nQ1eNIQatoXEByZ5dWgnKqbnqmTrBE5B4N7lrMJKQ2ytWMiTO2o0v6Ew/w==}
engines: {node: '>= 0.6'}
+ resolution: {integrity: sha512-9Kr/j4O16ISv8zBBhJoi4bXOYNTkFLOqSL3UDB0njXxCXNezjeyVrJyGOWtgfs/q2km1gwBcfH8q1yEGoMYunA==}
+ engines: {node: '>=18'}
+
resolution: {integrity: sha512-yCEafptTtb4bk7GLEQoM8KVJpxAfdBJYaXyzQEgQQQgYrZiDp8SJmGKlYza6CYjEDNstAdNdKA3UuoULlEbS6w==}
engines: {node: '>=12.13'}
@@ -3974,6 +4039,9 @@ packages:
resolution: {integrity: sha512-xUtoPkMggbz0MPyPiIWr1Kp4aeWJjDZ6SMvURhimjdZgsRuDplF5/s9hcgGhyXMhs+6vpnuoiZ2kFiu3FMnS8Q==}
engines: {node: '>=18'}
+ resolution: {integrity: sha512-5qucVt2XcuGMcEGgWI7i+yZpmpByQ8J1lHhcL7PwqCwu9FPP3VUXzT4ltHe5i2z9dePwEHcDVOAfSnHsOlCXRA==}
+
resolution: {integrity: sha512-e3nRfgfUZ4rNGL232gUgX06QNyyez04KdjFrF+LTRoOXmrOgFKDg4BCdsjW8EnT69eqdYGmRpJwiPVYNrCaW3g==}
engines: {node: '>= 0.4'}
@@ -4742,6 +4810,10 @@ packages:
resolution: {integrity: sha512-dcS1ul+9tmeD95T+x28/ehLgd9mENa3LsvDTtzm3vyBEO7RPptvAD+t44WVXaUjTBRcrpFeFlC8WCruUR456hw==}
engines: {node: '>=0.10.0'}
+ resolution: {integrity: sha512-o+NO+8WrRiQEE4/7nwRJhN1HWpVmJm511pBHUxPLtp0BUISzlBplORYSmTclCnJvQq2tKu/sgl3xVpkc7ZWuQQ==}
+ engines: {node: '>=6'}
+
resolution: {integrity: sha512-h4GGlBJNGU/Sj2PipW9kL+ewj7To3c+SnnNKH3HZaVHEqGPMHVB96T1lLjtCLcZCyUfabMr/zFIkLNI4War+Xg==}
engines: {node: '>=18'}
@@ -4965,6 +5037,11 @@ packages:
engines: {node: '>=18.0.0'}
hasBin: true
+ resolution: {integrity: sha512-7+/RQQ9dNsyGfR2XN2RDLultf7HHrJ5YltSXSeyQGUpzGU3iYlFhh9Smg+ygkkOJ3+trf0bgwixOnqnnWpc9ZQ==}
+ engines: {node: '>=18.0.0'}
+ hasBin: true
+
resolution: {integrity: sha512-J7p63hRiAjw1NDEww1W7i37+ByIrOWO5XQQAzZ3VOcL0PNybwpfmV/N05zFAzwQ9USyEcX6t3UO+K5aqBQOIHw==}
@@ -5894,6 +5971,10 @@ packages:
resolution: {integrity: sha512-5JRxVqC8I8NuOUjzBbvVJAKNM8qoVuH0O77h4WInc/qC2q5IreqKxYwgkga3PfA22OayK2ikceb/B26dztPl+Q==}
engines: {node: '>=16'}
+ resolution: {integrity: sha512-HRVVSbCCMbj7/kdWF9Q+bbckjBHLtHMEoJWlkmYzzdwhYMkjkOwubLM6t7NbWKjgKamGDrWL1++KrjUO1t9oAQ==}
+ engines: {node: '>=18'}
+
resolution: {integrity: sha512-qpCAvRl9stuOHveKsn7HncJRvv501qIacKzQlO/+Lwxc9+0q2wLyv4Dfvt80/DPn2pqOBsJdDiogXGR9+OvwRw==}
engines: {node: '>=8'}
@@ -6156,9 +6237,16 @@ packages:
resolution: {integrity: sha512-raqeBD6NQK4SkWhQzeYKd1KmIG6dllBOTt55Rmkt4HtI9mwdWtJljnrXjAFUBLTSN67HWrOIZ3EPF4kjUw80Bg==}
engines: {node: '>=14.0'}
+ resolution: {integrity: sha512-GrKEsc3ughskmGA9jevVlIOPMiiAHJ4OFUtaAH+NhfTUSiZ1wMPIQqQvAJUrJspFXJt3EBWgpAeoHEDVT1IBug==}
+ engines: {node: '>=20.18.1'}
+
resolution: {integrity: sha512-B06u0wXkEd+o5gOCMl/ZHl5cfpYbDZKAT+HWTL+Hws6jWu7dCiqBBXXXzMFcFVJb8D4ytAnYmxJA83uwOQRSsg==}
+ resolution: {integrity: sha512-t/OMHBNAkknVCI7bVB9OWjUUAwhVv9vsPIAGnNUxnu3FxPQN11rjh0sksLMzc3g7IlTgvHmOTl4JM7JHpcv5wA==}
+
resolution: {integrity: sha512-VGXBUVwxKMBUznyffQweQABPRRW1vHZAbadFZud4pLFAqRGvv/96vafgjWFqzourzr8YonlQiPgH0YCJfawoGQ==}
engines: {node: '>=12'}
@@ -6321,6 +6409,11 @@ packages:
engines: {node: '>=16'}
hasBin: true
+ resolution: {integrity: sha512-wDZqSKfIfQ2eVTUL6UawXdXEKPPyzRTnVdbhoKGq3NFrMxd+7v1cNH92u8775Qo1zO5S+GyWonQmZPFakXLvGw==}
+ engines: {node: '>=16'}
+ hasBin: true
+
resolution: {integrity: sha512-37xm0CG2qMvsJUNZYQKje6HbCsJFYuE8dQSnu7981iDRT4DLrEIL1DAUnZJG9HiXteKPvrSj96AkZyomi5sYHw==}
engines: {node: '>=18.0.0'}
@@ -6331,6 +6424,16 @@ packages:
'@cloudflare/workers-types':
optional: true
+ resolution: {integrity: sha512-zGFEtHrjTAWOngm+zwEvYCxFwMSIBrzHa3Yu6rAxYMEzsT8PPvo2rdswyUJiUkpE9s2Depr37opceaY7JxEYFw==}
+ engines: {node: '>=18.0.0'}
+ hasBin: true
+ peerDependencies:
+ '@cloudflare/workers-types': ^4.20250726.0
+ peerDependenciesMeta:
+ '@cloudflare/workers-types':
+ optional: true
+
resolution: {integrity: sha512-r6lPcBGxZXlIcymEu7InxDMhdW0KDxpLgoFLcguasxCaJ/SOIZwINatK9KY/tf+ZrlywOKU0UDj3ATXUBfxJXA==}
engines: {node: '>=8'}
@@ -6417,9 +6520,15 @@ packages:
resolution: {integrity: sha512-cYVsTjKl8b+FrnidjibDWskAv7UKOfcwaVZdp/it9n1s9fU3IkgDbhdIRKCW4JDsAlECJY0ytoVPT3sK6kideA==}
engines: {node: '>=18'}
+ resolution: {integrity: sha512-ho7XuGjLaJ2hWHoK8yFnsUGy2Y5uDpqSTq1FkHLK4/oqKtyUU1AFbOOxY4IpC9f0fTLjwYbslUz0Po5BpD1wrA==}
+
resolution: {integrity: sha512-UeVBXie8cA35DS6+nBkls68xaBBXCye0CNznrhszZjTbRVnJKQuNsyLKBTTL4ln1o1rh2PKtv35twV7irj5SEg==}
+ resolution: {integrity: sha512-rLfVLB4FgQneDr0dv1oddCVZmKjcJ6yX6mS4pU82Mq/Dt9a3cLZQ62pDBL4AUO+uVrCvtWz3ZFUL2HFAFJ/BXQ==}
+
resolution: {integrity: sha512-zYEoo86M1qpA1Pq6329oSyHLS785z/mTwfr9V1Xf/ZLhuuBGaMlDGu/pDVGVUe4H4oa1EFgWZT53DP0U3oT9CQ==}
@@ -7115,6 +7224,12 @@ snapshots:
optionalDependencies:
workerd: 1.20250617.0
+ dependencies:
+ unenv: 2.0.0-rc.19
+ optionalDependencies:
+ workerd: 1.20250726.0
+
dependencies:
'@vitest/runner': 3.2.4
@@ -7135,18 +7250,33 @@ snapshots:
'@cloudflare/[email protected]':
optional: true
+ '@cloudflare/[email protected]':
+ optional: true
+
'@cloudflare/[email protected]':
optional: true
+ '@cloudflare/[email protected]':
+ optional: true
+
'@cloudflare/[email protected]':
optional: true
+ '@cloudflare/[email protected]':
+ optional: true
+
'@cloudflare/[email protected]':
optional: true
+ '@cloudflare/[email protected]':
+ optional: true
+
'@cloudflare/[email protected]':
optional: true
+ '@cloudflare/[email protected]':
+ optional: true
+
'@cloudflare/[email protected]': {}
'@composio/[email protected]': {}
@@ -8904,6 +9034,18 @@ snapshots:
'@polka/[email protected]': {}
+ '@poppinss/[email protected]':
+ dependencies:
+ kleur: 4.1.5
+
+ '@poppinss/[email protected]':
+ dependencies:
+ '@poppinss/colors': 4.1.5
+ '@sindresorhus/is': 7.0.2
+ supports-color: 10.0.0
+
+ '@poppinss/[email protected]': {}
+
'@protobufjs/[email protected]': {}
'@protobufjs/[email protected]': {}
@@ -9007,6 +9149,8 @@ snapshots:
'@shikijs/[email protected]': {}
+ '@sindresorhus/[email protected]': {}
+
'@sindresorhus/[email protected]':
dependencies:
'@sindresorhus/transliterate': 1.6.0
@@ -9344,6 +9488,8 @@ snapshots:
'@smithy/types': 4.3.1
tslib: 2.8.1
+ '@speed-highlight/[email protected]': {}
+
'@standard-schema/[email protected]': {}
'@swc/[email protected]': {}
@@ -10079,6 +10225,8 @@ snapshots:
+
dependencies:
is-what: 4.1.16
@@ -10253,6 +10401,8 @@ snapshots:
+
@@ -11075,6 +11225,8 @@ snapshots:
+
dependencies:
@@ -11275,6 +11427,24 @@ snapshots:
- bufferutil
- utf-8-validate
+ dependencies:
+ '@cspotcode/source-map-support': 0.8.1
+ acorn: 8.14.0
+ acorn-walk: 8.3.2
+ exit-hook: 2.2.1
+ glob-to-regexp: 0.4.1
+ sharp: 0.33.5
+ stoppable: 1.1.0
+ undici: 7.12.0
+ workerd: 1.20250726.0
+ ws: 8.18.0
+ youch: 4.1.0-beta.10
+ zod: 3.22.3
+ transitivePeerDependencies:
+ - bufferutil
+ - utf-8-validate
+
dependencies:
brace-expansion: 1.1.12
@@ -12268,6 +12438,8 @@ snapshots:
dependencies:
copy-anything: 3.0.5
+
dependencies:
has-flag: 4.0.0
@@ -12536,6 +12708,8 @@ snapshots:
dependencies:
'@fastify/busboy': 2.1.1
+
dependencies:
defu: 6.1.4
@@ -12544,6 +12718,14 @@ snapshots:
pathe: 2.0.3
ufo: 1.6.1
+ dependencies:
+ defu: 6.1.4
+ exsolve: 1.0.7
+ ohash: 2.0.11
+ pathe: 2.0.3
+ ufo: 1.6.1
+
dependencies:
crypto-random-string: 4.0.0
@@ -12856,6 +13038,14 @@ snapshots:
'@cloudflare/workerd-linux-arm64': 1.20250617.0
'@cloudflare/workerd-windows-64': 1.20250617.0
+ optionalDependencies:
+ '@cloudflare/workerd-darwin-64': 1.20250726.0
+ '@cloudflare/workerd-darwin-arm64': 1.20250726.0
+ '@cloudflare/workerd-linux-64': 1.20250726.0
+ '@cloudflare/workerd-linux-arm64': 1.20250726.0
+ '@cloudflare/workerd-windows-64': 1.20250726.0
+
dependencies:
'@cloudflare/kv-asset-handler': 0.4.0
@@ -12873,6 +13063,22 @@ snapshots:
- bufferutil
- utf-8-validate
+ dependencies:
+ '@cloudflare/kv-asset-handler': 0.4.0
+ '@cloudflare/unenv-preset': 2.5.0([email protected])([email protected])
+ blake3-wasm: 2.1.5
+ esbuild: 0.25.4
+ miniflare: 4.20250726.0
+ path-to-regexp: 6.3.0
+ unenv: 2.0.0-rc.19
+ workerd: 1.20250726.0
+ optionalDependencies:
+ fsevents: 2.3.3
+ transitivePeerDependencies:
+ - bufferutil
+ - utf-8-validate
+
dependencies:
ansi-styles: 4.3.0
@@ -12938,12 +13144,25 @@ snapshots:
+ dependencies:
+ '@poppinss/exception': 1.2.2
+ error-stack-parser-es: 1.0.5
+
dependencies:
cookie: 0.7.2
mustache: 4.2.0
stacktracey: 2.1.8
+ dependencies:
+ '@poppinss/colors': 4.1.5
+ '@poppinss/dumper': 0.6.4
+ '@speed-highlight/core': 1.2.7
+ cookie: 1.0.2
+ youch-core: 0.3.3
+
dependencies:
zod: 3.25.67