Skip to content

Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

perf(cli): resolve toolkits from local knowledge, not the catalog

jkomyno5350b00d906c
diff --git a/ts/packages/cli/CHANGELOG.md b/ts/packages/cli/CHANGELOG.md
index b576e3617..238cc1987 100644
--- a/ts/packages/cli/CHANGELOG.md
+++ b/ts/packages/cli/CHANGELOG.md
@@ -44,6 +44,22 @@
layer treated the request's own failure as a cache failure and retried it,
so every failed toolkit, tool, or trigger listing cost two round trips and
twice the wait before reporting the same error.
+- Resolving a tool's toolkit no longer downloads the toolkit catalog. The CLI
+ ships with the toolkit slugs it knew at build time and remembers any it
+ learns since in `known-toolkit-slugs.json`, so `composio tools execute` only
+ reaches for the catalog when a slug matches nothing it knows — a toolkit
+ released after your CLI version. `FORCE_USE_CACHE` is unaffected.
+- Resolving a tool's toolkit now reads what the CLI knows locally once per run
+ rather than once per lookup. A single `composio tools execute` resolves the
+ same toolkit up to four times — and once per tool with `--parallel` — and
+ each of those re-read `known-toolkit-slugs.json`, re-parsed it, and rebuilt
+ the lookup table over it. The weekly background refresh now also runs once per
+ run instead of rewriting the file after every lookup.
+- The background catalog refresh is abandoned after ten seconds. A finished
+ command ends when the event loop drains, so a slow network could otherwise
+ hold an exiting `composio` process open until the refresh completed.
+- Cache files are now written atomically, so an interrupted run can no longer
+ leave a truncated `toolkits.json` or `tools.json` behind.
## 0.3.1
diff --git a/ts/packages/cli/README.md b/ts/packages/cli/README.md
index fcb46d4bf..1141d09c3 100644
--- a/ts/packages/cli/README.md
+++ b/ts/packages/cli/README.md
@@ -129,7 +129,7 @@ The CLI implements a file-based caching system for improved performance and offl
### Cache Features
- **Cache-first reads**: When `FORCE_USE_CACHE=true`, the CLI first checks for cached data before making API calls. If you already ran `composio generate` before, it will work even if you're offline.
-- **Best-effort writes**: All successful API responses are automatically cached to disk for future use.
+- **Best-effort writes**: All successful API responses are automatically cached to disk for future use. Writes are atomic — a run interrupted mid-write leaves the previous cache intact rather than a truncated file.
- **Graceful fallback**: If cache files are corrupted or missing, the CLI falls back to making API calls.
- **Parameter-aware caching**: Methods with parameters include those parameters in the cache key.
@@ -148,6 +148,14 @@ The following files are cached:
- `trigger-types-as-enums.json` - Results from trigger type enumerations
- `trigger-types.json` - Results from paginated trigger types payloads
+### Known toolkit slugs
+
+`known-toolkit-slugs.json` also lives in the cache directory, but it is not one of the files above.
+
+Running a tool means knowing which toolkit it belongs to, and the tool slug alone does not say: `GOOGLE_ANALYTICS_RUN_REPORT` belongs to `google_analytics`, not `google`. The CLI ships with the list of toolkit slugs that existed when it was built and records any it learns afterwards in this file, so resolving a toolkit costs a small local read instead of downloading the catalog.
+
+It holds derived data, not saved API responses, so it is read on every run regardless of `FORCE_USE_CACHE` — that variable keeps its meaning of opting in to replaying previously cached API responses. Deleting the file is safe: the CLI re-learns what it needs. Being out of date is also safe, because the backend never removes a toolkit; a slug the file has never seen simply costs one catalog fetch, after which it is remembered and refreshed weekly in the background.
+
## Development
### Installation
diff --git a/ts/packages/cli/src/cli-main.ts b/ts/packages/cli/src/cli-main.ts
index a0ac2f383..2b655af48 100644
--- a/ts/packages/cli/src/cli-main.ts
+++ b/ts/packages/cli/src/cli-main.ts
@@ -26,6 +26,7 @@ import { UpgradeBinary } from 'src/services/upgrade-binary';
import { TerminalUI, TerminalUILive } from 'src/services/terminal-ui';
import { TriggersRealtime } from 'src/services/triggers-realtime';
import { ToolsExecutorLive as _ToolsExecutorLive } from 'src/services/tools-executor';
+import { ToolkitSlugCatalog } from 'src/services/toolkit-slug-catalog';
import { ProjectContext } from 'src/services/project-context';
import { ProjectEnvironmentDetector } from 'src/services/project-environment-detector';
import { CommandRunner } from 'src/services/command-runner';
@@ -94,9 +95,16 @@ export const ComposioClientSingletonLive = Layer.provide(
Layer.mergeAll(BunFileSystem.layer, BunPath.layer, NodeOs.Default, ConfigLive)
) satisfies RequiredLayer;
+// Fed the cached repository so that the staleness refresh behind it shares the
+// one catalog fetch a run is allowed, rather than starting a second.
+export const ToolkitSlugCatalogLive = Layer.provide(
+ ToolkitSlugCatalog.Default,
+ ComposioToolkitsRepositoryCachedLive
+) satisfies RequiredLayer;
+
export const ToolsExecutorLive = Layer.provide(
_ToolsExecutorLive,
- ComposioClientSingletonLive
+ Layer.mergeAll(ComposioClientSingletonLive, ToolkitSlugCatalogLive)
) satisfies RequiredLayer;
export const ProjectContextLive = Layer.provide(
@@ -119,6 +127,7 @@ const layers = Layer.mergeAll(
ComposioSessionRepositoryLive,
ComposioClientSingletonLive,
ComposioToolkitsRepositoryCachedLive,
+ ToolkitSlugCatalogLive,
ToolsExecutorLive,
JsPackageManagerDetector.Default,
ProjectEnvironmentDetector.Default,
diff --git a/ts/packages/cli/src/effects/toolkit-from-tool-slug.ts b/ts/packages/cli/src/effects/toolkit-from-tool-slug.ts
index 04da7e4ae..35fedd945 100644
--- a/ts/packages/cli/src/effects/toolkit-from-tool-slug.ts
+++ b/ts/packages/cli/src/effects/toolkit-from-tool-slug.ts
@@ -1,23 +1,31 @@
import { Effect } from 'effect';
import { ComposioToolkitsRepository } from 'src/services/composio-clients';
+import { ToolkitSlugCatalog } from 'src/services/toolkit-slug-catalog';
import { isMetaToolSlug } from 'src/utils/meta-tool-slugs';
import {
guessToolkitFromToolSlug,
matchToolkitFromToolSlug,
+ toolkitFromMatchedPrefix,
} from 'src/utils/toolkit-from-tool-slug';
/**
- * Resolves the toolkit slug for a tool or trigger slug by matching the longest
- * known toolkit slug prefix from the toolkit list (served from the local cache
- * when warm). No string split can find where the toolkit name ends —
+ * Resolves the toolkit slug for a tool or trigger slug.
+ *
+ * No string split can find where the toolkit name ends —
* `GOOGLE_ANALYTICS_RUN_REPORT` belongs to `google_analytics`, not `google` —
- * so the known list is the only reliable source. Falls back to the
- * first-underscore guess when the list cannot be loaded (first run offline,
- * empty cache).
+ * so this matches the longest known toolkit slug prefix. "Known" is
+ * {@link ToolkitSlugCatalog}: the catalog baked in at build time plus whatever
+ * this machine has learned since, both local and memoized per run, so the
+ * common case costs no network at all.
+ *
+ * Only a slug that matches nothing known falls through to the catalog, which
+ * is what a toolkit released after this binary looks like. Everything else
+ * degrades rather than fails: an unreachable catalog still yields the
+ * first-underscore guess.
*/
export const toolkitFromToolSlug = (
toolSlug: string
-): Effect.Effect<string | undefined, never, ComposioToolkitsRepository> =>
+): Effect.Effect<string | undefined, never, ComposioToolkitsRepository | ToolkitSlugCatalog> =>
Effect.gen(function* () {
// Meta tools belong to the session rather than to a toolkit, and their
// slugs shadow real ones — `COMPOSIO_SEARCH_TOOLS` prefix-matches the
@@ -27,10 +35,19 @@ export const toolkitFromToolSlug = (
return undefined;
}
+ const catalog = yield* ToolkitSlugCatalog;
+ const local = yield* catalog.local;
+
+ const match = local.match(toolSlug);
+ if (match !== undefined) {
+ return toolkitFromMatchedPrefix(match);
+ }
+
const repository = yield* ComposioToolkitsRepository;
const toolkits = yield* repository.getToolkits();
- return matchToolkitFromToolSlug(
- toolSlug,
- toolkits.map(toolkit => toolkit.slug)
- );
+ const allSlugs = [...local.slugs, ...toolkits.map(toolkit => toolkit.slug)];
+
+ yield* catalog.remember(allSlugs);
+
+ return matchToolkitFromToolSlug(toolSlug, allSlugs);
}).pipe(Effect.catchAll(() => Effect.succeed(guessToolkitFromToolSlug(toolSlug))));
diff --git a/ts/packages/cli/src/effects/write-file-atomic.ts b/ts/packages/cli/src/effects/write-file-atomic.ts
new file mode 100644
index 000000000..7f44cbb09
--- /dev/null
+++ b/ts/packages/cli/src/effects/write-file-atomic.ts
@@ -0,0 +1,26 @@
+import { FileSystem } from '@effect/platform';
+import { Effect, Random } from 'effect';
+
+/**
+ * Writes `content` to a sibling temp file and renames it over `filePath`.
+ *
+ * Renaming within a filesystem is atomic, so a reader — another CLI process,
+ * or the next run after this one is killed mid-write — sees either the old
+ * file or the new one, never a truncated mix. The temp file is a sibling
+ * rather than a tmpdir entry so both paths stay on the same volume, where
+ * rename cannot degrade into a copy.
+ */
+export const writeFileAtomic = (filePath: string, content: string) =>
+ Effect.gen(function* () {
+ const fs = yield* FileSystem.FileSystem;
+
+ // Concurrent writers must not share a temp path, or one would rename the
+ // other's half-written file into place.
+ const suffix = yield* Random.nextIntBetween(0x100000, 0xffffff);
+ const tempPath = `${filePath}.tmp-${suffix.toString(16)}`;
+
+ yield* fs.writeFileString(tempPath, content);
+ yield* fs
+ .rename(tempPath, filePath)
+ .pipe(Effect.onError(() => fs.remove(tempPath).pipe(Effect.ignore)));
+ });
diff --git a/ts/packages/cli/src/models/tools.ts b/ts/packages/cli/src/models/tools.ts
index 6af94382f..031f4ff81 100644
--- a/ts/packages/cli/src/models/tools.ts
+++ b/ts/packages/cli/src/models/tools.ts
@@ -58,16 +58,6 @@ export const Tool = Schema.Struct({
* List of tags associated with the tool for categorization and filtering.
*/
tags: Schema.Array(Schema.String),
- /**
- * Parent toolkit of the tool. Absent in cache entries written before the
- * field was captured.
- */
- toolkit: Schema.optional(
- Schema.Struct({
- name: Schema.String,
- slug: Schema.String,
- })
- ),
}).annotations({ identifier: 'Tool' });
export type Tool = Schema.Schema.Type<typeof Tool>;
diff --git a/ts/packages/cli/src/services/composio-clients-cached.ts b/ts/packages/cli/src/services/composio-clients-cached.ts
index 25bbc6b2f..d591fdca1 100644
--- a/ts/packages/cli/src/services/composio-clients-cached.ts
+++ b/ts/packages/cli/src/services/composio-clients-cached.ts
@@ -3,6 +3,7 @@ import { FileSystem, Path } from '@effect/platform';
import { BunFileSystem } from '@effect/platform-bun';
import { setupCacheDir } from 'src/effects/setup-cache-dir';
import { FORCE_CONFIG } from 'src/effects/force-config';
+import { writeFileAtomic } from 'src/effects/write-file-atomic';
import { ComposioToolkitsRepository, InvalidToolkitsError } from './composio-clients';
import type { ToolkitVersionSpec } from 'src/effects/toolkit-version-overrides';
import { NodeOs } from './node-os';
@@ -90,9 +91,14 @@ function createCachedEffect<T, E, R>(
)
);
- const writeToCache = (fs: FileSystem.FileSystem, cacheFilePath: string, result: T) =>
+ /**
+ * Atomic: these files are hundreds of KB, and a run killed mid-write — or
+ * two CLI processes writing at once — would otherwise leave behind a
+ * truncated file that the next run reads back as a parse failure.
+ */
+ const writeToCache = (cacheFilePath: string, result: T) =>
encoder(result).pipe(
- Effect.flatMap(content => fs.writeFileString(cacheFilePath, content)),
+ Effect.flatMap(content => writeFileAtomic(cacheFilePath, content)),
Effect.catchAll(error =>
Effect.logWarning(`Failed to write to cache ${cacheFilePath}: ${error}`)
)
@@ -130,7 +136,7 @@ function createCachedEffect<T, E, R>(
// Write to cache only if we're fetching the full dataset (no cacheFilter).
// Filtered API calls fetch partial data that would corrupt the shared cache file.
if (!cacheFilter && Option.isSome(cacheFilePath)) {
- yield* writeToCache(fs, cacheFilePath.value, result);
+ yield* writeToCache(cacheFilePath.value, result);
}
return result;
diff --git a/ts/packages/cli/src/services/known-toolkit-slugs.ts b/ts/packages/cli/src/services/known-toolkit-slugs.ts
new file mode 100644
index 000000000..3db5ee11e
--- /dev/null
+++ b/ts/packages/cli/src/services/known-toolkit-slugs.ts
@@ -0,0 +1,96 @@
+import { FileSystem, Path } from '@effect/platform';
+import { BunFileSystem } from '@effect/platform-bun';
+import { DateTime, Effect, Layer, Option, Schema } from 'effect';
+import { setupCacheDir } from 'src/effects/setup-cache-dir';
+import { writeFileAtomic } from 'src/effects/write-file-atomic';
+import { JSONTransformSchema } from 'src/models/utils/json-transform-schema';
+import { NodeOs } from './node-os';
+
+/**
+ * Toolkit slugs this machine has learned since the CLI was built.
+ *
+ * The CLI ships with the catalog it knew at build time
+ * (`src/generated/toolkit-slugs.ts`). Toolkits released after that are
+ * discovered the first time someone runs one of their tools, and remembered
+ * here so the next run costs nothing.
+ *
+ * This is derived data, not an HTTP response cache: it is read unconditionally
+ * rather than under `FORCE_USE_CACHE`, which stays what it has always been —
+ * an opt-in to replaying previously cached API responses. Being wrong about
+ * this file can only cost a fetch, never an answer: an unknown slug falls
+ * through to the catalog, and the backend never removes a toolkit, so a slug
+ * recorded here never stops being real.
+ */
+
+export const KNOWN_TOOLKIT_SLUGS_FILE = 'known-toolkit-slugs.json';
+
+const KnownToolkitSlugs = Schema.Struct({
+ slugs: Schema.Array(Schema.String),
+ refreshedAt: Schema.DateTimeUtc,
+});
+
+export type KnownToolkitSlugs = Schema.Schema.Type<typeof KnownToolkitSlugs>;
+
+const KnownToolkitSlugsJSON = JSONTransformSchema(KnownToolkitSlugs);
+
+/**
+ * The file lives in the cache directory but is not part of the toolkit cache,
+ * so it self-provides its platform services: callers stay free of filesystem
+ * dependencies in their own environments.
+ */
+const provideFileSystem = <A, E>(
+ effect: Effect.Effect<A, E, FileSystem.FileSystem | Path.Path | NodeOs>
+): Effect.Effect<A, E> =>
+ effect.pipe(Effect.provide(Layer.mergeAll(BunFileSystem.layer, Path.layer, NodeOs.Default)));
+
+const knownToolkitSlugsPath = Effect.gen(function* () {
+ const path = yield* Path.Path;
+ const cacheDir = yield* setupCacheDir;
+ return path.join(cacheDir, KNOWN_TOOLKIT_SLUGS_FILE);
+});
+
+/**
+ * Reads the learned slugs. A missing, unreadable, or corrupted file is not a
+ * failure — it means "nothing learned yet", and the next refresh rewrites it.
+ */
+export const readKnownToolkitSlugs: Effect.Effect<Option.Option<KnownToolkitSlugs>> = Effect.gen(
+ function* () {
+ const fs = yield* FileSystem.FileSystem;
+ const filePath = yield* knownToolkitSlugsPath;
+ const content = yield* fs.readFileString(filePath);
+ return yield* Schema.decode(KnownToolkitSlugsJSON)(content);
+ }
+).pipe(
+ Effect.asSome,
+ Effect.catchAll(error =>
+ Effect.logDebug(`No usable ${KNOWN_TOOLKIT_SLUGS_FILE}: ${error}`).pipe(
+ Effect.as(Option.none<KnownToolkitSlugs>())
+ )
+ ),
+ provideFileSystem
+);
+
+/**
+ * Records `slugs`, sorted and deduped, stamped with the current time.
+ *
+ * Never fails: this is an optimization, and a machine that cannot write to its
+ * own cache directory should still be able to run tools. Concurrent CLI
+ * processes are safe by construction — the write is atomic, and since slugs
+ * are only ever added, whichever writer lands last leaves a usable file.
+ */
+export const writeKnownToolkitSlugs = (slugs: ReadonlyArray<string>): Effect.Effect<void> =>
+ Effect.gen(function* () {
+ const filePath = yield* knownToolkitSlugsPath;
+ const refreshedAt = yield* DateTime.now;
+
+ const content = yield* Schema.encode(KnownToolkitSlugsJSON)({
+ slugs: [...new Set(slugs.map(slug => slug.toLowerCase()))].sort(),
+ refreshedAt,
+ });
+
+ yield* writeFileAtomic(filePath, content);
+ yield* Effect.logDebug(`Recorded ${slugs.length} toolkit slugs in ${filePath}`);
+ }).pipe(
+ Effect.catchAll(error => Effect.logDebug(`Failed to record known toolkit slugs: ${error}`)),
+ provideFileSystem
+ );
diff --git a/ts/packages/cli/src/services/toolkit-slug-catalog.ts b/ts/packages/cli/src/services/toolkit-slug-catalog.ts
new file mode 100644
index 000000000..830c057af
--- /dev/null
+++ b/ts/packages/cli/src/services/toolkit-slug-catalog.ts
@@ -0,0 +1,127 @@
+import { DateTime, Duration, Effect, Option, Ref } from 'effect';
+import { BAKED_TOOLKIT_SLUGS } from 'src/generated/toolkit-slugs';
+import { ComposioToolkitsRepository } from 'src/services/composio-clients';
+import {
+ readKnownToolkitSlugs,
+ writeKnownToolkitSlugs,
+ type KnownToolkitSlugs,
+} from 'src/services/known-toolkit-slugs';
+import { makeLongestPrefixMatcher } from 'src/utils/toolkit-from-tool-slug';
+
+/**
+ * How long learned slugs are trusted before a background refresh is started.
+ * Only relevant for toolkits released after the last refresh: a slug already
+ * known is never wrong, so this bounds how long a *new* toolkit can be
+ * mistaken for a shorter one that shares its prefix (a hypothetical
+ * `google_analytics_v2` resolving as `google_analytics`).
+ */
+const REFRESH_AFTER = Duration.days(7);
+
+/**
+ * A successful run ends by event-loop drain rather than an explicit exit, so a
+ * daemon fiber mid-fetch holds the finished command open until the network
+ * answers. This bounds that lingering; an interrupted refresh costs a fetch on
+ * a later run, nothing more.
+ */
+const REFRESH_TIMEOUT = Duration.seconds(10);
+
+const learnedSlugs = (learned: Option.Option<KnownToolkitSlugs>): ReadonlyArray<string> =>
+ Option.match(learned, {
+ onNone: () => [],
+ onSome: known => known.slugs,
+ });
+
+/**
+ * Re-reads the catalog and records it, in the background. Failures are
+ * swallowed: a refresh that does not happen costs a fetch later, nothing more.
+ */
+const refreshKnownToolkitSlugs = Effect.gen(function* () {
+ const repository = yield* ComposioToolkitsRepository;
+ const toolkits = yield* repository.getToolkits();
+ yield* writeKnownToolkitSlugs([...BAKED_TOOLKIT_SLUGS, ...toolkits.map(t => t.slug)]);
+}).pipe(Effect.timeout(REFRESH_TIMEOUT), Effect.ignore);
+
+/**
+ * Starts a refresh when the learned slugs are missing or older than
+ * {@link REFRESH_AFTER}. A `refreshedAt` in the future (a clock that jumped)
+ * counts as fresh — the point is to bound staleness, not to police clocks.
+ */
+const refreshInBackgroundIfStale = (learned: Option.Option<KnownToolkitSlugs>) =>
+ Effect.gen(function* () {
+ const now = yield* DateTime.now;
+ const isFresh = Option.match(learned, {
+ onNone: () => false,
+ onSome: known =>
+ Duration.lessThan(
+ Duration.millis(DateTime.distance(known.refreshedAt, now)),
+ REFRESH_AFTER
+ ),
+ });
+
+ if (isFresh) return;
+
+ yield* Effect.forkDaemon(refreshKnownToolkitSlugs);
+ });
+
+/**
+ * The toolkit slugs a run can match against without asking the API: the
+ * catalog baked in at build time plus whatever this machine has learned since.
+ */
+export interface LocalToolkitSlugs {
+ readonly slugs: ReadonlyArray<string>;
+ /**
+ * Longest-prefix match over {@link slugs}. Returns the matched toolkit slug,
+ * or undefined when nothing local matches — which is what a toolkit released
+ * after this binary looks like.
+ */
+ readonly match: (toolSlug: string) => string | undefined;
+}
+
+const loadLocalToolkitSlugs = Effect.gen(function* () {
+ const learned = yield* readKnownToolkitSlugs;
+ const slugs = [...BAKED_TOOLKIT_SLUGS, ...learnedSlugs(learned)];
+
+ yield* refreshInBackgroundIfStale(learned);
+
+ return { slugs, match: makeLongestPrefixMatcher(slugs) } satisfies LocalToolkitSlugs;
+});
+
+/**
+ * Owns the local half of toolkit resolution, resolved once per process.
+ *
+ * Reading it costs a file read, a JSON parse, a schema decode, and a lookup set
+ * over a thousand-odd slugs — cheap next to the catalog fetch it replaces, but
+ * a single `composio execute` resolves a toolkit four times over (bootstrap
+ * telemetry, the execute context, the connection check, the execution itself),
+ * and `--parallel` multiplies that by the number of tools. None of it can
+ * change under a running process, so `Effect.cached` memoizes it for the
+ * lifetime of the layer: the first caller pays, the rest await the same result.
+ *
+ * Memoizing also gates the staleness refresh to one fork per run, where an
+ * unmemoized read forked — and rewrote the file — once per resolution.
+ */
+export class ToolkitSlugCatalog extends Effect.Service<ToolkitSlugCatalog>()(
+ 'services/ToolkitSlugCatalog',
+ {
+ effect: Effect.gen(function* () {
+ const local = yield* Effect.cached(loadLocalToolkitSlugs);
+ const hasRecorded = yield* Ref.make(false);
+
+ /**
+ * Records slugs learned from a catalog fetch, in the background, at most
+ * once per run. Every miss in a run merges the same memoized fetch into
+ * the same local list, so later calls would fork a daemon fiber only to
+ * rewrite an identical file — and each pending fiber holds the finished
+ * command open a little longer.
+ */
+ const remember = (slugs: ReadonlyArray<string>): Effect.Effect<void> =>
+ Effect.gen(function* () {
+ const alreadyRecorded = yield* Ref.getAndSet(hasRecorded, true);
+ if (alreadyRecorded) return;
+ yield* Effect.forkDaemon(writeKnownToolkitSlugs(slugs));
+ });
+
+ return { local, remember };
+ }),
+ }
+) {}
diff --git a/ts/packages/cli/src/services/tools-executor.ts b/ts/packages/cli/src/services/tools-executor.ts
index 2e1161524..0a04a7c8e 100644
--- a/ts/packages/cli/src/services/tools-executor.ts
+++ b/ts/packages/cli/src/services/tools-executor.ts
@@ -16,6 +16,7 @@ import {
import { getOrFetchToolInputDefinition } from 'src/services/tool-input-validation';
import { ToolFileUploadError, uploadToolInputFiles } from 'src/services/tool-file-uploads';
import { toolkitFromToolSlug } from 'src/effects/toolkit-from-tool-slug';
+import { ToolkitSlugCatalog } from 'src/services/toolkit-slug-catalog';
import { isMetaToolSlug } from 'src/utils/meta-tool-slugs';
import type { NodeOs } from 'src/services/node-os';
import type { NodeProcess } from 'src/services/node-process';
@@ -131,6 +132,11 @@ export const ToolsExecutorLive = Layer.effect(
// the raw Composio client on first call — no environment requirements.
const clientSingleton = yield* ComposioClientSingleton;
+ // Held rather than required per call: which slugs resolve locally is an
+ // implementation detail of toolkit resolution, not part of what a caller
+ // has to hand the executor.
+ const slugCatalog = yield* ToolkitSlugCatalog;
+
return ToolsExecutor.of({
execute: (slug, params) =>
Effect.gen(function* () {
@@ -252,7 +258,8 @@ export const ToolsExecutorLive = Layer.effect(
);
})
)
- )
+ ),
+ Effect.provideService(ToolkitSlugCatalog, slugCatalog)
),
});
})
diff --git a/ts/packages/cli/src/utils/toolkit-from-tool-slug.ts b/ts/packages/cli/src/utils/toolkit-from-tool-slug.ts
index b5126f88c..f8c1eab07 100644
--- a/ts/packages/cli/src/utils/toolkit-from-tool-slug.ts
+++ b/ts/packages/cli/src/utils/toolkit-from-tool-slug.ts
@@ -24,16 +24,38 @@ export const guessToolkitFromToolSlug = (toolSlug: string): string | undefined =
export const longestPrefix = (
toolSlug: string,
knownToolkitSlugs: ReadonlyArray<string>
-): string | undefined => {
+): string | undefined => makeLongestPrefixMatcher(knownToolkitSlugs)(toolSlug);
+
+/**
+ * {@link longestPrefix} against a list that does not change, with the lookup
+ * set built once and closed over. The known list runs to a thousand-odd slugs
+ * and a single command resolves several tool slugs against it, so callers that
+ * hold onto the returned function pay for the set once instead of per lookup.
+ */
+export const makeLongestPrefixMatcher = (
+ knownToolkitSlugs: ReadonlyArray<string>
+): ((toolSlug: string) => string | undefined) => {
const known = new Set(knownToolkitSlugs.map(slug => slug.toLowerCase()));
- const segments = toolSlug.toLowerCase().split('_');
- for (let end = segments.length - 1; end >= 1; end -= 1) {
- const candidate = segments.slice(0, end).join('_');
- if (known.has(candidate)) return candidate;
- }
- return undefined;
+
+ return toolSlug => {
+ const segments = toolSlug.toLowerCase().split('_');
+ for (let end = segments.length - 1; end >= 1; end -= 1) {
+ const candidate = segments.slice(0, end).join('_');
+ if (known.has(candidate)) return candidate;
+ }
+ return undefined;
+ };
};
+/**
+ * Bare `composio`-prefixed slugs are meta tools, not user-linkable toolkits;
+ * longer matches such as `composio_search` are real toolkits. A `composio`
+ * match is a resolved answer rather than a miss, so callers must not fall
+ * through to a wider search on the `undefined` it returns.
+ */
+export const toolkitFromMatchedPrefix = (match: string): string | undefined =>
+ match === 'composio' ? undefined : match;
+
/**
* `longestPrefix` with the toolkit-resolution policy applied on top: the
* `composio` meta guard, and `guessToolkitFromToolSlug` when no candidate is
@@ -45,9 +67,7 @@ export const matchToolkitFromToolSlug = (
): string | undefined => {
const match = longestPrefix(toolSlug, knownToolkitSlugs);
if (match !== undefined) {
- // Bare `composio`-prefixed slugs are meta tools, not user-linkable
- // toolkits; longer matches such as `composio_search` are real toolkits.
- return match === 'composio' ? undefined : match;
+ return toolkitFromMatchedPrefix(match);
}
return guessToolkitFromToolSlug(toolSlug);
};
diff --git a/ts/packages/cli/test/__utils__/services/test-layer.ts b/ts/packages/cli/test/__utils__/services/test-layer.ts
index 03db1ee04..58190ecb9 100644
--- a/ts/packages/cli/test/__utils__/services/test-layer.ts
+++ b/ts/packages/cli/test/__utils__/services/test-layer.ts
@@ -47,6 +47,7 @@ import { UpgradeBinary } from 'src/services/upgrade-binary';
import { NodeOs } from 'src/services/node-os';
import { TriggersRealtime } from 'src/services/triggers-realtime';
import { ToolsExecutor, ToolsExecutorLive } from 'src/services/tools-executor';
+import { ToolkitSlugCatalog } from 'src/services/toolkit-slug-catalog';
import type { ToolExecuteResponse } from 'src/services/tools-executor';
import type {
SessionCreateResponse,
@@ -1214,6 +1215,13 @@ export const TestLayer = (input?: TestLiveInput) =>
})
);
+ // Built per test layer, so each test resolves toolkit slugs against its own
+ // cache directory rather than inheriting a memo from an earlier test.
+ const ToolkitSlugCatalogTest = Layer.provide(
+ ToolkitSlugCatalog.Default,
+ ComposioToolkitsRepositoryTest
+ );
+
// --- ToolsExecutor ---
// When `input.toolsExecutor` is set, use a canned mock (bypasses Tool Router).
// Otherwise, use the real ToolsExecutorLive which flows through the mock ComposioClientSingleton.
@@ -1237,7 +1245,10 @@ export const TestLayer = (input?: TestLiveInput) =>
},
})
)
- : Layer.provide(ToolsExecutorLive, ComposioClientSingletonTest);
+ : Layer.provide(
+ ToolsExecutorLive,
+ Layer.mergeAll(ComposioClientSingletonTest, ToolkitSlugCatalogTest)
+ );
const CliConfigLive = CliConfig.layer(ComposioCliConfig);
@@ -1286,6 +1297,7 @@ export const TestLayer = (input?: TestLiveInput) =>
ComposioSessionRepositoryTest,
TriggersRealtimeTest,
ComposioToolkitsRepositoryTest,
+ ToolkitSlugCatalogTest,
JsPackageManagerDetector.Default,
ProjectEnvironmentDetector.Default,
CommandRunnerTest,
diff --git a/ts/packages/cli/test/src/effects/toolkit-from-tool-slug.test.ts b/ts/packages/cli/test/src/effects/toolkit-from-tool-slug.test.ts
index a09966844..54e7817d2 100644
--- a/ts/packages/cli/test/src/effects/toolkit-from-tool-slug.test.ts
+++ b/ts/packages/cli/test/src/effects/toolkit-from-tool-slug.test.ts
@@ -1,63 +1,301 @@
-import { describe, expect, layer } from '@effect/vitest';
-import { Effect } from 'effect';
+import { describe, expect, it } from '@effect/vitest';
+import { FileSystem } from '@effect/platform';
+import { BunFileSystem } from '@effect/platform-bun';
+import { ConfigProvider, DateTime, Effect, Layer, Schedule } from 'effect';
+import * as tempy from 'tempy';
import { toolkitFromToolSlug } from 'src/effects/toolkit-from-tool-slug';
import type { Toolkits } from 'src/models/toolkits';
-import { TestLive } from 'test/__utils__';
+import { ComposioToolkitsRepository, HttpServerError } from 'src/services/composio-clients';
+import { KNOWN_TOOLKIT_SLUGS_FILE } from 'src/services/known-toolkit-slugs';
+import { ToolkitSlugCatalog } from 'src/services/toolkit-slug-catalog';
import { makeToolkitFixture } from 'test/__utils__/models/toolkits';
+import {
+ countingToolkitsRepository,
+ type GetToolkitsError,
+} from 'test/__utils__/services/toolkits-repository-stub';
-const testToolkits: Toolkits = [
- makeToolkitFixture('google', 'Google'),
- makeToolkitFixture('google_analytics', 'Google Analytics'),
- makeToolkitFixture('gmail', 'Gmail'),
- makeToolkitFixture('microsoft', 'Microsoft'),
- makeToolkitFixture('microsoft_teams', 'Microsoft Teams'),
- // A real, linkable toolkit whose slug is also the prefix of a session meta
- // tool (`COMPOSIO_SEARCH_TOOLS`).
- makeToolkitFixture('composio_search', 'Composio Search'),
-];
+/**
+ * A toolkit that cannot be in the baked catalog, standing in for one released
+ * after the binary was built. Tests of the fetch path must use slugs like this
+ * — anything real resolves locally and never reaches the repository.
+ */
+const UNRELEASED_TOOLKIT = 'acme_analytics';
-describe('toolkitFromToolSlug', () => {
- layer(TestLive({ toolkitsData: { toolkits: testToolkits } }))('with known toolkits', it => {
- it.effect('resolves multi-word toolkit slugs by longest known prefix', () =>
- Effect.gen(function* () {
- expect(yield* toolkitFromToolSlug('GOOGLE_ANALYTICS_RUN_REPORT')).toBe('google_analytics');
- expect(yield* toolkitFromToolSlug('MICROSOFT_TEAMS_SEND_MESSAGE')).toBe('microsoft_teams');
- })
+const failingFetch = () =>
+ Effect.fail(new HttpServerError({ cause: 'catalog unavailable', status: 503 }));
+
+/**
+ * A learned-slugs file recorded `daysAgo` days ago. Anything under a week old
+ * keeps the resolver from starting a background refresh, which is what makes
+ * "no catalog fetch" assertions meaningful.
+ */
+const learnedFileContent = (slugs: ReadonlyArray<string>, daysAgo = 0) =>
+ JSON.stringify({
+ slugs,
+ refreshedAt: DateTime.formatIso(DateTime.subtract(DateTime.unsafeNow(), { days: daysAgo })),
+ });
+
+interface ResolverContext {
+ readonly calls: () => number;
+ readonly waitForLearnedFile: (
+ predicate: (content: string) => boolean
+ ) => Effect.Effect<string, unknown, FileSystem.FileSystem>;
+}
+
+/**
+ * Runs `program` against a private cache directory, so the resolver reads and
+ * writes a learned-slugs file belonging to this test alone. `seedLearnedFile`
+ * is written verbatim — including deliberately corrupt content.
+ */
+const withResolver = <A>(
+ options: {
+ readonly getToolkits?: () => Effect.Effect<Toolkits, GetToolkitsError>;
+ readonly seedLearnedFile?: string;
+ },
+ program: (
+ context: ResolverContext
+ ) => Effect.Effect<
+ A,
+ unknown,
+ FileSystem.FileSystem | ComposioToolkitsRepository | ToolkitSlugCatalog
+ >
+) =>
+ Effect.suspend(() => {
+ const cacheDir = tempy.temporaryDirectory();
+ return runInCacheDir(cacheDir, options, program);
+ });
+
+const runInCacheDir = <A>(
+ cacheDir: string,
+ options: {
+ readonly getToolkits?: () => Effect.Effect<Toolkits, GetToolkitsError>;
+ readonly seedLearnedFile?: string;
+ },
+ program: (
+ context: ResolverContext
+ ) => Effect.Effect<
+ A,
+ unknown,
+ FileSystem.FileSystem | ComposioToolkitsRepository | ToolkitSlugCatalog
+ >
+) =>
+ Effect.gen(function* () {
+ const fs = yield* FileSystem.FileSystem;
+ const learnedFile = `${cacheDir}/${KNOWN_TOOLKIT_SLUGS_FILE}`;
+
+ if (options.seedLearnedFile !== undefined) {
+ yield* fs.writeFileString(learnedFile, options.seedLearnedFile);
+ }
+
+ const repository = countingToolkitsRepository(
+ options.getToolkits ?? (() => Effect.succeed([] as Toolkits))
);
- it.effect('resolves single-word toolkit slugs', () =>
- Effect.gen(function* () {
- expect(yield* toolkitFromToolSlug('GMAIL_SEND_EMAIL')).toBe('gmail');
- })
+ const readLearnedFile = fs.readFileString(learnedFile);
+
+ return yield* program({
+ calls: repository.calls,
+ // The resolver records what it learns in the background, so a test that
+ // asserts on the file has to wait for it rather than read once.
+ waitForLearnedFile: predicate =>
+ readLearnedFile.pipe(
+ Effect.filterOrFail(predicate, () => 'learned file has not caught up yet' as const),
+ Effect.retry(Schedule.spaced('10 millis').pipe(Schedule.intersect(Schedule.recurs(200))))
+ ),
+ }).pipe(
+ // Built inside this test's cache directory, and fresh per test: the
+ // catalog memoizes what it reads, so a shared one would leak the first
+ // test's learned slugs into every later one.
+ Effect.provide(
+ Layer.mergeAll(
+ repository.layer,
+ Layer.provide(ToolkitSlugCatalog.Default, repository.layer)
+ )
+ )
);
+ }).pipe(
+ Effect.provide(BunFileSystem.layer),
+ Effect.withConfigProvider(ConfigProvider.fromMap(new Map([['CACHE_DIR', cacheDir]])))
+ );
- it.effect('falls back to the guess for slugs outside the known list', () =>
+describe('toolkitFromToolSlug', () => {
+ it.effect('resolves a baked multi-word toolkit without touching the network', () =>
+ withResolver({ seedLearnedFile: learnedFileContent([]) }, ({ calls }) =>
Effect.gen(function* () {
- expect(yield* toolkitFromToolSlug('NOTION_CREATE_PAGE')).toBe('notion');
+ expect(yield* toolkitFromToolSlug('GOOGLE_ANALYTICS_RUN_REPORT')).toBe('google_analytics');
+ expect(yield* toolkitFromToolSlug('GMAIL_SEND_EMAIL')).toBe('gmail');
+ expect(calls()).toBe(0);
})
- );
+ )
+ );
- it.effect('gives meta tools no toolkit, even when one shadows their slug', () =>
+ it.effect('gives meta tools no toolkit, even when one shadows their slug', () =>
+ withResolver({ seedLearnedFile: learnedFileContent([]) }, ({ calls }) =>
Effect.gen(function* () {
- // Attributing this to `composio_search` would make a failed meta call
- // tell the user to link an app that has nothing to do with it.
+ // `composio_search` is a real, linkable toolkit whose slug is a prefix
+ // of this session meta tool. Attributing the meta tool to it would
+ // send users to link an app they do not need.
expect(yield* toolkitFromToolSlug('COMPOSIO_SEARCH_TOOLS')).toBeUndefined();
expect(yield* toolkitFromToolSlug('COMPOSIO_MANAGE_CONNECTIONS')).toBeUndefined();
+ expect(calls()).toBe(0);
})
- );
+ )
+ );
- it.effect('still resolves ordinary tools of the shadowed toolkit', () =>
+ it.effect('treats a bare `composio` match as resolved, not as a miss', () =>
+ withResolver({ seedLearnedFile: learnedFileContent([]) }, ({ calls }) =>
Effect.gen(function* () {
- expect(yield* toolkitFromToolSlug('COMPOSIO_SEARCH_SEARCH')).toBe('composio_search');
+ expect(yield* toolkitFromToolSlug('COMPOSIO_ENABLE_TRIGGER')).toBeUndefined();
+ // A miss here would cost every `COMPOSIO_*` tool a catalog fetch.
+ expect(calls()).toBe(0);
})
- );
- });
+ )
+ );
+
+ it.live('answers from the baked list on a first run, and seeds the file behind it', () =>
+ withResolver(
+ { getToolkits: () => Effect.succeed([makeToolkitFixture(UNRELEASED_TOOLKIT)]) },
+ ({ waitForLearnedFile }) =>
+ Effect.gen(function* () {
+ expect(yield* toolkitFromToolSlug('GOOGLE_ANALYTICS_RUN_REPORT')).toBe(
+ 'google_analytics'
+ );
+
+ const learned = yield* waitForLearnedFile(content =>
+ content.includes(UNRELEASED_TOOLKIT)
+ );
+ expect(learned).toContain('google_analytics');
+ })
+ )
+ );
+
+ it.live('resolves a learned toolkit without touching the network', () =>
+ withResolver(
+ {
+ seedLearnedFile: learnedFileContent([UNRELEASED_TOOLKIT]),
+ },
+ ({ calls }) =>
+ Effect.gen(function* () {
+ expect(yield* toolkitFromToolSlug('ACME_ANALYTICS_RUN_REPORT')).toBe(UNRELEASED_TOOLKIT);
+ expect(calls()).toBe(0);
+ })
+ )
+ );
- layer(TestLive())('with an empty toolkit list', it => {
- it.effect('falls back to the first-underscore guess', () =>
+ it.live('fetches the catalog for an unknown slug, and remembers what it finds', () =>
+ withResolver(
+ {
+ getToolkits: () => Effect.succeed([makeToolkitFixture(UNRELEASED_TOOLKIT)]),
+ seedLearnedFile: learnedFileContent([]),
+ },
+ ({ calls, waitForLearnedFile }) =>
+ Effect.gen(function* () {
+ expect(yield* toolkitFromToolSlug('ACME_ANALYTICS_RUN_REPORT')).toBe(UNRELEASED_TOOLKIT);
+ expect(calls()).toBe(1);
+
+ const learned = yield* waitForLearnedFile(content =>
+ content.includes(UNRELEASED_TOOLKIT)
+ );
+ // Baked slugs are recorded alongside it, so the file stands alone.
+ expect(learned).toContain('google_analytics');
+ })
+ )
+ );
+
+ it.live('records learned slugs at most once per run', () =>
+ withResolver({ seedLearnedFile: learnedFileContent([]) }, ({ waitForLearnedFile }) =>
Effect.gen(function* () {
- expect(yield* toolkitFromToolSlug('GOOGLE_ANALYTICS_RUN_REPORT')).toBe('google');
+ const catalog = yield* ToolkitSlugCatalog;
+ // Every miss in a run merges the same memoized fetch, so a second
+ // recording could only rewrite the same file. Passing a different list
+ // here makes the gate observable: its slugs must never land.
+ yield* catalog.remember([UNRELEASED_TOOLKIT]);
+ yield* catalog.remember(['slug_from_a_second_recording']);
+
+ const learned = yield* waitForLearnedFile(content => content.includes(UNRELEASED_TOOLKIT));
+ expect(learned).not.toContain('slug_from_a_second_recording');
})
- );
- });
+ )
+ );
+
+ it.live('falls back to the first-underscore guess when the catalog is unreachable', () =>
+ withResolver(
+ {
+ getToolkits: failingFetch,
+ seedLearnedFile: learnedFileContent([]),
+ },
+ ({ calls }) =>
+ Effect.gen(function* () {
+ expect(yield* toolkitFromToolSlug('ACME_ANALYTICS_RUN_REPORT')).toBe('acme');
+ expect(calls()).toBe(1);
+ })
+ )
+ );
+
+ it.live('tolerates a corrupted learned-slugs file', () =>
+ withResolver({ seedLearnedFile: '{ not json' }, ({ calls }) =>
+ Effect.gen(function* () {
+ expect(yield* toolkitFromToolSlug('GOOGLE_ANALYTICS_RUN_REPORT')).toBe('google_analytics');
+ expect(calls()).toBeLessThanOrEqual(1);
+ })
+ )
+ );
+
+ it.live('refreshes learned slugs in the background once they go stale', () =>
+ withResolver(
+ {
+ getToolkits: () => Effect.succeed([makeToolkitFixture(UNRELEASED_TOOLKIT)]),
+ seedLearnedFile: learnedFileContent(['stale_toolkit'], 8),
+ },
+ ({ waitForLearnedFile }) =>
+ Effect.gen(function* () {
+ // Answered from the baked list — the refresh is what the fetch is for.
+ expect(yield* toolkitFromToolSlug('GOOGLE_ANALYTICS_RUN_REPORT')).toBe(
+ 'google_analytics'
+ );
+
+ const learned = yield* waitForLearnedFile(content =>
+ content.includes(UNRELEASED_TOOLKIT)
+ );
+ expect(learned).not.toContain('stale_toolkit');
+ })
+ )
+ );
+
+ it.live('reads local knowledge once, however many slugs a run resolves', () =>
+ withResolver(
+ {
+ getToolkits: () => Effect.succeed([makeToolkitFixture(UNRELEASED_TOOLKIT)]),
+ seedLearnedFile: learnedFileContent(['stale_toolkit'], 8),
+ },
+ ({ calls, waitForLearnedFile }) =>
+ Effect.gen(function* () {
+ expect(yield* toolkitFromToolSlug('GOOGLE_ANALYTICS_RUN_REPORT')).toBe(
+ 'google_analytics'
+ );
+ expect(yield* toolkitFromToolSlug('GMAIL_SEND_EMAIL')).toBe('gmail');
+ expect(yield* toolkitFromToolSlug('GITHUB_CREATE_AN_ISSUE')).toBe('github');
+
+ yield* waitForLearnedFile(content => content.includes(UNRELEASED_TOOLKIT));
+ // The staleness verdict cannot change under a running process, so the
+ // refresh it triggers belongs to the run, not to each resolution.
+ expect(calls()).toBe(1);
+ })
+ )
+ );
+
+ it.live('does not refresh while the learned slugs are fresh', () =>
+ withResolver(
+ {
+ getToolkits: failingFetch,
+ seedLearnedFile: learnedFileContent([UNRELEASED_TOOLKIT], 6),
+ },
+ ({ calls }) =>
+ Effect.gen(function* () {
+ expect(yield* toolkitFromToolSlug('ACME_ANALYTICS_RUN_REPORT')).toBe(UNRELEASED_TOOLKIT);
+ yield* Effect.sleep('50 millis');
+ expect(calls()).toBe(0);
+ })
+ )
+ );
});