🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
TypeScript analyticsautogenevaluationlangchainlarge-language-modelsllama-indexllmllm-evaluationllm-observabilityllmopsmonitoringobservabilityopen-sourceopenaiplaygroundprompt-engineeringprompt-managementself-hostedycombinator
Public API
How to add new api routes
Implementation
Wrap with
withMiddlewareType-safe and authed API Route with
createAuthedAPIRouteAdd zod types to
/features/public-api/typesfolder.- Use
coerceto handle primitives, such as dates, for use in your application and tests. - Use
strict()on all objects that should not return additional properties. Recommended as default. In these cases, the test utilitymakeZodVerifiedAPICallwill throw an error if the response contains additional properties. Also, we will log an error in production if the response contains additional properties.
- Use
Throw errors defined in
shared/src/errorswhich translate to HTTP status codes
Testing
- Add tests for all standard cases
- use
makeZodVerifiedAPICallto test the API response against the zod response schema
API Reference
- Add to
fernincludingdocsattributes - Export with
pnpm run openapi:export, then commit the changes to the API reference. CI fails if the served specs underweb/public/generated/drift fromfern/.
SDKs
- Copy/paste fern-generated types or api reference to Python and JS SDKs respectively
- Implement wrapping functions if needed
How to refactor existing apis to this pattern
- Move request types to
features/public-api/types - Create response types in
features/public-api/types - Validate response type with
validateZodSchemain the API route to get type warnings in case of mismatch - Refactor api route by passing current api route to LLM together with example of e.g. public/v2/datasets.ts