Skip to content

The agent engineering platform.

feat(docs): add truefoundry ai gateway (#32362)

rishiraj7f259863e1d5
diff --git a/docs/docs/integrations/providers/truefoundry.mdx b/docs/docs/integrations/providers/truefoundry.mdx
new file mode 100644
index 0000000000..a629c41c36
--- /dev/null
+++ b/docs/docs/integrations/providers/truefoundry.mdx
@@ -0,0 +1,101 @@
+# TrueFoundry
+
+TrueFoundry provides an enterprise-ready [AI Gateway](https://www.truefoundry.com/ai-gateway) to provide governance and observability to agentic frameworks like LangChain. TrueFoundry AI Gateway serves as a unified interface for LLM access, providing:
+
+- **Unified API Access**: Connect to 250+ LLMs (OpenAI, Claude, Gemini, Groq, Mistral) through one API
+- **Low Latency**: Sub-3ms internal latency with intelligent routing and load balancing
+- **Enterprise Security**: SOC 2, HIPAA, GDPR compliance with RBAC and audit logging
+- **Quota and cost management**: Token-based quotas, rate limiting, and comprehensive usage tracking
+- **Observability**: Full request/response logging, metrics, and traces with customizable retention
+
+
+## Prerequisites
+
+Before integrating LangChain with TrueFoundry, ensure you have:
+
+1. **TrueFoundry Account**: A [TrueFoundry account](https://www.truefoundry.com/register) with at least one model provider configured. Follow quick start guide [here](https://docs.truefoundry.com/gateway/quick-start)
+2. **Personal Access Token**: Generate a token by following the [TrueFoundry token generation guide](https://docs.truefoundry.com/gateway/authentication)
+
+## Quickstart
+
+You can connect to TrueFoundry's unified LLM gateway through the `ChatOpenAI` interface.
+
+- Set the `base_url` to your TrueFoundry endpoint (explained below)
+- Set the `api_key` to your TrueFoundry [PAT (Personal Access Token)](https://docs.truefoundry.com/gateway/authentication#personal-access-token-pat)
+- Use the same `model-name` as shown in the unified code snippet
+
+![TrueFoundry metrics](/img/unified-code-tfy.png)
+
+### Installation
+
+```bash
+pip install langchain-openai
+```
+
+### Basic Setup
+
+Connect to TrueFoundry by updating the `ChatOpenAI` model in LangChain:
+
+```python
+from langchain_openai import ChatOpenAI
+
+llm = ChatOpenAI(
+ api_key=TRUEFOUNDRY_API_KEY,
+ base_url=TRUEFOUNDRY_GATEWAY_BASE_URL,
+ model="openai-main/gpt-4o" # Similarly you can call any model from any model provider
+)
+
+llm.invoke("What is the meaning of life, universe and everything?")
+```
+
+The request is routed through your TrueFoundry gateway to the specified model provider. TrueFoundry automatically handles rate limiting, load balancing, and observability.
+
+### LangGraph Integration
+
+
+```python
+from langchain_openai import ChatOpenAI
+from langgraph.graph import StateGraph, MessagesState
+from langchain_core.messages import HumanMessage
+
+# Define your LangGraph workflow
+def call_model(state: MessagesState):
+ model = ChatOpenAI(
+ api_key=TRUEFOUNDRY_API_KEY,
+ base_url=TRUEFOUNDRY_GATEWAY_BASE_URL,
+ # Copy the exact model name from gateway
+ model="openai-main/gpt-4o"
+ )
+ response = model.invoke(state["messages"])
+ return {"messages": [response]}
+
+# Build workflow
+workflow = StateGraph(MessagesState)
+workflow.add_node("agent", call_model)
+workflow.set_entry_point("agent")
+workflow.set_finish_point("agent")
+
+app = workflow.compile()
+
+# Run agent through TrueFoundry
+result = app.invoke({"messages": [HumanMessage(content="Hello!")]})
+```
+
+
+## Observability and Governance
+
+![TrueFoundry metrics](/img/gateway-metrics.png)
+
+With the Metrics Dashboard, you can monitor and analyze:
+
+- **Performance Metrics**: Track key latency metrics like Request Latency, Time to First Token (TTFS), and Inter-Token Latency (ITL) with P99, P90, and P50 percentiles
+- **Cost and Token Usage**: Gain visibility into your application's costs with detailed breakdowns of input/output tokens and the associated expenses for each model
+- **Usage Patterns**: Understand how your application is being used with detailed analytics on user activity, model distribution, and team-based usage
+- **Rate Limiting & Load Balancing**: Configure limits, distribute traffic across models, and set up fallbacks
+
+## Support
+
+For questions, issues, or support:
+
+- **Email**: [[email protected]](mailto:[email protected])
+- **Documentation**: [https://docs.truefoundry.com/](https://docs.truefoundry.com/)
diff --git a/docs/sidebars.js b/docs/sidebars.js
index 26d7102f2d..3d175abe97 100644
--- a/docs/sidebars.js
+++ b/docs/sidebars.js
@@ -27,7 +27,7 @@ module.exports = {
},
{
type: "category",
- link: {type: 'doc', id: 'tutorials/index'},
+ link: { type: 'doc', id: 'tutorials/index' },
label: "Tutorials",
collapsible: false,
items: [{
@@ -38,7 +38,7 @@ module.exports = {
},
{
type: "category",
- link: {type: 'doc', id: 'how_to/index'},
+ link: { type: 'doc', id: 'how_to/index' },
label: "How-to guides",
collapsible: false,
items: [{
@@ -49,7 +49,7 @@ module.exports = {
},
{
type: "category",
- link: {type: 'doc', id: 'concepts/index'},
+ link: { type: 'doc', id: 'concepts/index' },
label: "Conceptual guide",
collapsible: false,
items: [{
@@ -103,7 +103,7 @@ module.exports = {
{
type: "category",
label: "Migrating from v0.0 chains",
- link: {type: 'doc', id: 'versions/migrating_chains/index'},
+ link: { type: 'doc', id: 'versions/migrating_chains/index' },
collapsible: false,
collapsed: false,
items: [{
@@ -115,7 +115,7 @@ module.exports = {
{
type: "category",
label: "Upgrading to LangGraph memory",
- link: {type: 'doc', id: 'versions/migrating_memory/index'},
+ link: { type: 'doc', id: 'versions/migrating_memory/index' },
collapsible: false,
collapsed: false,
items: [{
@@ -418,7 +418,7 @@ module.exports = {
},
],
},
-
+
],
link: {
type: "generated-index",
@@ -434,7 +434,7 @@ module.exports = {
},
{
type: "category",
- link: {type: 'doc', id: 'contributing/tutorials/index'},
+ link: { type: 'doc', id: 'contributing/tutorials/index' },
label: "Tutorials",
collapsible: false,
items: [{
@@ -445,7 +445,7 @@ module.exports = {
},
{
type: "category",
- link: {type: 'doc', id: 'contributing/how_to/index'},
+ link: { type: 'doc', id: 'contributing/how_to/index' },
label: "How-to guides",
collapsible: false,
items: [{
@@ -456,7 +456,7 @@ module.exports = {
},
{
type: "category",
- link: {type: 'doc', id: 'contributing/reference/index'},
+ link: { type: 'doc', id: 'contributing/reference/index' },
label: "Reference & FAQ",
collapsible: false,
items: [{
diff --git a/docs/static/img/gateway-metrics.png b/docs/static/img/gateway-metrics.png
new file mode 100644
index 0000000000..5c521c892a
Binary files /dev/null and b/docs/static/img/gateway-metrics.png differ
diff --git a/docs/static/img/unified-code-tfy.png b/docs/static/img/unified-code-tfy.png
new file mode 100644
index 0000000000..3f54355ec0
Binary files /dev/null and b/docs/static/img/unified-code-tfy.png differ