The agent engineering platform.
Python agentsaiai-agentsanthropicchatgptdeepagentsenterpriseframeworkgeminigenerative-ailangchainlanggraphllmmultiagentopen-sourceopenaipydanticpythonragtypescript
feat(docs): add truefoundry ai gateway (#32362)
rishiraj7f259863e1d5
diff --git a/docs/docs/integrations/providers/truefoundry.mdx b/docs/docs/integrations/providers/truefoundry.mdxnew file mode 100644index 0000000000..a629c41c36--- /dev/null+++ b/docs/docs/integrations/providers/truefoundry.mdx@@ -0,0 +1,101 @@+# TrueFoundry++TrueFoundry provides an enterprise-ready [AI Gateway](https://www.truefoundry.com/ai-gateway) to provide governance and observability to agentic frameworks like LangChain. TrueFoundry AI Gateway serves as a unified interface for LLM access, providing:++- **Unified API Access**: Connect to 250+ LLMs (OpenAI, Claude, Gemini, Groq, Mistral) through one API+- **Low Latency**: Sub-3ms internal latency with intelligent routing and load balancing+- **Enterprise Security**: SOC 2, HIPAA, GDPR compliance with RBAC and audit logging+- **Quota and cost management**: Token-based quotas, rate limiting, and comprehensive usage tracking+- **Observability**: Full request/response logging, metrics, and traces with customizable retention+++## Prerequisites++Before integrating LangChain with TrueFoundry, ensure you have:++1. **TrueFoundry Account**: A [TrueFoundry account](https://www.truefoundry.com/register) with at least one model provider configured. Follow quick start guide [here](https://docs.truefoundry.com/gateway/quick-start)+2. **Personal Access Token**: Generate a token by following the [TrueFoundry token generation guide](https://docs.truefoundry.com/gateway/authentication)++## Quickstart++You can connect to TrueFoundry's unified LLM gateway through the `ChatOpenAI` interface.++- Set the `base_url` to your TrueFoundry endpoint (explained below)+- Set the `api_key` to your TrueFoundry [PAT (Personal Access Token)](https://docs.truefoundry.com/gateway/authentication#personal-access-token-pat)+- Use the same `model-name` as shown in the unified code snippet++++### Installation++```bash+pip install langchain-openai+```++### Basic Setup++Connect to TrueFoundry by updating the `ChatOpenAI` model in LangChain:++```python+from langchain_openai import ChatOpenAI++llm = ChatOpenAI(+ api_key=TRUEFOUNDRY_API_KEY,+ base_url=TRUEFOUNDRY_GATEWAY_BASE_URL,+ model="openai-main/gpt-4o" # Similarly you can call any model from any model provider+)++llm.invoke("What is the meaning of life, universe and everything?")+```++The request is routed through your TrueFoundry gateway to the specified model provider. TrueFoundry automatically handles rate limiting, load balancing, and observability.++### LangGraph Integration+++```python+from langchain_openai import ChatOpenAI+from langgraph.graph import StateGraph, MessagesState+from langchain_core.messages import HumanMessage++# Define your LangGraph workflow+def call_model(state: MessagesState):+ model = ChatOpenAI(+ api_key=TRUEFOUNDRY_API_KEY,+ base_url=TRUEFOUNDRY_GATEWAY_BASE_URL,+ # Copy the exact model name from gateway+ model="openai-main/gpt-4o"+ )+ response = model.invoke(state["messages"])+ return {"messages": [response]}++# Build workflow+workflow = StateGraph(MessagesState)+workflow.add_node("agent", call_model)+workflow.set_entry_point("agent")+workflow.set_finish_point("agent")++app = workflow.compile()++# Run agent through TrueFoundry+result = app.invoke({"messages": [HumanMessage(content="Hello!")]})+```+++## Observability and Governance++++With the Metrics Dashboard, you can monitor and analyze:++- **Performance Metrics**: Track key latency metrics like Request Latency, Time to First Token (TTFS), and Inter-Token Latency (ITL) with P99, P90, and P50 percentiles+- **Cost and Token Usage**: Gain visibility into your application's costs with detailed breakdowns of input/output tokens and the associated expenses for each model+- **Usage Patterns**: Understand how your application is being used with detailed analytics on user activity, model distribution, and team-based usage+- **Rate Limiting & Load Balancing**: Configure limits, distribute traffic across models, and set up fallbacks++## Support++For questions, issues, or support:++- **Email**: [[email protected]](mailto:[email protected])+- **Documentation**: [https://docs.truefoundry.com/](https://docs.truefoundry.com/)diff --git a/docs/sidebars.js b/docs/sidebars.jsindex 26d7102f2d..3d175abe97 100644--- a/docs/sidebars.js+++ b/docs/sidebars.js@@ -27,7 +27,7 @@ module.exports = {},{type: "category",- link: {type: 'doc', id: 'tutorials/index'},+ link: { type: 'doc', id: 'tutorials/index' },label: "Tutorials",collapsible: false,items: [{@@ -38,7 +38,7 @@ module.exports = {},{type: "category",- link: {type: 'doc', id: 'how_to/index'},+ link: { type: 'doc', id: 'how_to/index' },label: "How-to guides",collapsible: false,items: [{@@ -49,7 +49,7 @@ module.exports = {},{type: "category",- link: {type: 'doc', id: 'concepts/index'},+ link: { type: 'doc', id: 'concepts/index' },label: "Conceptual guide",collapsible: false,items: [{@@ -103,7 +103,7 @@ module.exports = {{type: "category",label: "Migrating from v0.0 chains",- link: {type: 'doc', id: 'versions/migrating_chains/index'},+ link: { type: 'doc', id: 'versions/migrating_chains/index' },collapsible: false,collapsed: false,items: [{@@ -115,7 +115,7 @@ module.exports = {{type: "category",label: "Upgrading to LangGraph memory",- link: {type: 'doc', id: 'versions/migrating_memory/index'},+ link: { type: 'doc', id: 'versions/migrating_memory/index' },collapsible: false,collapsed: false,items: [{@@ -418,7 +418,7 @@ module.exports = {},],},-+],link: {type: "generated-index",@@ -434,7 +434,7 @@ module.exports = {},{type: "category",- link: {type: 'doc', id: 'contributing/tutorials/index'},+ link: { type: 'doc', id: 'contributing/tutorials/index' },label: "Tutorials",collapsible: false,items: [{@@ -445,7 +445,7 @@ module.exports = {},{type: "category",- link: {type: 'doc', id: 'contributing/how_to/index'},+ link: { type: 'doc', id: 'contributing/how_to/index' },label: "How-to guides",collapsible: false,items: [{@@ -456,7 +456,7 @@ module.exports = {},{type: "category",- link: {type: 'doc', id: 'contributing/reference/index'},+ link: { type: 'doc', id: 'contributing/reference/index' },label: "Reference & FAQ",collapsible: false,items: [{diff --git a/docs/static/img/gateway-metrics.png b/docs/static/img/gateway-metrics.pngnew file mode 100644index 0000000000..5c521c892aBinary files /dev/null and b/docs/static/img/gateway-metrics.png differdiff --git a/docs/static/img/unified-code-tfy.png b/docs/static/img/unified-code-tfy.pngnew file mode 100644index 0000000000..3f54355ec0Binary files /dev/null and b/docs/static/img/unified-code-tfy.png differ