A curated collection of practical AI projects implementing OCR systems, RAG, AI agents, and other AI use cases.
feat: add LangChain Data Agent for natural language SQL queries on Chinook database
Tiioluwaniabb1d326640d
diff --git a/ai_agents/langchain_data_agent/.env.example b/ai_agents/langchain_data_agent/.env.examplenew file mode 100644index 0000000..e45a62e--- /dev/null+++ b/ai_agents/langchain_data_agent/.env.example@@ -0,0 +1,6 @@+# Orq.ai API key (https://orq.ai)+ORQ_API_KEY=your_orq_api_key_here++# Optional: enable LangSmith tracing+# LANGSMITH_TRACING=true+# LANGSMITH_API_KEY=your_langsmith_key_herediff --git a/ai_agents/langchain_data_agent/README.md b/ai_agents/langchain_data_agent/README.mdnew file mode 100644index 0000000..77a1ae6--- /dev/null+++ b/ai_agents/langchain_data_agent/README.md@@ -0,0 +1,142 @@+# LangChain Data Agent++> Query the Chinook SQLite database in plain English through a conversational Streamlit chat.++## Overview++LangChain Data Agent turns natural language questions into safe, read-only SQL against the bundled Chinook music store database. A LangGraph workflow orchestrates schema discovery, query generation, validation, and execution, then returns a clear natural-language answer with an optional Plotly chart when results are tabular or numeric.++This project is useful for data analysts, engineers, and learners who want to explore SQL databases without writing queries by hand, while still seeing the generated SQL for transparency and learning.++## Demo++++## Features++- Conversational Streamlit chat with session-based history+- Natural language to SQL via a LangGraph agent and LangChain SQL toolkit+- Read-only query enforcement (SELECT / WITH only; no INSERT, UPDATE, DELETE, or DDL)+- Automatic Plotly charts for numeric and tabular results+- Expandable SQL panel showing the query the agent ran+- Result tables with real column names from the database (e.g. `ArtistName`, `TrackCount`)++## Tech Stack++**Frameworks & Libraries:**++- LangGraph — agent workflow orchestration+- LangChain — SQL toolkit, messages, and LLM integration+- langchain-openai — ChatOpenAI client for the Orq.ai router+- OpenAI SDK — API client (routed through Orq.ai)+- SQLAlchemy — database engine for SQLite+- Pandas — result parsing and chart data prep++**Additional Tools:**++- Database: SQLite (Chinook)+- Charts: Plotly+- Web Framework: Streamlit++## Prerequisites++- Python 3.10 or higher+- API keys for:+ - Orq.ai (`ORQ_API_KEY`)++## Installation++### 1. Clone the Repository++```bash+git clone https://github.com/Sumanth077/Hands-On-AI-Engineering.git+cd Hands-On-AI-Engineering/ai_agents/langchain_data_agent+```++### 2. Create Virtual Environment (Recommended)++```bash+python -m venv venv+# Windows:+venv\Scripts\activate+# macOS/Linux:+source venv/bin/activate+```++### 3. Install Dependencies++```bash+pip install -r requirements.txt+```++### 4. Set Up Environment Variables++```bash+cp .env.example .env+```++Edit `.env` and set your Orq.ai API key:++```env+ORQ_API_KEY=your_orq_api_key_here+```++Optional LangSmith tracing (uncomment in `.env` if needed):++```env+LANGSMITH_TRACING=true+LANGSMITH_API_KEY=your_langsmith_key_here+```++## Usage++### Running the Application++```bash+streamlit run app.py+```++Open the URL shown in the terminal (typically `http://localhost:8501`).++### Example Usage++Ask questions in the chat like:+++| Question | What you get |+| ------------------------------------------ | ----------------------------------------------------- |+| How many artists are in the database? | A count, the SQL used, and a simple result table |+| Which artist has the most tracks? | Ranked results with real column names and a bar chart |+| What are the top 5 best-selling genres? | Aggregated sales by genre with a chart |+| How many customers are in each country? | Grouped counts per country |+| What is the average track length by genre? | Aggregated duration stats by genre |+++The agent lists tables, fetches relevant schemas, generates and checks SQL, runs the query read-only, and summarizes the answer. Follow-up questions use conversation history from the session.++## Project Structure++```+langchain-data-agent/+├── app.py # Streamlit chat UI, charts, and result tables+├── agent.py # LangGraph SQL agent and read-only query tool+├── chinook.db # Chinook sample SQLite database+├── requirements.txt # Python dependencies+├── .env.example # Environment variable template+├── .gitignore+├── README.md+└── assets/+ └── demo.gif # Application demo recording+```++## How It Works++1. **User input** — You type a question in the Streamlit chat.+2. **List tables** — The agent loads available Chinook tables (Artist, Album, Track, Invoice, etc.).+3. **Get schema** — The LLM calls `sql_db_schema` for tables relevant to your question.+4. **Generate SQL** — The model writes a SQLite `SELECT` query from the schema and question.+5. **Check SQL** — A dedicated step reviews the query for common SQL mistakes.+6. **Execute** — `ReadOnlyQuerySQLDatabaseTool` validates and runs the query; results include column names.+7. **Respond** — The agent returns a natural-language answer; the UI shows the SQL, a data table, and a Plotly chart when appropriate.++The LLM (`kimi-k2.6`) is called through the [Orq.ai](https://orq.ai) router (`https://api.orq.ai/v3/router`). Message history is flattened for compatibility with the model’s tool-calling requirements, while the graph still runs the full LangGraph SQL workflow under the hood.\ No newline at end of filediff --git a/ai_agents/langchain_data_agent/agent.py b/ai_agents/langchain_data_agent/agent.pynew file mode 100644index 0000000..3b22f75--- /dev/null+++ b/ai_agents/langchain_data_agent/agent.py@@ -0,0 +1,297 @@+"""LangGraph SQL agent for the Chinook database (read-only)."""++from __future__ import annotations++import os+import re+from pathlib import Path+from typing import Literal++from dotenv import load_dotenv+from langchain.messages import AIMessage, HumanMessage+from langchain_community.agent_toolkits import SQLDatabaseToolkit+from langchain_community.tools.sql_database.tool import QuerySQLDatabaseTool+from langchain_community.utilities import SQLDatabase+from langchain_core.messages import BaseMessage, ToolMessage+from langchain_openai import ChatOpenAI+from langgraph.graph import END, START, MessagesState, StateGraph+from langgraph.prebuilt import ToolNode++load_dotenv()++DB_PATH = Path(__file__).parent / "chinook.db"+TOP_K = 10++FORBIDDEN_SQL = re.compile(+ r"\b(INSERT|UPDATE|DELETE|DROP|CREATE|ALTER|TRUNCATE|REPLACE|"+ r"GRANT|REVOKE|ATTACH|DETACH|MERGE|CALL|EXEC|EXECUTE)\b",+ re.IGNORECASE,+)+READ_ONLY_SQL = re.compile(r"^\s*(SELECT|WITH|PRAGMA|EXPLAIN)\b", re.IGNORECASE | re.DOTALL)+++def _llm_context(state: MessagesState) -> list[HumanMessage]:+ """Text-only context for Kimi — never replay assistant tool_calls to the API."""+ user_question: str | None = None+ blocks: list[str] = []++ for msg in state["messages"]:+ if isinstance(msg, HumanMessage):+ user_question = msg.content if isinstance(msg.content, str) else str(msg.content)+ elif isinstance(msg, AIMessage):+ if msg.tool_calls:+ summary = ", ".join(+ f"{tc.get('name')}({tc.get('args')})" for tc in msg.tool_calls+ )+ blocks.append(f"Assistant requested tools: {summary}")+ elif msg.content and str(msg.content).strip():+ blocks.append(str(msg.content).strip())+ elif isinstance(msg, ToolMessage):+ name = msg.name or "tool"+ body = msg.content if isinstance(msg.content, str) else str(msg.content)+ blocks.append(f"### {name}\n{body}")++ parts: list[str] = []+ if blocks:+ parts.append("Database context:\n" + "\n\n".join(blocks))+ if user_question:+ parts.append(f"User question: {user_question}")+ return [HumanMessage(content="\n\n".join(parts) if parts else "")]+++def _validate_read_only(query: str) -> str:+ cleaned = query.strip().rstrip(";")+ if FORBIDDEN_SQL.search(cleaned):+ raise ValueError("Only read-only SELECT queries are allowed.")+ if not READ_ONLY_SQL.match(cleaned):+ raise ValueError("Only read-only SELECT queries are allowed.")+ return cleaned+++class ReadOnlyQuerySQLDatabaseTool(QuerySQLDatabaseTool):+ """SQL query tool that rejects non-SELECT statements."""++ def _run(self, query: str, run_manager=None) -> str:+ safe_query = _validate_read_only(query)+ return self.db.run_no_throw(safe_query, include_columns=True)+++def _build_model() -> ChatOpenAI:+ api_key = os.getenv("ORQ_API_KEY")+ if not api_key:+ raise ValueError("ORQ_API_KEY is not set. Copy .env.example to .env and add your key.")+ return ChatOpenAI(+ model="kimi-k2.6",+ openai_api_key=api_key,+ openai_api_base="https://api.orq.ai/v3/router",+ temperature=1,+ # Kimi K2.6 defaults to thinking mode, which breaks multi-turn tool calling+ # unless reasoning_content is preserved on every assistant tool-call message.+ extra_body={"thinking": {"type": "disabled"}},+ )+++def _build_tools(db: SQLDatabase, model: ChatOpenAI) -> list:+ toolkit = SQLDatabaseToolkit(db=db, llm=model)+ tools = toolkit.get_tools()+ for i, tool in enumerate(tools):+ if tool.name == "sql_db_query":+ tools[i] = ReadOnlyQuerySQLDatabaseTool(db=db)+ return tools+++def _create_agent():+ if not DB_PATH.exists():+ raise FileNotFoundError(f"Database not found at {DB_PATH}")++ db = SQLDatabase.from_uri(f"sqlite:///{DB_PATH.resolve()}")+ model = _build_model()+ tools = _build_tools(db, model)++ get_schema_tool = next(t for t in tools if t.name == "sql_db_schema")+ get_schema_node = ToolNode([get_schema_tool], name="get_schema")++ run_query_tool = next(t for t in tools if t.name == "sql_db_query")+ run_query_node = ToolNode([run_query_tool], name="run_query")++ list_tables_tool = next(t for t in tools if t.name == "sql_db_list_tables")++ generate_query_system_prompt = f"""+You are an agent designed to interact with a SQL database.+Given an input question, create a syntactically correct {db.dialect} query to run,+then look at the results of the query and return the answer. Unless the user+specifies a specific number of examples they wish to obtain, always limit your+query to at most {TOP_K} results.++You can order the results by a relevant column to return the most interesting+examples in the database. Never query for all the columns from a specific table,+only ask for the relevant columns given the question.++DO NOT make any DML statements (INSERT, UPDATE, DELETE, DROP etc.) to the database.+When you have enough information to answer, respond in clear natural language.+""".strip()++ check_query_system_prompt = f"""+You are a SQL expert with a strong attention to detail.+Double check the {db.dialect} query for common mistakes, including:+- Using NOT IN with NULL values+- Using UNION when UNION ALL should have been used+- Using BETWEEN for exclusive ranges+- Data type mismatch in predicates+- Properly quoting identifiers+- Using the correct number of arguments for functions+- Casting to the correct data type+- Using the proper columns for joins++If there are any of the above mistakes, rewrite the query. If there are no mistakes,+just reproduce the original query.++You will call the appropriate tool to execute the query after running this check.+Only SELECT (read-only) queries are permitted.+""".strip()++ def list_tables(state: MessagesState):+ tool_call = {+ "name": "sql_db_list_tables",+ "args": {},+ "id": "list_tables_call",+ "type": "tool_call",+ }+ tool_message = list_tables_tool.invoke(tool_call)+ response = AIMessage(content=f"Available tables: {tool_message.content}")+ return {"messages": [response]}++ def _require_tool_call(llm_with_tools, messages, tool_name: str, retry_hint: str):+ response = llm_with_tools.invoke(messages)+ if response.tool_calls and any(tc.get("name") == tool_name for tc in response.tool_calls):+ return response+ return llm_with_tools.invoke([*messages, HumanMessage(content=retry_hint)])++ def call_get_schema(state: MessagesState):+ llm_with_tools = model.bind_tools([get_schema_tool])+ messages = [+ {+ "role": "system",+ "content": (+ "Call sql_db_schema with comma-separated table names relevant "+ "to the user question."+ ),+ },+ *_llm_context(state),+ ]+ response = _require_tool_call(+ llm_with_tools,+ messages,+ "sql_db_schema",+ "Call sql_db_schema now with the relevant table names.",+ )+ return {"messages": [response]}++ def generate_query(state: MessagesState):+ llm_with_tools = model.bind_tools([run_query_tool])+ messages = [+ {"role": "system", "content": generate_query_system_prompt},+ *_llm_context(state),+ ]+ response = llm_with_tools.invoke(messages)+ return {"messages": [response]}++ def check_query(state: MessagesState):+ system_message = {"role": "system", "content": check_query_system_prompt}+ tool_call = state["messages"][-1].tool_calls[0]+ user_message = {"role": "user", "content": tool_call["args"]["query"]}+ llm_with_tools = model.bind_tools([run_query_tool])+ response = _require_tool_call(+ llm_with_tools,+ [system_message, user_message],+ "sql_db_query",+ "Call sql_db_query with the validated read-only SELECT query.",+ )+ response.id = state["messages"][-1].id+ return {"messages": [response]}++ def should_continue(state: MessagesState) -> Literal["check_query", END]:+ last_message = state["messages"][-1]+ if not last_message.tool_calls:+ return END+ return "check_query"++ builder = StateGraph(MessagesState)+ builder.add_node("list_tables", list_tables)+ builder.add_node("call_get_schema", call_get_schema)+ builder.add_node("get_schema", get_schema_node)+ builder.add_node("generate_query", generate_query)+ builder.add_node("check_query", check_query)+ builder.add_node("run_query", run_query_node)++ builder.add_edge(START, "list_tables")+ builder.add_edge("list_tables", "call_get_schema")+ builder.add_edge("call_get_schema", "get_schema")+ builder.add_edge("get_schema", "generate_query")+ builder.add_conditional_edges("generate_query", should_continue)+ builder.add_edge("check_query", "run_query")+ builder.add_edge("run_query", "generate_query")++ return builder.compile()+++_agent = None+++def get_agent(*, reload: bool = False):+ global _agent+ if _agent is None or reload:+ _agent = _create_agent()+ return _agent+++def _to_langchain_messages(history: list[dict]) -> list[BaseMessage]:+ messages: list[BaseMessage] = []+ for item in history:+ role = item.get("role", "user")+ content = item.get("content", "")+ if role == "user":+ messages.append(HumanMessage(content=content))+ elif role == "assistant":+ messages.append(AIMessage(content=content))+ return messages+++def _extract_sql_metadata(messages: list[BaseMessage]) -> tuple[str | None, str | None]:+ last_sql: str | None = None+ last_results: str | None = None+ for message in messages:+ if isinstance(message, AIMessage) and message.tool_calls:+ for tool_call in message.tool_calls:+ if tool_call.get("name") == "sql_db_query":+ last_sql = tool_call.get("args", {}).get("query")+ if isinstance(message, ToolMessage) and message.name == "sql_db_query":+ last_results = message.content+ return last_sql, last_results+++def extract_answer(messages: list[BaseMessage]) -> str:+ for message in reversed(messages):+ if isinstance(message, AIMessage) and message.content and not message.tool_calls:+ text = message.content+ if isinstance(text, str) and text.strip():+ if not text.startswith("Available tables:"):+ return text.strip()+ return "I could not generate an answer. Please try rephrasing your question."+++def run_query(question: str, history: list[dict] | None = None) -> dict:+ """Run the SQL agent and return answer metadata for the UI."""+ agent = get_agent(reload=True)+ prior = _to_langchain_messages(history or [])+ prior.append(HumanMessage(content=question))++ result = agent.invoke({"messages": prior})++ last_sql, last_results = _extract_sql_metadata(result["messages"])+ return {+ "answer": extract_answer(result["messages"]),+ "sql": last_sql,+ "raw_results": last_results,+ "messages": result["messages"],+ }diff --git a/ai_agents/langchain_data_agent/app.py b/ai_agents/langchain_data_agent/app.pynew file mode 100644index 0000000..4c76796--- /dev/null+++ b/ai_agents/langchain_data_agent/app.py@@ -0,0 +1,236 @@+"""Streamlit chat UI for the LangChain Data Agent."""++from __future__ import annotations++import ast+import re++import pandas as pd+import plotly.express as px+import streamlit as st++from agent import run_query++st.set_page_config(+ page_title="LangChain Data Agent",+ page_icon="🎵",+ layout="wide",+)++st.title("LangChain Data Agent")+st.caption(+ "Ask questions about the Chinook music store database in plain English. "+ "Queries are translated to read-only SQL and answered with charts when relevant."+)++with st.sidebar:+ st.header("About")+ st.markdown(+ """+ **Database:** Chinook SQLite (artists, albums, tracks, sales)++ **Model:** kimi-k2.6 via [Orq.ai](https://orq.ai)+ """+ )+ if st.button("Clear conversation", use_container_width=True):+ st.session_state.messages = []+ st.rerun()+++def _column_names_from_sql(sql: str | None, num_cols: int) -> list[str]:+ """Derive display column names from the SELECT list when results are bare tuples."""+ if not sql or num_cols < 1:+ return [f"col_{i + 1}" for i in range(num_cols)]++ match = re.search(r"\bSELECT\b(.+?)\bFROM\b", sql, re.IGNORECASE | re.DOTALL)+ if not match:+ return [f"col_{i + 1}" for i in range(num_cols)]++ select_part = match.group(1).strip()+ if select_part == "*":+ return [f"col_{i + 1}" for i in range(num_cols)]++ parts: list[str] = []+ depth = 0+ buf: list[str] = []+ for char in select_part + ",":+ if char == "(":+ depth += 1+ buf.append(char)+ elif char == ")":+ depth = max(0, depth - 1)+ buf.append(char)+ elif char == "," and depth == 0:+ part = "".join(buf).strip()+ if part:+ parts.append(part)+ buf = []+ else:+ buf.append(char)++ names: list[str] = []+ for part in parts:+ alias = re.search(+ r"\bAS\s+(?:[`\"']?)([A-Za-z_][\w$]*)(?:[`\"']?)\s*$",+ part,+ re.IGNORECASE,+ )+ if alias:+ names.append(alias.group(1))+ continue+ token = part.strip().rstrip(",").split()[-1]+ token = token.strip("`\"'[]")+ names.append(token.split(".")[-1] if token else f"col_{len(names) + 1}")++ if len(names) >= num_cols:+ return names[:num_cols]+ return names + [f"col_{i + 1}" for i in range(len(names), num_cols)]+++def _parse_results(raw: str | None, sql: str | None = None) -> pd.DataFrame | None:+ if not raw or not raw.strip():+ return None++ text = raw.strip()+ if text.startswith("[") and text.endswith("]"):+ try:+ rows = ast.literal_eval(text)+ if not rows:+ return None+ if isinstance(rows[0], dict):+ return pd.DataFrame(rows)+ if isinstance(rows[0], tuple):+ cols = _column_names_from_sql(sql, len(rows[0]))+ return pd.DataFrame(rows, columns=cols)+ except (SyntaxError, ValueError):+ pass++ lines = [ln.strip() for ln in text.splitlines() if ln.strip()]+ if len(lines) < 2:+ return None++ header = re.split(r"\s*\|\s*", lines[0].strip("| "))+ data_rows = []+ for line in lines[2:]:+ if set(line.replace("|", "").strip()) <= {"-", ":"}:+ continue+ data_rows.append(re.split(r"\s*\|\s*", line.strip("| ")))++ if header and data_rows:+ width = len(header)+ normalized = [row[:width] + [""] * (width - len(row)) for row in data_rows]+ df = pd.DataFrame(normalized, columns=header)+ for col in df.columns:+ converted = pd.to_numeric(df[col], errors="coerce")+ if converted.notna().any():+ df[col] = converted+ return df++ return None+++def _build_chart(df: pd.DataFrame):+ if df.empty or len(df) < 2:+ return None++ numeric_cols = df.select_dtypes(include="number").columns.tolist()+ non_numeric = [c for c in df.columns if c not in numeric_cols]++ if not numeric_cols:+ if len(df.columns) >= 2:+ fig = px.bar(df, x=df.columns[0], y=df.columns[1], title="Query results")+ return fig+ return None++ value_col = numeric_cols[-1]+ label_col = non_numeric[0] if non_numeric else df.columns[0]++ if label_col == value_col:+ return None++ plot_df = df[[label_col, value_col]].head(20).copy()+ plot_df[value_col] = pd.to_numeric(plot_df[value_col], errors="coerce")+ plot_df = plot_df.dropna(subset=[value_col])+ if plot_df.empty:+ return None++ if len(plot_df) > 12:+ fig = px.bar(+ plot_df,+ x=label_col,+ y=value_col,+ title="Query results",+ labels={label_col: label_col, value_col: value_col},+ )+ else:+ fig = px.pie(+ plot_df,+ names=label_col,+ values=value_col,+ title="Query results",+ )+ fig.update_layout(xaxis_tickangle=-35, margin=dict(l=20, r=20, t=40, b=80))+ return fig+++if "messages" not in st.session_state:+ st.session_state.messages = []++for message in st.session_state.messages:+ with st.chat_message(message["role"]):+ st.markdown(message["content"])+ if message.get("sql"):+ with st.expander("SQL query"):+ st.code(message["sql"], language="sql")+ if message.get("chart_df") is not None:+ chart = _build_chart(message["chart_df"])+ if chart is not None:+ st.plotly_chart(chart, use_container_width=True)+ elif message.get("raw_results"):+ df = _parse_results(message["raw_results"], message.get("sql"))+ if df is not None and not df.empty:+ st.dataframe(df, use_container_width=True)++if prompt := st.chat_input("Ask a question about the Chinook database..."):+ st.session_state.messages.append({"role": "user", "content": prompt})+ with st.chat_message("user"):+ st.markdown(prompt)++ with st.chat_message("assistant"):+ with st.spinner("Thinking..."):+ try:+ history = [+ {"role": m["role"], "content": m["content"]}+ for m in st.session_state.messages[:-1]+ ]+ result = run_query(prompt, history=history)+ answer = result["answer"]+ sql = result.get("sql")+ raw = result.get("raw_results")+ df = _parse_results(raw, sql)++ st.markdown(answer)+ if sql:+ with st.expander("SQL query"):+ st.code(sql, language="sql")+ chart = _build_chart(df) if df is not None else None+ if chart is not None:+ st.plotly_chart(chart, use_container_width=True)+ elif df is not None and not df.empty:+ st.dataframe(df, use_container_width=True)++ st.session_state.messages.append(+ {+ "role": "assistant",+ "content": answer,+ "sql": sql,+ "raw_results": raw,+ "chart_df": df,+ }+ )+ except Exception as exc:+ error_text = f"Something went wrong: {exc}"+ st.error(error_text)+ st.session_state.messages.append(+ {"role": "assistant", "content": error_text}+ )diff --git a/ai_agents/langchain_data_agent/assets/demo.gif b/ai_agents/langchain_data_agent/assets/demo.gifnew file mode 100644index 0000000..0a2e731Binary files /dev/null and b/ai_agents/langchain_data_agent/assets/demo.gif differdiff --git a/ai_agents/langchain_data_agent/chinook.db b/ai_agents/langchain_data_agent/chinook.dbnew file mode 100644index 0000000..8c687ffBinary files /dev/null and b/ai_agents/langchain_data_agent/chinook.db differdiff --git a/ai_agents/langchain_data_agent/requirements.txt b/ai_agents/langchain_data_agent/requirements.txtnew file mode 100644index 0000000..b72db3a--- /dev/null+++ b/ai_agents/langchain_data_agent/requirements.txt@@ -0,0 +1,10 @@+streamlit>=1.32.0+langchain>=0.3.0+langchain-community>=0.3.0+langchain-openai>=0.2.0+langgraph>=0.2.0+openai>=1.40.0+sqlalchemy>=2.0.0+plotly>=5.18.0+pandas>=2.0.0+python-dotenv>=1.0.0