100+ AI Agents, Agent Skills and RAG Apps - Free and Open Source.
π§ Math Tutor Agent β Agentic RAG with Feedback Loop
This project implements an Agentic-RAG architecture to simulate a math professor that solves JEE-level math questions with step-by-step explanations. The system smartly routes queries between a vector database and web search, applies input/output guardrails, and incorporates human feedback for continuous learning.
π Features
- β Input Guardrails (DSPy): Accepts only academic math questions.
- π Knowledge Base Search: Uses Qdrant Vector DB with OpenAI Embeddings to match known questions.
- π Web Fallback: Integrates Tavily API when no good match is found.
- βοΈ GPT-4.1 Explanations: Generates step-by-step math solutions.
- π‘οΈ Output Guardrails: Filters for correctness and safety.
- π Human-in-the-Loop Feedback: Users rate answers (Yes/No), logged for future learning.
- π Benchmarking: Evaluated on JEEBench dataset with adjustable question limits.
- π» Streamlit UI: Interactive dashboard with multiple tabs.
π Architecture Flow
π Knowledge Base
- Dataset: JEEBench (HuggingFace)
- Vector DB: Qdrant (with OpenAI Embeddings)
- Storage: Built with
llama-indexto persist embeddings and perform top-1 similarity search
π Web Search
- Uses Tavily API for fallback search when the KB doesn't contain a good match
- Fetched content is piped into GPT-4o for clean explanation
π Guardrails
- Input Guardrail (DSPy): Accepts only math-related academic questions
- Output Guardrail (DSPy): Blocks hallucinated or off-topic content
π¨βπ« Human-in-the-Loop Feedback
- Streamlit UI allows students to give π / π after seeing the answer
- Feedback is logged to a local JSON file for future improvement
π Benchmarking
- Evaluated on 50 random JEEBench Math Questions
- Current Accuracy: 66%
- Benchmark results saved to:
benchmark/results.csv
π Demo
To run the app with Streamlit:
streamlit run app/streamlit.py