OpenRAG is a comprehensive, single package Retrieval-Augmented Generation platform built on Langflow, Docling, and Opensearch.
Python
chore: update docs with current way of contributing to OpenRAG (#2539)
Lucas Oliveira6fc819d3ec10
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.mdindex 73396b808..574f41dda 100644--- a/CONTRIBUTING.md+++ b/CONTRIBUTING.md@@ -32,15 +32,17 @@ This guide will help you set up your development environment and start contribut## Quickstart-Get OpenRAG running in three commands:+For active development, run the infrastructure in containers and the backend and frontend from your checkout:```bashmake check_tools # Verify you have all prerequisitesmake setup # Install dependencies and create .env-make dev # Start OpenRAG+# Populate the required values in .env, then:+make dev-local-cpu # Start local development infrastructure+# In separate terminals: make backend, make frontend, and make docling```-OpenRAG is now running locally on the following ports:+OpenRAG is then running locally on the following ports:- **Frontend**: http://localhost:3000- **Langflow**: http://localhost:7860@@ -155,27 +157,23 @@ You should see: `All required tools are installed.`make setup```-2. Configure the required environment variables before starting OpenRAG:+2. Configure the required environment variables before starting OpenRAG. Start with `.env.example`; `make dev-local`, `make dev-local-cpu`, and the full-stack Make targets validate these values:```env- OPENAI_API_KEY=- OPENSEARCH_PASSWORD=LANGFLOW_SUPERUSER=adminLANGFLOW_SUPERUSER_PASSWORD=+ LANGFLOW_SECRET_KEY=+ OPENSEARCH_PASSWORD=```+ Configure a model provider, such as `OPENAI_API_KEY`, as required for the functionality you want to use.The `OPENSEARCH_PASSWORD` must adhere to the [OpenSearch password complexity requirements](https://docs.opensearch.org/latest/security/configuration/demo-configuration/#setting-up-a-custom-admin-password).- If `LANGFLOW_SUPERUSER_PASSWORD` isn't set, then the Langflow instance starts without authentication enabled.+ You can reuse the credentials and generated secrets from a `.env` created by the OpenRAG terminal UI. Do not reuse its `~/.openrag` path settings for checkout development: leave the repository-relative defaults in place for `OPENRAG_DOCUMENTS_PATH(S)`, `OPENRAG_KEYS_PATH`, `OPENRAG_FLOWS_PATH`, `OPENRAG_CONFIG_PATH`, `OPENRAG_DATA_PATH`, and `LANGFLOW_DATA_PATH`.For more information, see the [OpenRAG environment variables reference](https://docs.openr.ag/reference/configuration).-3. Start OpenRAG using one of the options described in the next section.- ```bash- make dev # With GPU support- # or- make dev-cpu # CPU only- ```+3. Start OpenRAG using one of the workflows described in the next section. For normal code changes, use [Local Development](#local-development-recommended-for-development).---@@ -184,13 +182,22 @@ You should see: `All required tools are installed.`There are multiple ways to start OpenRAG based on your use case:* Local development environment: Recommended for development.-* Full Docker stack: Simple build that runs everything in containers. Not ideal for development. Best for testing the full system.+* Full Docker stack: Run published images or build all images from the checkout. The latter is useful for testing the full system.* Branch development: Build OpenRAG with a fork or branch of the [Langflow repository](https://github.com/langflow-ai/langflow).* Docling only: Run the Docling service by itself.-### Full Docker Stack (Simplest)+### Keep terminal-managed versions and checked-out flows compatible++When you run `uv run openrag` from this repository, the OpenRAG terminal UI reads the version from `pyproject.toml` (via the installed package metadata) and writes that value to `OPENRAG_VERSION` before it starts services. Docker Compose then uses that value to pull the OpenRAG images from Docker Hub.++The flows used during development come from the checked-out source, not from those pulled images. Therefore, when starting services through `uv run openrag`, check out the release tag that matches the version in `pyproject.toml`. For example, a checkout containing version `0.5.0` must use the corresponding `0.5.0` release tag and images.++> [!WARNING]+> Do not run a newer or older branch's flows with a differently-versioned OpenRAG image. Flow components and their expected behavior can change between releases, so this mismatch can cause flows to fail or behave incorrectly.-Everything runs in containers. Best for testing the full system.+### Full Docker Stack (Published Images)++Everything runs in containers using the image tag configured by `OPENRAG_VERSION` (or `OPENRAG_IMAGE_TAG`). This is convenient for running a released version, but it does not use local backend or frontend code.```bashmake dev # Start with GPU support@@ -198,15 +205,16 @@ make dev-cpu # Start with CPU onlymake stop # Stop and remove all containers```-### B) Local Development (Recommended for Development)+### Local Development (Recommended for Development)> [!TIP]> This is the **recommended workflow** for active development. It provides faster code reloading and easier debugging.-Run infrastructure in Docker, but backend/frontend locally for faster iteration.+Run the infrastructure in Docker, but the backend and frontend from the current checkout. `make dev-local` and `make dev-local-cpu` rebuild the Langflow and OpenSearch images from the Dockerfiles in your branch, so the infrastructure stays aligned with the source you are developing.```bash-# Terminal 1: Start infrastructure (OpenSearch, Langflow, Dashboards)+# Terminal 1: Start infrastructure (OpenSearch, Langflow, Dashboards).+# Use make dev-local for GPU support.make dev-local-cpu# Terminal 2: Run backend locally@@ -215,18 +223,69 @@ make backend# Terminal 3: Run frontend locallymake frontend-# Terminal 4 (optional): Start docling for document processing+# Terminal 4: Start Docling for document processingmake docling```-**Benefits:**-- Faster code reloading-- Direct access to logs and debugging-- Easier testing and iteration+The frontend automatically picks up changes. Restart `make backend` after backend changes.> [!TIP]> When running the backend with `make backend`, you can access the interactive API documentation at http://localhost:8000/docs.+### Run two local OpenRAG instances++Use this configuration only when you have **two separate clones** of the OpenRAG repository on the same machine and want to run both at the same time. Each clone has its own local files, including its Langflow database.++> [!IMPORTANT]+> Do not run two OpenRAG stacks from the same clone. Even with different ports and Compose project names, they share the local database and other state, so this configuration is unsupported.++In the second clone, set a unique Compose project name and host ports in its `.env` file so its containers and services do not conflict with the first clone:++```env+# second OpenRAG clone: .env+OPENRAG_BACKEND_PORT=8001+COMPOSE_PROJECT_NAME=openrag-1++FRONTEND_PORT=3001+LANGFLOW_PORT=7861+OPENSEARCH_PORT=9201+OPENSEARCH_DASHBOARDS_PORT=5602+OPENSEARCH_PERF_PORT=9601++# Target the second frontend when running frontend end-to-end tests.+BASE_URL=http://localhost:3001++# Keep the Langflow link in the OpenRAG UI correct.+LANGFLOW_PUBLIC_URL=http://localhost:7861+```++Start the second clone with the same local-development commands, from that clone's repository root:++```bash+make dev-local-cpu+make backend+make frontend+```++Both instances can use the same host Docling process; start it once with `make docling`.++`BASE_URL` is used by the frontend end-to-end test tooling; it does not configure the running application.++### Full Docker Stack (Build from This Checkout)++To build and run every OpenRAG container from the current checkout, use a build target. The backend and frontend run in containers, so only Docling needs to be started separately on the host.++```bash+# GPU support+make dev-build++# Or CPU only+make dev-build-cpu++# In another terminal: start Docling+make docling+```+### C) Branch Development (Custom Langflow)If you need to test a Langflow change that hasn't shipped yet — your own fork/branch, or an upstream feature branch — use `dev-branch` instead of `dev`/`dev-cpu`. Instead of pulling the published Langflow image, it clones `REPO` at `BRANCH` and builds the Langflow image from source, then starts the rest of the stack normally:@@ -280,11 +339,12 @@ A quick-reference cheat sheet of the commands you'll reach for most while contri|---|---|| `make check_tools` | Verify Docker/Podman/Colima, Python, uv, Node.js, and npm are installed and meet version requirements. || `make setup` | Install backend + frontend dependencies and scaffold `.env` from `.env.example`. Run once, and again after pulling changes that touch dependencies. |-| `make dev` / `make dev-cpu` | Start the **full stack** in containers (GPU or CPU). Simplest option, best for just trying OpenRAG out — not ideal for iterating on code. |-| `make dev-local-cpu` | Start **infra only** (OpenSearch, Dashboards, Langflow) in containers, with the backend/frontend run on the host. This is the recommended loop for active development — see [Local Development](#b-local-development-recommended-for-development). |-| `make backend` | Run the FastAPI backend on the host (`uv run python src/main.py`), hot-reloading on code changes. Requires infra started via `make dev-local-cpu` first. |-| `make frontend` | Run the Next.js frontend on the host (`npx next dev`), hot-reloading on code changes. |-| `make docling` / `make docling-stop` | Start/stop the Docling service used for document parsing and OCR. |+| `make dev` / `make dev-cpu` | Start the **full stack** from the configured published images (GPU or CPU). |+| `make dev-build` / `make dev-build-cpu` | Build the complete stack from the current checkout, then run it in containers (GPU or CPU). Start Docling separately with `make docling`. |+| `make dev-local` / `make dev-local-cpu` | Build and start **infra only** (OpenSearch, Dashboards, Langflow) with the source in the current checkout; run backend/frontend on the host. This is the recommended active-development loop — see [Local Development](#local-development-recommended-for-development). |+| `make backend` | Run the FastAPI backend on the host (`uv run python src/main.py`). Restart it after backend changes. Requires infrastructure started via `make dev-local` or `make dev-local-cpu` first. |+| `make frontend` | Run the Next.js frontend on the host (`npx next dev`), with automatic reload on changes. |+| `make docling` / `make docling-stop` | Start/stop the host Docling service used for document parsing and OCR. || `make dev-branch BRANCH=<name>` | Build and run the full stack with a custom Langflow branch instead of the published image — see [Branch Development](#c-branch-development-custom-langflow). || `make stop` | Stop and remove all OpenRAG containers. || `make clean` | Stop containers and delete volumes (data is wiped). |@@ -351,13 +411,15 @@ make clear-os-data # Clear OpenSearch data directory completely### Full Factory Reset> [!CAUTION]-> This will delete all data, containers, and volumes. Use only when you need a completely fresh start.+> This stops and removes containers and volumes, then deletes local Langflow data, backend configuration and data, and generated JWT keys. Use only when you need a completely fresh start.++Before resetting, stop any host `make backend` and `make frontend` processes with <kbd>Ctrl</kbd>+<kbd>C</kbd> in their terminals. `make factory-reset` stops the infrastructure containers, but it does not stop those host processes; leaving them running prevents the replacement backend and frontend from binding to their ports.```bashmake factory-reset # Complete reset: containers, volumes, and data```-The reset also removes any legacy `opensearch-data` directory, so old OpenSearch index files do not linger after cleanup.+After confirming the prompt, start the development workflow again with `make dev-local` or `make dev-local-cpu`, then run `make backend`, `make frontend`, and `make docling` in separate terminals.---diff --git a/docs/docs/core-components/agents.mdx b/docs/docs/core-components/agents.mdxindex b3e747edc..eddd238fd 100644--- a/docs/docs/core-components/agents.mdx+++ b/docs/docs/core-components/agents.mdx@@ -24,13 +24,13 @@ All OpenRAG flows are designed to be modular, performant, and provider-agnostic.To view and modify a flow in OpenRAG, click <Icon name="Settings2" aria-hidden="true"/> **Settings**.From here, you can manage cloud storage connectors, model providers, and common parameters for the **Agent** and **Knowledge Ingestion** flows.-To further explore and edit flows, click **Edit in Langflow** to launch the embedded [Langflow visual editor](https://docs.langflow.org/concepts-overview) where you can fully [customize the flow](https://docs.langflow.org/concepts-flows) to suit your use case.+To further explore and edit flows, click **Edit in Langflow** to launch the embedded [Langflow visual editor](https://docs.langflow.org/concepts-overview), where you can customize the flow within the supported limits. For customization limits, unlocking instructions, and testing guidance, see [Editing flows in Langflow](/editing-in-langflow).:::tipAfter you click **Edit in Langflow**, you can access and edit all of OpenRAG's built-in flows from the Langflow editor's [**Projects** page](https://docs.langflow.org/concepts-flows#projects).You can also create your own flows.-To edit any flow, you must first [unlock the flow](https://docs.langflow.org/concepts-flows#lock-a-flow).+To edit any flow, you must first [unlock the flow](https://docs.langflow.org/concepts-flows#lock-a-flow). For the exact steps, see [Editing flows in Langflow](/editing-in-langflow).If you edit any flows other than the **Agent** or **Knowledge Ingestion** flows, it is recommended that you [export the flows](https://docs.langflow.org/concepts-flows-import) before editing so you can revert them to their original state if needed.:::@@ -49,9 +49,9 @@ To restore these flows to their original state, you must reimport the flow from## Build custom flows and use other Langflow functionality-In addition to OpenRAG's built-in flows, all Langflow features are available through OpenRAG, including the ability to [create your own flows](https://docs.langflow.org/concepts-flows) and popular extensibility features such as the following:+In addition to OpenRAG's built-in flows, OpenRAG provides access to Langflow functionality such as the ability to [create your own flows](https://docs.langflow.org/concepts-flows) and popular extensibility features such as the following. Available customizations vary by deployment; see [Editing flows in Langflow](/editing-in-langflow).-* [Create custom components](https://docs.langflow.org/components-custom-components).+* In supported local deployments, [create custom components](https://docs.langflow.org/components-custom-components).* Integrate with many third-party services through [bundles](https://docs.langflow.org/components-bundle-components).* Use [MCP clients](https://docs.langflow.org/mcp-client) and [MCP servers](https://docs.langflow.org/mcp-server), and serve flows as MCP tools for your agentic flows.@@ -73,4 +73,4 @@ If necessary, you can set a specific Langflow version with the `LANGFLOW_VERSION* Components might break, including components in OpenRAG's built-in flows.-* Default settings and behaviors might change causing unexpected results when OpenRAG expects a newer default.\ No newline at end of file+* Default settings and behaviors might change causing unexpected results when OpenRAG expects a newer default.diff --git a/docs/docs/core-components/editing-in-langflow.mdx b/docs/docs/core-components/editing-in-langflow.mdxnew file mode 100644index 000000000..f8581af40--- /dev/null+++ b/docs/docs/core-components/editing-in-langflow.mdx@@ -0,0 +1,25 @@+---+title: Editing flows in Langflow+slug: /editing-in-langflow+---++You can edit OpenRAG flows in the Langflow visual editor, subject to the following customization limits.++## Supported customization++* **SaaS:** Edit flows using components already available in Langflow. Custom components and custom component code are not supported.+* **Local deployments:** Custom components and custom component code are available only when you set `LANGFLOW_ALLOW_CUSTOM_COMPONENTS=true`. Otherwise, use components already available in Langflow.+* **Ingestion:** Adding custom metadata to the **OpenSearch** component in Langflow is not supported.+* **Recommended components:** Use **OpenRAG LLM** and **OpenRAG Embeddings** for language models and embeddings.++## Edit a flow++1. In OpenRAG, open **Settings** and click **Edit in Langflow**.+2. Open the **Projects** page and select the flow you want to edit.+3. Disable lock mode before editing: click the flow name in the header, then turn off the **Lock** toggle.+4. Save the unlock change by pressing <kbd>Command</kbd>+<kbd>S</kbd> (macOS) or <kbd>Ctrl</kbd>+<kbd>S</kbd> (Windows/Linux).+5. Modify the flow within the limits above, then save it with the same keyboard shortcut.+6. Test the flow through OpenRAG. The Langflow **Playground** and **Run** controls do not work for OpenRAG flows because OpenRAG passes the values required to run a flow at runtime. For example, test an Agent-flow change by starting a new conversation in OpenRAG.+7. After editing the **Agent** flow, start a new conversation in OpenRAG so previous conversation context is not reused.++For restore options, see [Use Langflow in OpenRAG](/agents#revert-a-built-in-flow-to-its-original-configuration).diff --git a/docs/docs/get-started/install-uv.mdx b/docs/docs/get-started/install-uv.mdxindex e5afbfd90..7e4566c5a 100644--- a/docs/docs/get-started/install-uv.mdx+++ b/docs/docs/get-started/install-uv.mdx@@ -97,6 +97,10 @@ If you encounter errors during installation, see [Troubleshoot OpenRAG](/support<PartialStartTui />+ :::note+ This terminal-managed workflow is for an installed OpenRAG package. To develop OpenRAG from a cloned source checkout, use the [checkout development workflow](/support/contribute#run-openrag-from-a-checkout) instead.+ :::+## Configure OpenRAG in the terminal {#setup}When you install OpenRAG with `uv`, you manage the OpenRAG services in an OpenRAG terminal session.@@ -111,4 +115,4 @@ Container definitions are stored in the `docker-compose` files in the same direc<PartialOnboarding />-<PartialInstallNextSteps />\ No newline at end of file+<PartialInstallNextSteps />diff --git a/docs/docs/get-started/tui.mdx b/docs/docs/get-started/tui.mdxindex 0cc3ee205..6426481d3 100644--- a/docs/docs/get-started/tui.mdx+++ b/docs/docs/get-started/tui.mdx@@ -19,6 +19,8 @@ If you installed OpenRAG with `uv`, start a terminal session with `uv run openraIf you installed OpenRAG with `uvx`, start a terminal session with `uvx openrag`.+If you are developing from a cloned OpenRAG source checkout, use the [checkout development workflow](/support/contribute#run-openrag-from-a-checkout) instead of the terminal-managed workflow.+### Start a GUI terminal session:::info@@ -66,4 +68,4 @@ If you are using the TUI (`--tui`), you can press <kbd>q</kbd> to quit the TUI.Exiting the terminal session doesn't stop your OpenRAG services.Your OpenRAG services continue to run until they are stopped from within the terminal session or by another process that inadvertently stops them.-To restart the terminal session, see [Start a terminal session](#access-the-tui).\ No newline at end of file+To restart the terminal session, see [Start a terminal session](#access-the-tui).diff --git a/docs/docs/support/contribute.mdx b/docs/docs/support/contribute.mdxindex eb87fad7b..02de72674 100644--- a/docs/docs/support/contribute.mdx+++ b/docs/docs/support/contribute.mdx@@ -68,6 +68,43 @@ If you want to contribute code to OpenRAG, you can do so by submitting a pull reSee [CONTRIBUTING.md](https://github.com/langflow-ai/openrag/blob/main/CONTRIBUTING.md) to set up your development environment and learn about the contribution process.+### Run OpenRAG from a checkout++For day-to-day development, run the infrastructure in containers and run the backend and frontend from your checked-out repository.++1. Copy `.env.example` to `.env`, and populate the required values. If you have a `.env` created by the OpenRAG terminal UI, you can reuse its credentials and generated secrets, but do not reuse the path variables that point to `~/.openrag`. Keep the repository-relative defaults for the flow, data, configuration, key, document, and Langflow-data paths.++2. In four terminals at the root of the repository, run the following commands:++ ```bash+ # Terminal 1: build Langflow and OpenSearch from this checkout, then start infra.+ # Use make dev-local instead when you need GPU support.+ make dev-local-cpu++ # Terminal 2+ make backend++ # Terminal 3+ make frontend++ # Terminal 4+ make docling+ ```++ The frontend reloads automatically when you edit it. Restart the backend process after backend changes.++3. To build the full container stack from the current checkout instead, run `make dev-build` (GPU) or `make dev-build-cpu` (CPU), then run `make docling` separately. In this mode, do not run `make backend` or `make frontend` because they are already running in containers.++4. To discard local service state and start over, first stop any host `make backend` and `make frontend` processes with <kbd>Ctrl</kbd>+<kbd>C</kbd> in their terminals. `make factory-reset` stops the infrastructure containers, but it does not stop those host processes; leaving them running prevents the replacement backend and frontend from binding to their ports. Then run `make factory-reset` and confirm the prompt. This removes the containers and volumes, local Langflow data, backend configuration and data, and generated JWT keys. Finally, repeat the local development commands.++:::warning Keep image versions and flows compatible++When you launch the terminal UI with `uv run openrag`, it takes the OpenRAG version from the package metadata for the checkout (derived from `pyproject.toml`) and writes it to `OPENRAG_VERSION`. Docker Compose uses that value to pull the OpenRAG images from Docker Hub.++The development flows come from the source you have checked out, rather than from the pulled OpenRAG image. Check out the release tag that corresponds to the version in `pyproject.toml` before you start services this way. Running an outdated image with newer checked-out flows, or the reverse, can break flow components and behavior.++:::+### Tips for successful submissions* Explain the motivation for your submission in the PR description.@@ -152,4 +189,4 @@ This can help you find broken links before creating a PR.10. Keep an eye on your pull request in case an OpenRAG maintainer requests changes or asks questions.- OpenRAG technical writers can directly edit documentation PRs to enforce style guidelines and fix errors.\ No newline at end of file+ OpenRAG technical writers can directly edit documentation PRs to enforce style guidelines and fix errors.diff --git a/docs/sidebars.js b/docs/sidebars.jsindex 5a65e51a5..d1d6f9313 100644--- a/docs/sidebars.js+++ b/docs/sidebars.js@@ -56,6 +56,7 @@ const sidebars = {id: "core-components/agents",label: "Flows",},+ "core-components/editing-in-langflow",{type: "category",label: "Knowledge",@@ -95,4 +96,4 @@ const sidebars = {],};-export default sidebars;\ No newline at end of file+export default sidebars;