Skip to content

LLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows

[MISC] Add worktree setup script for automated env file copying (#1756)

Chandrasekharan M9e2f01c81bb9
diff --git a/.claude/skills/worktree/SKILL.md b/.claude/skills/worktree/SKILL.md
new file mode 100644
index 000000000..50f8fb5e5
--- /dev/null
+++ b/.claude/skills/worktree/SKILL.md
@@ -0,0 +1,96 @@
+---
+name: worktree
+description: Creates git worktrees for isolated development. Use when user wants to "create a worktree", "start a new feature branch", "work on a fix in isolation", or needs a parallel development environment.
+---
+
+# Git Worktree Skill
+
+Create isolated worktrees for parallel development. Ends by providing commands to continue in the new worktree.
+
+## Workflow
+
+1. **Detect repo info**
+ ```bash
+ REPO_NAME=$(basename -s .git $(git config --get remote.origin.url) 2>/dev/null || basename $(pwd))
+ SOURCE_REPO=$(git rev-parse --show-toplevel)
+ ```
+
+2. **Parse user description**
+ - Type: `fix` (bug, patch, fix) | `feat` (feature, add, implement) | `misc` (default)
+ - Branch: `{type}/{kebab-description}` (max 30 chars)
+ - Folder: `{type}-{kebab-description}`
+ - Path: `{SOURCE_REPO}/../{REPO_NAME}-worktrees/{folder}/`
+ - Version: `{kebab-description}` (for docker image tag)
+
+3. **Create worktree**
+ ```bash
+ git fetch origin main
+ mkdir -p "$(dirname "$WORKTREE_PATH")"
+ git worktree add -b "{branch}" "$WORKTREE_PATH" origin/main
+ ```
+
+4. **Copy config files**
+ - **unstract repo:** Run `"$SOURCE_REPO/.claude/skills/worktree/worktree-setup.sh" "$WORKTREE_PATH" "$SOURCE_REPO"`
+ - **Other repos:** Ask user which config files to copy, then copy them
+
+5. **Detect services from request** (unstract only)
+
+ Match keywords in user's request to services:
+
+ | Keywords | Services |
+ | ------------------------------ | -------------------------- |
+ | frontend, UI, react, component | frontend |
+ | backend, API, django | backend |
+ | platform, adapter, LLM | platform-service |
+ | prompt, studio | prompt-service |
+ | worker, celery, task | backend (includes workers) |
+ | x2text, extraction | x2text-service |
+
+ Default if unclear: `backend frontend`
+
+6. **Print summary with commands**
+
+ ```text
+ Worktree created!
+
+ Branch: {branch}
+ Path: {worktree_path}
+
+ ─────────────────────────────────────────────────────────
+ Continue in new worktree and paste your prompt:
+
+ cd {worktree_path} && claude
+
+ {user_request_slightly_made_better_for_new_agent_to_understand}
+ ─────────────────────────────────────────────────────────
+ Build & run services:
+
+ cd {worktree_path}/docker && \
+ VERSION={version} docker compose -f docker-compose.yaml -f compose.override.yaml build {services} && \
+ VERSION={version} docker compose -f docker-compose.yaml -f compose.override.yaml watch {services}
+ ─────────────────────────────────────────────────────────
+
+ Cleanup later: git worktree remove {worktree_path}
+ ```
+
+ IMPORTANT: Escape any quotes in `{user_request}` for shell safety.
+
+## Examples
+
+**User:** "worktree for fixing login validation in frontend"
+- Branch: `fix/login-validation`
+- Version: `login-validation`
+- Services: `frontend`
+
+**User:** "create worktree to add workflow export API"
+- Branch: `feat/workflow-export-api`
+- Version: `workflow-export-api`
+- Services: `backend`
+
+## Reference
+
+```bash
+git worktree list # List worktrees
+git worktree remove {path} # Remove worktree
+git worktree prune # Prune stale entries
+```
diff --git a/.claude/skills/worktree/worktree-setup.sh b/.claude/skills/worktree/worktree-setup.sh
new file mode 100755
index 000000000..db51c4e58
--- /dev/null
+++ b/.claude/skills/worktree/worktree-setup.sh
@@ -0,0 +1,194 @@
+#!/bin/bash
+#
+# Worktree Setup Script for Unstract
+# Copies necessary config files from source repo to a new worktree
+#
+# Usage: ./worktree-setup.sh <target_worktree_path> [source_repo_path]
+#
+
+set -euo pipefail
+
+show_usage() {
+ echo "Usage: $0 <target_worktree_path> [source_repo_path]"
+ echo ""
+ echo "Arguments:"
+ echo " target_worktree_path Path to the new worktree"
+ echo " source_repo_path Path to source repo (defaults to script's parent dir)"
+ exit "${1:-0}"
+}
+
+# Handle help flag
+if [ "$1" = "-h" ] || [ "$1" = "--help" ]; then
+ show_usage 0
+fi
+
+TARGET_PATH="$1"
+SOURCE_PATH="${2:-$(cd "$(dirname "$0")/../../.." && pwd)}"
+
+if [ -z "$TARGET_PATH" ]; then
+ show_usage 1
+fi
+
+# Convert to absolute paths (handle non-existent target)
+if [ -d "$TARGET_PATH" ]; then
+ TARGET_PATH=$(cd "$TARGET_PATH" && pwd)
+else
+ # Portable: construct absolute path relative to CWD without requiring realpath
+ TARGET_PATH="$(cd "$(dirname "$TARGET_PATH")" 2>/dev/null && pwd)/$(basename "$TARGET_PATH")" \
+ || TARGET_PATH="$(pwd)/$TARGET_PATH"
+fi
+if [ ! -d "$SOURCE_PATH" ]; then
+ echo "Error: source repo path does not exist: $SOURCE_PATH"
+ show_usage 1
+fi
+SOURCE_PATH=$(cd "$SOURCE_PATH" && pwd)
+
+echo "Copying config files from unstract repo..."
+echo " Source: $SOURCE_PATH"
+echo " Target: $TARGET_PATH"
+echo ""
+
+# Files to copy (relative to repo root)
+CONFIG_FILES=(
+ # Service .env files
+ "backend/.env"
+ "frontend/.env"
+ "platform-service/.env"
+ "prompt-service/.env"
+ "runner/.env"
+ "workers/.env"
+ "x2text-service/.env"
+ # Docker config files
+ "docker/.env"
+ "docker/essentials.env"
+ "docker/compose.override.yaml"
+ "docker/proxy_overrides.yaml"
+)
+
+copied=0
+skipped=0
+missing=0
+
+for file in "${CONFIG_FILES[@]}"; do
+ src="$SOURCE_PATH/$file"
+ dest="$TARGET_PATH/$file"
+
+ if [ -f "$src" ]; then
+ # Create directory if needed
+ mkdir -p "$(dirname "$dest")"
+
+ if [ -f "$dest" ]; then
+ # File exists, check if different
+ if ! diff -q "$src" "$dest" > /dev/null 2>&1; then
+ cp "$src" "$dest"
+ echo " [updated] $file"
+ copied=$((copied + 1))
+ else
+ echo " [skipped] $file (unchanged)"
+ skipped=$((skipped + 1))
+ fi
+ else
+ cp "$src" "$dest"
+ echo " [copied] $file"
+ copied=$((copied + 1))
+ fi
+ else
+ echo " [missing] $file (not found in source)"
+ missing=$((missing + 1))
+ fi
+done
+
+echo ""
+echo "Env files: Copied=$copied, Skipped=$skipped, Missing=$missing"
+echo ""
+
+# ============================================================
+# Step 2: Copy gitignored backend settings
+# ============================================================
+echo "Copying gitignored backend settings..."
+
+SETTINGS_DIR="backend/backend/settings"
+TRACKED_SETTINGS=("__init__.py" "base.py" "dev.py" "test.py")
+
+settings_copied=0
+settings_skipped=0
+
+if [ -d "$SOURCE_PATH/$SETTINGS_DIR" ]; then
+ for file in "$SOURCE_PATH/$SETTINGS_DIR"/*.py; do
+ [ -f "$file" ] || continue
+ filename=$(basename "$file")
+
+ # Skip tracked files (these come from git)
+ skip=false
+ for tracked in "${TRACKED_SETTINGS[@]}"; do
+ if [ "$filename" = "$tracked" ]; then
+ skip=true
+ break
+ fi
+ done
+ $skip && continue
+
+ # Copy gitignored settings file
+ dest="$TARGET_PATH/$SETTINGS_DIR/$filename"
+ mkdir -p "$(dirname "$dest")"
+
+ if [ -f "$dest" ]; then
+ if ! diff -q "$file" "$dest" > /dev/null 2>&1; then
+ cp "$file" "$dest"
+ echo " [updated] $filename"
+ settings_copied=$((settings_copied + 1))
+ else
+ echo " [skipped] $filename (unchanged)"
+ settings_skipped=$((settings_skipped + 1))
+ fi
+ else
+ cp "$file" "$dest"
+ echo " [copied] $filename"
+ settings_copied=$((settings_copied + 1))
+ fi
+ done
+else
+ echo " [warning] $SETTINGS_DIR not found in source"
+fi
+
+echo ""
+echo "Settings: Copied=$settings_copied, Skipped=$settings_skipped"
+echo ""
+
+# ============================================================
+# Step 3: Copy project-local Claude skills (gitignored ones)
+# ============================================================
+SKILLS_DIR=".claude/skills"
+skills_copied=0
+skills_skipped=0
+
+if [ -d "$SOURCE_PATH/$SKILLS_DIR" ]; then
+ echo "Copying project-local Claude skills..."
+
+ for skill_dir in "$SOURCE_PATH/$SKILLS_DIR"/*/; do
+ [ -d "$skill_dir" ] || continue
+ skill_name=$(basename "$skill_dir")
+ dest_dir="$TARGET_PATH/$SKILLS_DIR/$skill_name"
+
+ # Skip if already present and identical
+ if [ -d "$dest_dir" ] && diff -rq "$skill_dir" "$dest_dir" > /dev/null 2>&1; then
+ echo " [skipped] $skill_name/ (unchanged)"
+ skills_skipped=$((skills_skipped + 1))
+ continue
+ fi
+
+ mkdir -p "$dest_dir"
+ # Trailing-dot trick: copies all contents including hidden files, but not . or ..
+ cp -r "$skill_dir." "$dest_dir/" 2>/dev/null || true
+ echo " [copied] $skill_name/"
+ skills_copied=$((skills_copied + 1))
+ done
+
+ echo ""
+ echo "Skills: Copied=$skills_copied, Skipped=$skills_skipped"
+else
+ echo "No $SKILLS_DIR directory found in source, skipping."
+fi
+
+echo ""
+echo "Done!"
diff --git a/.gitignore b/.gitignore
index 80c568a0f..c96f27b53 100644
--- a/.gitignore
+++ b/.gitignore
@@ -8,6 +8,7 @@
!workers/run-worker.sh
!workers/run-worker-docker.sh
!workers/log_consumer/scheduler.sh
+!.claude/skills/worktree/worktree-setup.sh
### Django ###
*.log
@@ -688,6 +689,7 @@ prompting/
# Claude
CLAUDE.md
.claude/*
+!.claude/skills/
CONTRIBUTION_GUIDE.md
.mcp.json