High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
feat: smart API reference updater - merge code changes with user metadata without overwriting
Mario86513ce2a2b4
diff --git a/scripts/README.md b/scripts/README.mdindex c29d2faf..b916f92d 100644--- a/scripts/README.md+++ b/scripts/README.md@@ -6,39 +6,63 @@ Utility scripts for PinchTab development, testing, and documentation.### Overview-Automatically generate a complete JSON API reference from Go code.+Keep the API reference in sync with code changes **without losing user metadata**.### Quick Start```bash-# Generate endpoint list as JSON-go run scripts/gen-api-docs/main.go > docs/references/api-reference.json+# Check what changed (dry-run)+go run scripts/update-api-reference.go --dry-run-# Pretty-print-go run scripts/gen-api-docs/main.go | jq .+# Apply changes (adds new endpoints, removes old ones, keeps metadata)+go run scripts/update-api-reference.go-# Count endpoints-go run scripts/gen-api-docs/main.go | jq '.count'+# Query the result+jq '.endpoints | map(select(.cli == true)) | length' docs/references/api-reference.json+jq '.endpoints[] | select(.method == "POST") | .path' docs/references/api-reference.json+```++### Tools++#### `update-api-reference.go` (Main Tool - USE THIS)++**Smart merge tool** that updates the API reference without overwriting user-added metadata.-# Filter by method-go run scripts/gen-api-docs/main.go | jq '.endpoints[] | select(.method == "POST")'+**Features:**+- ✅ Scans code for current endpoints+- ✅ Reads existing `api-reference.json`+- ✅ **Merges intelligently:**+ - Keeps all user-added fields (description, examples, params, implementation status)+ - Adds new endpoints found in code+ - Removes endpoints no longer in code+ - Updates only method/path/handler+- ✅ Shows what changed (added/removed)+- ✅ Preserves all manual metadata++**Usage:**+```bash+# Dry-run first (show changes, no write)+go run scripts/update-api-reference.go --dry-run-# Count by method-go run scripts/gen-api-docs/main.go | jq '.endpoints | group_by(.method) | map({method: .[0].method, count: length})'+# Apply changes+go run scripts/update-api-reference.go```-### Tool: `gen-api-docs/main.go`+**Example Output:**+```+Found 66 endpoints in code+Found 66 endpoints in existing file+Merged to 66 endpoints+✓ No changes needed-Go-based tool using AST parsing. **Outputs structured JSON.**+Wrote docs/references/api-reference.json+```-**Features:**-- ✅ Parses all handler files (handlers, profiles, orchestrator, dashboard)-- ✅ Extracts HTTP method, path, and handler name-- ✅ Removes duplicate endpoints-- ✅ Sorts by method then path-- ✅ Outputs clean JSON for programmatic use+#### `gen-api-docs/main.go` (Reference)++Go-based tool using AST parsing. Use this **only** if you want to regenerate from scratch.-**Output Format:**+**Output:**```json{"version": "1.0",@@ -49,89 +73,102 @@ Go-based tool using AST parsing. **Outputs structured JSON.**"method": "GET","path": "/snapshot","handler": "HandleSnapshot"- },- {- "method": "POST",- "path": "/navigate",- "handler": "HandleNavigate"}]}```-### JSON File Structure+### Workflow: Keep API Reference in Sync-**Location:** `docs/references/api-reference.json`--**Fields:**-- `version` (string) — API reference version (e.g., "1.0")-- `generated` (string) — Generation info-- `count` (number) — Total number of endpoints-- `endpoints` (array) — List of endpoint objects- - `method` — HTTP method (GET, POST, PUT, DELETE, PATCH)- - `path` — URL path pattern (e.g., `/snapshot`, `/instances/{id}/action`)- - `handler` — Go handler function name+**Scenario 1: New endpoint added to code**+```bash+go run scripts/update-api-reference.go --dry-run+# Output:+# ✨ New endpoints:+# + POST /new-endpoint-### Use Cases+go run scripts/update-api-reference.go+# Adds /new-endpoint with basic fields, preserves all existing metadata+```-**Programmatic Access:**+**Scenario 2: Endpoint removed from code**```bash-# Find all POST endpoints-jq '.endpoints[] | select(.method == "POST") | .path'--# Find endpoints by path-jq '.endpoints[] | select(.path | contains("instances"))'+go run scripts/update-api-reference.go --dry-run+# Output:+# ❌ Removed endpoints:+# - DELETE /old-endpoint-# Find endpoints by handler-jq '.endpoints[] | select(.handler | contains("Handle"))'+go run scripts/update-api-reference.go+# Removes /old-endpoint, preserves all other data```-**Validation:**-```bash-# Verify total count-TOTAL=$(jq '.count' < docs/references/api-reference.json)-ACTUAL=$(jq '.endpoints | length' < docs/references/api-reference.json)-[ "$TOTAL" = "$ACTUAL" ] && echo "Count matches" || echo "Mismatch"+**Scenario 3: User adds metadata manually**+Edit `docs/references/api-reference.json` to add:+```json+{+ "method": "GET",+ "path": "/snapshot",+ "handler": "HandleSnapshot",+ "description": "Get accessibility tree",+ "cli": true,+ "curl": true,+ "examples": {+ "curl": "curl http://localhost:9867/snapshot?tabId=abc"+ }+}```-**CI/CD:**+Then run:```bash-# Generate in CI pipeline-go run scripts/gen-api-docs/main.go > /tmp/api-ref.json-# Commit if changed-git diff docs/references/api-reference.json && git add . && git commit -m "docs: update api reference"+go run scripts/update-api-reference.go+# Preserves all your edits, updates code-derived fields only```-### Technical Details+### JSON File Structure++**Location:** `docs/references/api-reference.json`++**Fields per endpoint:**+- `method` — HTTP method (GET, POST, PUT, DELETE, PATCH)+- `path` — URL path pattern (e.g., `/snapshot`, `/instances/{id}/action`)+- `handler` — Go handler function name+- `description` (optional) — What this endpoint does+- `cli` (optional) — CLI support flag+- `curl` (optional) — Curl support flag+- `params` (optional) — Parameter list with types/descriptions+- `examples` (optional) — curl/cli/payload examples+- `implemented` (optional) — Implementation status-The Go tool:-- Uses `go/parser` to parse Go source files-- Uses `go/ast` to traverse the syntax tree-- Finds all functions named `RegisterHandlers` or `RegisterRoutes`-- Extracts `mux.HandleFunc()` and `mux.Handle()` calls-- Parses route patterns with regex: `"METHOD /path"`-- Removes duplicates (same method + path)-- Sorts results for consistency-- Outputs JSON with proper formatting+### Query Examples-### Why JSON?+```bash+# List all endpoints with CLI support+jq '.endpoints[] | select(.cli == true) | {method, path}' docs/references/api-reference.json-- ✅ **Structured data** — Easy to parse programmatically-- ✅ **No styling** — Plain facts, no markdown formatting to maintain-- ✅ **Tooling** — Use `jq` or any JSON tool to query/filter-- ✅ **Automation** — CI/CD can validate endpoint counts, check for breaking changes-- ✅ **Flexibility** — Can generate docs in any format from JSON+# Count endpoints by method+jq '.endpoints | group_by(.method) | map({method: .[0].method, count: length})' docs/references/api-reference.json++# Find endpoints with documented examples+jq '.endpoints[] | select(.examples.curl != null) | .path' docs/references/api-reference.json++# Validate total count+TOTAL=$(jq '.count' docs/references/api-reference.json)+ACTUAL=$(jq '.endpoints | length' docs/references/api-reference.json)+[ "$TOTAL" = "$ACTUAL" ] && echo "✓ Count matches" || echo "✗ Mismatch"+```-### For Complete Documentation+### Why This Approach?-For full endpoint documentation with parameters, examples, response types, see:-- `docs/examples/API_DOCUMENTATION_EXAMPLES.md` — Complete workflow examples-- `docs/references/endpoints.md` — Full manual API reference-- `docs/references/curl-commands.md` — cURL examples+- ✅ **Automatic sync** — Code changes reflected without manual updates+- ✅ **User data preserved** — Descriptions, examples, status stay intact+- ✅ **Conflict-free** — Smart merge avoids overwriting metadata+- ✅ **Auditable** — Shows what changed with dry-run+- ✅ **Git-friendly** — Only changed endpoints in commits---## Other Scripts-Bash-based tools:-- `generate-api-docs.sh` — Regex-based endpoint extraction (fallback)+- `gen-api-docs/main.go` — Generate endpoints from scratch+- `generate-api-docs.sh` — Bash alternative (legacy)+- `ENDPOINT_METADATA.md` — Guide for adding structured endpoint docsdiff --git a/scripts/update-api-reference.go b/scripts/update-api-reference.gonew file mode 100644index 00000000..70e0dde6--- /dev/null+++ b/scripts/update-api-reference.go@@ -0,0 +1,364 @@+package main++import (+ "encoding/json"+ "flag"+ "fmt"+ "go/ast"+ "go/parser"+ "go/token"+ "io/ioutil"+ "log"+ "os"+ "regexp"+ "sort"+ "strings"+)++type Endpoint struct {+ Method string `json:"method"`+ Path string `json:"path"`+ Handler string `json:"handler"`+ Description string `json:"description,omitempty"`+ Implemented *bool `json:"implemented,omitempty"`+ CLI bool `json:"cli,omitempty"`+ Curl bool `json:"curl,omitempty"`+ Params []Parameter `json:"params,omitempty"`+ Examples ExampleGroup `json:"examples,omitempty"`+}++type Parameter struct {+ Name string `json:"name"`+ Type string `json:"type"`+ Location string `json:"location"`+ Required bool `json:"required"`+ Description string `json:"description"`+}++type ExampleGroup struct {+ Curl string `json:"curl,omitempty"`+ CLI string `json:"cli,omitempty"`+ Payload string `json:"payload,omitempty"`+}++type APIReference struct {+ Version string `json:"version"`+ Generated string `json:"generated"`+ Count int `json:"count"`+ Endpoints []Endpoint `json:"endpoints"`+}++func main() {+ dryRun := flag.Bool("dry-run", false, "Show changes without writing")+ flag.Parse()++ // Generate current endpoints from code+ currentEndpoints := generateEndpoints()+ fmt.Printf("Found %d endpoints in code\n", len(currentEndpoints))++ // Read existing API reference+ existingData := readExistingReference()+ fmt.Printf("Found %d endpoints in existing file\n", len(existingData.Endpoints))++ // Merge endpoints+ mergedEndpoints := mergeEndpoints(existingData.Endpoints, currentEndpoints)+ fmt.Printf("Merged to %d endpoints\n", len(mergedEndpoints))++ // Sort by method then path+ sort.Slice(mergedEndpoints, func(i, j int) bool {+ if mergedEndpoints[i].Method != mergedEndpoints[j].Method {+ return mergedEndpoints[i].Method < mergedEndpoints[j].Method+ }+ return mergedEndpoints[i].Path < mergedEndpoints[j].Path+ })++ // Build final reference+ result := APIReference{+ Version: "2.0",+ Generated: "auto-merged from code + user metadata",+ Count: len(mergedEndpoints),+ Endpoints: mergedEndpoints,+ }++ // Show changes+ showChanges(existingData.Endpoints, mergedEndpoints)++ if !*dryRun {+ // Write to file+ outputPath := "docs/references/api-reference.json"+ data, err := json.MarshalIndent(result, "", " ")+ if err != nil {+ log.Fatalf("Error marshaling JSON: %v", err)+ }++ err = ioutil.WriteFile(outputPath, append(data, '\n'), 0644)+ if err != nil {+ log.Fatalf("Error writing file: %v", err)+ }+ fmt.Printf("\nWrote %s\n", outputPath)+ } else {+ fmt.Println("\n(dry-run mode - no changes written)")+ }+}++func generateEndpoints() []Endpoint {+ var endpoints []Endpoint++ // Parse all handler files+ endpoints = append(endpoints, extractEndpoints("internal/handlers/handlers.go")...)+ endpoints = append(endpoints, extractEndpoints("internal/dashboard/dashboard.go")...)++ profiles, err := findProfilesFile()+ if err == nil {+ endpoints = append(endpoints, extractEndpoints(profiles)...)+ }++ orchestrator, err := findOrchestratorFile()+ if err == nil {+ endpoints = append(endpoints, extractEndpoints(orchestrator)...)+ }++ // Remove duplicates+ endpointMap := make(map[string]Endpoint)+ for _, ep := range endpoints {+ key := ep.Method + " " + ep.Path+ if _, exists := endpointMap[key]; !exists {+ endpointMap[key] = ep+ }+ }++ endpoints = make([]Endpoint, 0, len(endpointMap))+ for _, ep := range endpointMap {+ endpoints = append(endpoints, ep)+ }++ return endpoints+}++func readExistingReference() APIReference {+ path := "docs/references/api-reference.json"+ data, err := ioutil.ReadFile(path)+ if err != nil {+ fmt.Printf("Could not read existing file (%v), starting fresh\n", err)+ return APIReference{+ Version: "2.0",+ Generated: "auto-merged from code + user metadata",+ Endpoints: []Endpoint{},+ }+ }++ var ref APIReference+ err = json.Unmarshal(data, &ref)+ if err != nil {+ fmt.Printf("Error parsing existing file (%v), starting fresh\n", err)+ return APIReference{+ Version: "2.0",+ Generated: "auto-merged from code + user metadata",+ Endpoints: []Endpoint{},+ }+ }++ return ref+}++func mergeEndpoints(existing, current []Endpoint) []Endpoint {+ // Index existing endpoints by key+ existingMap := make(map[string]Endpoint)+ for _, ep := range existing {+ key := ep.Method + " " + ep.Path+ existingMap[key] = ep+ }++ // Index current endpoints by key+ currentMap := make(map[string]Endpoint)+ for _, ep := range current {+ key := ep.Method + " " + ep.Path+ currentMap[key] = ep+ }++ // Merge: keep existing fields, update method/path/handler+ merged := []Endpoint{}+ for _, current := range current {+ key := current.Method + " " + current.Path++ if existing, ok := existingMap[key]; ok {+ // Endpoint exists: keep all user metadata, update only code-derived fields+ merged = append(merged, Endpoint{+ Method: current.Method,+ Path: current.Path,+ Handler: current.Handler,+ Description: existing.Description, // Keep existing+ Implemented: existing.Implemented, // Keep existing+ CLI: existing.CLI, // Keep existing+ Curl: existing.Curl, // Keep existing+ Params: existing.Params, // Keep existing+ Examples: existing.Examples, // Keep existing+ })+ } else {+ // New endpoint: add with basic fields+ merged = append(merged, Endpoint{+ Method: current.Method,+ Path: current.Path,+ Handler: current.Handler,+ CLI: false,+ Curl: true,+ })+ }+ }++ return merged+}++func showChanges(existing, merged []Endpoint) {+ existingMap := make(map[string]bool)+ for _, ep := range existing {+ key := ep.Method + " " + ep.Path+ existingMap[key] = true+ }++ mergedMap := make(map[string]bool)+ added := []string{}+ removed := []string{}++ for _, ep := range merged {+ key := ep.Method + " " + ep.Path+ mergedMap[key] = true++ if !existingMap[key] {+ added = append(added, fmt.Sprintf("%s %s", ep.Method, ep.Path))+ }+ }++ for _, ep := range existing {+ key := ep.Method + " " + ep.Path+ if !mergedMap[key] {+ removed = append(removed, fmt.Sprintf("%s %s", ep.Method, ep.Path))+ }+ }++ if len(added) > 0 {+ fmt.Println("\n✨ New endpoints:")+ for _, ep := range added {+ fmt.Printf(" + %s\n", ep)+ }+ }++ if len(removed) > 0 {+ fmt.Println("\n❌ Removed endpoints:")+ for _, ep := range removed {+ fmt.Printf(" - %s\n", ep)+ }+ }++ if len(added) == 0 && len(removed) == 0 {+ fmt.Println("✓ No changes needed")+ }+}++func extractEndpoints(filePath string) []Endpoint {+ fset := token.NewFileSet()+ file, err := parser.ParseFile(fset, filePath, nil, parser.ParseComments)+ if err != nil {+ return []Endpoint{}+ }++ var endpoints []Endpoint++ for _, decl := range file.Decls {+ funcDecl, ok := decl.(*ast.FuncDecl)+ if !ok {+ continue+ }++ if funcDecl.Name.Name == "RegisterHandlers" || funcDecl.Name.Name == "RegisterRoutes" {+ endpoints = append(endpoints, extractHandleFuncCalls(funcDecl.Body)...)+ }+ }++ return endpoints+}++func extractHandleFuncCalls(body *ast.BlockStmt) []Endpoint {+ var endpoints []Endpoint+ routePattern := regexp.MustCompile(`^(GET|POST|PUT|DELETE|PATCH)\s+(/[^\s]*)`)++ for _, stmt := range body.List {+ exprStmt, ok := stmt.(*ast.ExprStmt)+ if !ok {+ continue+ }++ callExpr, ok := exprStmt.X.(*ast.CallExpr)+ if !ok {+ continue+ }++ selector, ok := callExpr.Fun.(*ast.SelectorExpr)+ if !ok {+ continue+ }++ if selector.Sel.Name != "HandleFunc" && selector.Sel.Name != "Handle" {+ continue+ }++ if len(callExpr.Args) < 1 {+ continue+ }++ basicLit, ok := callExpr.Args[0].(*ast.BasicLit)+ if !ok {+ continue+ }++ route := strings.Trim(basicLit.Value, "\"")+ matches := routePattern.FindStringSubmatch(route)+ if len(matches) < 3 {+ continue+ }++ method := matches[1]+ path := matches[2]++ var handler string+ if len(callExpr.Args) >= 2 {+ if selectorExpr, ok := callExpr.Args[1].(*ast.SelectorExpr); ok {+ handler = selectorExpr.Sel.Name+ }+ }++ endpoints = append(endpoints, Endpoint{+ Method: method,+ Path: path,+ Handler: handler,+ })+ }++ return endpoints+}++func findProfilesFile() (string, error) {+ matches := []string{+ "internal/profiles/handlers.go",+ "internal/profiles/profiles.go",+ }+ for _, path := range matches {+ if _, err := os.Stat(path); err == nil {+ return path, nil+ }+ }+ return "", fmt.Errorf("profiles file not found")+}++func findOrchestratorFile() (string, error) {+ matches := []string{+ "internal/orchestrator/handlers.go",+ "internal/orchestrator/orchestrator.go",+ }+ for _, path := range matches {+ if _, err := os.Stat(path); err == nil {+ return path, nil+ }+ }+ return "", fmt.Errorf("orchestrator file not found")+}