Skip to content

Open source CapCut alternative.

TypeScript

feat(site): remove donkey vision

David Le7206990f3d69
diff --git a/README.md b/README.md
index 6939a987..2dc8b6b6 100644
--- a/README.md
+++ b/README.md
@@ -10,13 +10,9 @@
<a href="https://github.com/DonkeyUseCorp/Donkey/releases/latest"><img alt="Latest release" src="https://img.shields.io/github/v/release/DonkeyUseCorp/Donkey?label=release&color=EC7868" /></a>
<a href="LICENSE"><img alt="License: Apache 2.0" src="https://img.shields.io/badge/License-Apache_2.0-blue.svg" /></a>
<img alt="Platform: macOS" src="https://img.shields.io/badge/Platform-macOS-black.svg" />
- <a href="https://donkeycut.com/donkeyvision"><img alt="Vision API" src="https://img.shields.io/badge/Vision-API-EC7868.svg" /></a>
</p>
-Donkey is two things in one repository:
-
-- **Donkey Cut** — a free browser video editor at [donkeycut.com](https://donkeycut.com), powered by a local engine that ships inside the companion Mac app.
-- **Donkey Vision** — a screen-understanding API that turns any screenshot into structured, clickable UI, so you can build computer-use tools of your own.
+Donkey is **Donkey Cut** — a free browser video editor at [donkeycut.com](https://donkeycut.com), powered by a local engine that ships inside the companion Mac app.
---
@@ -84,59 +80,13 @@ The editor is free. Pay only for AI-generated media: the Pro plan ($20/month) ad
---
-## Donkey Vision
-
-Donkey Vision turns a screenshot into structured UI the way the app does. Send an image, get back every interactable element — buttons, icons, inputs, rows, text — each with a bounding box, a center point, and a label. Because it reads pixels, it works on software that exposes no API at all: native apps, the web, Electron, games, remote desktops, and VNC sessions.
-
-Pass a natural-language `instruction` like `click the play button` and it returns the matching target with a ready-to-click point.
-
-| Apple Music | Desktop |
-| --- | --- |
-| ![Apple Music parsed into UI elements](vision/bench/results/apple-music/apple-music.content.png) | ![macOS desktop parsed into UI elements](vision/bench/results/desktop/desktop.content.png) |
-
-### Example
-
-```bash
-curl "https://donkeycut.com/api/vision" \
- -H "Authorization: Bearer $DONKEY_API_KEY" \
- -H "Content-Type: application/json" \
- -d '{
- "image": "iVBORw0KGgo...", # base64 png/jpeg/webp, no "data:" prefix
- "instruction": "click the play button",
- "returnElements": true
- }'
-```
-
-```jsonc
-{
- "image": { "width": 1440, "height": 900 },
- "elements": [
- {
- "label": "Play",
- "kind": "button",
- "interactive": true,
- "box": { "x": 618, "y": 816, "width": 42, "height": 42 },
- "point": { "x": 639, "y": 837 }, // ready to click
- "confidence": 0.82
- }
- ],
- "target": { "label": "Play", "point": { "x": 639, "y": 837 }, "confidence": 0.91 }
-}
-```
-
-A warm parse takes about **0.7s** server-side. See [donkeycut.com/donkeyvision](https://donkeycut.com/donkeyvision) for the full reference and to get a key. The parsing model and its latency notes live in [`docs/guides/donkey-vision.md`](docs/guides/donkey-vision.md) and [`vision/`](vision/).
-
----
-
## Repository layout
| Path | What's there |
| --- | --- |
| [`apps/Donkey`](apps/Donkey) | The macOS companion app; it ships and supervises the Cut engine. |
-| [`site`](site) | The Next.js site, the Cut editor and engine, and hosted API routes (including Vision). |
-| [`vision`](vision) | The screenshot-parsing worker and its benchmarks. |
+| [`site`](site) | The Next.js site, the Cut editor and engine, and hosted API routes. |
| [`docs`](docs/README.md) | Supported product behavior and engineering guides. |
-| [`plans`](plans) | Active and historical implementation planning. |
## Build and run
@@ -172,7 +122,6 @@ The site uses Supabase Postgres through Prisma. Keep local credentials in `.env`
[`docs/README.md`](docs/README.md) is the source of truth for supported behavior. Good starting points:
- [Cut](docs/guides/cut/README.md) — the editor, its local engine, and the boundary between them.
-- [Donkey Vision](docs/guides/donkey-vision.md) — the screen-understanding layer.
- [Install Donkey Locally](docs/guides/install-donkey.md) — building the app bundle.
## License
diff --git a/docs/guides/backend-apis.md b/docs/guides/backend-apis.md
index 34cbd6f7..049dc269 100644
--- a/docs/guides/backend-apis.md
+++ b/docs/guides/backend-apis.md
@@ -1,8 +1,7 @@
# Backend API Guide
Backend APIs are the Next.js route handlers under the site project. They serve
-two callers: the site's own client views, signed in with a session cookie; and
-third-party Vision API developers, who authenticate with an API key.
+the site's own client views, signed in with a session cookie.
**The one rule:** every route handler is wrapped in `withDonkeyAuth`. A public
endpoint is a deliberate exception with a product reason — today only Better
@@ -12,15 +11,10 @@ a handler without the wrapper and the endpoint is open to anyone.
## Authentication
-`withDonkeyAuth` takes a session cookie by default. A route accepts a
-third-party API key only when it opts in with `allowApiKey: true`. That typed
-allowlist is the only way a key reaches a handler. If a handler instead branches
-on whether the request path starts with `/api/vision`, the allowlist has already
-been bypassed.
-
-Inside the handler, `request.donkey` carries who the caller is. Its `method`
-field says how they authenticated — by session cookie, API key, or dev bypass —
-and the handler branches on that, never on the path.
+`withDonkeyAuth` takes a session cookie. Inside the handler, `request.donkey`
+carries who the caller is. Its `method` field says how they authenticated — by
+session cookie or dev bypass — and the handler branches on that, never on the
+path.
A route that requires more than being signed in declares the role in the same
wrapper: `withSuperUser(handler)` rejects everyone else with a plain 403 before
@@ -183,37 +177,6 @@ usage metadata and normalized billable units. They must not keep prompts,
request bodies, screenshots, generated assets, provider output, output
references, or any other user content.
-## Third-Party Vision API
-
-`POST /api/vision` is also a standalone product for outside developers, sold
-separately from the Mac app. One handler serves both audiences, branching on how
-the caller authenticated:
-
-| Caller | `request.donkey.method` | Gate |
-|---|---|---|
-| Mac app | `session-cookie` | hosted credit balance |
-| Third-party developer | `api-key` | active subscription + monthly call quota |
-
-A third-party developer's calls never touch the money-credit balance.
-
-- Developers sign in with Google and subscribe through Stripe. The plan is a
- flat monthly subscription that grants a quota of API calls; the included count
- comes from the Stripe price metadata.
-- Keys come from Better Auth's API-key plugin, managed from account settings.
- Only a hash is stored, the secret is shown once, and creating a key requires
- an active subscription.
-- The route opts in with `allowApiKey: true`, enforces a per-key burst limit,
- and counts succeeded vision calls in the current period against the quota. A
- covered call is recorded as `billingMode: "included"` — it shows up in usage
- at zero money cost.
-- Stripe is the source of truth for subscription state. Its webhook,
- `POST /api/billing/webhook`, is signature-verified and therefore the public
- exception named in the one rule; it syncs the subscription lifecycle and the
- quota. Checkout, portal, subscription, and usage all live under
- `/api/billing/`.
-- The account views are client-rendered, and every data read goes through the
- audited TanStack Query hooks.
-
## Pattern
```typescript
diff --git a/docs/guides/frontend-nextjs-guidelines.md b/docs/guides/frontend-nextjs-guidelines.md
index 61035d53..6f84f07a 100644
--- a/docs/guides/frontend-nextjs-guidelines.md
+++ b/docs/guides/frontend-nextjs-guidelines.md
@@ -81,7 +81,7 @@ inline, the call belongs in a hook.
`src/queries/apiClient.ts`. Define each query's key as a constant alongside
its hook in the same module (export it if another module needs to invalidate
it). Mount `QueryProvider` once at the root layout.
-- Per-user account views (`/app/settings`, including API keys and usage) are
+- Per-user account views (`/app/settings`, including usage) are
client-rendered and read their data through these hooks. The route handlers
still enforce auth server-side, so a client guard is for UX, not security.
- Use database clients only from server-side code, and never run migrations or
diff --git a/site/.env.example b/site/.env.example
index a52f0024..f95b0f08 100644
--- a/site/.env.example
+++ b/site/.env.example
@@ -25,23 +25,14 @@ GOOGLE_APPLICATION_CREDENTIALS_JSON='{"type":"service_account",...}'
# keyless Vertex path above.
GEMINI_API_KEY="replace-with-gemini-api-key"
-# RunPod-hosted vision worker, served by /api/inference/vision (see
-# vision/README.md). Required for that route; unrelated to the gemini-flash
-# screenshot parser at /api/inference/screenshots/parse.
-RUNPOD_API_KEY="replace-with-runpod-api-key"
-RUNPOD_VISION_ENDPOINT_ID="replace-with-endpoint-id"
-
-# Stripe. Two separate subscription products plus one-time credit top-ups all
-# run through STRIPE_SECRET_KEY and the /api/billing/webhook endpoint (signed by
+# Stripe. The Pro subscription plus one-time credit top-ups run through
+# STRIPE_SECRET_KEY and the /api/billing/webhook endpoint (signed by
# STRIPE_WEBHOOK_SECRET).
-# STRIPE_VISION_PRICE_ID — recurring price for the Vision API plan; metadata
-# key "monthlyCallQuota" tunes the included call quota without a deploy.
# STRIPE_PRO_PRICE_ID — recurring price for the Donkey Pro app plan; the
# included monthly inference allowance equals this price (a $20/mo plan
# includes $20 of inference), read straight from the price's amount.
STRIPE_SECRET_KEY="replace-with-stripe-secret-key"
STRIPE_WEBHOOK_SECRET="replace-with-stripe-webhook-signing-secret"
-STRIPE_VISION_PRICE_ID="replace-with-stripe-price-id"
STRIPE_PRO_PRICE_ID="replace-with-stripe-pro-price-id"
STRIPE_PORTAL_CONFIGURATION_ID="replace-with-stripe-portal-configuration-id"
diff --git a/site/package-lock.json b/site/package-lock.json
index 8731adc4..f26773e5 100644
--- a/site/package-lock.json
+++ b/site/package-lock.json
@@ -18,7 +18,6 @@
"@aws-sdk/client-s3": "^3.1094.0",
"@aws-sdk/s3-request-presigner": "^3.1094.0",
"@base-ui/react": "^1.6.0",
- "@better-auth/api-key": "1.6.11",
"@better-auth/core": "1.6.11",
"@better-auth/kysely-adapter": "1.6.11",
"@better-auth/utils": "0.4.0",
@@ -1282,20 +1281,6 @@
}
}
},
- "node_modules/@better-auth/api-key": {
- "version": "1.6.11",
- "resolved": "https://registry.npmjs.org/@better-auth/api-key/-/api-key-1.6.11.tgz",
- "integrity": "sha512-717Bmbs1Y2h0KrPIwrutxI/HwdqKlga1a1OHlL5TrRbN35IxHeY3GilqvyI/92n7RxFGa/AQhIajX0OL+FJkvw==",
- "license": "MIT",
- "dependencies": {
- "zod": "^4.3.6"
- },
- "peerDependencies": {
- "@better-auth/core": "^1.6.11",
- "@better-auth/utils": "0.4.0",
- "better-auth": "^1.6.11"
- }
- },
"node_modules/@better-auth/core": {
"version": "1.6.11",
"resolved": "https://registry.npmjs.org/@better-auth/core/-/core-1.6.11.tgz",
diff --git a/site/package.json b/site/package.json
index aa61a241..a8b6bde1 100644
--- a/site/package.json
+++ b/site/package.json
@@ -34,7 +34,6 @@
"@aws-sdk/client-s3": "^3.1094.0",
"@aws-sdk/s3-request-presigner": "^3.1094.0",
"@base-ui/react": "^1.6.0",
- "@better-auth/api-key": "1.6.11",
"@better-auth/core": "1.6.11",
"@better-auth/kysely-adapter": "1.6.11",
"@better-auth/utils": "0.4.0",
diff --git a/site/prisma/Auth.prisma b/site/prisma/Auth.prisma
index b14a6f85..a909e6be 100644
--- a/site/prisma/Auth.prisma
+++ b/site/prisma/Auth.prisma
@@ -9,7 +9,7 @@ model User {
image String?
superUser Boolean @default(false)
// Stripe customer id, one per user, shared across every billing product
- // (Pro, Vision, one-time credit top-ups) so a customer is never duplicated.
+ // (Pro, one-time credit top-ups) so a customer is never duplicated.
stripeCustomerId String? @unique
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
@@ -22,7 +22,6 @@ model User {
creditLimits UserCreditLimit[]
creditAutoReload CreditAutoReload?
apiKeys Apikey[]
- visionApiSubscription VisionApiSubscription?
proSubscription ProSubscription?
avatar UserAvatar?
onboarding UserOnboarding?
@@ -88,3 +87,37 @@ model Verification {
@@index([identifier])
@@map("verification")
}
+
+// Developer API keys, held for a future API product. The shape mirrors the
+// Better Auth @better-auth/api-key plugin schema (generated via the Better
+// Auth CLI) so the plugin can own reads/writes when that product ships; a key
+// links to its owner via referenceId (= User.id). The one addition over the
+// generated shape is the User relation on referenceId, which gives
+// cascade-delete without changing any plugin field.
+model Apikey {
+ id String @id
+ configId String @default("default")
+ name String?
+ start String?
+ referenceId String
+ user User @relation(fields: [referenceId], references: [id], onDelete: Cascade)
+ prefix String?
+ key String
+ refillInterval Int?
+ refillAmount Int?
+ lastRefillAt DateTime?
+ enabled Boolean? @default(true)
+ rateLimitEnabled Boolean? @default(true)
+ rateLimitTimeWindow Int? @default(86400000)
+ rateLimitMax Int? @default(10)
+ requestCount Int? @default(0)
+ remaining Int?
+ lastRequest DateTime?
+ expiresAt DateTime?
+ createdAt DateTime
+ updatedAt DateTime
+ permissions String?
+ metadata String?
+
+ @@index([key])
+}
diff --git a/site/prisma/Billing.prisma b/site/prisma/Billing.prisma
index 99bd053d..5a1f7bd7 100644
--- a/site/prisma/Billing.prisma
+++ b/site/prisma/Billing.prisma
@@ -1,34 +1,7 @@
-// Third-party Vision API product: paid access tier gated by a Stripe
-// subscription that grants a monthly quota of API calls, plus API keys issued
-// to developers. This is separate from the Mac app's credit-funded inference;
-// VisionApiSubscription does not touch UserCreditAccount balances.
-
-model VisionApiSubscription {
- id String @id @default(cuid())
- userId String @unique
- user User @relation(fields: [userId], references: [id], onDelete: Cascade)
- stripeCustomerId String
- stripeSubscriptionId String? @unique
- status String @default("incomplete")
- planKey String @default("vision")
- monthlyCallQuota Int @default(0)
- // Atomic per-period call counter. Incremented under a conditional UPDATE so
- // concurrent requests can't exceed the quota; reset to 0 by the webhook when
- // the Stripe billing period rolls over. This is the authoritative API-call
- // count (never mixes in the Mac app's credit-billed vision calls).
- periodCallCount Int @default(0)
- currentPeriodStart DateTime?
- currentPeriodEnd DateTime?
- cancelAtPeriodEnd Boolean @default(false)
- createdAt DateTime @default(now())
- updatedAt DateTime @updatedAt
-}
-
-// Donkey Pro: the Mac app subscription. Kept entirely separate from the Vision
-// API product above. While active it includes a monthly inference allowance
-// equal to the plan price, granted into the user's credit balance as an
-// expiring grant (source "pro_subscription") so it is drawn down before
-// never-expiring purchased credits and does not roll over.
+// Donkey Pro: the Mac app subscription. While active it includes a monthly
+// inference allowance equal to the plan price, granted into the user's credit
+// balance as an expiring grant (source "pro_subscription") so it is drawn down
+// before never-expiring purchased credits and does not roll over.
// monthlyAllowanceMicros mirrors the included amount (the price), for display.
model ProSubscription {
id String @id @default(cuid())
@@ -47,37 +20,3 @@ model ProSubscription {
@@map("pro_subscription")
}
-
-// Mirrors the Better Auth @better-auth/api-key plugin schema (generated via the
-// Better Auth CLI). The plugin owns reads/writes through auth.api.* and links a
-// key to its owner via referenceId (= User.id). We keep verification
-// authoritative and run our own monthly quota, so the rate-limit/refill columns
-// stay unused. The only addition over the generated shape is the User relation
-// on referenceId, which gives cascade-delete without changing any plugin field.
-model Apikey {
- id String @id
- configId String @default("default")
- name String?
- start String?
- referenceId String
- user User @relation(fields: [referenceId], references: [id], onDelete: Cascade)
- prefix String?
- key String
- refillInterval Int?
- refillAmount Int?
- lastRefillAt DateTime?
- enabled Boolean? @default(true)
- rateLimitEnabled Boolean? @default(true)
- rateLimitTimeWindow Int? @default(86400000)
- rateLimitMax Int? @default(10)
- requestCount Int? @default(0)
- remaining Int?
- lastRequest DateTime?
- expiresAt DateTime?
- createdAt DateTime
- updatedAt DateTime
- permissions String?
- metadata String?
-
- @@index([key])
-}
diff --git a/site/prisma/Credits.prisma b/site/prisma/Credits.prisma
index ec327b75..2d52c90f 100644
--- a/site/prisma/Credits.prisma
+++ b/site/prisma/Credits.prisma
@@ -23,9 +23,8 @@ model UserCreditGrant {
source String
sourceId String?
// What the amounts below are denominated in. "credit" = USD micros that feed
- // UserCreditAccount.balanceMicros. "vision_call" = a raw integer count of
- // Vision API calls that lives off-balance (never touches balanceMicros and is
- // consumed by the Vision API quota path, not by dollar inference).
+ // UserCreditAccount.balanceMicros. Historical rows carry retired units, so
+ // balance paths filter on this column.
unit String @default("credit")
originalAmountMicros BigInt
remainingAmountMicros BigInt
diff --git a/site/public/donkeyvision/apple-music.jpg b/site/public/donkeyvision/apple-music.jpg
deleted file mode 100644
index 9e3c57ac..00000000
Binary files a/site/public/donkeyvision/apple-music.jpg and /dev/null differ
diff --git a/site/public/donkeyvision/desktop.jpg b/site/public/donkeyvision/desktop.jpg
deleted file mode 100644
index 830a1edc..00000000
Binary files a/site/public/donkeyvision/desktop.jpg and /dev/null differ
diff --git a/site/public/donkeyvision/spotify.jpg b/site/public/donkeyvision/spotify.jpg
deleted file mode 100644
index c3551d40..00000000
Binary files a/site/public/donkeyvision/spotify.jpg and /dev/null differ
diff --git a/site/src/app/_components/landing/ContactSalesButton.tsx b/site/src/app/_components/landing/ContactSalesButton.tsx
deleted file mode 100644
index dd3bd02e..00000000
--- a/site/src/app/_components/landing/ContactSalesButton.tsx
+++ /dev/null
@@ -1,32 +0,0 @@
-"use client";
-
-import type { ReactNode } from "react";
-
-type Props = {
- children: ReactNode;
- className: string;
-};
-
-const contactEmail = "[email protected]";
-const contactSubject = "Donkey Vision API";
-
-export function ContactSalesButton({ children, className }: Props) {
- const handleClick = () => {
- const gmailURL = new URL("https://mail.google.com/mail/");
- gmailURL.searchParams.set("view", "cm");
- gmailURL.searchParams.set("fs", "1");
- gmailURL.searchParams.set("to", contactEmail);
- gmailURL.searchParams.set("su", contactSubject);
-
- const opened = window.open(gmailURL.toString(), "_blank", "noopener,noreferrer");
- if (!opened) {
- window.location.href = `mailto:${contactEmail}?subject=${encodeURIComponent(contactSubject)}`;
- }
- };
-
- return (
- <button className={className} onClick={handleClick} type="button">
- {children}
- </button>
- );
-}
diff --git a/site/src/app/_components/landing/Footer.tsx b/site/src/app/_components/landing/Footer.tsx
index c018643b..f6a9767d 100644
--- a/site/src/app/_components/landing/Footer.tsx
+++ b/site/src/app/_components/landing/Footer.tsx
@@ -25,10 +25,7 @@ export function Footer() {
const linkGroups: FooterColumn[] = [
{
title: "Product",
- links: [
- { href: "/donkeyvision", label: "Donkey Vision API" },
- { href: GITHUB_REPO_URL, label: "GitHub" },
- ],
+ links: [{ href: GITHUB_REPO_URL, label: "GitHub" }],
},
{
title: "Legal",
diff --git a/site/src/app/_components/landing/TopNav.tsx b/site/src/app/_components/landing/TopNav.tsx
index 1d0af237..5449f896 100644
--- a/site/src/app/_components/landing/TopNav.tsx
+++ b/site/src/app/_components/landing/TopNav.tsx
@@ -12,8 +12,7 @@ const NAV_ICON_SIZE = 59;
type Props = {
homeHref?: string;
- // Wordmark next to the logo. Donkey Vision is its own B2B product, so that
- // page overrides the default "Donkey" with "Donkey Vision".
+ // Wordmark next to the logo.
wordmark?: string;
// Sign-in/up pages show a single toggle to the other mode. Auth otherwise
// lives on donkeycut.com, so the marketing nav carries no auth entry points.
diff --git a/site/src/app/api/api-keys/[id]/route.ts b/site/src/app/api/api-keys/[id]/route.ts
deleted file mode 100644
index c81f3669..00000000
--- a/site/src/app/api/api-keys/[id]/route.ts
+++ /dev/null
@@ -1,23 +0,0 @@
-import { NextResponse } from "next/server";
-
-import { auth } from "@/lib/auth";
-import { withDonkeyAuth } from "@/lib/donkey-api-auth";
-
-export const dynamic = "force-dynamic";
-
-type RouteContext = { params: Promise<{ id: string }> };
-
-// Revoke (delete) one of the signed-in user's API keys. Better Auth enforces
-// ownership against the session, so a user can only delete their own keys.
-export const DELETE = withDonkeyAuth(
- async (request, context: RouteContext) => {
- const { id } = await context.params;
-
- await auth.api.deleteApiKey({
- body: { keyId: id },
- headers: request.headers,
- });
-
- return NextResponse.json({ deleted: true });
- },
-);
diff --git a/site/src/app/api/api-keys/route.ts b/site/src/app/api/api-keys/route.ts
deleted file mode 100644
index d12bbbe7..00000000
--- a/site/src/app/api/api-keys/route.ts
+++ /dev/null
@@ -1,94 +0,0 @@
-import { NextResponse } from "next/server";
-import { z } from "zod";
-
-import { auth, visionApiKeyPrefix } from "@/lib/auth";
-import {
- donkeySessionUserId,
- withDonkeyAuth,
-} from "@/lib/donkey-api-auth";
-import { validationErrorResponse } from "@/lib/inference/responses";
-
-export const dynamic = "force-dynamic";
-
-function unauthorized() {
- return NextResponse.json({ error: "Unauthorized" }, { status: 401 });
-}
-
-const createApiKeySchema = z.object({
- name: z.string().trim().min(1).max(120),
-});
-
-type ApiKeyRecord = {
- id: string;
- name: string | null;
- start: string | null;
- prefix: string | null;
- enabled: boolean | null;
- referenceId: string;
- createdAt: Date;
- lastRequest: Date | null;
- expiresAt: Date | null;
-};
-
-function toSafeApiKey(key: ApiKeyRecord) {
- return {
- createdAt: key.createdAt,
- enabled: key.enabled ?? true,
- expiresAt: key.expiresAt,
- id: key.id,
- lastRequest: key.lastRequest,
- name: key.name,
- // start + prefix let the UI show e.g. "dk_live_abc…" without the secret.
- prefix: key.prefix,
- start: key.start,
- };
-}
-
-// List the signed-in user's Vision API keys (secrets never returned). The
-// referenceId filter is defense-in-depth: the plugin already scopes by session,
-// but we never return a key the session user does not own.
-export const GET = withDonkeyAuth(async (request) => {
- const userId = donkeySessionUserId(request);
- if (!userId) {
- return unauthorized();
- }
-
- const result = (await auth.api.listApiKeys({
- headers: request.headers,
- })) as { apiKeys: ApiKeyRecord[] } | ApiKeyRecord[];
- const keys = Array.isArray(result) ? result : result.apiKeys;
- const ownKeys = keys.filter((key) => key.referenceId === userId);
-
- return NextResponse.json({ apiKeys: ownKeys.map(toSafeApiKey) });
-});
-
-// Create a new key. Any signed-in user can mint one; the key only returns data
-// when the user has capacity (an active subscription or remaining vision-call
-// grants), which the Vision API route enforces per call. The full secret is
-// returned exactly once here and never again.
-export const POST = withDonkeyAuth(async (request) => {
- const userId = donkeySessionUserId(request);
- if (!userId) {
- return unauthorized();
- }
-
- const body = await request.json().catch(() => null);
- const parsed = createApiKeySchema.safeParse(body);
- if (!parsed.success) {
- return validationErrorResponse(parsed.error);
- }
-
- const created = await auth.api.createApiKey({
- body: {
- name: parsed.data.name,
- prefix: visionApiKeyPrefix,
- },
- headers: request.headers,
- });
-
- return NextResponse.json({
- apiKey: toSafeApiKey(created as ApiKeyRecord),
- // The plaintext secret — display once, store securely client-side, then drop.
- secret: created.key,
- });
-});
diff --git a/site/src/app/api/billing/checkout/route.ts b/site/src/app/api/billing/checkout/route.ts
index 74e12eca..24f6648f 100644
--- a/site/src/app/api/billing/checkout/route.ts
+++ b/site/src/app/api/billing/checkout/route.ts
@@ -2,11 +2,7 @@ import { NextResponse } from "next/server";
import { auth } from "@/lib/auth";
import { proPriceId } from "@/lib/billing/pro-subscription";
-import {
- ensureStripeCustomer,
- getStripe,
- visionPriceId,
-} from "@/lib/billing/stripe";
+import { ensureStripeCustomer, getStripe } from "@/lib/billing/stripe";
import {
notFoundResponse,
unauthorizedResponse,
@@ -15,35 +11,17 @@ import {
export const dynamic = "force-dynamic";
-// Resolve the Stripe price for a self-serve subscription plan, or null if the
-// plan is unknown or not configured. Pro and Vision are separate products.
-function subscriptionPriceId(planKey: string): string | null {
- if (planKey === "vision") {
- return visionPriceId();
- }
- if (planKey === "pro") {
- return proPriceId() ?? null;
- }
- return null;
-}
-
-// Start a Stripe Checkout session for a subscription (Vision API or Pro).
-// Session-only (API keys are not accepted here, the default for withDonkeyAuth).
-// This route keeps its getSession call because it needs the user's email/name.
+// Start a Stripe Checkout session for a Pro subscription. This route keeps its
+// getSession call because it needs the user's email/name.
export const POST = withDonkeyAuth(async (request) => {
const session = await auth.api.getSession({ headers: request.headers });
if (!session) {
return unauthorizedResponse();
}
- const body = (await request.json().catch(() => ({}))) as {
- planKey?: unknown;
- };
- const planKey =
- typeof body.planKey === "string" ? body.planKey : "vision";
- // An unknown or not-yet-configured plan stays a 404 (not 401) so the landing
- // card does not treat it as a sign-in prompt.
- const priceId = subscriptionPriceId(planKey);
+ // A not-yet-configured price stays a 404 (not 401) so the landing card does
+ // not treat it as a sign-in prompt.
+ const priceId = proPriceId();
if (!priceId) {
return notFoundResponse();
}
diff --git a/site/src/app/api/billing/portal/route.ts b/site/src/app/api/billing/portal/route.ts
index d7ba21c3..a9f0d2af 100644
--- a/site/src/app/api/billing/portal/route.ts
+++ b/site/src/app/api/billing/portal/route.ts
@@ -1,6 +1,6 @@
import { NextResponse } from "next/server";
-import { getStripe, visionPortalConfigurationId } from "@/lib/billing/stripe";
+import { getStripe, portalConfigurationId } from "@/lib/billing/stripe";
import {
donkeySessionUserId,
notFoundResponse,
@@ -18,26 +18,17 @@ export const POST = withDonkeyAuth(async (request) => {
return unauthorizedResponse();
}
- // Pro customers carry their Stripe id on User (ensureStripeCustomer);
- // early Vision-only customers may still have it on their subscription row.
const user = await prisma.user.findUnique({
where: { id: userId },
select: { stripeCustomerId: true },
});
- let customerId = user?.stripeCustomerId ?? null;
- if (!customerId) {
- const vision = await prisma.visionApiSubscription.findUnique({
- where: { userId },
- select: { stripeCustomerId: true },
- });
- customerId = vision?.stripeCustomerId ?? null;
- }
+ const customerId = user?.stripeCustomerId ?? null;
if (!customerId) {
return notFoundResponse();
}
const stripe = getStripe();
- const configuration = visionPortalConfigurationId();
+ const configuration = portalConfigurationId();
const portal = await stripe.billingPortal.sessions.create({
customer: customerId,
return_url: `${request.nextUrl.origin}/app/settings`,
diff --git a/site/src/app/api/billing/subscription/route.ts b/site/src/app/api/billing/subscription/route.ts
deleted file mode 100644
index b0bbefec..00000000
--- a/site/src/app/api/billing/subscription/route.ts
+++ /dev/null
@@ -1,39 +0,0 @@
-import { NextResponse } from "next/server";
-
-import { isActiveVisionStatus } from "@/lib/billing/vision-subscription";
-import {
- donkeySessionUserId,
- unauthorizedResponse,
- withDonkeyAuth,
-} from "@/lib/donkey-api-auth";
-import { prisma } from "@/lib/prisma";
-
-export const dynamic = "force-dynamic";
-
-// Current Vision API subscription for the signed-in user, or null. Returns
-// isActive so clients don't re-derive the active-status rule.
-export const GET = withDonkeyAuth(async (request) => {
- const userId = donkeySessionUserId(request);
- if (!userId) {
- return unauthorizedResponse();
- }
-
- const subscription = await prisma.visionApiSubscription.findUnique({
- where: { userId },
- });
-
- if (!subscription?.stripeSubscriptionId) {
- return NextResponse.json({ subscription: null });
- }
-
- return NextResponse.json({
- subscription: {
- cancelAtPeriodEnd: subscription.cancelAtPeriodEnd,
- currentPeriodEnd: subscription.currentPeriodEnd?.toISOString() ?? null,
- isActive: isActiveVisionStatus(subscription.status),
- monthlyCallQuota: subscription.monthlyCallQuota,
- planKey: subscription.planKey,
- status: subscription.status,
- },
- });
-});
diff --git a/site/src/app/api/billing/usage/route.ts b/site/src/app/api/billing/usage/route.ts
index c5e45147..7c6f03b9 100644
--- a/site/src/app/api/billing/usage/route.ts
+++ b/site/src/app/api/billing/usage/route.ts
@@ -4,7 +4,6 @@ import { Prisma } from "@/generated/prisma/client";
import { creditMicrosToString } from "@/lib/credits/amounts";
import { readInferenceUsageBreakdown } from "@/lib/credits/inference";
-import { visionCallGrantRemaining } from "@/lib/credits/vision-grants";
import {
donkeySessionUserId,
unauthorizedResponse,
@@ -14,9 +13,8 @@ import { prisma } from "@/lib/prisma";
export const dynamic = "force-dynamic";
-// Up to 5 pages of 25 in the usage tab — fetched per product so a busy product
-// can't starve the other tab out of the recent window.
-const recentPerProduct = 125;
+// Up to 5 pages of 25 in the usage tab.
+const recentLimit = 125;
const recentSelect = {
billingStatus: true,
@@ -34,7 +32,7 @@ type RecentEvent = Prisma.InferenceUsageEventGetPayload<{
select: typeof recentSelect;
}>;
-function toRecentCall(event: RecentEvent, product: "app" | "vision") {
+function toRecentCall(event: RecentEvent) {
return {
billingStatus: event.billingStatus,
// The app conversation this call belongs to; null for background work and
@@ -45,7 +43,6 @@ function toRecentCall(event: RecentEvent, product: "app" | "vision") {
createdAt: event.createdAt.toISOString(),
errorCode: event.errorCode,
model: event.model,
- product,
requestKind: event.requestKind,
status: event.status,
// Per-call token breakdown — what actually drove the cost. Large input =
@@ -54,50 +51,21 @@ function toRecentCall(event: RecentEvent, product: "app" | "vision") {
};
}
-// Current-period Vision API call usage vs quota, plus recent calls, for the
-// settings UI. "included" billingStatus is the third-party Vision API (API-key)
-// product; everything else is app usage billed to the user's credits — including
-// the app's own vision_parse calls on the same route. Fetching the two products
-// separately keeps each tab's history independent.
+// Recent credit-billed inference calls for the settings usage UI.
export const GET = withDonkeyAuth(async (request) => {
const userId = donkeySessionUserId(request);
if (!userId) {
return unauthorizedResponse();
}
- const [subscription, appRecent, visionRecent, extraRemaining] =
- await Promise.all([
- prisma.visionApiSubscription.findUnique({ where: { userId } }),
- prisma.inferenceUsageEvent.findMany({
- orderBy: { createdAt: "desc" },
- select: recentSelect,
- take: recentPerProduct,
- where: { billingStatus: { not: "included" }, userId },
- }),
- prisma.inferenceUsageEvent.findMany({
- orderBy: { createdAt: "desc" },
- select: recentSelect,
- take: recentPerProduct,
- where: { billingStatus: "included", userId },
- }),
- // One-time vision-call grants (signup bonus, top-ups) on top of any plan.
- visionCallGrantRemaining(userId),
- ]);
-
- const limit = subscription?.monthlyCallQuota ?? 0;
- const used = subscription?.periodCallCount ?? 0;
+ const recent = await prisma.inferenceUsageEvent.findMany({
+ orderBy: { createdAt: "desc" },
+ select: recentSelect,
+ take: recentLimit,
+ where: { billingStatus: { not: "included" }, userId },
+ });
return NextResponse.json({
- // Extra calls from grants, spent after the subscription quota runs out.
- extraRemaining: Number(extraRemaining),
- limit,
- periodEnd: subscription?.currentPeriodEnd?.toISOString() ?? null,
- periodStart: subscription?.currentPeriodStart?.toISOString() ?? null,
- recent: [
- ...appRecent.map((event) => toRecentCall(event, "app")),
- ...visionRecent.map((event) => toRecentCall(event, "vision")),
- ],
- remaining: Math.max(0, limit - used),
- used,
+ recent: recent.map(toRecentCall),
});
});
diff --git a/site/src/app/api/billing/webhook/route.ts b/site/src/app/api/billing/webhook/route.ts
index 9c0c35a7..9bc695e7 100644
--- a/site/src/app/api/billing/webhook/route.ts
+++ b/site/src/app/api/billing/webhook/route.ts
@@ -11,23 +11,17 @@ import {
subscriptionIsPro,
syncProSubscription,
} from "@/lib/billing/pro-subscription";
-import {
- getStripe,
- stripeId,
- syncVisionSubscription,
-} from "@/lib/billing/stripe";
+import { getStripe, stripeId } from "@/lib/billing/stripe";
import { notFoundResponse } from "@/lib/donkey-api-auth";
export const dynamic = "force-dynamic";
-// Two subscription products share these webhook events; route each Stripe
-// subscription to the right sync by its price (Pro vs Vision, the default).
+// Pro is the only subscription product; ignore any other Stripe subscription
+// that reaches these webhook events.
async function syncSubscription(subscription: Stripe.Subscription) {
if (subscriptionIsPro(subscription)) {
await syncProSubscription(subscription);
- return;
}
- await syncVisionSubscription(subscription);
}
// Public, signature-verified Stripe webhook. This is an intentional exception to
diff --git a/site/src/app/api/vision/route.ts b/site/src/app/api/vision/route.ts
deleted file mode 100644
index 8f70e19a..00000000
--- a/site/src/app/api/vision/route.ts
+++ /dev/null
@@ -1,242 +0,0 @@
-import { NextResponse } from "next/server";
-
-import {
- creditUsageHeaders,
- inferenceUsageRoutes,
- recordFailedInferenceUsage,
- recordInferenceUsage,
- requireInferenceCredits,
-} from "@/lib/credits/inference";
-import {
- getActiveVisionSubscription,
- releaseVisionApiCall,
- reserveVisionApiCall,
-} from "@/lib/billing/vision-subscription";
-import {
- releaseVisionCallGrant,
- reserveVisionCallGrant,
-} from "@/lib/credits/vision-grants";
-import {
- type DonkeyAuthenticatedRequest,
- shouldBypassDonkeyInferenceCredits,
- withDonkeyAuth,
-} from "@/lib/donkey-api-auth";
-import { InferenceProviderError } from "@/lib/inference/providers";
-import {
- checkInMemoryRateLimit,
- rateLimitResponse,
-} from "@/lib/inference/rate-limit";
-import { readLimitedJsonBody } from "@/lib/inference/request-body";
-import {
- inferenceErrorCode,
- inferenceProviderErrorResponse,
- requireInferenceClientId,
- validationErrorResponse,
-} from "@/lib/inference/responses";
-import { groundInstruction } from "@/lib/inference/vision/grounding";
-import { parseImage } from "@/lib/inference/vision/parse";
-import {
- visionRequestSchema,
- type VisionResponse,
-} from "@/lib/inference/vision/schema";
-
-export const dynamic = "force-dynamic";
-
-const visionMaxBodyBytes = 6_250_000;
-const visionRateLimit = {
- limit: 4,
- windowMs: 3_000,
-};
-// Third-party API keys are capped at 3 requests/second (the monthly call quota
-// is the real spend cap; this is a burst guard).
-const visionApiKeyRateLimit = {
- limit: 3,
- windowMs: 1_000,
-};
-const inferenceProvider = "omniparser";
-const parseModel = "omniparser-v2";
-
-// How an api-key call's capacity was reserved, so the right bucket is credited
-// back if the request fails: the subscription's monthly quota or a one-time
-// vision-call grant.
-type VisionReservation =
- | { kind: "subscription"; limit: number; remaining: number }
- | { kind: "grant"; grantId: string; remaining: number };
-
-// This route accepts third-party API keys in addition to app sessions.
-export const POST = withDonkeyAuth(visionParseHandler, { allowApiKey: true });
-
-// B2B vision API: parse a screenshot into UI elements, and optionally ground a
-// natural-language instruction to a click target. Single-shot only.
-//
-// Two auth paths share this handler:
-// - api-key: third-party developers, gated by an active Vision API
-// subscription + monthly call quota (no money-credit charge).
-// - session-cookie / dev-bypass: the Mac app, gated by the credit balance.
-async function visionParseHandler(request: DonkeyAuthenticatedRequest) {
- const isApiKey = request.donkey.method === "api-key";
-
- const client = requireInferenceClientId(request.donkey.clientId);
- if (!client.ok) {
- return client.response;
- }
-
- // Bucket API-key callers by their key id, not the client-supplied
- // x-donkey-client-id header — otherwise a caller could rotate the header to
- // get a fresh burst bucket per request and defeat the limit.
- const rateLimitKey = isApiKey
- ? `apikey:${request.donkey.apiKeyId}`
- : `${request.donkey.userId}:${client.clientId}`;
- const rateLimit = checkInMemoryRateLimit({
- key: rateLimitKey,
- ...(isApiKey ? visionApiKeyRateLimit : visionRateLimit),
- });
- if (!rateLimit.ok) {
- return rateLimitResponse(rateLimit.retryAfterSeconds);
- }
-
- const body = await readLimitedJsonBody(request, visionMaxBodyBytes);
- if (!body.ok) {
- return body.response;
- }
-
- const parsed = visionRequestSchema.safeParse(body.json);
- if (!parsed.success) {
- return validationErrorResponse(parsed.error);
- }
- const { image, instruction, model, returnElements, options } = parsed.data;
- const billedModel = instruction ? model : parseModel;
-
- // The dev-bypass user has no real account, so skip the credit preflight and
- // usage recording (both reference a real user row) when bypassing auth.
- const bypassCredits = shouldBypassDonkeyInferenceCredits(request.donkey);
- // For the api-key path, the reserved call is released if the request later
- // fails, so a failed parse doesn't burn the developer's subscription quota or
- // their grant allotment.
- let visionReservation: VisionReservation | null = null;
- if (isApiKey) {
- const userId = request.donkey.userId;
- // Spend the subscription's monthly quota first (it resets each period), then
- // fall back to one-time vision-call grants.
- const subscription = await getActiveVisionSubscription(userId);
- if (subscription) {
- const reserved = await reserveVisionApiCall({
- monthlyCallQuota: subscription.monthlyCallQuota,
- userId,
- });
- if (reserved.ok) {
- visionReservation = {
- kind: "subscription",
- limit: subscription.monthlyCallQuota,
- remaining: reserved.remaining,
- };
- }
- }
- if (!visionReservation) {
- const granted = await reserveVisionCallGrant(userId);
- if (granted.ok) {
- visionReservation = {
- grantId: granted.grantId,
- kind: "grant",
- remaining: Number(granted.remaining),
- };
- }
- }
- if (!visionReservation) {
- // Valid key, but no capacity on either the subscription or grants.
- // Operational limit, not an auth failure — keep it distinct from a bad key.
- return NextResponse.json(
- {
- error: "quota_exceeded",
- ...(subscription ? { limit: subscription.monthlyCallQuota } : {}),
- },
- { status: 402 },
- );
- }
- } else if (!bypassCredits) {
- const credits = await requireInferenceCredits({
- model: billedModel,
- provider: inferenceProvider,
- route: inferenceUsageRoutes.vision,
- userId: request.donkey.userId,
- });
- if (!credits.ok) {
- return credits.response;
- }
- }
-
- try {
- const parseResult = await parseImage(image, options);
-
- const grounding = instruction
- ? await groundInstruction(parseResult.elements, instruction, model)
- : null;
-
- const includeElements = returnElements ?? instruction === undefined;
- const result: VisionResponse = {
- image: parseResult.image,
- ...(includeElements ? { elements: parseResult.elements } : {}),
- ...(grounding
- ? { target: grounding.target, alternates: grounding.alternates, model }
- : {}),
- };
-
- let usageHeaders: Record<string, string> = {};
- if (bypassCredits) {
- usageHeaders["X-Donkey-Dev-Auth-Bypass"] = "true";
- } else {
- const recordedUsage = await recordInferenceUsage({
- billingMode: isApiKey ? "included" : "credits",
- clientId: client.clientId,
- conversationId: request.donkey.conversationId,
- metadata: { grounded: String(instruction !== undefined) },
- model: billedModel,
- provider: inferenceProvider,
- requestKind: "vision_parse",
- route: inferenceUsageRoutes.vision,
- status: "succeeded",
- userId: request.donkey.userId,
- });
- usageHeaders = visionReservation
- ? {
- "X-Donkey-Calls-Remaining": String(visionReservation.remaining),
- ...(visionReservation.kind === "subscription"
- ? { "X-Donkey-Calls-Limit": String(visionReservation.limit) }
- : {}),
- }
- : creditUsageHeaders(recordedUsage);
- }
-
- return NextResponse.json(result, {
- headers: usageHeaders,
- });
- } catch (error) {
- // Give the reserved call back — a failed parse shouldn't be billed.
- if (visionReservation) {
- if (visionReservation.kind === "subscription") {
- await releaseVisionApiCall(request.donkey.userId);
- } else {
- await releaseVisionCallGrant(visionReservation.grantId);
- }
- }
- if (!bypassCredits) {
- await recordFailedInferenceUsage({
- billingMode: isApiKey ? "included" : "credits",
- clientId: client.clientId,
- conversationId: request.donkey.conversationId,
- errorCode: inferenceErrorCode(error),
- metadata: { grounded: String(instruction !== undefined) },
- model: billedModel,
- provider: inferenceProvider,
- requestKind: "vision_parse",
- route: inferenceUsageRoutes.vision,
- userId: request.donkey.userId,
- });
- }
- if (error instanceof InferenceProviderError) {
- return inferenceProviderErrorResponse(error);
- }
-
- throw error;
- }
-}
diff --git a/site/src/app/cut/_components/landing/CutFooter.tsx b/site/src/app/cut/_components/landing/CutFooter.tsx
index fef45b97..16c3b718 100644
--- a/site/src/app/cut/_components/landing/CutFooter.tsx
+++ b/site/src/app/cut/_components/landing/CutFooter.tsx
@@ -11,8 +11,7 @@ const DISCORD_URL = "https://discord.gg/Xv6qGax7sT";
// Cut's own footer, in the shared landing Footer's grouped-column design. The
// shared Footer links routes that don't exist on donkeycut.com (/sign-in,
-// /use-cases, /donkeyvision), so this one carries only links that resolve on
-// both hosts.
+// /use-cases), so this one carries only links that resolve on both hosts.
export function CutFooter() {
const linkGroups: FooterColumn[] = [
{
diff --git a/site/src/app/cut/app/(home)/settings/_components/ProCard.tsx b/site/src/app/cut/app/(home)/settings/_components/ProCard.tsx
index e38a5fd2..5e83d676 100644
--- a/site/src/app/cut/app/(home)/settings/_components/ProCard.tsx
+++ b/site/src/app/cut/app/(home)/settings/_components/ProCard.tsx
@@ -94,7 +94,7 @@ export function ProCard() {
disabled={checkout.isPending}
onClick={() => {
track("pro_checkout_started");
- checkout.mutate("pro", {
+ checkout.mutate(undefined, {
onSuccess: (result) => window.location.assign(result.url),
});
}}
diff --git a/site/src/app/cut/app/(home)/settings/page.tsx b/site/src/app/cut/app/(home)/settings/page.tsx
index 66e0c5a4..812e9d51 100644
--- a/site/src/app/cut/app/(home)/settings/page.tsx
+++ b/site/src/app/cut/app/(home)/settings/page.tsx
@@ -4,8 +4,7 @@ import { CreditsCard } from "@/app/cut/app/(home)/settings/_components/CreditsCa
import { ProCard } from "@/app/cut/app/(home)/settings/_components/ProCard";
// Cut's billing page: the Pro subscription that carries the monthly AI
-// credits, plus the credit balance and top-ups. The Vision API plan is a
-// Donkey-only surface and stays on the apex settings.
+// credits, plus the credit balance and top-ups.
export default function CutBillingPage() {
return (
<div className="max-w-2xl space-y-6 pb-9">
diff --git a/site/src/app/cut/app/(home)/settings/usage/page.tsx b/site/src/app/cut/app/(home)/settings/usage/page.tsx
index b7084f0c..098c5fd0 100644
--- a/site/src/app/cut/app/(home)/settings/usage/page.tsx
+++ b/site/src/app/cut/app/(home)/settings/usage/page.tsx
@@ -3,5 +3,5 @@
import { UsageHistoryCard } from "@/components/UsageHistoryCard";
export default function CutUsagePage() {
- return <UsageHistoryCard showVision={false} plain />;
+ return <UsageHistoryCard />;
}
diff --git a/site/src/app/cut/sitemap.ts b/site/src/app/cut/sitemap.ts
index 8475f995..e1cafcf5 100644
--- a/site/src/app/cut/sitemap.ts
+++ b/site/src/app/cut/sitemap.ts
@@ -16,11 +16,6 @@ export default function sitemap(): MetadataRoute.Sitemap {
changeFrequency: "monthly",
priority: 0.8,
},
- {
- url: `${DONKEYCUT_CANONICAL}/donkeyvision`,
- changeFrequency: "monthly",
- priority: 0.6,
- },
{
url: `${DONKEYCUT_CANONICAL}/privacy`,
changeFrequency: "yearly",
diff --git a/site/src/app/donkeyvision/ApiReference.tsx b/site/src/app/donkeyvision/ApiReference.tsx
deleted file mode 100644
index 163c4ae3..00000000
--- a/site/src/app/donkeyvision/ApiReference.tsx
+++ /dev/null
@@ -1,203 +0,0 @@
-"use client";
-
-import { Check, Copy } from "lucide-react";
-import { useState } from "react";
-
-import {
- API_ENDPOINT,
- BODY_PARAMS,
- CODE_SAMPLES,
- RESPONSE_FIELDS,
- RESPONSE_SAMPLE,
- type ApiField,
-} from "@/app/donkeyvision/apiContract";
-import { HighlightedCode, InlineMarkup } from "@/app/donkeyvision/codeHighlight";
-
-function CopyButton({ value, label }: { value: string; label: string }) {
- const [copied, setCopied] = useState(false);
-
- return (
- <button
- type="button"
- aria-label={`Copy ${label}`}
- onClick={() => {
- void navigator.clipboard.writeText(value).then(() => {
- setCopied(true);
- setTimeout(() => setCopied(false), 1500);
- });
- }}
- className="inline-flex items-center justify-center rounded-md border border-white/20 p-1.5 text-white/70 transition hover:bg-white/10 hover:text-white"
- >
- {copied ? (
- <Check size={14} aria-hidden="true" />
- ) : (
- <Copy size={14} aria-hidden="true" />
- )}
- </button>
- );
-}
-
-function CodePanel({ children }: { children: string }) {
- return (
- <pre className="max-w-full overflow-x-auto px-4 py-4 text-xs leading-6 text-white md:text-[13px]">
- <code>
- <HighlightedCode code={children} />
- </code>
- </pre>
- );
-}
-
-function RequestExamples() {
- const [active, setActive] = useState(CODE_SAMPLES[0].key);
- const sample =
- CODE_SAMPLES.find((s) => s.key === active) ?? CODE_SAMPLES[0];
-
- return (
- <div className="overflow-hidden rounded-lg border-2 border-[#0F0E0D] bg-[#0F0E0D]">
- <div className="flex items-center justify-between gap-2 border-b border-white/15 pl-2 pr-3">
- <div className="flex items-center">
- {CODE_SAMPLES.map((s) => {
- const isActive = s.key === active;
- return (
- <button
- key={s.key}
- type="button"
- onClick={() => setActive(s.key)}
- aria-pressed={isActive}
- className={`relative px-3 py-3 text-sm font-semibold transition-colors ${
- isActive
- ? "text-white"
- : "text-white/45 hover:text-white/75"
- }`}
- >
- {s.label}
- {isActive ? (
- <span className="absolute inset-x-2 -bottom-px h-0.5 rounded-full bg-[#F5D875]" />
- ) : null}
- </button>
- );
- })}
- </div>
- <CopyButton value={sample.code} label={`${sample.label} request`} />
- </div>
- <CodePanel>{sample.code}</CodePanel>
- </div>
- );
-}
-
-function ResponseExample() {
- return (
- <div className="overflow-hidden rounded-lg border-2 border-[#0F0E0D] bg-[#0F0E0D]">
- <div className="flex items-center justify-between border-b border-white/15 px-4 py-2.5">
- <div className="flex items-center gap-2 text-sm font-semibold text-white">
- <span className="inline-flex items-center rounded-full bg-[#34C759]/15 px-2 py-0.5 font-mono text-xs text-[#34C759]">
- 200
- </span>
- Response
- </div>
- <CopyButton value={RESPONSE_SAMPLE} label="response" />
- </div>
- <CodePanel>{RESPONSE_SAMPLE}</CodePanel>
- </div>
- );
-}
-
-function FieldList({ fields }: { fields: ApiField[] }) {
- return (
- <dl className="divide-y divide-[#0F0E0D]/10 rounded-lg border-2 border-[#0F0E0D] bg-white">
- {fields.map((field) => (
- <div key={field.name} className="px-4 py-3.5">
- <dt className="flex flex-wrap items-center gap-2">
- <span className="font-mono text-sm font-semibold text-[#0F0E0D]">
- {field.name}
- </span>
- <span className="rounded bg-[#0F0E0D]/8 px-1.5 py-0.5 font-mono text-[11px] text-[#0F0E0D]/70">
- {field.type}
- </span>
- <span
- className={`rounded px-1.5 py-0.5 text-[11px] font-semibold ${
- field.required
- ? "bg-[#EC7868]/20 text-[#C0432F]"
- : "bg-[#0F0E0D]/5 text-[#0F0E0D]/45"
- }`}
- >
- {field.required ? "required" : "optional"}
- </span>
- </dt>
- <dd className="mt-1 text-sm leading-6 text-[#555]">
- <InlineMarkup text={field.description} />
- </dd>
- </div>
- ))}
- </dl>
- );
-}
-
-export function ApiReference() {
- return (
- <div className="mt-16">
- {/* Endpoint */}
- <div className="flex flex-col gap-3 rounded-lg border-2 border-[#0F0E0D] bg-white p-4 sm:flex-row sm:items-center sm:justify-between">
- <div className="flex min-w-0 items-center gap-3">
- <span className="inline-flex shrink-0 items-center rounded-md border-2 border-[#0F0E0D] bg-[#F5D875] px-2.5 py-1 text-xs font-bold uppercase tracking-[0.08em]">
- {API_ENDPOINT.method}
- </span>
- <code className="truncate font-mono text-sm font-semibold text-[#0F0E0D] md:text-base">
- {API_ENDPOINT.path}
- </code>
- </div>
- <p className="shrink-0 text-sm text-[#555]">
- Auth:{" "}
- <code className="rounded bg-[#0F0E0D]/8 px-1.5 py-0.5 font-mono text-[0.85em] text-[#0F0E0D]">
- Authorization: Bearer dk_live_...
- </code>
- </p>
- </div>
-
- <div className="mt-6 grid gap-8 lg:grid-cols-2">
- {/* Request side */}
- <div className="min-w-0">
- <h3 className="text-sm font-bold uppercase tracking-[0.12em] text-[#0F0E0D]">
- Request
- </h3>
- <p className="mt-2 text-sm leading-6 text-[#555]">
- Submit a base64 screenshot. Add an optional{" "}
- <code className="rounded bg-[#0F0E0D]/8 px-1 py-0.5 font-mono text-[0.85em] text-[#0F0E0D]">
- instruction
- </code>{" "}
- to return a matching click target.
- </p>
- <div className="mt-4">
- <RequestExamples />
- </div>
- <h4 className="mt-8 text-sm font-bold uppercase tracking-[0.12em] text-[#0F0E0D]">
- Body parameters
- </h4>
- <div className="mt-4">
- <FieldList fields={BODY_PARAMS} />
- </div>
- </div>
-
- {/* Response side */}
- <div className="min-w-0">
- <h3 className="text-sm font-bold uppercase tracking-[0.12em] text-[#0F0E0D]">
- Response
- </h3>
- <p className="mt-2 text-sm leading-6 text-[#555]">
- Coordinates are pixel values from the submitted screenshot, measured
- from the top-left corner.
- </p>
- <div className="mt-4">
- <ResponseExample />
- </div>
- <h4 className="mt-8 text-sm font-bold uppercase tracking-[0.12em] text-[#0F0E0D]">
- Response fields
- </h4>
- <div className="mt-4">
- <FieldList fields={RESPONSE_FIELDS} />
- </div>
- </div>
- </div>
- </div>
- );
-}
diff --git a/site/src/app/donkeyvision/ApiSection.tsx b/site/src/app/donkeyvision/ApiSection.tsx
deleted file mode 100644
index c48d6e13..00000000
--- a/site/src/app/donkeyvision/ApiSection.tsx
+++ /dev/null
@@ -1,72 +0,0 @@
-import { ApiReference } from "@/app/donkeyvision/ApiReference";
-import { InlineMarkup } from "@/app/donkeyvision/codeHighlight";
-import { features } from "@/app/donkeyvision/data";
-import type { Feature } from "@/app/donkeyvision/types";
-
-type FeatureLineProps = {
- feature: Feature;
-};
-
-export function ApiSection() {
- return (
- <section
- className="border-b-2 border-[#0F0E0D] bg-[#FAF6EC] py-20"
- id="api"
- >
- <div className="mx-auto max-w-[1400px] px-6 md:px-12">
- <div className="max-w-3xl">
- <h2 className="text-4xl font-semibold leading-none md:text-6xl">
- How the API works
- </h2>
- <p className="mt-6 text-lg leading-8 text-[#454545]">
- Send a screenshot to <Code>/api/vision</Code>. The response
- includes detected UI elements with IDs, labels, types, bounding
- boxes, center points, and confidence scores.
- </p>
- <p className="mt-4 text-lg leading-8 text-[#454545]">
- Add an optional text instruction, such as{" "}
- <Code>click the play button</Code>, to return the matching click
- target. Model selection supports ChatGPT, Claude, Gemini, or a custom
- model.
- </p>
- </div>
-
- <div className="mt-10 grid gap-x-10 gap-y-8 sm:grid-cols-2 lg:grid-cols-3">
- {features.map((feature) => (
- <FeatureLine feature={feature} key={feature.title} />
- ))}
- </div>
-
- <ApiReference />
- </div>
- </section>
- );
-}
-
-function FeatureLine({ feature }: FeatureLineProps) {
- const Icon = feature.icon;
-
- return (
- <div className="flex gap-3">
- <Icon
- size={20}
- aria-hidden="true"
- className="mt-0.5 shrink-0 text-[#0F0E0D]"
- />
- <div>
- <h3 className="text-base font-semibold">{feature.title}</h3>
- <p className="mt-1 text-sm leading-6 text-[#555]">
- <InlineMarkup text={feature.description} />
- </p>
- </div>
- </div>
- );
-}
-
-function Code({ children }: { children: string }) {
- return (
- <code className="rounded bg-[#0F0E0D]/8 px-1.5 py-0.5 font-mono text-[0.85em] text-[#0F0E0D]">
- {children}
- </code>
- );
-}
diff --git a/site/src/app/donkeyvision/HeroSection.tsx b/site/src/app/donkeyvision/HeroSection.tsx
deleted file mode 100644
index 1e1c8d64..00000000
--- a/site/src/app/donkeyvision/HeroSection.tsx
+++ /dev/null
@@ -1,34 +0,0 @@
-import { Braces } from "lucide-react";
-
-import { VisionPreview } from "@/app/donkeyvision/VisionPreview";
-
-export function HeroSection() {
- return (
- <section className="mx-auto grid w-full max-w-[1400px] gap-10 px-6 pb-16 pt-10 md:grid-cols-[minmax(0,1.02fr)_minmax(420px,0.98fr)] md:px-12 md:pb-24 md:pt-16">
- <div className="flex min-w-0 flex-col justify-center">
- <h1 className="max-w-4xl break-words text-[46px] font-semibold leading-[0.92] sm:text-[54px] md:text-[76px] md:leading-[0.9] lg:text-[92px]">
- Open source <span className="italic">computer use</span> API
- </h1>
- <p className="mt-7 max-w-2xl break-words text-lg leading-8 text-[#454545] md:text-xl">
- Donkey Vision finds every interactable element in a screenshot —
- buttons, icons, inputs, rows — and returns each one&rsquo;s box, center
- point, and label. It reads pixels, so it works on software that exposes
- no API at all.
- </p>
- <div className="mt-9 flex max-w-full flex-wrap gap-3">
- <a
- className="inline-flex min-h-14 items-center justify-center gap-2 rounded-full border-2 border-[#0F0E0D] bg-white px-7 text-base font-semibold text-[#0F0E0D] transition hover:-translate-y-0.5"
- href="#api"
- >
- See the API <Braces size={18} aria-hidden="true" />
- </a>
- </div>
- <p className="mt-7 text-sm text-[#666]">
- The same screen-understanding layer Donkey uses to read and drive apps
- on your Mac.
- </p>
- </div>
- <VisionPreview />
- </section>
- );
-}
diff --git a/site/src/app/donkeyvision/MediaSection.tsx b/site/src/app/donkeyvision/MediaSection.tsx
deleted file mode 100644
index 71c86f07..00000000
--- a/site/src/app/donkeyvision/MediaSection.tsx
+++ /dev/null
@@ -1,32 +0,0 @@
-import { surfaces } from "@/app/donkeyvision/data";
-
-export function MediaSection() {
- return (
- <section className="border-y-2 border-[#0F0E0D] bg-white py-20">
- <div className="mx-auto max-w-[1400px] px-6 md:px-12">
- <h2 className="text-4xl font-semibold leading-none md:text-6xl 2xl:whitespace-nowrap">
- Works anywhere a screenshot can be captured.
- </h2>
- <p className="mt-6 text-lg leading-8 text-[#454545] 2xl:whitespace-nowrap">
- Donkey Vision analyzes pixels directly. No DOM access, private
- integration, app-specific setup, or brittle selectors required.
- </p>
- <p className="mt-4 text-lg leading-8 text-[#454545] 2xl:whitespace-nowrap">
- The same API request can process native apps, browser tabs, Electron
- apps, remote desktops, and other screenshot-based environments.
- </p>
- <ul className="mt-10 grid max-w-3xl gap-x-10 gap-y-3 sm:grid-cols-2">
- {surfaces.map((surface) => (
- <li className="flex items-center gap-3" key={surface}>
- <span
- aria-hidden="true"
- className="h-1.5 w-1.5 shrink-0 rounded-full bg-[#0F0E0D]"
- />
- <span className="text-base font-semibold leading-6">{surface}</span>
- </li>
- ))}
- </ul>
- </div>
- </section>
- );
-}
diff --git a/site/src/app/donkeyvision/PricingSection.tsx b/site/src/app/donkeyvision/PricingSection.tsx
deleted file mode 100644
index 6726c875..00000000
--- a/site/src/app/donkeyvision/PricingSection.tsx
+++ /dev/null
@@ -1,57 +0,0 @@
-import Link from "next/link";
-
-import { ContactSalesButton } from "@/app/_components/landing/ContactSalesButton";
-
-const planFeatures = [
- "5,000 API calls per month",
- "3 requests per second",
- "Detect UI elements, boxes, center points, and labels",
- "Prompt-to-click targeting for actions like “click the play button”",
-];
-
-export function PricingSection() {
- return (
- <section className="bg-[#0F0E0D] text-white" id="contact">
- <div className="mx-auto grid w-full max-w-[1400px] gap-8 px-6 py-20 md:grid-cols-[1fr_auto] md:px-12">
- <div>
- <h2 className="text-4xl font-semibold leading-none md:text-6xl">
- Get Donkey Vision API access.
- </h2>
- <p className="mt-4 text-xl font-semibold text-[#EC7868] md:text-2xl">
- Starts at $50/month
- </p>
- <p className="mt-6 max-w-2xl text-lg leading-8 text-white/75">
- Start detecting clickable elements from screenshots in minutes.
- Donkey Vision returns UI boxes, center points, labels, and
- prompt-matched click targets for computer-use agents.
- </p>
- <ul className="mt-8 grid max-w-2xl gap-2 text-base text-white/85 sm:grid-cols-2">
- {planFeatures.map((feature) => (
- <li key={feature} className="flex gap-2">
- <span aria-hidden="true" className="leading-6 text-[#EC7868]">
- ●
- </span>
- {feature}
- </li>
- ))}
- </ul>
- <p className="mt-24 text-sm leading-6 text-white/55">
- Need higher call volume or rate limits?{" "}
- <ContactSalesButton className="font-semibold text-white/80 underline underline-offset-2 transition hover:text-white">
- Contact us
- </ContactSalesButton>{" "}
- for increased limits.
- </p>
- </div>
- <div className="flex flex-col justify-center md:w-60">
- <Link
- href="/donkeyvision/settings"
- className="flex min-h-14 w-full items-center justify-center gap-2 rounded-full bg-[#EC7868] px-7 text-base font-semibold text-[#0F0E0D] transition hover:-translate-y-0.5"
- >
- Start building
- </Link>
- </div>
- </div>
- </section>
- );
-}
diff --git a/site/src/app/donkeyvision/ProofSection.tsx b/site/src/app/donkeyvision/ProofSection.tsx
deleted file mode 100644
index dccd3b96..00000000
--- a/site/src/app/donkeyvision/ProofSection.tsx
+++ /dev/null
@@ -1,47 +0,0 @@
-import { ArrowUpRight } from "lucide-react";
-
-import { stats } from "@/app/donkeyvision/data";
-
-const cardStyles = [
- { bg: "bg-[#EC7868]", rotate: "-rotate-2", tape: "rotate-[-6deg]" },
- { bg: "bg-[#A8D5E8]", rotate: "rotate-2", tape: "rotate-[5deg]" },
-];
-
-export function ProofSection() {
- return (
- <section className="mx-auto w-full max-w-[1400px] px-6 pb-20 md:px-12">
- <div className="grid gap-8 sm:grid-cols-2 md:gap-10">
- {stats.map((stat, index) => {
- const style = cardStyles[index % cardStyles.length];
-
- return (
- <div
- className={`group relative rounded-2xl border-2 border-[#0F0E0D] ${style.bg} ${style.rotate} p-7 shadow-[6px_6px_0_0_#0F0E0D] transition duration-200 hover:-translate-y-1 hover:rotate-0 md:p-8`}
- key={stat.label}
- >
- <span
- aria-hidden="true"
- className={`absolute -top-3 left-1/2 h-6 w-20 -translate-x-1/2 ${style.tape} rounded-sm border-2 border-[#0F0E0D] bg-white/70`}
- />
- <div className="flex items-center gap-1.5 text-xs font-bold uppercase tracking-[0.12em] text-[#0F0E0D]">
- <ArrowUpRight size={14} aria-hidden="true" />
- {stat.eyebrow}
- </div>
- <div className="mt-4 text-5xl font-bold tracking-tight md:text-6xl">
- {stat.value}
- </div>
- <p className="mt-4 text-sm font-medium leading-6 text-[#0F0E0D]/80">
- {stat.label}
- </p>
- </div>
- );
- })}
- </div>
- <p className="mt-8 max-w-3xl text-sm leading-6 text-[#0F0E0D]/60">
- Latency numbers show server processing time only. Total request time
- also depends on image size, upload speed, network latency, and any
- queueing.
- </p>
- </section>
- );
-}
diff --git a/site/src/app/donkeyvision/UseCasesSection.tsx b/site/src/app/donkeyvision/UseCasesSection.tsx
deleted file mode 100644
index 5b69e13c..00000000
--- a/site/src/app/donkeyvision/UseCasesSection.tsx
+++ /dev/null
@@ -1,45 +0,0 @@
-import { InlineMarkup } from "@/app/donkeyvision/codeHighlight";
-import { useCases } from "@/app/donkeyvision/data";
-import type { Feature } from "@/app/donkeyvision/types";
-
-type Props = {
- feature: Feature;
-};
-
-export function UseCasesSection() {
- return (
- <section className="mx-auto w-full max-w-[1400px] px-6 py-20 md:px-12">
- <div className="mb-10">
- <h2 className="text-4xl font-semibold leading-none md:whitespace-nowrap md:text-6xl">
- Use screenshots when there is no API.
- </h2>
- <p className="mt-5 max-w-2xl text-lg leading-8 text-[#454545]">
- Send a screenshot from any application. Donkey Vision returns detected
- UI elements, labels, bounding boxes, center points, and optional
- prompt-matched click targets.
- </p>
- </div>
- <div className="grid gap-5 md:grid-cols-2 lg:grid-cols-4">
- {useCases.map((feature) => (
- <UseCaseCard feature={feature} key={feature.title} />
- ))}
- </div>
- </section>
- );
-}
-
-function UseCaseCard({ feature }: Props) {
- const Icon = feature.icon;
-
- return (
- <div className="rounded-lg border-2 border-[#0F0E0D] bg-white p-5">
- <div className="mb-5 flex h-12 w-12 items-center justify-center rounded-md border-2 border-[#0F0E0D] bg-[#F2B5C4]">
- <Icon size={22} aria-hidden="true" />
- </div>
- <h3 className="text-xl font-semibold leading-7">{feature.title}</h3>
- <p className="mt-3 text-sm leading-6 text-[#555]">
- <InlineMarkup text={feature.description} />
- </p>
- </div>
- );
-}
diff --git a/site/src/app/donkeyvision/VisionCompareSection.tsx b/site/src/app/donkeyvision/VisionCompareSection.tsx
deleted file mode 100644
index e8eef6e6..00000000
--- a/site/src/app/donkeyvision/VisionCompareSection.tsx
+++ /dev/null
@@ -1,57 +0,0 @@
-"use client";
-
-import { useState } from "react";
-
-import { VisionCompareSlider } from "@/app/donkeyvision/VisionCompareSlider";
-import { VISION_DATASETS } from "@/app/donkeyvision/visionData";
-
-export function VisionCompareSection() {
- const [activeKey, setActiveKey] = useState(VISION_DATASETS[0].key);
-
- const active =
- VISION_DATASETS.find((d) => d.key === activeKey) ?? VISION_DATASETS[0];
-
- return (
- <section
- id="demo"
- className="border-y-2 border-[#0F0E0D] bg-[#F5EFE0] py-20"
- >
- <div className="mx-auto max-w-[1400px] px-6 md:px-12">
- <h2 className="max-w-3xl text-4xl font-semibold leading-none md:text-6xl">
- Drag to see what it reads.
- </h2>
- <p className="mt-6 max-w-2xl text-lg leading-8 text-[#454545]">
- Left is the raw screenshot. Right is the same pixels with every
- detected control boxed and labeled. Drag the handle — no DOM, no
- integration, just the image.
- </p>
-
- <div className="relative mt-10">
- <VisionCompareSlider key={active.key} dataset={active} />
- </div>
-
- {/* Dataset buttons. */}
- <div className="mt-6 flex flex-wrap gap-3">
- {VISION_DATASETS.map((d) => {
- const isActive = d.key === active.key;
- return (
- <button
- key={d.key}
- type="button"
- onClick={() => setActiveKey(d.key)}
- aria-pressed={isActive}
- className={`rounded-md border-2 border-[#0F0E0D] px-5 py-2.5 text-sm font-semibold transition-colors ${
- isActive
- ? "bg-[#0F0E0D] text-white"
- : "bg-white text-[#0F0E0D] hover:bg-[#F5D875]"
- }`}
- >
- {d.title}
- </button>
- );
- })}
- </div>
- </div>
- </section>
- );
-}
diff --git a/site/src/app/donkeyvision/VisionCompareSlider.tsx b/site/src/app/donkeyvision/VisionCompareSlider.tsx
deleted file mode 100644
index c8617aad..00000000
--- a/site/src/app/donkeyvision/VisionCompareSlider.tsx
+++ /dev/null
@@ -1,84 +0,0 @@
-"use client";
-
-import { ChevronLeft, ChevronRight } from "lucide-react";
-import Image from "next/image";
-import { useCallback, useRef, useState } from "react";
-
-import { VisionOverlay } from "@/app/donkeyvision/VisionOverlay";
-import type { VisionDataset } from "@/app/donkeyvision/visionData";
-
-// Drag-to-reveal compare widget: left is the raw screenshot, right is the same
-// pixels with detected controls boxed. Internal handle position resets whenever
-// the component remounts, so callers can pass a changing `key` to reset it.
-export function VisionCompareSlider({ dataset }: { dataset: VisionDataset }) {
- const [pos, setPos] = useState(50);
- const frameRef = useRef<HTMLDivElement>(null);
- const dragging = useRef(false);
-
- const moveTo = useCallback((clientX: number) => {
- const frame = frameRef.current;
- if (!frame) return;
- const rect = frame.getBoundingClientRect();
- const ratio = (clientX - rect.left) / rect.width;
- setPos(Math.min(100, Math.max(0, ratio * 100)));
- }, []);
-
- return (
- <div className="relative w-full max-w-full overflow-hidden rounded-lg border-2 border-[#0F0E0D] bg-[#FAF6EC]">
- <div className="flex items-center justify-between border-b-2 border-[#0F0E0D] bg-white px-4 py-[9px]">
- <div className="flex items-center gap-2">
- <span className="h-3 w-3 rounded-full border-2 border-[#0F0E0D] bg-[#EC7868]" />
- <span className="h-3 w-3 rounded-full border-2 border-[#0F0E0D] bg-[#F5D875]" />
- <span className="h-3 w-3 rounded-full border-2 border-[#0F0E0D] bg-[#B7E4C7]" />
- </div>
- <span className="text-[10px] font-semibold text-[#0F0E0D]">
- {dataset.title}
- </span>
- </div>
-
- <div
- ref={frameRef}
- className="relative w-full touch-none select-none"
- style={{ aspectRatio: `${dataset.width} / ${dataset.height}` }}
- onPointerDown={(e) => {
- dragging.current = true;
- e.currentTarget.setPointerCapture(e.pointerId);
- moveTo(e.clientX);
- }}
- onPointerMove={(e) => {
- if (dragging.current) moveTo(e.clientX);
- }}
- onPointerUp={() => {
- dragging.current = false;
- }}
- >
- <Image
- src={dataset.image}
- alt={`${dataset.title} screenshot`}
- fill
- draggable={false}
- sizes="(max-width: 1100px) 100vw, 1100px"
- className="object-cover"
- />
- {/* Reveal the overlay only to the right of the handle. */}
- <div
- className="absolute inset-0"
- style={{ clipPath: `inset(0 0 0 ${pos}%)` }}
- >
- <VisionOverlay dataset={dataset} />
- </div>
-
- {/* Divider + handle. */}
- <div
- className="absolute inset-y-0 w-0.5 -translate-x-1/2 bg-[#007AFF]"
- style={{ left: `${pos}%` }}
- >
- <div className="absolute top-1/2 left-1/2 flex h-10 w-10 -translate-x-1/2 -translate-y-1/2 items-center justify-center rounded-full border-2 border-[#007AFF] bg-white">
- <ChevronLeft size={16} aria-hidden="true" />
- <ChevronRight size={16} aria-hidden="true" />
- </div>
- </div>
- </div>
- </div>
- );
-}
diff --git a/site/src/app/donkeyvision/VisionOverlay.tsx b/site/src/app/donkeyvision/VisionOverlay.tsx
deleted file mode 100644
index d187bf8b..00000000
--- a/site/src/app/donkeyvision/VisionOverlay.tsx
+++ /dev/null
@@ -1,66 +0,0 @@
-import type { VisionDataset } from "@/app/donkeyvision/visionData";
-
-// Per-index palette (matches the worker overlay + the Mac app's vision overlay),
-// so adjacent boxes read apart instead of sharing one color.
-const PALETTE = [
- "#FF3B30", "#FF9500", "#FFCC00", "#34C759",
- "#00C7BE", "#30B0C7", "#007AFF", "#5856D6",
- "#AF52DE", "#FF2D55", "#A2845E", "#8E8E93",
-];
-
-function textOn(hex: string): string {
- const r = parseInt(hex.slice(1, 3), 16);
- const g = parseInt(hex.slice(3, 5), 16);
- const b = parseInt(hex.slice(5, 7), 16);
- return 0.299 * r + 0.587 * g + 0.114 * b > 150 ? "#0F0E0D" : "#FFFFFF";
-}
-
-export function VisionOverlay({ dataset }: { dataset: VisionDataset }) {
- const { width: W, height: H, elements } = dataset;
- const fs = W / 85;
- return (
- <svg
- className="absolute inset-0 h-full w-full"
- viewBox={`0 0 ${W} ${H}`}
- preserveAspectRatio="none"
- aria-hidden="true"
- >
- {elements.map((el, i) => {
- const color = PALETTE[i % PALETTE.length];
- const x = el.box[0] * W;
- const y = el.box[1] * H;
- const w = (el.box[2] - el.box[0]) * W;
- const h = (el.box[3] - el.box[1]) * H;
- const text = `AI ${el.label}`.slice(0, 26);
- const chipH = fs + 5;
- const chipW = text.length * fs * 0.56 + 6;
- const chipY = y - chipH >= 0 ? y - chipH : y;
- return (
- <g key={i}>
- <rect
- x={x}
- y={y}
- width={w}
- height={h}
- fill="none"
- stroke={color}
- strokeWidth={2}
- vectorEffect="non-scaling-stroke"
- />
- <rect x={x} y={chipY} width={chipW} height={chipH} fill={color} />
- <text
- x={x + 3}
- y={chipY + chipH - 4}
- fontSize={fs}
- fontFamily="-apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif"
- fontWeight={600}
- fill={textOn(color)}
- >
- {text}
- </text>
- </g>
- );
- })}
- </svg>
- );
-}
diff --git a/site/src/app/donkeyvision/VisionPreview.tsx b/site/src/app/donkeyvision/VisionPreview.tsx
deleted file mode 100644
index 228d471d..00000000
--- a/site/src/app/donkeyvision/VisionPreview.tsx
+++ /dev/null
@@ -1,38 +0,0 @@
-import Image from "next/image";
-
-import { VisionOverlay } from "@/app/donkeyvision/VisionOverlay";
-import { VISION_DATASETS } from "@/app/donkeyvision/visionData";
-
-const spotify =
- VISION_DATASETS.find((d) => d.key === "spotify") ?? VISION_DATASETS[0];
-
-export function VisionPreview() {
- return (
- <div className="relative w-full min-w-0 max-w-full self-center">
- <div className="relative w-full max-w-full overflow-hidden rounded-lg border-2 border-[#0F0E0D] bg-[#FAF6EC]">
- <div className="flex items-center justify-between border-b-2 border-[#0F0E0D] bg-white px-4 py-[9px]">
- <div className="flex items-center gap-2">
- <span className="h-3 w-3 rounded-full border-2 border-[#0F0E0D] bg-[#EC7868]" />
- <span className="h-3 w-3 rounded-full border-2 border-[#0F0E0D] bg-[#F5D875]" />
- <span className="h-3 w-3 rounded-full border-2 border-[#0F0E0D] bg-[#B7E4C7]" />
- </div>
- <span className="text-[10px] font-semibold">Spotify</span>
- </div>
- <div
- className="relative w-full"
- style={{ aspectRatio: `${spotify.width} / ${spotify.height}` }}
- >
- <Image
- src={spotify.image}
- alt={`${spotify.title} screenshot with detected controls`}
- fill
- priority
- sizes="(max-width: 1100px) 100vw, 700px"
- className="object-cover"
- />
- <VisionOverlay dataset={spotify} />
- </div>
- </div>
- </div>
- );
-}
diff --git a/site/src/app/donkeyvision/apiContract.ts b/site/src/app/donkeyvision/apiContract.ts
deleted file mode 100644
index b6af2196..00000000
--- a/site/src/app/donkeyvision/apiContract.ts
+++ /dev/null
@@ -1,182 +0,0 @@
-// Public Donkey Vision API contract for POST /api/vision.
-// Keep this in sync with src/lib/inference/vision/schema.ts.
-
-export const API_ENDPOINT = {
- method: "POST",
- path: "/api/vision",
- host: "https://donkeycut.com",
-} as const;
-
-export type CodeLanguage = {
- key: string;
- label: string;
- code: string;
-};
-
-const url = `${API_ENDPOINT.host}${API_ENDPOINT.path}`;
-
-export const CODE_SAMPLES: CodeLanguage[] = [
- {
- key: "typescript",
- label: "TypeScript",
- code: `const res = await fetch("${url}", {
- method: "POST",
- headers: {
- Authorization: \`Bearer \${process.env.DONKEY_API_KEY}\`,
- "Content-Type": "application/json",
- },
- body: JSON.stringify({
- // base64 png/jpeg/webp, no "data:" prefix
- image: screenshotBase64,
- instruction: "click the play button",
- returnElements: true,
- }),
-});
-
-const { target } = await res.json();
-console.log(target.point); // { x, y } — ready to click`,
- },
- {
- key: "python",
- label: "Python",
- code: `import os, requests
-
-res = requests.post(
- "${url}",
- headers={"Authorization": f"Bearer {os.environ['DONKEY_API_KEY']}"},
- json={
- # base64 png/jpeg/webp, no "data:" prefix
- "image": screenshot_base64,
- "instruction": "click the play button",
- "returnElements": True,
- },
-)
-
-target = res.json()["target"]
-print(target["point"]) # { x, y } — ready to click`,
- },
- {
- key: "swift",
- label: "Swift",
- code: `var request = URLRequest(url: URL(string: "${url}")!)
-request.httpMethod = "POST"
-request.setValue("Bearer \\(apiKey)", forHTTPHeaderField: "Authorization")
-request.setValue("application/json", forHTTPHeaderField: "Content-Type")
-request.httpBody = try JSONSerialization.data(withJSONObject: [
- // base64 png/jpeg/webp, no "data:" prefix
- "image": screenshotBase64,
- "instruction": "click the play button",
- "returnElements": true,
-])
-
-let (data, _) = try await URLSession.shared.data(for: request)
-let result = try JSONDecoder().decode(VisionResponse.self, from: data)
-print(result.target?.point) // { x, y } — ready to click`,
- },
- {
- key: "curl",
- label: "cURL",
- code: `# image is base64 png/jpeg/webp, no "data:" prefix
-curl "${url}" \\
- -H "Authorization: Bearer $DONKEY_API_KEY" \\
- -H "Content-Type: application/json" \\
- -d '{
- "image": "iVBORw0KGgo...",
- "instruction": "click the play button",
- "returnElements": true
- }'
-
-# target.point in the response is { x, y } — ready to click`,
- },
-];
-
-export const RESPONSE_SAMPLE = `{
- "image": { "width": 1440, "height": 900 },
- "elements": [
- {
- "id": "a92kfq",
- "label": "Play",
- "kind": "button",
- "interactive": true,
- "box": { "x": 618, "y": 816, "width": 42, "height": 42 },
- "point": { "x": 639, "y": 837 },
- "confidence": 0.82
- }
- ],
- "target": {
- "elementId": "a92kfq",
- "label": "Play",
- "kind": "button",
- "box": { "x": 618, "y": 816, "width": 42, "height": 42 },
- "point": { "x": 639, "y": 837 },
- "confidence": 0.91
- },
- "alternates": [],
- "model": "gemini-3.1-flash-lite"
-}`;
-
-export type ApiField = {
- name: string;
- type: string;
- required?: boolean;
- description: string;
-};
-
-export const BODY_PARAMS: ApiField[] = [
- {
- name: "image",
- type: "string",
- required: true,
- description:
- "Base64-encoded screenshot. Supports PNG, JPEG, and WebP. For best results, use JPEG quality `0.8` and resize the longest side to `1568px`.",
- },
- {
- name: "instruction",
- type: "string",
- description:
- "Natural language click instruction, such as `click the play button`. When provided, the response includes a matching click target.",
- },
- {
- name: "model",
- type: "string",
- description:
- "Model used for prompt-based targeting. Supported options include `gemini-3.5-flash` and `gemini-3.1-flash-lite`. Defaults to `gemini-3.1-flash-lite`.",
- },
- {
- name: "returnElements",
- type: "boolean",
- description:
- "Controls whether the full detected element list is returned. Defaults to `true`; disabled automatically when an instruction is provided.",
- },
-];
-
-export const RESPONSE_FIELDS: ApiField[] = [
- {
- name: "image",
- type: "object",
- description: "Screenshot dimensions in pixels: `{ width, height }`.",
- },
- {
- name: "elements",
- type: "array",
- description:
- "Detected UI elements. Each element includes `id`, `label`, `kind`, `interactive`, `box`, `point`, and `confidence`. Omitted when `returnElements` is `false`.",
- },
- {
- name: "target",
- type: "object | null",
- description:
- "Best matching click target for the provided instruction. Includes the target element, bounding box, and click point. Returned only when `instruction` is provided.",
- },
- {
- name: "alternates",
- type: "array",
- description:
- "Additional possible matches for the instruction, sorted by confidence.",
- },
- {
- name: "model",
- type: "string",
- description: "Model used for prompt-based target selection.",
- },
-];
diff --git a/site/src/app/donkeyvision/codeHighlight.tsx b/site/src/app/donkeyvision/codeHighlight.tsx
deleted file mode 100644
index 0590bcb0..00000000
--- a/site/src/app/donkeyvision/codeHighlight.tsx
+++ /dev/null
@@ -1,99 +0,0 @@
-import type { ReactNode } from "react";
-
-// Lightweight, language-agnostic syntax highlighter for the API reference code
-// samples (TypeScript, Swift, Python, cURL) and the JSON response. It is not a full
-// grammar — it tokenizes the shapes these snippets share (comments, strings,
-// numbers, keywords, JSON keys, calls) and paints them with editor-style
-// colors, which is enough to read like a code editor without a dependency.
-
-// VS Code Dark+ palette, tuned for the near-black (#0F0E0D) panel background.
-const COLOR = {
- comment: "#6A9955",
- string: "#CE9178",
- number: "#B5CEA8",
- keyword: "#569CD6",
- property: "#9CDCFE",
- func: "#DCDCAA",
-};
-
-const KEYWORDS = new Set([
- // shared across the three languages used in the samples
- "const", "let", "var", "await", "async", "function", "return", "try",
- "import", "from", "new", "def", "print", "true", "false", "null",
- "True", "False", "None", "as", "in",
-]);
-
-const TOKENIZER_SOURCE =
- "(\\/\\/[^\\n]*|#[^\\n]*|\\/\\*[\\s\\S]*?\\*\\/)|(\"(?:\\\\.|[^\"\\\\])*\"(?=\\s*:))|(`(?:\\\\.|[^`\\\\])*`|\"(?:\\\\.|[^\"\\\\])*\"|'(?:\\\\.|[^'\\\\])*')|(\\b\\d+(?:\\.\\d+)?\\b)|([A-Za-z_$][\\w$]*)";
-
-// Renders a plain string, turning `backtick` spans into inline <code> chips.
-// Used for prose (feature cards, parameter descriptions) where the source copy
-// marks code-like tokens with backticks.
-export function InlineMarkup({ text }: { text: string }): ReactNode {
- return text.split(/(`[^`]+`)/g).map((part, i) =>
- part.startsWith("`") && part.endsWith("`") ? (
- <code
- key={i}
- className="rounded bg-[#0F0E0D]/8 px-1 py-0.5 font-mono text-[0.85em] text-[#0F0E0D]"
- >
- {part.slice(1, -1)}
- </code>
- ) : (
- part
- ),
- );
-}
-
-export function HighlightedCode({ code }: { code: string }): ReactNode {
- const tokenizer = new RegExp(TOKENIZER_SOURCE, "g");
- const nodes: ReactNode[] = [];
- let lastIndex = 0;
- let key = 0;
- let match: RegExpExecArray | null;
-
- while ((match = tokenizer.exec(code)) !== null) {
- if (match.index > lastIndex) {
- nodes.push(code.slice(lastIndex, match.index));
- }
-
- const [full, comment, property, string, number, ident] = match;
- let color: string | null = null;
-
- if (comment) {
- color = COLOR.comment;
- } else if (property) {
- color = COLOR.property;
- } else if (string) {
- color = COLOR.string;
- } else if (number) {
- color = COLOR.number;
- } else if (ident) {
- if (KEYWORDS.has(ident)) {
- color = COLOR.keyword;
- } else {
- // A trailing "(" makes it read as a call; otherwise leave it default.
- let j = tokenizer.lastIndex;
- while (j < code.length && /\s/.test(code[j])) j += 1;
- if (code[j] === "(") color = COLOR.func;
- }
- }
-
- nodes.push(
- color ? (
- <span key={key} style={{ color }}>
- {full}
- </span>
- ) : (
- full
- ),
- );
- key += 1;
- lastIndex = tokenizer.lastIndex;
- }
-
- if (lastIndex < code.length) {
- nodes.push(code.slice(lastIndex));
- }
-
- return nodes;
-}
diff --git a/site/src/app/donkeyvision/data.ts b/site/src/app/donkeyvision/data.ts
deleted file mode 100644
index 839a2431..00000000
--- a/site/src/app/donkeyvision/data.ts
+++ /dev/null
@@ -1,98 +0,0 @@
-import {
- Boxes,
- Crosshair,
- ImageIcon,
- ListChecks,
- MousePointerClick,
- SlidersHorizontal,
- Sparkles,
- Type,
- Video,
- Zap,
-} from "lucide-react";
-
-import type { Feature, Stat } from "@/app/donkeyvision/types";
-
-export const stats: Stat[] = [
- {
- eyebrow: "Server-side",
- label:
- "Process a screenshot into structured UI elements — boxes, labels, OCR, and click coordinates — in under a second.",
- value: "~0.7s",
- },
-];
-
-export const features: Feature[] = [
- {
- description:
- "Buttons, icons, inputs, links, rows, text targets, and other visible UI regions.",
- icon: Boxes,
- title: "Detected elements",
- },
- {
- description:
- "Bounding boxes and center points are returned in the original screenshot coordinate space.",
- icon: Crosshair,
- title: "Coordinates",
- },
- {
- description:
- "Each element includes a readable label and kind, such as `button`, `input`, `icon`, or `text`.",
- icon: Type,
- title: "Labels and types",
- },
- {
- description:
- "Natural language instructions return the matching element, click point, and region.",
- icon: MousePointerClick,
- title: "Prompt-to-click",
- },
- {
- description:
- "Per-request thresholds control detection sensitivity and element merging.",
- icon: SlidersHorizontal,
- title: "Detection options",
- },
- {
- description:
- "Prompt-based targeting supports ChatGPT, Claude, Gemini, or a custom model.",
- icon: Sparkles,
- title: "Model choice",
- },
-];
-
-export const useCases: Feature[] = [
- {
- description:
- "Give agents the current screen state before taking action. Return clickable elements, coordinates, labels, and prompt-matched targets for tasks like `click the play button`.",
- icon: Zap,
- title: "Computer-use agents",
- },
- {
- description:
- "Convert screenshots into structured UI data: detected elements, text labels, element types, bounding boxes, and center points.",
- icon: ImageIcon,
- title: "Screenshot parsing",
- },
- {
- description:
- "Parse video frames into UI element timelines. Track visible controls, labels, and screen changes across demos, workflows, and regression tests.",
- icon: Video,
- title: "Video frame analysis",
- },
- {
- description:
- "Find click targets across native apps, web apps, Electron apps, VNC sessions, and remote desktops without DOM access or app-specific selectors.",
- icon: ListChecks,
- title: "Cross-app automation",
- },
-];
-
-export const surfaces: string[] = [
- "Native macOS apps",
- "Web apps in any browser",
- "Electron and hybrid apps",
- "Remote desktops and VNC sessions",
- "Enterprise and internal tools",
- "Games and media players",
-];
diff --git a/site/src/app/donkeyvision/page.tsx b/site/src/app/donkeyvision/page.tsx
deleted file mode 100644
index 1e727e54..00000000
--- a/site/src/app/donkeyvision/page.tsx
+++ /dev/null
@@ -1,80 +0,0 @@
-import type { Metadata } from "next";
-
-import { Footer } from "@/app/_components/landing/Footer";
-import { TopNav } from "@/app/_components/landing/TopNav";
-import { ApiSection } from "@/app/donkeyvision/ApiSection";
-import { HeroSection } from "@/app/donkeyvision/HeroSection";
-import { MediaSection } from "@/app/donkeyvision/MediaSection";
-import { PricingSection } from "@/app/donkeyvision/PricingSection";
-import { ProofSection } from "@/app/donkeyvision/ProofSection";
-import { UseCasesSection } from "@/app/donkeyvision/UseCasesSection";
-import { VisionCompareSection } from "@/app/donkeyvision/VisionCompareSection";
-
-export const dynamic = "force-static";
-
-export const metadata: Metadata = {
- title: "OmniParser API for UI Element Detection | Donkey Vision",
- description:
- "Donkey Vision is a fast, OmniParser-compatible API for detecting interactable UI elements in screenshots — bounding boxes, center points, and labels.",
- keywords: [
- "OmniParser API",
- "OmniParser",
- "OmniParser-compatible API",
- "UI element detection API",
- "screenshot UI parsing API",
- "computer use vision API",
- "Donkey Vision",
- ],
- alternates: {
- canonical: "https://donkeycut.com/donkeyvision",
- },
- openGraph: {
- type: "website",
- url: "https://donkeycut.com/donkeyvision",
- siteName: "Donkey",
- title: "OmniParser API for UI Element Detection | Donkey Vision",
- description:
- "Donkey Vision is a fast, OmniParser-compatible API for detecting interactable UI elements in screenshots — bounding boxes, center points, and labels.",
- },
- twitter: {
- card: "summary_large_image",
- title: "OmniParser API for UI Element Detection | Donkey Vision",
- description:
- "Donkey Vision is a fast, OmniParser-compatible API for detecting interactable UI elements in screenshots — bounding boxes, center points, and labels.",
- },
-};
-
-const structuredData = {
- "@context": "https://schema.org",
- "@type": "WebAPI",
- name: "Donkey Vision",
- alternateName: "OmniParser API",
- url: "https://donkeycut.com/donkeyvision",
- description:
- "Donkey Vision is a fast, OmniParser-compatible API for detecting interactable UI elements in screenshots — bounding boxes, center points, and labels.",
- provider: {
- "@type": "Organization",
- name: "Donkey",
- url: "https://donkeycut.com",
- },
-};
-
-export default function DonkeyVisionPage() {
- return (
- <main className="min-h-screen w-full overflow-x-clip bg-[#F5EFE0] font-[-apple-system,BlinkMacSystemFont,'Segoe_UI',sans-serif] text-[#0F0E0D]">
- <script
- type="application/ld+json"
- dangerouslySetInnerHTML={{ __html: JSON.stringify(structuredData) }}
- />
- <TopNav wordmark="Donkey Vision" />
- <HeroSection />
- <ProofSection />
- <VisionCompareSection />
- <ApiSection />
- <UseCasesSection />
- <MediaSection />
- <PricingSection />
- <Footer />
- </main>
- );
-}
diff --git a/site/src/app/donkeyvision/settings/SettingsShell.tsx b/site/src/app/donkeyvision/settings/SettingsShell.tsx
deleted file mode 100644
index 5e779455..00000000
--- a/site/src/app/donkeyvision/settings/SettingsShell.tsx
+++ /dev/null
@@ -1,103 +0,0 @@
-"use client";
-
-import Link from "next/link";
-import { usePathname, useRouter } from "next/navigation";
-import { useEffect, type ReactNode } from "react";
-
-import { useAppLoaded } from "@/lib/analytics";
-import { authClient } from "@/lib/auth-client";
-import { cn } from "@/lib/utils";
-import { Skeleton } from "@/components/ui/skeleton";
-
-const navItems = [
- { href: "/donkeyvision/settings/usage", label: "Usage" },
- { href: "/donkeyvision/settings/api-keys", label: "API keys" },
-];
-
-// The Donkey Vision settings: API keys and usage for the Vision API. Donkey
-// Cut has its own billing and usage pages inside the Cut app
-// (src/app/cut/app/(home)/settings).
-// The settings UI is fully client-rendered. We guard on the client session and,
-// if signed out, send the user to Google sign-in (API routes also enforce auth).
-export function SettingsShell({ children }: { children: ReactNode }) {
- const router = useRouter();
- const pathname = usePathname();
- const { data: session, isPending } = authClient.useSession();
-
- useAppLoaded("settings", session?.user);
-
- useEffect(() => {
- if (!isPending && !session) {
- void authClient.signIn.social({
- callbackURL: "/donkeyvision/settings",
- provider: "google",
- });
- }
- }, [isPending, session]);
-
- if (isPending || !session) {
- return (
- <div className="mx-auto max-w-5xl px-6 py-16">
- <Skeleton className="h-8 w-48" />
- <Skeleton className="mt-6 h-40 w-full" />
- </div>
- );
- }
-
- return (
- <div className="min-h-screen">
- <header className="border-b">
- <div className="mx-auto flex max-w-5xl items-center justify-between px-6 py-4">
- <Link className="flex items-center gap-2 text-lg font-semibold" href="/donkeyvision">
- <div className="flex h-8 w-8 items-center justify-center overflow-hidden rounded-md">
- <img
- src="/donkey-logo.svg"
- alt="Donkey Logo"
- width={32}
- height={32}
- className="block h-full w-full object-contain"
- />
- </div>
- <span>Donkey</span>
- </Link>
- <nav className="flex items-center gap-1">
- {navItems.map((item) => (
- <Link
- key={item.href}
- href={item.href}
- className={cn(
- "rounded-md px-3 py-1.5 text-sm font-medium transition-colors",
- pathname === item.href
- ? "bg-muted text-foreground"
- : "text-muted-foreground hover:text-foreground",
- )}
- >
- {item.label}
- </Link>
- ))}
- <button
- type="button"
- onClick={() => {
- // Sign out everywhere: revoke every session for this user (so the Mac app signs out
- // too), then clear this browser's session and leave. signOut + redirect always run,
- // even if the revoke call fails, so the user is never stranded signed-in locally.
- void (async () => {
- try {
- await authClient.revokeSessions();
- } finally {
- await authClient.signOut();
- router.push("/");
- }
- })();
- }}
- className="ml-2 rounded-md px-3 py-1.5 text-sm font-medium text-muted-foreground hover:text-foreground"
- >
- Sign out
- </button>
- </nav>
- </div>
- </header>
- <main className="mx-auto max-w-5xl px-6 py-10">{children}</main>
- </div>
- );
-}
diff --git a/site/src/app/donkeyvision/settings/_components/ApiKeysManager.tsx b/site/src/app/donkeyvision/settings/_components/ApiKeysManager.tsx
deleted file mode 100644
index a448932e..00000000
--- a/site/src/app/donkeyvision/settings/_components/ApiKeysManager.tsx
+++ /dev/null
@@ -1,188 +0,0 @@
-"use client";
-
-import { Info } from "lucide-react";
-import Link from "next/link";
-import { useState } from "react";
-
-import {
- useApiKeys,
- useCreateApiKey,
- useDeleteApiKey,
- type CreatedApiKey,
-} from "@/queries/apiKeys";
-import { Button } from "@/components/ui/button";
-import { Card, CardContent } from "@/components/ui/card";
-import {
- Dialog,
- DialogContent,
- DialogDescription,
- DialogFooter,
- DialogHeader,
- DialogTitle,
-} from "@/components/ui/dialog";
-import { Input } from "@/components/ui/input";
-import { Label } from "@/components/ui/label";
-import { Skeleton } from "@/components/ui/skeleton";
-import {
- Table,
- TableBody,
- TableCell,
- TableHead,
- TableHeader,
- TableRow,
-} from "@/components/ui/table";
-
-export function ApiKeysManager() {
- const keys = useApiKeys();
- const createKey = useCreateApiKey();
- const deleteKey = useDeleteApiKey();
-
- const [name, setName] = useState("");
- const [createOpen, setCreateOpen] = useState(false);
- const [createdKey, setCreatedKey] = useState<CreatedApiKey | null>(null);
-
- const handleCreate = () => {
- const trimmed = name.trim();
- if (!trimmed) {
- return;
- }
- createKey.mutate(trimmed, {
- onSuccess: (result) => {
- setCreatedKey(result);
- setName("");
- setCreateOpen(false);
- },
- });
- };
-
- return (
- <div className="space-y-8">
- <div className="flex items-center justify-between gap-4">
- <h1 className="text-2xl font-semibold">API Keys</h1>
- <Button onClick={() => setCreateOpen(true)}>Create key</Button>
- </div>
-
- <Card>
- <CardContent className="pt-6">
- {keys.isLoading ? (
- <Skeleton className="h-24 w-full" />
- ) : keys.data && keys.data.length > 0 ? (
- <Table>
- <TableHeader>
- <TableRow>
- <TableHead>Name</TableHead>
- <TableHead>Key</TableHead>
- <TableHead>Created</TableHead>
- <TableHead>Last used</TableHead>
- <TableHead className="text-right">Actions</TableHead>
- </TableRow>
- </TableHeader>
- <TableBody>
- {keys.data.map((key) => (
- <TableRow key={key.id}>
- <TableCell>{key.name ?? "—"}</TableCell>
- <TableCell className="font-mono text-xs">
- {(key.prefix ?? "") + (key.start ?? "")}…
- </TableCell>
- <TableCell>
- {new Date(key.createdAt).toLocaleDateString()}
- </TableCell>
- <TableCell>
- {key.lastRequest
- ? new Date(key.lastRequest).toLocaleDateString()
- : "Never"}
- </TableCell>
- <TableCell className="text-right">
- <Button
- disabled={deleteKey.isPending}
- onClick={() => deleteKey.mutate(key.id)}
- size="sm"
- variant="destructive"
- >
- Revoke
- </Button>
- </TableCell>
- </TableRow>
- ))}
- </TableBody>
- </Table>
- ) : (
- <p className="text-sm text-muted-foreground">No API keys yet.</p>
- )}
- </CardContent>
- </Card>
-
- <Link
- href="/donkeyvision#api"
- className="inline-flex items-center gap-1.5 text-sm text-muted-foreground transition-colors hover:text-foreground"
- >
- <Info className="h-4 w-4" />
- Learn how to use your API keys
- </Link>
-
- <Dialog open={createOpen} onOpenChange={setCreateOpen}>
- <DialogContent aria-describedby={undefined}>
- <DialogHeader>
- <DialogTitle>Create API key</DialogTitle>
- </DialogHeader>
- <div className="grid gap-1.5">
- <Label htmlFor="key-name">Name</Label>
- <Input
- id="key-name"
- required
- value={name}
- onChange={(event) => setName(event.target.value)}
- onKeyDown={(event) => {
- if (event.key === "Enter") {
- handleCreate();
- }
- }}
- />
- </div>
- <DialogFooter>
- <Button
- disabled={createKey.isPending || !name.trim()}
- onClick={handleCreate}
- >
- {createKey.isPending ? "Creating…" : "Create key"}
- </Button>
- </DialogFooter>
- </DialogContent>
- </Dialog>
-
- <Dialog
- open={createdKey !== null}
- onOpenChange={(open) => {
- if (!open) {
- setCreatedKey(null);
- }
- }}
- >
- <DialogContent>
- <DialogHeader>
- <DialogTitle>API key created</DialogTitle>
- <DialogDescription>
- Secrets are shown only once at creation. Store them securely.
- </DialogDescription>
- </DialogHeader>
- <div className="rounded-md border bg-muted p-3 font-mono text-sm break-all">
- {createdKey?.secret}
- </div>
- <DialogFooter>
- <Button
- onClick={() => {
- if (createdKey) {
- void navigator.clipboard.writeText(createdKey.secret);
- }
- }}
- variant="secondary"
- >
- Copy
- </Button>
- <Button onClick={() => setCreatedKey(null)}>Done</Button>
- </DialogFooter>
- </DialogContent>
- </Dialog>
- </div>
- );
-}
diff --git a/site/src/app/donkeyvision/settings/_components/UsageCard.tsx b/site/src/app/donkeyvision/settings/_components/UsageCard.tsx
deleted file mode 100644
index 1498d81f..00000000
--- a/site/src/app/donkeyvision/settings/_components/UsageCard.tsx
+++ /dev/null
@@ -1,108 +0,0 @@
-"use client";
-
-import { useProSubscription, useUsage } from "@/queries/billing";
-import { Card, CardContent } from "@/components/ui/card";
-import { Skeleton } from "@/components/ui/skeleton";
-import {
- Table,
- TableBody,
- TableCell,
- TableRow,
-} from "@/components/ui/table";
-import { formatUsd } from "@/lib/credits/format-usd";
-
-// One labeled usage line (product on the left, status/amount on the right).
-function UsageRow({ label, value }: { label: string; value: string }) {
- return (
- <TableRow>
- <TableCell className="font-medium text-foreground">{label}</TableCell>
- <TableCell className="text-right text-muted-foreground">
- {value}
- </TableCell>
- </TableRow>
- );
-}
-
-export function UsageCard() {
- const usage = useUsage();
- const pro = useProSubscription();
-
- // Only gate on the Vision usage query; the Pro row degrades on its own so a
- // slow or failing Pro request never blocks the Vision summary from rendering.
- if (usage.isLoading) {
- return (
- <Card>
- <CardContent>
- <Skeleton className="h-24 w-full" />
- </CardContent>
- </Card>
- );
- }
-
- const data = usage.data;
- const limit = data?.limit ?? 0;
- const extra = data?.extraRemaining ?? 0;
- const pct =
- data && limit > 0
- ? Math.min(100, Math.round((data.used / limit) * 100))
- : 0;
-
- // Donkey Pro is the app-inference subscription; its included allowance is spent
- // separately from the Vision API quota below. Used = allowance − remaining.
- const proData = pro.data;
- const proActive = proData?.isActive ?? false;
- const proAllowance =
- proData?.monthlyAllowance != null
- ? Number.parseFloat(proData.monthlyAllowance)
- : null;
- const proRemaining = Number.parseFloat(proData?.allowanceRemaining ?? "0");
- const proUsed =
- proAllowance != null ? Math.max(0, proAllowance - proRemaining) : null;
-
- // Don't assert "No Pro plan" while the Pro query is still loading or errored —
- // that would mislabel an actual Pro user. Only claim it once we have data.
- const proStatus = pro.isLoading
- ? "Loading…"
- : proActive && proUsed != null && proData?.monthlyAllowance != null
- ? `${formatUsd(String(proUsed))} of ${formatUsd(proData.monthlyAllowance)} used this month`
- : pro.isError
- ? "Unavailable"
- : "No Pro plan — using credits";
-
- const visionStatus =
- data && limit > 0
- ? `${data.used.toLocaleString()} of ${limit.toLocaleString()} calls used · ${data.remaining.toLocaleString()} remaining`
- : extra > 0
- ? "No subscription — using your extra calls"
- : "No active plan — subscribe to start making calls";
-
- return (
- <Card>
- <CardContent className="space-y-4">
- <Table>
- <TableBody>
- <UsageRow label="Donkey Pro (app)" value={proStatus} />
- <UsageRow label="Vision API" value={visionStatus} />
- {extra > 0 ? (
- <UsageRow
- label="Extra Vision API calls"
- value={`${extra.toLocaleString()} available`}
- />
- ) : null}
- </TableBody>
- </Table>
- {data && limit > 0 ? (
- <div
- aria-label="Vision API quota used"
- className="h-2 w-full overflow-hidden rounded-full bg-muted"
- >
- <div
- className="h-full rounded-full bg-primary"
- style={{ width: `${pct}%` }}
- />
- </div>
- ) : null}
- </CardContent>
- </Card>
- );
-}
diff --git a/site/src/app/donkeyvision/settings/api-keys/page.tsx b/site/src/app/donkeyvision/settings/api-keys/page.tsx
deleted file mode 100644
index c0927b1e..00000000
--- a/site/src/app/donkeyvision/settings/api-keys/page.tsx
+++ /dev/null
@@ -1,5 +0,0 @@
-import { ApiKeysManager } from "@/app/donkeyvision/settings/_components/ApiKeysManager";
-
-export default function ApiKeysPage() {
- return <ApiKeysManager />;
-}
diff --git a/site/src/app/donkeyvision/settings/layout.tsx b/site/src/app/donkeyvision/settings/layout.tsx
deleted file mode 100644
index c9d9ef55..00000000
--- a/site/src/app/donkeyvision/settings/layout.tsx
+++ /dev/null
@@ -1,23 +0,0 @@
-import type { Metadata } from "next";
-import type { ReactNode } from "react";
-
-import { SettingsShell } from "@/app/donkeyvision/settings/SettingsShell";
-import { AppSurfaceBackground } from "@/components/AppSurfaceBackground";
-
-export const metadata: Metadata = {
- title: "Settings | Donkey",
- description: "Manage your Vision API subscription and API keys.",
-};
-
-// QueryProvider is mounted once at the root layout, so the settings UI just
-// needs its session-guarded shell here. The white product surface is this
-// layout's own: the Vision marketing page one segment up renders on the cream
-// landing background.
-export default function SettingsLayout({ children }: { children: ReactNode }) {
- return (
- <div className="min-h-screen bg-white font-system text-foreground">
- <AppSurfaceBackground />
- <SettingsShell>{children}</SettingsShell>
- </div>
- );
-}
diff --git a/site/src/app/donkeyvision/settings/page.tsx b/site/src/app/donkeyvision/settings/page.tsx
deleted file mode 100644
index 81b2a70e..00000000
--- a/site/src/app/donkeyvision/settings/page.tsx
+++ /dev/null
@@ -1,6 +0,0 @@
-import { redirect } from "next/navigation";
-
-// Vision settings carry no payment UI; settings opens on Usage.
-export default function SettingsIndexPage() {
- redirect("/donkeyvision/settings/usage");
-}
diff --git a/site/src/app/donkeyvision/settings/usage/page.tsx b/site/src/app/donkeyvision/settings/usage/page.tsx
deleted file mode 100644
index 194d96f9..00000000
--- a/site/src/app/donkeyvision/settings/usage/page.tsx
+++ /dev/null
@@ -1,19 +0,0 @@
-"use client";
-
-import { UsageCard } from "@/app/donkeyvision/settings/_components/UsageCard";
-import { UsageHistoryCard } from "@/components/UsageHistoryCard";
-
-export default function UsagePage() {
- return (
- <div className="space-y-8">
- <div>
- <h1 className="text-2xl font-semibold">Usage</h1>
- <p className="mt-1 text-sm text-muted-foreground">
- Your recent Donkey app and Vision API calls.
- </p>
- </div>
- <UsageCard />
- <UsageHistoryCard />
- </div>
- );
-}
diff --git a/site/src/app/donkeyvision/types.ts b/site/src/app/donkeyvision/types.ts
deleted file mode 100644
index 455fa7fe..00000000
--- a/site/src/app/donkeyvision/types.ts
+++ /dev/null
@@ -1,13 +0,0 @@
-import type { LucideIcon } from "lucide-react";
-
-export type Stat = {
- eyebrow: string;
- label: string;
- value: string;
-};
-
-export type Feature = {
- description: string;
- icon: LucideIcon;
- title: string;
-};
diff --git a/site/src/app/donkeyvision/visionData.ts b/site/src/app/donkeyvision/visionData.ts
deleted file mode 100644
index d95b9b83..00000000
--- a/site/src/app/donkeyvision/visionData.ts
+++ /dev/null
@@ -1,332 +0,0 @@
-// AUTO-GENERATED by the vision benchmark. Detection data for the
-// /donkeyvision before/after demo: boxes in 0-1 ratio space + content.
-export type VisionElement = { box: [number, number, number, number]; label: string; kind: string };
-export type VisionDataset = { key: string; title: string; image: string; width: number; height: number; elements: VisionElement[] };
-
-export const VISION_DATASETS: VisionDataset[] = [
- {
- key: "spotify", title: "Spotify", image: "/donkeyvision/spotify.jpg",
- width: 3456, height: 2234,
- elements: [
- { box: [0.0695, 0.0089, 0.0835, 0.0227], label: "File", kind: "text" },
- { box: [0.1486, 0.0089, 0.183, 0.0227], label: "Playback", kind: "text" },
- { box: [0.192, 0.0089, 0.2226, 0.0227], label: "Window", kind: "text" },
- { box: [0.2315, 0.0089, 0.2506, 0.0227], label: "Help", kind: "text" },
- { box: [0.956, 0.0089, 0.993, 0.0227], label: "10:04AM", kind: "text" },
- { box: [0.0159, 0.1076, 0.0695, 0.1273], label: "Your Library", kind: "text" },
- { box: [0.7602, 0.1066, 0.8017, 0.1264], label: "Coldplay", kind: "text" },
- { box: [0.5466, 0.1214, 0.6307, 0.1412], label: "HOT SUMMER POP", kind: "text" },
- { box: [0.5485, 0.1441, 0.6065, 0.1609], label: "Advertisement", kind: "text" },
- { box: [0.3527, 0.1866, 0.3795, 0.2004], label: "Spotify", kind: "text" },
- { box: [0.1894, 0.1925, 0.2213, 0.2083], label: "Recents", kind: "text" },
- { box: [0.5121, 0.1846, 0.729, 0.2261], label: "30I3 #zw4JH4I--Fot3f", kind: "text" },
- { box: [0.0568, 0.2537, 0.0861, 0.2695], label: "Playlist", kind: "text" },
- { box: [0.0899, 0.2537, 0.1231, 0.2695], label: "DavidLe", kind: "text" },
- { box: [0.5223, 0.2596, 0.5657, 0.2754], label: "Listennow", kind: "text" },
- { box: [0.0478, 0.311, 0.0721, 0.3248], label: "Artist", kind: "text" },
- { box: [0.3087, 0.3218, 0.3335, 0.3396], label: "Music", kind: "text" },
- { box: [0.3501, 0.3228, 0.3858, 0.3386], label: "Podcasts", kind: "text" },
- { box: [0.6881, 0.463, 0.7226, 0.4768], label: "Showall", kind: "text" },
- { box: [0.7602, 0.5094, 0.8329, 0.537], label: "Angelsong", kind: "text" },
- { box: [0.7608, 0.538, 0.8004, 0.5538], label: "Coldplay", kind: "text" },
- { box: [0.3871, 0.5893, 0.419, 0.6051], label: "Release", kind: "text" },
- { box: [0.7698, 0.5933, 0.8412, 0.6111], label: "About the artist", kind: "text" },
- { box: [0.3871, 0.6051, 0.4126, 0.619], label: "Radar", kind: "text" },
- { box: [0.3782, 0.6446, 0.4483, 0.6604], label: "Catch all the latest", kind: "text" },
- { box: [0.3769, 0.6624, 0.4011, 0.6762], label: "music", kind: "text" },
- { box: [0.419, 0.6624, 0.4611, 0.6762], label: "artists YOa", kind: "text" },
- { box: [0.2736, 0.7058, 0.2953, 0.7196], label: "Made", kind: "text" },
- { box: [0.273, 0.7226, 0.3329, 0.7463], label: "David Le", kind: "text" },
- { box: [0.6881, 0.7335, 0.7226, 0.7493], label: "Showall", kind: "text" },
- { box: [0.7698, 0.8263, 0.8106, 0.844], label: "Coldplay", kind: "text" },
- { box: [0.7685, 0.8539, 0.8948, 0.8736], label: "92,932,665 monthly listeners", kind: "text" },
- { box: [0.4841, 0.8815, 0.5057, 0.8954], label: "Daily", kind: "text" },
- { box: [0.5408, 0.8746, 0.5676, 0.8983], label: "03", kind: "text" },
- { box: [0.5855, 0.8786, 0.6244, 0.8983], label: "Daily Mix]", kind: "text" },
- { box: [0.6894, 0.8815, 0.7124, 0.8954], label: "Daily E", kind: "text" },
- { box: [0.7698, 0.8914, 0.8055, 0.9072], label: "Coldplay", kind: "text" },
- { box: [0.817, 0.8934, 0.8335, 0.9072], label: "one", kind: "text" },
- { box: [0.8552, 0.8934, 0.9241, 0.9072], label: "most popular acts", kind: "text" },
- { box: [0.9305, 0.8914, 0.9713, 0.9072], label: "the world;", kind: "text" },
- { box: [0.7698, 0.9062, 0.9739, 0.925], label: "having sold more than I00 million albums and releasing", kind: "text" },
- { box: [0.442, 0.9309, 0.5593, 0.9664], label: "X K 0 M Q", kind: "text" },
- { box: [0.3157, 0.9724, 0.3323, 0.9862], label: "0.58", kind: "text" },
- { box: [0.3999, 0.6594, 0.419, 0.6762], label: "from", kind: "text" },
- { box: [0.5801, 0.4934, 0.677, 0.696], label: "THISIS FREQUENCY best in Black musicl Cover:", kind: "icon" },
- { box: [0.4783, 0.4949, 0.5719, 0.6988], label: "Singled Out buzzing here. The hottest newa", kind: "icon" },
- { box: [0.3751, 0.029, 0.6598, 0.0868], label: "doyou want to play? What", kind: "icon" },
- { box: [0.0126, 0.1447, 0.0594, 0.1768], label: "Playlists", kind: "icon" },
- { box: [0.6835, 0.4928, 0.7522, 0.697], label: "Dance Popa 7oetimjich", kind: "icon" },
- { box: [0.4226, 0.3606, 0.5745, 0.4074], label: "Lovo Letter", kind: "icon" },
- { box: [0.2711, 0.3604, 0.4237, 0.4085], label: "Coldplay", kind: "icon" },
- { box: [0.274, 0.494, 0.3693, 0.6912], label: "RDA Ariana Grande, Z4eiN3530t_", kind: "icon" },
- { box: [0.0103, 0.451, 0.2324, 0.5082], label: "Cigarettes After Sex Album Cigarettes After Sex", kind: "icon" },
- { box: [0.0092, 0.623, 0.2261, 0.6801], label: "Eyes Wide Album Sabrina Carpenter Open", kind: "icon" },
- { box: [0.0609, 0.1452, 0.1062, 0.1768], label: "Albums", kind: "icon" },
- { box: [0.0105, 0.3979, 0.2414, 0.4458], label: "Playlist *", kind: "icon" },
- { box: [0.0097, 0.3377, 0.2406, 0.3947], label: "You're Love Single Mood;", kind: "icon" },
- { box: [0.0099, 0.5662, 0.2267, 0.6231], label: "Discover Weekly Playlist Made for DavidLe", kind: "icon" },
- { box: [0.0098, 0.5118, 0.2302, 0.5655], label: "Album Cigarettes After Sox Cry", kind: "icon" },
- { box: [0.0476, 0.2868, 0.0757, 0.3061], label: "Mood:", kind: "icon" },
- { box: [0.1076, 0.1456, 0.1489, 0.1763], label: "Artists", kind: "icon" },
- { box: [0.2733, 0.7663, 0.3702, 0.9195], label: "Daily", kind: "icon" },
- { box: [0.0473, 0.2289, 0.1015, 0.2493], label: "Liked Songs", kind: "icon" },
- { box: [0.2738, 0.4497, 0.4131, 0.4777], label: "It's New Music Fridayl", kind: "icon" },
- { box: [0.0217, 0, 0.0647, 0.0308], label: "Spotify", kind: "icon" },
- { box: [0.9304, 0.8478, 0.9784, 0.8805], label: "Follow", kind: "icon" },
- { box: [0.1542, 0.099, 0.2148, 0.1337], label: "Create", kind: "icon" },
- { box: [0.0109, 0.6817, 0.2294, 0.7326], label: "Return Album ODESZA", kind: "icon" },
- { box: [0.0466, 0.9402, 0.1205, 0.9783], label: "Angelsong Coldplay", kind: "icon" },
- { box: [0.0896, 0, 0.1431, 0.0276], label: "Edit View", kind: "icon" },
- { box: [0.3761, 0.7656, 0.47, 0.9196], label: "Daily", kind: "icon" },
- { box: [0.3449, 0.032, 0.3757, 0.0826], label: "Home", kind: "icon" },
- { box: [0.0049, 0.036, 0.0527, 0.0772], label: "Color picker", kind: "icon" },
- { box: [0.6986, 0, 0.719, 0.029], label: "Toggleoggleoggle", kind: "icon" },
- { box: [0.7628, 0, 0.7828, 0.0295], label: "Mute", kind: "icon" },
- { box: [0.965, 0.0329, 0.9967, 0.0813], label: "Currency", kind: "icon" },
- { box: [0.7195, 0, 0.7396, 0.0294], label: "Toggleoggleoggle", kind: "icon" },
- { box: [0.7842, 0, 0.803, 0.0297], label: "Navigation Bar", kind: "icon" },
- { box: [0.7409, 0, 0.7617, 0.0295], label: "Panther", kind: "icon" },
- { box: [0.9419, 0.0355, 0.9639, 0.0793], label: "28, September, 2024", kind: "icon" },
- { box: [0.8046, 0, 0.8262, 0.0301], label: "Text formatting options.", kind: "icon" },
- { box: [0.9197, 0.0373, 0.9417, 0.0779], label: "Notifications", kind: "icon" },
- { box: [0.825, 0.9455, 0.843, 0.9755], label: "Fill", kind: "icon" },
- { box: [0.8948, 0, 0.9153, 0.0314], label: "More Options", kind: "icon" },
- { box: [0.0531, 0.0373, 0.0713, 0.0763], label: "Previous", kind: "icon" },
- { box: [0.8984, 0.9447, 0.9302, 0.973], label: "Line", kind: "icon" },
- { box: [0.87, 0, 0.895, 0.0311], label: "Link", kind: "icon" },
- { box: [0.8447, 0.9456, 0.8617, 0.9741], label: "Center", kind: "icon" },
- { box: [0.0721, 0.0369, 0.0906, 0.0761], label: "Next", kind: "icon" },
- { box: [0.4397, 0.9343, 0.4576, 0.9703], label: "Shuffle", kind: "icon" },
- { box: [0.0106, 0.2823, 0.0432, 0.3286], label: "a person with purple hair.", kind: "icon" },
- { box: [0.8641, 0.9454, 0.8801, 0.9736], label: "Switch between plug-in and switch between plug ins", kind: "icon" },
- { box: [0.9579, 0.9452, 0.9762, 0.9751], label: "Copy", kind: "icon" },
- { box: [0.8269, 0, 0.86, 0.0305], label: "a power or power-up function.", kind: "icon" },
- { box: [0.883, 0.945, 0.8975, 0.9735], label: "Diamond", kind: "icon" },
- { box: [0.7599, 0.1434, 0.9877, 0.5136], label: "Siri", kind: "icon" },
- { box: [0.512, 0.118, 0.544, 0.1666], label: "Duck", kind: "icon" },
- { box: [0.0118, 0.1825, 0.0333, 0.2174], label: "Find", kind: "icon" },
- { box: [0.2638, 0.1017, 0.2789, 0.1318], label: "A logo for the Volkswagen company.", kind: "icon" },
- { box: [0.0048, 0, 0.0255, 0.03], label: "Apple Store", kind: "icon" },
- { box: [0.0103, 0.2255, 0.0436, 0.2743], label: "Heart", kind: "icon" },
- { box: [0.9776, 0.9449, 0.9948, 0.975], label: "Open Menu", kind: "icon" },
- { box: [0.9686, 0.515, 0.987, 0.5459], label: "Add", kind: "icon" },
- { box: [0.0, 0, 0.0055, 0.0297], label: "a field or field with trees.", kind: "icon" },
- { box: [0.2735, 0.3128, 0.2999, 0.3478], label: "All", kind: "icon" },
- { box: [0.2177, 0.0997, 0.2408, 0.1379], label: "Decrease", kind: "icon" },
- { box: [0.9319, 0.9451, 0.9557, 0.9738], label: "Increase", kind: "icon" },
- { box: [0.007, 0.9328, 0.0427, 0.9868], label: "Microsoft Edge browser.", kind: "icon" },
- ],
- },
- {
- key: "apple-music", title: "Apple Music", image: "/donkeyvision/apple-music.jpg",
- width: 3456, height: 2234,
- elements: [
- { box: [0.1422, 0.1234, 0.2264, 0.1431], label: "Top Picks for You", kind: "text" },
- { box: [0.1435, 0.5084, 0.2213, 0.5262], label: "Recently Played", kind: "text" },
- { box: [0.1422, 0.7394, 0.2047, 0.7532], label: "David Le's Station", kind: "text" },
- { box: [0.2838, 0.7394, 0.3004, 0.7532], label: "New", kind: "text" },
- { box: [0.308, 0.7394, 0.3297, 0.7532], label: "K-Pop", kind: "text" },
- { box: [0.4241, 0.7394, 0.4636, 0.7532], label: "Parachutes", kind: "text" },
- { box: [0.5644, 0.7394, 0.5938, 0.7532], label: "Inspired", kind: "text" },
- { box: [0.618, 0.7394, 0.6384, 0.7532], label: "Punk", kind: "text" },
- { box: [0.706, 0.7394, 0.7417, 0.7532], label: "Discovery", kind: "text" },
- { box: [0.1626, 0.7532, 0.1881, 0.767], label: "Station", kind: "text" },
- { box: [0.2838, 0.7532, 0.3476, 0.767], label: "Apple Music K-Pop", kind: "text" },
- { box: [0.4241, 0.7532, 0.4547, 0.767], label: "Coldplay", kind: "text" },
- { box: [0.5644, 0.7532, 0.6422, 0.767], label: "Apple Music Electronic", kind: "text" },
- { box: [0.706, 0.7532, 0.7226, 0.767], label: "Daft", kind: "text" },
- { box: [0.1728, 0.8144, 0.1932, 0.8282], label: "More", kind: "text" },
- { box: [0.4292, 0.8578, 0.4764, 0.8776], label: "Ihe verve", kind: "text" },
- { box: [0.3061, 0.8687, 0.3941, 0.8963], label: "COOLNEW", kind: "text" },
- { box: [0.3291, 0.8944, 0.3712, 0.92], label: "MOM", kind: "text" },
- { box: [0.0261, 0.9704, 0.0606, 0.9842], label: "David Le", kind: "text" },
- { box: [0.1418, 0.1489, 0.3018, 0.4813], label: "Rold Dinner Nlall", kind: "icon" },
- { box: [0.8167, 0.1482, 0.9796, 0.481], label: "Music Energy moving' nonetol Hlah-voltogo", kind: "icon" },
- { box: [0.8431, 0.5307, 0.9802, 0.7723], label: "Music SUNNERTIME Marshmello'$ Summer Sounds Apple Music Summ", kind: "icon" },
- { box: [0.7041, 0.532, 0.8382, 0.7384], label: "dopt pum", kind: "icon" },
- { box: [0.4238, 0.5319, 0.5567, 0.7378], label: "COLDMLAY", kind: "icon" },
- { box: [0.48, 0.1521, 0.6402, 0.4799], label: "BITch @", kind: "icon" },
- { box: [0.141, 0.5317, 0.2782, 0.7394], label: "Music", kind: "icon" },
- { box: [0.5636, 0.5313, 0.6981, 0.7485], label: "#Music Inspired By", kind: "icon" },
- { box: [0.282, 0.5304, 0.4166, 0.7399], label: "Music", kind: "icon" },
- { box: [0.1406, 0.8553, 0.2768, 1], label: "STEREOPHONICS Vou Gotta THERE COME BAck", kind: "icon" },
- { box: [0.3101, 0.1513, 0.471, 0.4807], label: "EMusic Vov Davld Lo's Statlon", kind: "icon" },
- { box: [0.0091, 0.1041, 0.1168, 0.1345], label: "Home", kind: "icon" },
- { box: [0.6484, 0.1508, 0.8098, 0.4822], label: "#Music plobor Biobar Similar Artists Station", kind: "icon" },
- { box: [0.0093, 0.133, 0.1172, 0.1631], label: "New", kind: "icon" },
- { box: [0.0077, 0.2472, 0.1177, 0.2778], label: "Artists", kind: "icon" },
- { box: [0.0086, 0.2763, 0.1193, 0.3064], label: "Albums", kind: "icon" },
- { box: [0.0079, 0.3049, 0.1192, 0.3368], label: "Songs", kind: "icon" },
- { box: [0.008, 0.3606, 0.1189, 0.3924], label: "Playlists", kind: "icon" },
- { box: [0.0085, 0.1619, 0.118, 0.1913], label: "Radio", kind: "icon" },
- { box: [0.0079, 0.3907, 0.1192, 0.4211], label: "Favorite Songs", kind: "icon" },
- { box: [0.0089, 0.2178, 0.119, 0.2491], label: "Recently Added", kind: "icon" },
- { box: [0.0089, 0.071, 0.1169, 0.1055], label: "Search", kind: "icon" },
- { box: [0.1076, 0, 0.1395, 0.0286], label: "Song", kind: "icon" },
- { box: [0.1679, 0, 0.2105, 0.029], label: "Controls", kind: "icon" },
- { box: [0.139, 0, 0.1687, 0.0283], label: "Vicw", kind: "icon" },
- { box: [0.2496, 0, 0.29, 0.0278], label: "Window", kind: "icon" },
- { box: [0.29, 0, 0.3189, 0.0284], label: "Help", kind: "icon" },
- { box: [0.2095, 0, 0.2511, 0.0277], label: "Account", kind: "icon" },
- { box: [0.025, 0, 0.0594, 0.0309], label: "Music", kind: "icon" },
- { box: [0.1434, 0.0676, 0.2017, 0.1045], label: "Home", kind: "icon" },
- { box: [0.0092, 0.1917, 0.1188, 0.222], label: "Library", kind: "icon" },
- { box: [0.1722, 0.8255, 0.2384, 0.8464], label: "Parachutes", kind: "icon" },
- { box: [0.8464, 0.8555, 0.979, 1], label: "A night time or night-time-related function.", kind: "icon" },
- { box: [0.7028, 0, 0.7232, 0.0287], label: "Toggleoggleoggle", kind: "icon" },
- { box: [0.7245, 0, 0.7445, 0.029], label: "Toggleoggleoggle", kind: "icon" },
- { box: [0.7694, 0, 0.7869, 0.0301], label: "Navigator", kind: "icon" },
- { box: [0.084, 0, 0.1091, 0.0287], label: "Edit", kind: "icon" },
- { box: [0.7455, 0, 0.7671, 0.0296], label: "Pill", kind: "icon" },
- { box: [0.7881, 0, 0.8071, 0.0301], label: "Navigation Bar", kind: "icon" },
- { box: [0.6728, 0.9424, 0.6938, 0.977], label: "Comment", kind: "icon" },
- { box: [0.8081, 0, 0.8302, 0.0305], label: "Text formatting.", kind: "icon" },
- { box: [0.6946, 0.9429, 0.7138, 0.9775], label: "Bullets", kind: "icon" },
- { box: [0.005, 0, 0.0252, 0.0304], label: "Apple", kind: "icon" },
- { box: [0.7363, 0.9377, 0.7598, 0.9812], label: "Mute", kind: "icon" },
- { box: [0.5635, 0.8542, 0.698, 0.9592], label: "a calendar or a calendar application.", kind: "icon" },
- { box: [0.006, 0.0351, 0.0515, 0.0709], label: "More options", kind: "icon" },
- { box: [0.9, 0, 0.9216, 0.0311], label: "More Options", kind: "icon" },
- { box: [0.7162, 0.938, 0.7348, 0.9814], label: "Podcast", kind: "icon" },
- { box: [0.8757, 0, 0.8985, 0.0315], label: "WiFi connectivity", kind: "icon" },
- { box: [0.0085, 0.336, 0.1197, 0.3647], label: "View as full screen", kind: "icon" },
- { box: [0.8302, 0, 0.8646, 0.0311], label: "Shuffle or randomize.", kind: "icon" },
- { box: [0.3823, 0.9348, 0.4142, 0.9956], label: "Play video.", kind: "icon" },
- { box: [0.5002, 0.9794, 0.5652, 1], label: "A filter or filter element.", kind: "icon" },
- { box: [0.9879, 0.3786, 1, 0.4791], label: "System", kind: "icon" },
- { box: [0.3596, 0.9364, 0.3837, 0.9854], label: "a code block or command line.", kind: "icon" },
- { box: [0.9894, 0.5312, 1, 0.7462], label: "a button to close a window or dialog box.", kind: "icon" },
- { box: [0.1419, 0.8093, 0.169, 0.8494], label: "GoldGold", kind: "icon" },
- { box: [0.6415, 0.9452, 0.662, 0.985], label: "a person.", kind: "icon" },
- { box: [0.0, 0, 0.0054, 0.0294], label: "a field or field with trees.", kind: "icon" },
- { box: [0.9176, 0, 0.9971, 0.0308], label: "Fullscreen", kind: "icon" },
- { box: [0.9882, 0.8543, 1, 1], label: "a file.", kind: "icon" },
- { box: [0.4142, 0.9349, 0.4452, 0.9836], label: "Play", kind: "icon" },
- { box: [0.0063, 0.963, 0.0243, 0.9921], label: "DL (Ctrl+F4)", kind: "icon" },
- { box: [0.5997, 0.9765, 0.696, 1], label: "a paintbrush tool.", kind: "icon" },
- ],
- },
- {
- key: "desktop", title: "Desktop", image: "/donkeyvision/desktop.jpg",
- width: 3456, height: 2234,
- elements: [
- { box: [0.1665, 0.0089, 0.1983, 0.0227], label: "Window", kind: "text" },
- { box: [0.2073, 0.0089, 0.2264, 0.0227], label: "Help", kind: "text" },
- { box: [0.9611, 0.0069, 0.9917, 0.0227], label: "4;05PM", kind: "text" },
- { box: [0.354, 0.1194, 0.3909, 0.1352], label: "Desktop", kind: "text" },
- { box: [0.845, 0.1214, 0.8718, 0.1352], label: "spotify", kind: "text" },
- { box: [0.5261, 0.155, 0.5568, 0.1668], label: "Modified", kind: "text" },
- { box: [0.3272, 0.1826, 0.3744, 0.1964], label: "torm options", kind: "text" },
- { box: [0.5249, 0.1807, 0.5555, 0.1964], label: "5, 2026", kind: "text" },
- { box: [0.5631, 0.1807, 0.595, 0.1964], label: "2.43PM", kind: "text" },
- { box: [0.6709, 0.1797, 0.6958, 0.1974], label: "Folder", kind: "text" },
- { box: [0.3272, 0.2004, 0.3693, 0.2142], label: "rough-cuts", kind: "text" },
- { box: [0.5096, 0.1984, 0.558, 0.2142], label: "Feb 16, 2026", kind: "text" },
- { box: [0.567, 0.1984, 0.6001, 0.2142], label: "10:12PM", kind: "text" },
- { box: [0.1996, 0.2221, 0.2328, 0.2359], label: "Favorites", kind: "text" },
- { box: [0.3272, 0.2182, 0.3476, 0.232], label: "temp", kind: "text" },
- { box: [0.5236, 0.2162, 0.5593, 0.232], label: "20, 2026", kind: "text" },
- { box: [0.567, 0.2162, 0.5989, 0.232], label: "4*43PM", kind: "text" },
- { box: [0.6716, 0.2182, 0.6958, 0.232], label: "Folder", kind: "text" },
- { box: [0.3272, 0.2359, 0.3514, 0.2498], label: "anime", kind: "text" },
- { box: [0.5096, 0.2359, 0.5274, 0.2498], label: "May", kind: "text" },
- { box: [0.5338, 0.234, 0.5555, 0.2498], label: "2026", kind: "text" },
- { box: [0.5631, 0.2359, 0.5989, 0.2498], label: "10:53AM", kind: "text" },
- { box: [0.6703, 0.2359, 0.6958, 0.2498], label: "Folder", kind: "text" },
- { box: [0.3272, 0.2537, 0.3463, 0.2675], label: "saas", kind: "text" },
- { box: [0.5096, 0.2537, 0.5274, 0.2675], label: "May", kind: "text" },
- { box: [0.5389, 0.2517, 0.5593, 0.2675], label: "2026", kind: "text" },
- { box: [0.5682, 0.2517, 0.5989, 0.2675], label: "6*35PM", kind: "text" },
- { box: [0.6716, 0.2537, 0.6958, 0.2675], label: "Folder", kind: "text" },
- { box: [0.3272, 0.2715, 0.3489, 0.2853], label: "other", kind: "text" },
- { box: [0.5096, 0.2695, 0.5899, 0.2853], label: "May 29, 2026 at 11:50", kind: "text" },
- { box: [0.6703, 0.2715, 0.6958, 0.2853], label: "Folder", kind: "text" },
- { box: [0.3272, 0.2892, 0.3795, 0.3031], label: "donkey-vision", kind: "text" },
- { box: [0.5338, 0.2873, 0.5542, 0.3031], label: "2026", kind: "text" },
- { box: [0.5619, 0.2892, 0.5938, 0.3031], label: "2.47PM", kind: "text" },
- { box: [0.6384, 0.2892, 0.6665, 0.3031], label: "3,6 MB", kind: "text" },
- { box: [0.6716, 0.2883, 0.7124, 0.305], label: "PNG image", kind: "text" },
- { box: [0.3272, 0.307, 0.354, 0.3208], label: "spotify", kind: "text" },
- { box: [0.5108, 0.307, 0.5644, 0.3208], label: "Today at 10.04", kind: "text" },
- { box: [0.6384, 0.307, 0.6665, 0.3208], label: "5.4 MB", kind: "text" },
- { box: [0.6716, 0.306, 0.7124, 0.3228], label: "PNG image", kind: "text" },
- { box: [0.3272, 0.3248, 0.3744, 0.3406], label: "apple-music", kind: "text" },
- { box: [0.5108, 0.3248, 0.5733, 0.3386], label: "Today at 3*43PM", kind: "text" },
- { box: [0.6511, 0.3248, 0.6665, 0.3386], label: "MB", kind: "text" },
- { box: [0.6716, 0.3238, 0.7124, 0.3406], label: "PNG image", kind: "text" },
- { box: [0.1996, 0.3643, 0.2341, 0.3781], label: "Locatlong", kind: "text" },
- { box: [0.2175, 0.5874, 0.2366, 0.6012], label: "Blue", kind: "text" },
- { box: [0.2175, 0.7295, 0.243, 0.7433], label: "Yellow", kind: "text" },
- { box: [0.229, 0.7858, 0.2519, 0.8036], label: "Tags ..", kind: "text" },
- { box: [0.9407, 0.1574, 1, 0.2531], label: "rough cuts", kind: "icon" },
- { box: [0.1967, 0.463, 0.2971, 0.4942], label: "AirDrop", kind: "icon" },
- { box: [0.1955, 0.4353, 0.2972, 0.4652], label: "dle", kind: "icon" },
- { box: [0.944, 0.2516, 0.9989, 0.3299], label: "Tor options", kind: "icon" },
- { box: [0.1963, 0.2623, 0.2966, 0.294], label: "Desktop", kind: "icon" },
- { box: [0.9434, 0.064, 0.9949, 0.1539], label: "temp", kind: "icon" },
- { box: [0.1951, 0.1455, 0.2993, 0.1786], label: "Recents", kind: "icon" },
- { box: [0.1943, 0.1761, 0.2993, 0.2089], label: "Shared", kind: "icon" },
- { box: [0.1968, 0.4923, 0.2978, 0.5229], label: "Network", kind: "icon" },
- { box: [0.1969, 0.2907, 0.2982, 0.3225], label: "Documents", kind: "icon" },
- { box: [0.1965, 0.5208, 0.2984, 0.5524], label: "Trash", kind: "icon" },
- { box: [0.8342, 0.1568, 0.8826, 0.2444], label: "applc", kind: "icon" },
- { box: [0.1963, 0.3774, 0.2972, 0.4088], label: "iCloud Drive", kind: "icon" },
- { box: [0.8968, 0.2486, 0.942, 0.3322], label: "other", kind: "icon" },
- { box: [0.8273, 0.2445, 0.8871, 0.3261], label: "donkey-vision", kind: "icon" },
- { box: [0.1956, 0.3204, 0.2983, 0.3549], label: "Downloads", kind: "icon" },
- { box: [0.1964, 0.5522, 0.2971, 0.584], label: "Tags", kind: "icon" },
- { box: [0.196, 0.2256, 0.2973, 0.2643], label: "Applications", kind: "icon" },
- { box: [0.1964, 0.6081, 0.2982, 0.6383], label: "Work", kind: "icon" },
- { box: [0.1977, 0.6353, 0.2982, 0.6659], label: "Important", kind: "icon" },
- { box: [0.1977, 0.6921, 0.2987, 0.7227], label: "Orange", kind: "icon" },
- { box: [0.0851, 0, 0.1394, 0.0264], label: "Edit View", kind: "icon" },
- { box: [0.6664, 0.1945, 0.7395, 0.2153], label: "Folder", kind: "icon" },
- { box: [0.0261, 0, 0.0606, 0.0284], label: "Finder", kind: "icon" },
- { box: [0.1976, 0.6627, 0.2981, 0.6947], label: "Red", kind: "icon" },
- { box: [0.1983, 0.7498, 0.2992, 0.7812], label: "Purple", kind: "icon" },
- { box: [0.6907, 0.1071, 0.7142, 0.1432], label: "a dropdown menu.", kind: "icon" },
- { box: [0.5388, 0.1075, 0.5692, 0.1428], label: "Decrease", kind: "icon" },
- { box: [0.576, 0.1072, 0.6076, 0.1422], label: "a search engine or more options.", kind: "icon" },
- { box: [0.6119, 0.1078, 0.6329, 0.1421], label: "Share", kind: "icon" },
- { box: [0.716, 0.1055, 0.7417, 0.1438], label: "Search", kind: "icon" },
- { box: [0.8968, 0.1576, 0.9428, 0.2464], label: "Save", kind: "icon" },
- { box: [0.7023, 0, 0.7235, 0.0319], label: "Toggleoggleoggle", kind: "icon" },
- { box: [0.1954, 0.4051, 0.2972, 0.4372], label: "View as full", kind: "icon" },
- { box: [0.7246, 0, 0.745, 0.0316], label: "Toggleoggleoggle", kind: "icon" },
- { box: [0.8872, 0.0649, 0.9443, 0.1579], label: "online", kind: "icon" },
- { box: [0.633, 0.1075, 0.6556, 0.142], label: "Toggle Pane", kind: "icon" },
- { box: [0.7453, 0, 0.7669, 0.0316], label: "Pill", kind: "icon" },
- { box: [0.6554, 0.1075, 0.6764, 0.1419], label: "More options", kind: "icon" },
- { box: [0.788, 0, 0.8079, 0.031], label: "Navigation", kind: "icon" },
- { box: [0.8086, 0, 0.8312, 0.0306], label: "A text input or translation option.", kind: "icon" },
- { box: [0.305, 0.1085, 0.3268, 0.1423], label: "Back", kind: "icon" },
- { box: [0.7683, 0, 0.7878, 0.0315], label: "Navigator", kind: "icon" },
- { box: [0.3269, 0.1084, 0.3481, 0.142], label: "Next", kind: "icon" },
- { box: [0.1941, 0.1108, 0.2186, 0.1407], label: "Color picker", kind: "icon" },
- { box: [0.3054, 0.2678, 0.326, 0.2859], label: "Undo (Ctrl+T)", kind: "icon" },
- { box: [0.8775, 0, 0.8992, 0.0335], label: "Wi-Fi connectivity.", kind: "icon" },
- { box: [0.3055, 0.3218, 0.3261, 0.342], label: "Image", kind: "icon" },
- { box: [0.3064, 0.2504, 0.3259, 0.267], label: "Undo (Ctrl+F)", kind: "icon" },
- { box: [0.3068, 0.1987, 0.3252, 0.214], label: "System", kind: "icon" },
- { box: [0.3067, 0.233, 0.326, 0.2501], label: "Undo (Ctrl+F)", kind: "icon" },
- { box: [0.6554, 0.197, 0.6711, 0.2144], label: "More options", kind: "icon" },
- { box: [0.2239, 0.1148, 0.2353, 0.136], label: "Green", kind: "icon" },
- { box: [0.0047, 0, 0.0249, 0.0294], label: "Apple's logo.", kind: "icon" },
- { box: [0.4239, 0.8663, 0.5138, 0.983], label: "Slide", kind: "icon" },
- { box: [0.3481, 0.8697, 0.4112, 1], label: "Lavender", kind: "icon" },
- { box: [0.0, 0, 0.0056, 0.027], label: "a field or field.", kind: "icon" },
- { box: [0.307, 0.2149, 0.3261, 0.232], label: "Undo", kind: "icon" },
- { box: [0.1988, 0.7216, 0.2182, 0.7509], label: "Yellow", kind: "icon" },
- { box: [0.3072, 0.1802, 0.3148, 0.1948], label: "Next", kind: "icon" },
- { box: [0.9908, 0.0644, 1, 0.1496], label: "a user profile or account.", kind: "icon" },
- { box: [0.3148, 0.1798, 0.3258, 0.1953], label: "a folder.", kind: "icon" },
- { box: [0.305, 0.3029, 0.3265, 0.3206], label: "Photo", kind: "icon" },
- { box: [0.2021, 0.5859, 0.2137, 0.6025], label: "Blue", kind: "icon" },
- { box: [0.9004, 0, 0.922, 0.0326], label: "View as full screen", kind: "icon" },
- { box: [0.8285, 0, 0.8563, 0.0334], label: "a secure or secure connection.", kind: "icon" },
- { box: [0.305, 0.2866, 0.3264, 0.3035], label: "Toggleoggleoggle", kind: "icon" },
- { box: [0.6563, 0.2514, 0.6668, 0.2672], label: "More options", kind: "icon" },
- ],
- },
-];
diff --git a/site/src/components/UsageHistoryCard.tsx b/site/src/components/UsageHistoryCard.tsx
index 38c92f22..da7fc70a 100644
--- a/site/src/components/UsageHistoryCard.tsx
+++ b/site/src/components/UsageHistoryCard.tsx
@@ -2,14 +2,8 @@
import { Fragment, useRef, useState, type KeyboardEvent } from "react";
-import { useUsage, type VisionUsage } from "@/queries/billing";
+import { useUsage, type UsageHistory } from "@/queries/billing";
import { Badge } from "@/components/ui/badge";
-import {
- Card,
- CardContent,
- CardHeader,
- CardTitle,
-} from "@/components/ui/card";
import { Skeleton } from "@/components/ui/skeleton";
import {
Pagination,
@@ -29,12 +23,12 @@ import {
} from "@/components/ui/table";
import { cn } from "@/lib/utils";
-type RecentCall = VisionUsage["recent"][number];
+type RecentCall = UsageHistory["recent"][number];
// Stable identity for a call so expansion state survives a background refetch
// (which can reorder/prepend rows) instead of being pinned to an array index.
function callKey(call: RecentCall): string {
- return `${call.createdAt}-${call.product}-${call.requestKind}-${call.costCredits}`;
+ return `${call.createdAt}-${call.requestKind}-${call.costCredits}`;
}
// Cents by default, but show a third decimal when a value carries sub-cent
@@ -148,13 +142,6 @@ function formatDuration(millis: number): string {
return millis >= 1000 ? `${(millis / 1000).toFixed(1)}s` : `${millis}ms`;
}
-const tabs = [
- { key: "app", label: "App" },
- { key: "vision", label: "Vision API" },
-] as const;
-
-type TabKey = (typeof tabs)[number]["key"];
-
// Page the table client-side, capped so the control never grows past 5 pages.
const ROWS_PER_PAGE = 25;
const MAX_PAGES = 5;
@@ -164,10 +151,6 @@ function CallDetail({ call }: { call: RecentCall }) {
const imageBilled = usage.generationCount > 0;
const tokenBilled =
usage.inputTokens > 0 || usage.outputTokens > 0 || usage.totalTokens > 0;
- // Vision parses report no token/image usage — they're a flat per-call charge,
- // so there's no breakdown to show, just the pricing model.
- const flatRate =
- !imageBilled && !tokenBilled && call.requestKind === "vision_parse";
const stats: { label: string; value: string }[] = [];
if (imageBilled) {
@@ -204,9 +187,7 @@ function CallDetail({ call }: { call: RecentCall }) {
? "Image generation is priced per image, not by tokens."
: tokenBilled
? "Cost is driven by tokens, and input dominates here. Cached input is an unchanged prompt prefix the provider reuses at a fraction of the rate; a low cached share on a repeated call means the cache isn't being hit."
- : flatRate
- ? "Flat rate — vision parses are billed per call."
- : "No billable usage was recorded for this call.";
+ : "No billable usage was recorded for this call.";
return (
<div className="space-y-3 px-2 py-1 text-sm">
@@ -228,18 +209,10 @@ function CallDetail({ call }: { call: RecentCall }) {
);
}
-// showVision=false (Cut's usage page) drops the Vision API tab and shows only
-// app calls. plain=true drops the card chrome entirely — the table renders
-// directly on the page and stretches to the space its parent gives it.
-export function UsageHistoryCard({
- showVision = true,
- plain = false,
-}: {
- showVision?: boolean;
- plain?: boolean;
-}) {
+// The table renders directly on the page and stretches to the space its
+// parent gives it; the surrounding pane owns scrolling.
+export function UsageHistoryCard() {
const usage = useUsage();
- const [tab, setTab] = useState<TabKey>("app");
const [page, setPage] = useState(1);
// Keyed by call identity (not row index) so a refetch can't leave the open
// detail pinned to whatever row now sits at that index.
@@ -248,41 +221,22 @@ export function UsageHistoryCard({
const rowRefs = useRef<(HTMLTableRowElement | null)[]>([]);
if (usage.isLoading) {
- if (plain) {
- return <Skeleton className="h-64 w-full" />;
- }
- return (
- <Card>
- <CardHeader>
- <Skeleton className="h-6 w-40" />
- </CardHeader>
- <CardContent>
- <Skeleton className="h-64 w-full" />
- </CardContent>
- </Card>
- );
+ return <Skeleton className="h-64 w-full" />;
}
- const recent = usage.data?.recent ?? [];
- const tabRows = recent.filter((call) => call.product === tab);
+ const rows = usage.data?.recent ?? [];
// Annotate in the server's chronological order so conversations group in place
// and individual (no-conversation) calls stay interleaved between them.
- const annotations = annotateRows(tabRows);
+ const annotations = annotateRows(rows);
const totalPages = Math.min(
MAX_PAGES,
- Math.max(1, Math.ceil(tabRows.length / ROWS_PER_PAGE)),
+ Math.max(1, Math.ceil(rows.length / ROWS_PER_PAGE)),
);
- // Clamp in case rows shrank (refetch / tab change) below the current page.
+ // Clamp in case rows shrank on a refetch below the current page.
const currentPage = Math.min(page, totalPages);
const pageStart = (currentPage - 1) * ROWS_PER_PAGE;
- const pageRows = tabRows.slice(pageStart, pageStart + ROWS_PER_PAGE);
-
- const selectTab = (next: TabKey) => {
- setTab(next);
- setPage(1);
- setExpanded(null);
- };
+ const pageRows = rows.slice(pageStart, pageStart + ROWS_PER_PAGE);
const goToPage = (next: number) => {
setPage(next);
@@ -314,53 +268,17 @@ export function UsageHistoryCard({
}
};
- // In a height-constrained flex column (Cut's usage page) the wrapper fills
- // the remaining space, the table body scrolls on its own with the column
- // header row pinned, and the pagination stays visible below. In normal flow
- // (the apex settings) these classes are inert and the card grows as before.
- const body = (
- <>
- {showVision ? (
- <div className="inline-flex rounded-md border p-0.5">
- {tabs.map((item) => (
- <button
- key={item.key}
- onClick={() => selectTab(item.key)}
- type="button"
- className={cn(
- "rounded px-3 py-1 text-sm font-medium transition-colors",
- tab === item.key
- ? "bg-muted text-foreground"
- : "text-muted-foreground hover:text-foreground",
- )}
- >
- {item.label}
- </button>
- ))}
- </div>
- ) : null}
-
+ return (
+ <div className="space-y-4">
{/* The table's own container must stay overflow-visible so the sticky
- header row pins against the real scroll ancestor: this wrapper in
- card mode, the surrounding pane in plain mode. */}
- <div
- className={cn(
- "[&_[data-slot=table-container]]:overflow-visible",
- !plain && "min-h-0 flex-1 overflow-y-auto",
- )}
- >
- {tabRows.length === 0 ? (
- <p className="text-sm text-muted-foreground">
- No {tab === "app" ? "app" : "Vision API"} calls yet.
- </p>
+ header row pins against the real scroll ancestor, the surrounding
+ pane. */}
+ <div className="[&_[data-slot=table-container]]:overflow-visible">
+ {rows.length === 0 ? (
+ <p className="text-sm text-muted-foreground">No calls yet.</p>
) : (
<Table>
- <TableHeader
- className={cn(
- "sticky top-0 z-10",
- plain ? "bg-background" : "bg-card",
- )}
- >
+ <TableHeader className="sticky top-0 z-10 bg-background">
<TableRow>
<TableHead>Date</TableHead>
<TableHead>Kind</TableHead>
@@ -461,7 +379,7 @@ export function UsageHistoryCard({
</div>
{totalPages > 1 ? (
- <div className={cn(plain && "sticky bottom-0 z-10 bg-background py-3")}>
+ <div className="sticky bottom-0 z-10 bg-background py-3">
<Pagination>
<PaginationContent>
<PaginationItem>
@@ -500,20 +418,6 @@ export function UsageHistoryCard({
</Pagination>
</div>
) : null}
- </>
- );
-
- if (plain) {
- return <div className="space-y-4">{body}</div>;
- }
- return (
- <Card className="min-h-0 flex-1">
- <CardHeader>
- <CardTitle>Usage history</CardTitle>
- </CardHeader>
- <CardContent className="flex min-h-0 flex-1 flex-col gap-4">
- {body}
- </CardContent>
- </Card>
+ </div>
);
}
diff --git a/site/src/cut/components/ProjectsHome.tsx b/site/src/cut/components/ProjectsHome.tsx
index 132856cc..97757d1f 100644
--- a/site/src/cut/components/ProjectsHome.tsx
+++ b/site/src/cut/components/ProjectsHome.tsx
@@ -189,7 +189,7 @@ function GraceBanner({ enabled }: { enabled: boolean }) {
onClick={async () => {
track("pro_checkout_started");
try {
- const { url } = await checkout.mutateAsync("pro");
+ const { url } = await checkout.mutateAsync();
window.location.assign(url);
} catch {
window.location.assign(`${base}/settings`);
diff --git a/site/src/cut/lib/proUpgrade.ts b/site/src/cut/lib/proUpgrade.ts
index ed4e0eeb..cffefd6b 100644
--- a/site/src/cut/lib/proUpgrade.ts
+++ b/site/src/cut/lib/proUpgrade.ts
@@ -13,7 +13,7 @@ export function useUpgradeToPro() {
track("pro_checkout_started");
void (async () => {
try {
- const { url } = await checkout.mutateAsync("pro");
+ const { url } = await checkout.mutateAsync();
window.location.assign(url);
} catch {
// Checkout can't start (an unconfigured plan 404s); the billing page
diff --git a/site/src/lib/auth.ts b/site/src/lib/auth.ts
index 9c0cb329..59afa65d 100644
--- a/site/src/lib/auth.ts
+++ b/site/src/lib/auth.ts
@@ -1,4 +1,3 @@
-import { apiKey } from "@better-auth/api-key";
import { betterAuth } from "better-auth";
import { prismaAdapter } from "better-auth/adapters/prisma";
@@ -6,10 +5,6 @@ import { DONKEYCUT_CANONICAL } from "@/cut/lib/hosts";
import { provisionSignupGrants } from "@/lib/onboarding/signup-grants";
import { prisma } from "@/lib/prisma";
-// Prefix for issued Vision API keys. The full secret is shown to the developer
-// once at creation; only a hash is stored (handled by the apiKey plugin).
-export const visionApiKeyPrefix = "dk_live_";
-
// donkeycut.com is the single production host: the sign-in pages, the auth
// API, the Google OAuth callback, and the session all live on that one origin
// (the proxy 308s www. to the apex before anything serves), so auth cookies
@@ -59,15 +54,4 @@ export const auth = betterAuth({
prompt: "select_account",
},
},
- plugins: [
- apiKey({
- // We enforce our own monthly call quota and per-key rate limit on the
- // vision route, so the plugin's built-in rate limiting stays off.
- defaultPrefix: visionApiKeyPrefix,
- enableMetadata: true,
- rateLimit: {
- enabled: false,
- },
- }),
- ],
});
diff --git a/site/src/lib/billing/pro-subscription.ts b/site/src/lib/billing/pro-subscription.ts
index aa48672b..c9ce31bf 100644
--- a/site/src/lib/billing/pro-subscription.ts
+++ b/site/src/lib/billing/pro-subscription.ts
@@ -9,7 +9,7 @@ import { creditMicrosPerCent, zeroCreditMicros } from "@/lib/credits/amounts";
import { grantCredits } from "@/lib/credits/inference";
import { prisma } from "@/lib/prisma";
-// Donkey Pro: the Mac app subscription, separate from the Vision API product.
+// Donkey Pro: the Mac app subscription.
export const proPlanKey = "pro";
// Stripe subscription statuses that include the Pro allowance.
diff --git a/site/src/lib/billing/stripe.ts b/site/src/lib/billing/stripe.ts
index 37ab0428..79c0b596 100644
--- a/site/src/lib/billing/stripe.ts
+++ b/site/src/lib/billing/stripe.ts
@@ -2,12 +2,6 @@ import Stripe from "stripe";
import { prisma } from "@/lib/prisma";
-// The Vision API product is a single self-serve plan. The monthly call quota is
-// read from the Stripe price metadata (key "monthlyCallQuota") so it can be
-// tuned in the Stripe dashboard without a deploy; this constant is the fallback.
-export const visionPlanKey = "vision";
-export const defaultVisionMonthlyQuota = 5_000;
-
let cachedStripe: Stripe | null = null;
export function getStripe(): Stripe {
@@ -31,36 +25,14 @@ export class StripeNotConfiguredError extends Error {
}
}
-export function visionPriceId(): string {
- const priceId = process.env.STRIPE_VISION_PRICE_ID;
- if (!priceId) {
- throw new StripeNotConfiguredError();
- }
-
- return priceId;
-}
-
// Optional: a specific billing portal configuration (bpc_...). When unset,
// Stripe uses the account's default portal configuration.
-export function visionPortalConfigurationId(): string | undefined {
+export function portalConfigurationId(): string | undefined {
return process.env.STRIPE_PORTAL_CONFIGURATION_ID || undefined;
}
-export function quotaFromPrice(price: Stripe.Price | null | undefined): number {
- const raw = price?.metadata?.monthlyCallQuota;
- if (raw) {
- const parsed = Number.parseInt(raw, 10);
- if (Number.isFinite(parsed) && parsed > 0) {
- return parsed;
- }
- }
-
- return defaultVisionMonthlyQuota;
-}
-
// Reuse one Stripe customer per user, stored on User.stripeCustomerId so every
-// billing product (Pro, Vision, top-ups) shares it and a customer is never
-// duplicated — and a Pro-only or credit-only user gets no phantom Vision row.
+// billing product (Pro, top-ups) shares it and a customer is never duplicated.
export async function ensureStripeCustomer(input: {
userId: string;
email: string;
@@ -74,20 +46,6 @@ export async function ensureStripeCustomer(input: {
return user.stripeCustomerId;
}
- // Legacy rows stored the customer id on the Vision subscription; reuse it (and
- // backfill the user) so an existing Vision customer is never re-created.
- const legacyVision = await prisma.visionApiSubscription.findUnique({
- select: { stripeCustomerId: true },
- where: { userId: input.userId },
- });
- if (legacyVision?.stripeCustomerId) {
- await prisma.user.update({
- data: { stripeCustomerId: legacyVision.stripeCustomerId },
- where: { id: input.userId },
- });
- return legacyVision.stripeCustomerId;
- }
-
const stripe = getStripe();
const customer = await stripe.customers.create({
email: input.email,
@@ -102,8 +60,8 @@ export async function ensureStripeCustomer(input: {
}
// Resolve which user a Stripe subscription belongs to: the userId stamped into
-// subscription metadata at checkout, falling back to the per-user Stripe customer
-// id. Shared by the Vision and Pro sync paths so they can't resolve differently.
+// subscription metadata at checkout, falling back to the per-user Stripe
+// customer id.
export async function resolveSubscriptionUserId(
subscription: Stripe.Subscription,
): Promise<string | null> {
@@ -124,63 +82,6 @@ export async function resolveSubscriptionUserId(
return null;
}
-// Source of truth for a subscription's lifecycle. Called from webhook handlers
-// with a fully-expanded Stripe subscription; maps it onto our row.
-export async function syncVisionSubscription(
- subscription: Stripe.Subscription,
-): Promise<void> {
- const userId = await resolveSubscriptionUserId(subscription);
- const customerId = stripeId(subscription.customer);
- if (!userId || !customerId) {
- console.error("[billing] could not resolve user/customer for subscription", {
- customer: customerId,
- subscription: subscription.id,
- userId,
- });
- return;
- }
-
- const item = subscription.items.data[0];
- const price = item?.price;
- const periodStart = unixToDate(item?.current_period_start ?? null);
- const periodEnd = unixToDate(item?.current_period_end ?? null);
-
- // Reset the per-period call counter when the billing period rolls over, so a
- // new period starts the quota fresh.
- const existing = await prisma.visionApiSubscription.findUnique({
- select: { currentPeriodStart: true },
- where: { userId },
- });
- const periodRolledOver =
- periodStart !== null &&
- existing?.currentPeriodStart?.getTime() !== periodStart.getTime();
-
- await prisma.visionApiSubscription.upsert({
- create: {
- cancelAtPeriodEnd: subscription.cancel_at_period_end,
- currentPeriodEnd: periodEnd,
- currentPeriodStart: periodStart,
- monthlyCallQuota: quotaFromPrice(price),
- planKey: visionPlanKey,
- status: subscription.status,
- stripeCustomerId: customerId,
- stripeSubscriptionId: subscription.id,
- userId,
- },
- update: {
- cancelAtPeriodEnd: subscription.cancel_at_period_end,
- currentPeriodEnd: periodEnd,
- currentPeriodStart: periodStart,
- monthlyCallQuota: quotaFromPrice(price),
- status: subscription.status,
- stripeCustomerId: customerId,
- stripeSubscriptionId: subscription.id,
- ...(periodRolledOver ? { periodCallCount: 0 } : {}),
- },
- where: { userId },
- });
-}
-
// Normalize a Stripe field that may be an id string or an expanded object (or
// absent) to its id. Shared by the webhook handlers.
export function stripeId(
@@ -193,8 +94,7 @@ export function stripeId(
return typeof value === "string" ? value : value.id;
}
-// Epoch seconds (Stripe's unit) to Date, or null when absent. Shared by the
-// Vision and Pro subscription sync paths.
+// Epoch seconds (Stripe's unit) to Date, or null when absent.
export function unixToDate(seconds: number | null | undefined): Date | null {
if (!seconds) {
return null;
diff --git a/site/src/lib/billing/vision-subscription.ts b/site/src/lib/billing/vision-subscription.ts
deleted file mode 100644
index 3d26154d..00000000
--- a/site/src/lib/billing/vision-subscription.ts
+++ /dev/null
@@ -1,66 +0,0 @@
-import type { VisionApiSubscription } from "@/generated/prisma/client";
-import { prisma } from "@/lib/prisma";
-
-// Stripe subscription statuses that grant Vision API access.
-const activeStatuses = new Set(["active", "trialing"]);
-
-export type ActiveVisionSubscription = VisionApiSubscription;
-
-export function isActiveVisionStatus(status: string) {
- return activeStatuses.has(status);
-}
-
-export async function getActiveVisionSubscription(
- userId: string,
-): Promise<ActiveVisionSubscription | null> {
- const subscription = await prisma.visionApiSubscription.findUnique({
- where: { userId },
- });
- if (!subscription || !isActiveVisionStatus(subscription.status)) {
- return null;
- }
-
- return subscription;
-}
-
-// Atomically reserve one call against the monthly quota. The conditional
-// UPDATE only increments when the row is still under quota, so concurrent
-// requests can never push periodCallCount past monthlyCallQuota — the count and
-// the check are a single statement. Returns the remaining calls after this one.
-export async function reserveVisionApiCall(input: {
- userId: string;
- monthlyCallQuota: number;
-}) {
- const reserved = await prisma.visionApiSubscription.updateMany({
- data: { periodCallCount: { increment: 1 } },
- where: {
- periodCallCount: { lt: input.monthlyCallQuota },
- userId: input.userId,
- },
- });
-
- if (reserved.count === 0) {
- return { ok: false as const };
- }
-
- const row = await prisma.visionApiSubscription.findUnique({
- select: { periodCallCount: true },
- where: { userId: input.userId },
- });
- const used = row?.periodCallCount ?? input.monthlyCallQuota;
-
- return {
- ok: true as const,
- remaining: Math.max(0, input.monthlyCallQuota - used),
- };
-}
-
-// Release a previously reserved call when the request ultimately fails, so a
-// failed parse doesn't consume the developer's quota (mirrors how the credit
-// path only charges successful calls). Never drops below zero.
-export async function releaseVisionApiCall(userId: string) {
- await prisma.visionApiSubscription.updateMany({
- data: { periodCallCount: { decrement: 1 } },
- where: { periodCallCount: { gt: 0 }, userId },
- });
-}
diff --git a/site/src/lib/credits/inference.ts b/site/src/lib/credits/inference.ts
index d9f099d0..d80c2f9c 100644
--- a/site/src/lib/credits/inference.ts
+++ b/site/src/lib/credits/inference.ts
@@ -4,7 +4,6 @@ import { Prisma, type PrismaClient } from "@/generated/prisma/client";
import { prisma } from "@/lib/prisma";
import {
creditMicrosToString,
- creditStringToMicros,
zeroCreditMicros,
} from "@/lib/credits/amounts";
import {
@@ -14,12 +13,10 @@ import {
import { isJsonObject, toJsonValue } from "@/lib/inference/json";
import type { JsonObject, JsonValue } from "@/lib/inference/providers";
-// UserCreditGrant.unit values. Dollar grants ("credit") feed the account
-// balance and pay for hosted inference; vision-call grants ("vision_call") are
-// an off-balance integer count consumed only by the Vision API quota path.
+// UserCreditGrant.unit value. Dollar grants ("credit") feed the account
+// balance and pay for hosted inference. Balance queries filter on this unit
+// because historical rows carry retired units.
export const creditGrantUnit = "credit";
-export const visionCallGrantUnit = "vision_call";
-export type CreditGrantUnit = typeof creditGrantUnit | typeof visionCallGrantUnit;
export const inferenceUsageRoutes = {
assets: "/api/inference/assets/",
@@ -28,7 +25,6 @@ export const inferenceUsageRoutes = {
chatCompletions: "/api/inference/chat/completions/",
responses: "/api/inference/responses/",
screenshotParse: "/api/inference/screenshots/parse/",
- vision: "/api/vision/",
} as const;
type CreditsDatabase = PrismaClient | Prisma.TransactionClient;
@@ -64,7 +60,7 @@ type InferenceUsageInput = {
userId: string;
clientId: string | null;
// The app conversation this call belongs to, for grouping usage history. Null
- // for background work and non-app (Vision API) callers.
+ // for background work.
conversationId?: string | null;
route: string;
requestKind: string;
@@ -74,9 +70,9 @@ type InferenceUsageInput = {
usage?: JsonValue;
errorCode?: string;
metadata?: JsonObject;
- // "included" records the event for usage/quota tracking without charging the
- // money-credit balance (the call is covered by a subscription, e.g. the
- // Vision API product). Defaults to "credits".
+ // "included" records the event for usage tracking without charging the
+ // money-credit balance (the call is covered by a subscription). Defaults
+ // to "credits".
billingMode?: "credits" | "included";
};
@@ -143,7 +139,6 @@ export async function grantCredits(input: {
amountMicros: bigint;
source: string;
sourceId?: string;
- unit?: CreditGrantUnit;
expiresAt?: Date;
periodStart?: Date;
periodEnd?: Date;
@@ -154,18 +149,12 @@ export async function grantCredits(input: {
throw new Error("Credit grants must be positive.");
}
- const unit = input.unit ?? creditGrantUnit;
- // Only dollar grants move the account balance and write a balance-tracking
- // ledger entry. Vision-call grants are an off-balance count, so they create
- // just the grant row.
- const affectsBalance = unit === creditGrantUnit;
-
if (input.sourceId) {
const existing = await prisma.userCreditGrant.findFirst({
where: {
source: input.source,
sourceId: input.sourceId,
- unit,
+ unit: creditGrantUnit,
userId: input.userId,
},
});
@@ -181,24 +170,20 @@ export async function grantCredits(input: {
// An account already carrying an overdraft — one that predates the
// write-off at charge time — starts from zero here, so the grant lands
// whole. Funding $25 credits $25.
- if (affectsBalance) {
- await forgiveOverdraft(tx, account.id, input.userId, account.balanceMicros, null);
- }
- const updatedAccount = affectsBalance
- ? await tx.userCreditAccount.update({
- data: {
- balanceMicros: {
- increment: input.amountMicros,
- },
- lifetimeGrantedMicros: {
- increment: input.amountMicros,
- },
- },
- where: {
- id: account.id,
- },
- })
- : account;
+ await forgiveOverdraft(tx, account.id, input.userId, account.balanceMicros, null);
+ const updatedAccount = await tx.userCreditAccount.update({
+ data: {
+ balanceMicros: {
+ increment: input.amountMicros,
+ },
+ lifetimeGrantedMicros: {
+ increment: input.amountMicros,
+ },
+ },
+ where: {
+ id: account.id,
+ },
+ });
const grant = await tx.userCreditGrant.create({
data: {
@@ -212,27 +197,25 @@ export async function grantCredits(input: {
remainingAmountMicros: input.amountMicros,
source: input.source,
sourceId: input.sourceId,
- unit,
+ unit: creditGrantUnit,
userId: input.userId,
},
});
- if (affectsBalance) {
- await tx.userCreditLedgerEntry.create({
- data: {
- accountId: account.id,
- amountMicros: input.amountMicros,
- balanceAfterMicros: updatedAccount.balanceMicros,
- description: input.description,
- grantId: grant.id,
- metadata: prismaJson(input.metadata),
- source: input.source,
- sourceId: input.sourceId,
- type: "grant",
- userId: input.userId,
- },
- });
- }
+ await tx.userCreditLedgerEntry.create({
+ data: {
+ accountId: account.id,
+ amountMicros: input.amountMicros,
+ balanceAfterMicros: updatedAccount.balanceMicros,
+ description: input.description,
+ grantId: grant.id,
+ metadata: prismaJson(input.metadata),
+ source: input.source,
+ sourceId: input.sourceId,
+ type: "grant",
+ userId: input.userId,
+ },
+ });
return grant;
},
@@ -250,7 +233,7 @@ export async function grantCredits(input: {
where: {
source: input.source,
sourceId: input.sourceId,
- unit,
+ unit: creditGrantUnit,
userId: input.userId,
},
});
@@ -555,8 +538,7 @@ export async function getCreditBalance(userId: string) {
gt: zeroCreditMicros,
},
status: "active",
- // The dollar balance view lists dollar grants only; vision-call grants
- // are surfaced through the Vision API quota path.
+ // Dollar grants only; historical rows can carry retired units.
unit: creditGrantUnit,
userId,
OR: [
@@ -749,8 +731,8 @@ async function expireCreditsForAccount(
gt: zeroCreditMicros,
},
status: "active",
- // Dollar grants only — vision-call grants are off-balance and expire via
- // their own path, so they must never decrement balanceMicros here.
+ // Dollar grants only — a historical row with a retired unit must never
+ // decrement balanceMicros here.
unit: creditGrantUnit,
},
});
@@ -858,8 +840,8 @@ async function debitGrants(
gt: zeroCreditMicros,
},
status: "active",
- // Dollar inference only spends dollar grants — vision-call grants are a
- // separate denomination and must never be debited here.
+ // Dollar inference only spends dollar grants — a historical row with a
+ // retired unit must never be debited here.
unit: creditGrantUnit,
},
});
@@ -1083,7 +1065,7 @@ async function resolveCreditRate(
const globalRate = rates.find((rate) => {
return rate.route === null && rate.provider === null && rate.model === null;
});
- const providerDefaultRate = defaultCreditRate(route, provider, model);
+ const providerDefaultRate = defaultCreditRate(provider, model);
if (exactRate || providerModelRate || providerDefaultRate.providerPricing) {
return rateSnapshot(
@@ -1096,11 +1078,6 @@ async function resolveCreditRate(
return rateSnapshot(globalRate, providerDefaultRate);
}
- // Flat-priced routes (vision) don't need per-model pricing.
- if (routeHasFlatPrice(route)) {
- return providerDefaultRate;
- }
-
// Every billable model must have a price — a code-level rate in provider-pricing.ts or a DB
// override. Fail loudly instead of silently charging a default so a missing price is caught.
throw new CreditRateNotConfiguredError(route, provider, model);
@@ -1135,35 +1112,21 @@ function rateSnapshot(
};
}
-// Flat per-call price for an app vision parse. Vision reports no token usage, so
-// each successful app (credits-billed) call is charged this flat amount. A DB
-// InferenceCreditRate row still overrides it.
-export const visionCallDefaultMicros = creditStringToMicros("0.004");
-
-// Routes priced per call regardless of model (no per-model pricing required): a vision parse is
-// a flat per-call charge. Every other route bills against a model price.
-function routeHasFlatPrice(route: string): boolean {
- return route === inferenceUsageRoutes.vision;
-}
-
function defaultCreditRate(
- route: string,
provider: string,
model: string,
): CreditRateSnapshot {
- const flatRequestMicros =
- route === inferenceUsageRoutes.vision ? visionCallDefaultMicros : zeroCreditMicros;
const providerPricing = providerCreditPricing(provider, model);
return {
id: null,
version: null,
- baseCostMicros: providerPricing ? zeroCreditMicros : flatRequestMicros,
+ baseCostMicros: zeroCreditMicros,
inputTokenCostMicros: zeroCreditMicros,
outputTokenCostMicros: zeroCreditMicros,
totalTokenCostMicros: zeroCreditMicros,
characterCostMicros: zeroCreditMicros,
- fallbackCostMicros: flatRequestMicros,
+ fallbackCostMicros: zeroCreditMicros,
providerPricing,
};
}
diff --git a/site/src/lib/credits/vision-grants.ts b/site/src/lib/credits/vision-grants.ts
deleted file mode 100644
index b89414a5..00000000
--- a/site/src/lib/credits/vision-grants.ts
+++ /dev/null
@@ -1,137 +0,0 @@
-import { Prisma } from "@/generated/prisma/client";
-
-import { zeroCreditMicros } from "@/lib/credits/amounts";
-import { grantCredits, visionCallGrantUnit } from "@/lib/credits/inference";
-import { prisma } from "@/lib/prisma";
-
-// Vision-call grants live in UserCreditGrant under unit="vision_call": an
-// off-balance integer count (stored in remainingAmountMicros) that the dollar
-// credit engine never touches. Consumption here is the *only* path that reads
-// these rows, so the two denominations can never cross.
-
-const oneCall = BigInt(1);
-const maxReserveAttempts = 5;
-
-function activeVisionGrantWhere(
- userId: string,
- now: Date,
-): Prisma.UserCreditGrantWhereInput {
- return {
- remainingAmountMicros: { gt: zeroCreditMicros },
- status: "active",
- unit: visionCallGrantUnit,
- userId,
- OR: [{ expiresAt: null }, { expiresAt: { gt: now } }],
- };
-}
-
-// Grant a one-time allotment of Vision API calls (signup bonus, promo, support
-// comp). Idempotent via (userId, unit, source, sourceId); optionally expiring.
-export async function grantVisionCalls(input: {
- userId: string;
- calls: number;
- source: string;
- sourceId?: string;
- expiresAt?: Date;
- description?: string;
-}) {
- if (!Number.isInteger(input.calls) || input.calls <= 0) {
- throw new Error("Vision call grants must be a positive integer.");
- }
-
- return grantCredits({
- amountMicros: BigInt(input.calls),
- description: input.description,
- expiresAt: input.expiresAt,
- source: input.source,
- sourceId: input.sourceId,
- unit: visionCallGrantUnit,
- userId: input.userId,
- });
-}
-
-// Flip expired vision-call grants to "expired". These grants never moved the
-// dollar balance, so a plain status update is the whole story (unlike dollar
-// expiry, which must also decrement balanceMicros).
-export async function expireVisionCallGrants(userId: string, now = new Date()) {
- await prisma.userCreditGrant.updateMany({
- data: { status: "expired" },
- where: {
- expiresAt: { lte: now },
- remainingAmountMicros: { gt: zeroCreditMicros },
- status: "active",
- unit: visionCallGrantUnit,
- userId,
- },
- });
-}
-
-// Total Vision API calls left across a user's active, unexpired grants.
-export async function visionCallGrantRemaining(userId: string): Promise<bigint> {
- const now = new Date();
- await expireVisionCallGrants(userId, now);
- const aggregate = await prisma.userCreditGrant.aggregate({
- _sum: { remainingAmountMicros: true },
- where: activeVisionGrantWhere(userId, now),
- });
-
- return aggregate._sum.remainingAmountMicros ?? zeroCreditMicros;
-}
-
-// Atomically consume one call from the user's grants, oldest-expiring first.
-// Mirrors the subscription quota reservation: a conditional decrement keyed to a
-// specific grant row, so concurrent calls can't double-spend, retried against
-// the next grant if a race empties the chosen one. Returns the debited grant id
-// so a failed request can hand the call back.
-export async function reserveVisionCallGrant(
- userId: string,
-): Promise<{ ok: true; grantId: string; remaining: bigint } | { ok: false }> {
- const now = new Date();
- await expireVisionCallGrants(userId, now);
-
- for (let attempt = 0; attempt < maxReserveAttempts; attempt += 1) {
- const grant = await prisma.userCreditGrant.findFirst({
- orderBy: [
- { expiresAt: { sort: "asc", nulls: "last" } },
- { createdAt: "asc" },
- ],
- where: activeVisionGrantWhere(userId, now),
- });
- if (!grant) {
- return { ok: false };
- }
-
- const debited = await prisma.userCreditGrant.updateMany({
- data: { remainingAmountMicros: { decrement: oneCall } },
- where: {
- id: grant.id,
- remainingAmountMicros: { gt: zeroCreditMicros },
- status: "active",
- },
- });
- if (debited.count === 0) {
- // Another request emptied this grant first; try the next candidate.
- continue;
- }
-
- // Drop the grant out of future selection once it reaches zero.
- await prisma.userCreditGrant.updateMany({
- data: { status: "exhausted" },
- where: { id: grant.id, remainingAmountMicros: { lte: zeroCreditMicros } },
- });
-
- return { grantId: grant.id, ok: true, remaining: await visionCallGrantRemaining(userId) };
- }
-
- return { ok: false };
-}
-
-// Hand a reserved call back to its grant when the request ultimately fails, so a
-// failed parse doesn't burn the allotment. Re-activates a grant the reservation
-// had just exhausted (but never resurrects an expired one).
-export async function releaseVisionCallGrant(grantId: string) {
- await prisma.userCreditGrant.updateMany({
- data: { remainingAmountMicros: { increment: oneCall }, status: "active" },
- where: { id: grantId, status: { not: "expired" } },
- });
-}
diff --git a/site/src/lib/donkey-api-auth.ts b/site/src/lib/donkey-api-auth.ts
index 0ea5274f..e1a8a66d 100644
--- a/site/src/lib/donkey-api-auth.ts
+++ b/site/src/lib/donkey-api-auth.ts
@@ -6,13 +6,12 @@ import { prisma } from "@/lib/prisma";
export type DonkeyAuthContext = {
platform: "api";
app: "donkey";
- method: "session-cookie" | "dev-bypass" | "api-key";
+ method: "session-cookie" | "dev-bypass";
clientId: string | null;
// The app's active conversation for this request, from x-donkey-conversation-id.
- // Null for background work (vision warming) and non-app callers (Vision API keys).
+ // Null for background work.
conversationId: string | null;
userId: string;
- apiKeyId: string | null;
};
// The known roles a route can require beyond being signed in. Add new roles
@@ -20,10 +19,6 @@ export type DonkeyAuthContext = {
export type DonkeyRole = "SUPER_USER";
export type DonkeyAuthOptions = {
- // Routes are session-only by default. Set true to also accept a Vision API
- // key as a bearer token. This is the typed allowlist for "which routes
- // support API keys" — no path string matching.
- allowApiKey?: boolean;
// When set, the authenticated user must hold this role or the request is
// rejected with 403 before the handler runs.
role?: DonkeyRole;
@@ -33,9 +28,9 @@ export type DonkeyAuthenticatedRequest = NextRequest & {
donkey: DonkeyAuthContext;
};
-// The real signed-in user's id, or null for api-key / dev-bypass callers. Use
-// this in session-only product routes (billing, API-key management) instead of
-// re-calling auth.api.getSession — withDonkeyAuth already authenticated.
+// The real signed-in user's id, or null for dev-bypass callers. Use this in
+// session-only product routes (billing) instead of re-calling
+// auth.api.getSession — withDonkeyAuth already authenticated.
export function donkeySessionUserId(
request: DonkeyAuthenticatedRequest,
): string | null {
@@ -91,11 +86,6 @@ export async function getDonkeyAuthContext(
return devBypass;
}
- const apiKeyContext = await apiKeyAuthContext(headers);
- if (apiKeyContext) {
- return apiKeyContext;
- }
-
const session = await auth.api.getSession({
headers,
});
@@ -112,47 +102,6 @@ export async function getDonkeyAuthContext(
clientId: clientId ? clientId : null,
conversationId: conversationIdFromHeaders(headers),
userId: session.user.id,
- apiKeyId: null,
- };
-}
-
-// Vision API keys are sent as a bearer token; that is the only accepted format.
-function apiKeyFromHeaders(headers: Headers): string | null {
- const authorization = headers.get("authorization")?.trim();
- if (!authorization?.toLowerCase().startsWith("bearer ")) {
- return null;
- }
-
- const token = authorization.slice("bearer ".length).trim();
- return token ? token : null;
-}
-
-async function apiKeyAuthContext(
- headers: Headers,
-): Promise<DonkeyAuthContext | null> {
- const key = apiKeyFromHeaders(headers);
- if (!key) {
- return null;
- }
-
- const verified = await auth.api.verifyApiKey({ body: { key } });
- if (!verified.valid || !verified.key) {
- return null;
- }
-
- const clientId = headers.get(clientIdHeader)?.trim();
-
- return {
- platform: "api",
- app: "donkey",
- method: "api-key",
- // The Vision API does not require x-donkey-client-id; default it to the key
- // id so downstream usage records and rate-limit buckets stay per-key.
- clientId: clientId ? clientId : verified.key.id,
- // Vision API keys have no app conversation; honor the header if sent, else null.
- conversationId: conversationIdFromHeaders(headers),
- userId: verified.key.referenceId,
- apiKeyId: verified.key.id,
};
}
@@ -179,7 +128,6 @@ function devAuthBypassContext(headers: Headers): DonkeyAuthContext | null {
clientId: clientId ? clientId : null,
conversationId: conversationIdFromHeaders(headers),
userId: devAuthBypassUserID,
- apiKeyId: null,
};
}
@@ -202,18 +150,6 @@ export function withDonkeyAuth<
);
}
- if (authContext.method === "api-key" && !options.allowApiKey) {
- return NextResponse.json(
- {
- error: "api_key_not_permitted_for_route",
- message: "API keys are not accepted on this route.",
- },
- {
- status: 401,
- },
- );
- }
-
if (
options.role === "SUPER_USER" &&
!(await isDonkeySuperUser(authContext.userId))
diff --git a/site/src/lib/inference/gemini-models.ts b/site/src/lib/inference/gemini-models.ts
index 2578405a..25128895 100644
--- a/site/src/lib/inference/gemini-models.ts
+++ b/site/src/lib/inference/gemini-models.ts
@@ -72,8 +72,6 @@ export const geminiModelRoles = {
computerUse: geminiModels.flash,
// Screenshot parsing into read-only UI evidence.
screenshotParse: geminiModels.flash,
- // Vision grounding: a cheap structured pick over already-parsed elements.
- visionGrounding: geminiModels.flashLite,
// Generative image editing and generation.
imageGeneration: geminiModels.proImage,
// Production review: the director judging rendered takes and minted frames
diff --git a/site/src/lib/inference/vision/grounding.ts b/site/src/lib/inference/vision/grounding.ts
deleted file mode 100644
index d3a12bfc..00000000
--- a/site/src/lib/inference/vision/grounding.ts
+++ /dev/null
@@ -1,143 +0,0 @@
-import { GoogleGenAI, Type } from "@google/genai";
-import { z } from "zod";
-
-import { ensureConfigured } from "@/lib/inference/http";
-import { InferenceProviderError } from "@/lib/inference/providers";
-import { geminiClientConfig } from "@/lib/inference/screenshot-parsing/gemini-flash";
-import type {
- VisionElement,
- VisionModel,
- VisionTarget,
-} from "@/lib/inference/vision/schema";
-
-export type GroundingResult = {
- target: VisionTarget | null;
- alternates: VisionTarget[];
-};
-
-const groundingOutputSchema = z.object({
- elementId: z.string(),
- confidence: z.number().min(0).max(1).optional().default(0.5),
- alternates: z.array(z.object({
- elementId: z.string(),
- confidence: z.number().min(0).max(1).optional().default(0.5),
- })).optional().default([]),
-});
-
-const geminiResponseSchema = {
- type: Type.OBJECT,
- required: ["elementId", "confidence"],
- properties: {
- elementId: {
- type: Type.STRING,
- description: "id of the single best-matching element, or empty string if none match.",
- },
- confidence: { type: Type.NUMBER },
- alternates: {
- type: Type.ARRAY,
- items: {
- type: Type.OBJECT,
- required: ["elementId", "confidence"],
- properties: {
- elementId: { type: Type.STRING },
- confidence: { type: Type.NUMBER },
- },
- },
- },
- },
-};
-
-// Resolve a natural-language instruction to a click target by letting the model
-// pick from the already-parsed elements (text only — no image). Coordinates
-// always come from the parser, never the model.
-export async function groundInstruction(
- elements: VisionElement[],
- instruction: string,
- model: VisionModel,
-): Promise<GroundingResult> {
- const config = geminiClientConfig();
- ensureConfigured(config.configured, "Grounding model credentials are not configured.");
-
- const byID = new Map(elements.map((element) => [element.id, element]));
-
- const client = new GoogleGenAI(config.options);
- let rawResponse: { text?: string };
- try {
- rawResponse = await client.models.generateContent({
- model,
- contents: groundingPrompt(elements, instruction),
- config: {
- responseMimeType: "application/json",
- responseSchema: geminiResponseSchema,
- temperature: 0,
- },
- });
- } catch (error) {
- throw new InferenceProviderError("Grounding request failed.", {
- statusCode: 502,
- code: "provider_error",
- details: { message: error instanceof Error ? error.message : String(error) },
- });
- }
-
- const parsed = groundingOutputSchema.safeParse(safeJsonParse(rawResponse.text));
- if (!parsed.success) {
- return { target: null, alternates: [] };
- }
-
- const target = targetFor(byID, parsed.data.elementId, parsed.data.confidence);
- const alternates = parsed.data.alternates
- .map((alternate) => targetFor(byID, alternate.elementId, alternate.confidence))
- .filter((value): value is VisionTarget => value !== null);
-
- return { target, alternates };
-}
-
-function groundingPrompt(elements: VisionElement[], instruction: string): string {
- const catalog = elements.map((element) => ({
- id: element.id,
- label: element.label,
- kind: element.kind,
- interactive: element.interactive,
- }));
- return [
- "You select the single UI element that best satisfies a user instruction.",
- "You are given a JSON list of detected on-screen elements.",
- "Return the id of the best-matching element, or an empty string if none match.",
- "Optionally include a few ranked alternates.",
- "",
- `Instruction: ${instruction}`,
- "",
- `Elements: ${JSON.stringify(catalog)}`,
- ].join("\n");
-}
-
-function targetFor(
- byID: Map<string, VisionElement>,
- elementId: string,
- confidence: number,
-): VisionTarget | null {
- const element = byID.get(elementId.trim());
- if (!element) {
- return null;
- }
- return {
- elementId: element.id,
- label: element.label,
- kind: element.kind,
- box: element.box,
- point: element.point,
- confidence,
- };
-}
-
-function safeJsonParse(text: string | undefined): unknown {
- if (!text) {
- return null;
- }
- try {
- return JSON.parse(text);
- } catch {
- return null;
- }
-}
diff --git a/site/src/lib/inference/vision/parse.ts b/site/src/lib/inference/vision/parse.ts
deleted file mode 100644
index fbb591e6..00000000
--- a/site/src/lib/inference/vision/parse.ts
+++ /dev/null
@@ -1,169 +0,0 @@
-import { z } from "zod";
-
-import { InferenceProviderError } from "@/lib/inference/providers";
-import type {
- VisionElement,
- VisionParseOptions,
-} from "@/lib/inference/vision/schema";
-
-type FetchImpl = typeof fetch;
-
-// Read RunPod config at module load so a missing var fails the deploy, not the
-// first request.
-const apiKey = requireEnv("RUNPOD_API_KEY");
-const endpointId = requireEnv("RUNPOD_VISION_ENDPOINT_ID");
-
-function requireEnv(name: string): string {
- const value = process.env[name]?.trim();
- if (!value) {
- throw new Error(`${name} must be set to use the vision API.`);
- }
- return value;
-}
-
-const defaultBoxThreshold = 0.05;
-// 0.7 keeps adjacent/dense boxes (e.g. file-list rows) distinct; a low value
-// over-merges neighbors. Matches the worker default in vision/handler.py.
-const defaultIouThreshold = 0.7;
-const defaultImgsz = 640;
-
-// Raw worker output. bbox is [x1, y1, x2, y2] in ratio space (0-1).
-const workerElementSchema = z.object({
- type: z.string().optional(),
- bbox: z.tuple([z.number(), z.number(), z.number(), z.number()]),
- interactivity: z.boolean().optional(),
- content: z.string().nullable().optional(),
-});
-
-const workerOutputSchema = z.object({
- image_size: z.object({ width: z.number().positive(), height: z.number().positive() }),
- parsed_content_list: z.array(workerElementSchema).default([]),
-});
-
-const runpodResponseSchema = z.object({
- status: z.string().optional(),
- error: z.unknown().optional(),
- output: z.unknown().optional(),
-});
-
-export type VisionParseResult = {
- image: { width: number; height: number };
- elements: VisionElement[];
-};
-
-export async function parseImage(
- imageBase64: string,
- options: VisionParseOptions = {},
- fetchImpl: FetchImpl = fetch,
-): Promise<VisionParseResult> {
- const rawOutput = await callWorker(imageBase64, options, fetchImpl);
- const parsed = workerOutputSchema.safeParse(rawOutput);
- if (!parsed.success) {
- throw new InferenceProviderError("Vision worker output was malformed.", {
- statusCode: 502,
- code: "provider_error",
- });
- }
-
- const { width, height } = parsed.data.image_size;
- const usedIDs = new Set<string>();
- const elements: VisionElement[] = [];
-
- for (const element of parsed.data.parsed_content_list) {
- const [x1, y1, x2, y2] = element.bbox;
- // Drop degenerate boxes; they carry no usable geometry.
- if (x2 <= x1 || y2 <= y1) {
- continue;
- }
-
- const boxX = x1 * width;
- const boxY = y1 * height;
- const boxW = (x2 - x1) * width;
- const boxH = (y2 - y1) * height;
- const content = element.content?.trim() ?? "";
- const interactive = element.interactivity ?? false;
-
- elements.push({
- id: uniqueShortID(usedIDs),
- label: content || element.type || "element",
- kind: interactive ? "button" : element.type === "text" ? "text" : "icon",
- interactive,
- box: { x: boxX, y: boxY, width: boxW, height: boxH },
- point: { x: boxX + boxW / 2, y: boxY + boxH / 2 },
- confidence: 0.5,
- });
- }
-
- return { image: { width, height }, elements };
-}
-
-async function callWorker(
- imageBase64: string,
- options: VisionParseOptions,
- fetchImpl: FetchImpl,
-): Promise<unknown> {
- let response: Response;
- try {
- response = await fetchImpl(`https://api.runpod.ai/v2/${endpointId}/runsync`, {
- method: "POST",
- headers: {
- Authorization: `Bearer ${apiKey}`,
- "Content-Type": "application/json",
- },
- body: JSON.stringify({
- input: {
- image: imageBase64,
- box_threshold: options.boxThreshold ?? defaultBoxThreshold,
- iou_threshold: options.iouThreshold ?? defaultIouThreshold,
- imgsz: defaultImgsz,
- },
- }),
- });
- } catch (error) {
- throw new InferenceProviderError("Vision worker request failed.", {
- statusCode: 502,
- code: "provider_request_failed",
- details: { message: error instanceof Error ? error.message : String(error) },
- });
- }
-
- if (!response.ok) {
- throw new InferenceProviderError(`Vision worker returned HTTP ${response.status}.`, {
- statusCode: 502,
- code: "provider_error",
- details: { httpStatus: response.status },
- });
- }
-
- const body = runpodResponseSchema.safeParse(await response.json());
- if (!body.success) {
- throw new InferenceProviderError("Vision worker response was malformed.", {
- statusCode: 502,
- code: "provider_error",
- });
- }
-
- const { status, output, error } = body.data;
- if (status && status !== "COMPLETED") {
- throw new InferenceProviderError(`Vision worker job did not complete (status ${status}).`, {
- statusCode: 502,
- code: "provider_error",
- details: typeof error === "string" ? { error } : undefined,
- });
- }
-
- return output;
-}
-
-function uniqueShortID(used: Set<string>): string {
- let id = shortID();
- while (used.has(id)) {
- id = shortID();
- }
- used.add(id);
- return id;
-}
-
-function shortID(): string {
- return Math.random().toString(36).slice(2, 8).padEnd(6, "0");
-}
diff --git a/site/src/lib/inference/vision/schema.ts b/site/src/lib/inference/vision/schema.ts
deleted file mode 100644
index 7fd2f4e6..00000000
--- a/site/src/lib/inference/vision/schema.ts
+++ /dev/null
@@ -1,73 +0,0 @@
-import { z } from "zod";
-
-import { geminiModels, geminiModelRoles } from "@/lib/inference/gemini-models";
-
-// Public B2B vision API contract for POST /api/vision.
-//
-// Two stages: parse a screenshot into UI elements (always), and optionally
-// ground a natural-language instruction ("click the play button") to a click
-// target using an LLM. Coordinates are pixels relative to the uploaded image,
-// origin top-left.
-
-// Allow-list of grounding models, sourced from the central Gemini registry.
-// Extend deliberately — each must be wired in vision/grounding.ts.
-export const supportedVisionModels = [
- geminiModels.flash,
- geminiModels.flashLite,
-] as const;
-export type VisionModel = (typeof supportedVisionModels)[number];
-export const defaultVisionModel: VisionModel = geminiModelRoles.visionGrounding;
-
-export const visionParseOptionsSchema = z.object({
- boxThreshold: z.number().min(0).max(1).optional(),
- iouThreshold: z.number().min(0).max(1).optional(),
-});
-
-export type VisionParseOptions = z.infer<typeof visionParseOptionsSchema>;
-
-export const visionRequestSchema = z.object({
- // Base64-encoded screenshot (png/jpeg/webp), no data: prefix.
- image: z.string().min(1).max(6_000_000),
- // When present, the LLM grounding stage runs and a target is returned.
- instruction: z.string().min(1).max(2_000).optional(),
- // Grounding model; only used when instruction is present.
- model: z.enum(supportedVisionModels).default(defaultVisionModel),
- // Whether to include the full parsed element list in the response. Defaults
- // to true for parse-only requests and false when an instruction is grounded.
- returnElements: z.boolean().optional(),
- options: visionParseOptionsSchema.optional().default({}),
-});
-
-export type VisionRequest = z.infer<typeof visionRequestSchema>;
-
-export type VisionPoint = { x: number; y: number };
-export type VisionBox = { x: number; y: number; width: number; height: number };
-
-export type VisionElement = {
- id: string;
- label: string;
- kind: string;
- interactive: boolean;
- box: VisionBox;
- point: VisionPoint;
- confidence: number;
-};
-
-export type VisionTarget = {
- elementId: string;
- label: string;
- kind: string;
- box: VisionBox;
- point: VisionPoint;
- confidence: number;
-};
-
-export type VisionResponse = {
- image: { width: number; height: number };
- // Present unless explicitly suppressed via returnElements: false.
- elements?: VisionElement[];
- // Present only when an instruction was grounded.
- target?: VisionTarget | null;
- alternates?: VisionTarget[];
- model?: VisionModel;
-};
diff --git a/site/src/proxy.ts b/site/src/proxy.ts
index 070d02a7..4cb8515d 100644
--- a/site/src/proxy.ts
+++ b/site/src/proxy.ts
@@ -11,8 +11,8 @@ import {
// site app: the marketing landing at /cut and the app under /cut/app. Every
// host gets the same mapping — "/" → landing, "/app/…" → editor app (generic
// "/…" → "/cut/…" rewrite) — with donkeycut.com as the one production host.
-// The auth pages (/sign-in, /sign-up), "/install", "/donkeyvision", and the
-// legal pages are real root-level routes and pass through the rewrite.
+// The auth pages (/sign-in, /sign-up), "/install", and the legal pages are
+// real root-level routes and pass through the rewrite.
// www. 308s to the apex; retired domains redirect to donkeycut.com at the
// edge (Cloudflare) and never reach this app.
//
@@ -53,17 +53,15 @@ function cutApi(req: NextRequest): NextResponse {
}
// Root-level routes the generic "/…" → "/cut/…" rewrite must not capture:
-// auth pages, the Mac download, Donkey Vision (marketing, settings, and API
-// docs), and the legal pages. "/app/settings" is not among them: Cut ships its
-// own billing and usage pages under /cut/app/settings, which the generic
-// rewrite serves at /app/settings.
+// auth pages, the Mac download, and the legal pages. "/app/settings" is not
+// among them: Cut ships its own billing and usage pages under
+// /cut/app/settings, which the generic rewrite serves at /app/settings.
const PASSTHROUGH = [
"/install",
"/privacy",
"/terms",
"/sign-in",
"/sign-up",
- "/donkeyvision",
// Email-footer unsubscribe page.
"/unsubscribe",
];
diff --git a/site/src/queries/apiKeys.ts b/site/src/queries/apiKeys.ts
deleted file mode 100644
index 87dbb474..00000000
--- a/site/src/queries/apiKeys.ts
+++ /dev/null
@@ -1,61 +0,0 @@
-"use client";
-
-import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
-
-import { apiFetch } from "@/queries/apiClient";
-
-export const apiKeysQueryKey = ["api-keys"] as const;
-
-export type ApiKey = {
- id: string;
- name: string | null;
- start: string | null;
- prefix: string | null;
- enabled: boolean;
- createdAt: string;
- lastRequest: string | null;
- expiresAt: string | null;
-};
-
-export type CreatedApiKey = {
- apiKey: ApiKey;
- // The plaintext secret, returned exactly once at creation.
- secret: string;
-};
-
-export function useApiKeys() {
- return useQuery({
- queryFn: () =>
- apiFetch<{ apiKeys: ApiKey[] }>("/api/api-keys").then(
- (response) => response.apiKeys,
- ),
- queryKey: apiKeysQueryKey,
- });
-}
-
-export function useCreateApiKey() {
- const queryClient = useQueryClient();
- return useMutation({
- mutationFn: (name: string) =>
- apiFetch<CreatedApiKey>("/api/api-keys", {
- body: JSON.stringify({ name }),
- method: "POST",
- }),
- onSuccess: () => {
- void queryClient.invalidateQueries({ queryKey: apiKeysQueryKey });
- },
- });
-}
-
-export function useDeleteApiKey() {
- const queryClient = useQueryClient();
- return useMutation({
- mutationFn: (id: string) =>
- apiFetch<{ deleted: boolean }>(`/api/api-keys/${id}`, {
- method: "DELETE",
- }),
- onSuccess: () => {
- void queryClient.invalidateQueries({ queryKey: apiKeysQueryKey });
- },
- });
-}
diff --git a/site/src/queries/billing.ts b/site/src/queries/billing.ts
index 1672e05b..21326d9f 100644
--- a/site/src/queries/billing.ts
+++ b/site/src/queries/billing.ts
@@ -4,7 +4,6 @@ import { useMutation, useQuery } from "@tanstack/react-query";
import { apiFetch } from "@/queries/apiClient";
-export const subscriptionQueryKey = ["billing", "subscription"] as const;
export const usageQueryKey = ["billing", "usage"] as const;
export const proSubscriptionQueryKey = ["billing", "pro"] as const;
@@ -25,41 +24,17 @@ export function useProSubscription() {
});
}
-// Plans that the checkout route understands. Only "vision" is self-serve today;
-// "pro" (the Mac app plan) is accepted by the landing card but the route returns
-// "not available" until it is wired to Stripe.
-export type BillingPlanKey = "vision" | "pro";
-
-export type VisionSubscription = {
- status: string;
- isActive: boolean;
- planKey: string;
- monthlyCallQuota: number;
- currentPeriodEnd: string | null;
- cancelAtPeriodEnd: boolean;
-};
-
-export type VisionUsage = {
- used: number;
- limit: number;
- remaining: number;
- // Extra calls from one-time grants (signup bonus, top-ups), spent after the
- // subscription quota is exhausted.
- extraRemaining: number;
- periodStart: string | null;
- periodEnd: string | null;
- // Recent inference calls across products (app + Vision API), newest first.
+export type UsageHistory = {
+ // Recent credit-billed inference calls, newest first.
recent: {
createdAt: string;
// The app conversation this call belongs to; null for background/warm calls
// and rows recorded before grouping existed. Drives the grouped rendering.
conversationId: string | null;
- // "app" = Pro/credit-billed app inference; "vision" = Vision API call.
- product: "app" | "vision";
requestKind: string;
model: string;
status: string;
- // USD cost charged to credits; "included" calls (Vision quota/grants) are 0.
+ // USD cost charged to credits.
costCredits: string;
billingStatus: string;
errorCode: string | null;
@@ -76,19 +51,9 @@ export type VisionUsage = {
}[];
};
-export function useSubscription() {
- return useQuery({
- queryFn: () =>
- apiFetch<{ subscription: VisionSubscription | null }>(
- "/api/billing/subscription",
- ).then((response) => response.subscription),
- queryKey: subscriptionQueryKey,
- });
-}
-
export function useUsage() {
return useQuery({
- queryFn: () => apiFetch<VisionUsage>("/api/billing/usage"),
+ queryFn: () => apiFetch<UsageHistory>("/api/billing/usage"),
queryKey: usageQueryKey,
});
}
@@ -97,11 +62,8 @@ export function useUsage() {
// invalidate here because the user leaves the page for Stripe.
export function useStartCheckout() {
return useMutation({
- mutationFn: (planKey?: BillingPlanKey) =>
- apiFetch<{ url: string }>("/api/billing/checkout", {
- body: JSON.stringify({ planKey: planKey ?? "vision" }),
- method: "POST",
- }),
+ mutationFn: () =>
+ apiFetch<{ url: string }>("/api/billing/checkout", { method: "POST" }),
});
}
diff --git a/vision/Dockerfile b/vision/Dockerfile
deleted file mode 100644
index bfa996d8..00000000
--- a/vision/Dockerfile
+++ /dev/null
@@ -1,83 +0,0 @@
-# OmniParser V2 RunPod Serverless worker.
-#
-# RunPod builds this image from GitHub (Serverless -> Import Git Repository ->
-# Dockerfile Path = vision/Dockerfile). The build context is the repo ROOT, not
-# vision/, so COPY sources must be repo-relative (e.g. vision/handler.py).
-# Build for linux/amd64 if building locally on Apple Silicon, from the repo
-# root: docker build --platform linux/amd64 -f vision/Dockerfile .
-FROM runpod/pytorch:1.0.3-cu1281-torch260-ubuntu2204
-
-WORKDIR /app
-
-RUN apt-get update && apt-get install -y \
- git \
- ffmpeg \
- libgl1 \
- libglib2.0-0 \
- && rm -rf /var/lib/apt/lists/*
-
-# Pin OmniParser for reproducible builds. Bump deliberately and retest.
-ARG OMNIPARSER_COMMIT=b0d5c9f5701f7e2be4771872e6e928da77759df3
-RUN git clone https://github.com/microsoft/OmniParser.git \
- && cd OmniParser \
- && git checkout ${OMNIPARSER_COMMIT}
-
-WORKDIR /app/OmniParser
-
-RUN pip install --upgrade pip \
- && pip install --ignore-installed blinker \
- && pip install -r requirements.txt \
- && pip install runpod huggingface_hub
-
-# OmniParser's util/utils.py builds a PaddleOCR() at import time with kwargs
-# (use_gpu, show_log, max_batch_size, use_dilation, det_db_score_mode) that
-# only exist in PaddleOCR 2.x; requirements.txt leaves paddleocr unpinned, so
-# the build pulls 3.x and the constructor raises ValueError at import, crashing
-# the worker. We run EasyOCR (the handler passes use_paddleocr=False) and never
-# touch paddle_ocr, so stub the eager construction to None instead of pinning
-# the whole paddle stack we don't use. Fails the build loudly if upstream
-# changes and the pattern stops matching.
-RUN python -c "import re,pathlib,sys; p=pathlib.Path('util/utils.py'); s=p.read_text(); s2=re.sub(r'paddle_ocr = PaddleOCR\(.*?\)','paddle_ocr = None',s,count=1,flags=re.S); p.write_text(s2) if s2!=s else sys.exit('paddle_ocr stub failed: pattern not found in util/utils.py')"
-
-# remove_overlap_new appends a raw [x1,y1,x2,y2] list (box1) instead of a dict in
-# its no-OCR branch. When YOLO finds icons but OCR returns zero text boxes (the
-# 'no ocr bbox!!!' path, common on sparse/textless UI regions), get_som_labeled_img
-# then does `sorted(filtered_boxes, key=lambda x: x['content'] ...)` and crashes with
-# TypeError: list indices must be integers or slices, not str. Patch that one branch
-# to append the same dict shape the OCR branch uses (content=None, captioning fills
-# it in). Anchor on the preceding dict-close + else so we touch only remove_overlap_new,
-# not the unused legacy remove_overlap. Fails the build loudly if upstream changes.
-RUN python -c "import re,pathlib,sys; p=pathlib.Path('util/utils.py'); s=p.read_text(); pat=r'(\}\)\n else:\n filtered_boxes\.append\()box1(\))'; repl=r\"\1{'type': 'icon', 'bbox': box1_elem['bbox'], 'interactivity': True, 'content': None, 'source':'box_yolo_content_yolo'}\2\"; s2,n=re.subn(pat,repl,s,count=1); p.write_text(s2) if n==1 else sys.exit('remove_overlap_new no-ocr patch failed: pattern not found in util/utils.py')"
-
-# Bake OmniParser V2 weights into the image so workers don't download on cold
-# start. icon_detect (YOLO) weights are AGPL-3.0; icon_caption is MIT.
-RUN mkdir -p weights && \
- for f in icon_detect/train_args.yaml icon_detect/model.pt icon_detect/model.yaml \
- icon_caption/config.json icon_caption/generation_config.json icon_caption/model.safetensors; do \
- hf download microsoft/OmniParser-v2.0 "$f" --local-dir weights; \
- done && \
- mv weights/icon_caption weights/icon_caption_florence
-
-# Prefetch EasyOCR's English models (detection + recognition) so the first
-# request doesn't pay the ~150MB download. EasyOCR's own downloader pulls from
-# github release assets, which the RunPod build sandbox can't reach (it stalled
-# builds here for 10min+). HuggingFace IS reachable (the OmniParser weights above
-# pull in ~12s), so fetch the byte-identical weights from a mirror, pinned to a
-# commit and MD5-verified against JaidedAI's official checksums so a changed or
-# corrupt file fails the build loudly. EASYOCR_MODULE_PATH makes the lookup dir
-# deterministic at runtime regardless of HOME.
-ENV EASYOCR_MODULE_PATH=/root/.EasyOCR
-ARG EASYOCR_MIRROR_REV=17cdb173ef73b32eb6e9d1270f33b30154b03908
-RUN mkdir -p "$EASYOCR_MODULE_PATH/model" && \
- hf download xiaoyao9184/easyocr craft_mlt_25k.pth --revision "$EASYOCR_MIRROR_REV" --local-dir /tmp/easyocr && \
- hf download xiaoyao9184/easyocr english_g2.pth --revision "$EASYOCR_MIRROR_REV" --local-dir /tmp/easyocr && \
- cp /tmp/easyocr/craft_mlt_25k.pth /tmp/easyocr/english_g2.pth "$EASYOCR_MODULE_PATH/model/" && \
- echo "2f8227d2def4037cdb3b34389dcf9ec1 $EASYOCR_MODULE_PATH/model/craft_mlt_25k.pth" | md5sum -c && \
- echo "5864788e1821be9e454ec108d61b887d $EASYOCR_MODULE_PATH/model/english_g2.pth" | md5sum -c && \
- rm -rf /tmp/easyocr
-
-COPY vision/handler.py /app/handler.py
-
-WORKDIR /app
-
-CMD ["python", "-u", "handler.py"]
diff --git a/vision/README.md b/vision/README.md
deleted file mode 100644
index c26f1dad..00000000
--- a/vision/README.md
+++ /dev/null
@@ -1,114 +0,0 @@
-# Vision RunPod Serverless Worker
-
-A [RunPod Serverless](https://docs.runpod.io/serverless/overview) worker that
-wraps Microsoft [OmniParser V2](https://github.com/microsoft/OmniParser) — YOLO
-icon detection + OCR + Florence-2 captioning — and parses a screenshot into
-labeled UI regions.
-
-The worker returns the model's **raw** output. Normalization into Donkey's
-screenshot-parser contract lives in the site adapter
-(`site/src/lib/inference/screenshot-parsing/vision.ts`), not here. The site
-serves it on its own route, `POST /api/inference/vision`, separate from the
-gemini-flash parser at `/api/inference/screenshots/parse`.
-
-## Files
-
-| File | Purpose |
-| --- | --- |
-| `handler.py` | RunPod handler: base64 image in, raw model output out |
-| `Dockerfile` | Builds the worker image; bakes V2 weights in at build time |
-| `requirements-worker.txt` | Worker-only deps (`runpod`, `huggingface_hub`) |
-| `test_input.json` | Sample payload for local handler testing |
-
-## Request / response contract
-
-Request `input`:
-
-```jsonc
-{
- "image": "<base64 screenshot, no data: prefix>",
- "box_threshold": 0.05, // optional
- "iou_threshold": 0.1, // optional
- "imgsz": 640 // optional
-}
-```
-
-Response:
-
-```jsonc
-{
- "labeled_image_base64": "<base64 PNG with boxes drawn>",
- "label_coordinates": { "0": [x, y, w, h], ... }, // ratio space (0-1)
- "parsed_content_list": [
- {
- "type": "text" | "icon",
- "bbox": [x1, y1, x2, y2], // ratio space (0-1), NOT pixels
- "interactivity": true,
- "content": "Sign in",
- "source": "box_ocr_content_ocr" // present for OCR text elements
- }
- ]
-}
-```
-
-Boxes are image-size-independent ratios. To get Donkey's
-`box_2d = [ymin, xmin, ymax, xmax]` in 0-1000 space, multiply by 1000 and
-reorder: `[y1*1000, x1*1000, y2*1000, x2*1000]` (the site adapter does this).
-
-## Deploy via RunPod GitHub integration
-
-RunPod builds and hosts the image directly from this repo — no Docker Hub push.
-
-1. RunPod console → **Settings → Connections → GitHub → Connect** and authorize
- access to this repository.
-2. **Serverless → New Endpoint → Import Git Repository** → select this repo.
-3. Set:
- - **Branch**: `main` (RunPod rebuilds on every push to it)
- - **Dockerfile Path**: `vision/Dockerfile`
-4. Pick a GPU (RTX 4090 / A40 / L40S is a good start).
-5. **Active workers = 0** for cheap testing (cold starts apply) or `1` for low
- latency. Set sensible execution/idle timeouts.
-6. Deploy. Note the **Endpoint ID** — the site adapter needs it.
-
-See RunPod's [GitHub deployment docs](https://docs.runpod.io/serverless/github-integration).
-
-## Local handler test
-
-You don't need a GPU to exercise the handler shape (CPU works, just slow).
-
-```bash
-# Put a real screenshot's base64 into test_input.json (replace the placeholder).
-base64 -i screenshot.png | tr -d '\n' # macOS: copy into "image"
-
-# Inside the built image (or a venv with OmniParser + its requirements):
-python handler.py # RunPod SDK auto-runs ./test_input.json
-```
-
-Confirm `parsed_content_list` is non-empty and each `bbox` is four values in
-0-1. Build the image for local runs with:
-
-```bash
-docker build --platform linux/amd64 -t vision-worker:dev vision/
-```
-
-## Calling the deployed endpoint
-
-```bash
-curl -s -X POST \
- "https://api.runpod.ai/v2/${RUNPOD_VISION_ENDPOINT_ID}/runsync" \
- -H "Authorization: Bearer ${RUNPOD_API_KEY}" \
- -H "Content-Type: application/json" \
- -d '{"input":{"image":"<base64>","box_threshold":0.05,"iou_threshold":0.1,"imgsz":640}}'
-```
-
-## Licensing
-
-OmniParser's `icon_detect` (YOLO) weights are **AGPL-3.0**; `icon_caption`
-(Florence-2) is **MIT**. This worker bakes both into the image. Keep that in
-mind before redistributing the built image — it carries AGPL-3.0 weights.
-
-## Latency note
-
-Serverless inference with zero active workers has real cold-start cost. Treat
-it as a fallback/evaluation parser; for low-latency UX run with active
-workers ≥ 1 or a persistent pod.
diff --git a/vision/bench/.gitignore b/vision/bench/.gitignore
deleted file mode 100644
index efc8f9f9..00000000
--- a/vision/bench/.gitignore
+++ /dev/null
@@ -1,5 +0,0 @@
-# Local venv for overlay rendering (Pillow).
-.venv/
-
-# Python bytecode cache.
-__pycache__/
diff --git a/vision/bench/bench_endpoint.py b/vision/bench/bench_endpoint.py
deleted file mode 100644
index 6f339c1c..00000000
--- a/vision/bench/bench_endpoint.py
+++ /dev/null
@@ -1,339 +0,0 @@
-#!/usr/bin/env python3
-"""Measure OmniParser latency over HTTP against any /runsync endpoint.
-
-Use this to answer "do I need to rent a GPU?" and "where does the latency go?":
-run it against the local CPU worker (docker compose up) and against your RunPod
-GPU endpoint, then compare the broken-out numbers.
-
-By default the image is compressed exactly the way the Mac app sends it
-(downscale to a 896px max dimension, JPEG quality 0.48 — see
-apps/.../ScreenshotCompression.swift). The old behaviour of uploading the raw
-file is available with --no-compress.
-
-Each run is broken into:
- compress local downscale + JPEG encode (client CPU)
- queue RunPod delayTime: dispatch / cold-start wait before the worker runs
- compute RunPod executionTime: actual work inside the worker (GPU)
- transport remainder: upload + download + network round trip
- total end-to-end wall clock the caller feels
-
-Compression needs Pillow, or falls back to macOS `sips` (no install). Timing
-breakout needs the endpoint to report delayTime/executionTime (RunPod does;
-the local API server may not — those fields show as n/a).
-
-After the runs it saves a results folder under vision/bench/results/
-(named <image>-<timestamp>/, override with --out-dir):
- <stem>.elements.json parsed_content_list + label_coordinates + image_size
- summary.json timing breakout for every run + warm medians
- summary.txt the console report, verbatim
-Render an overlay from the elements with render_content_overlay.py.
-Use --no-save to skip.
-
-Examples:
- # Local CPU worker (docker compose -f vision/docker-compose.yml up)
- python bench_endpoint.py --image shot.png
-
- # RunPod Cloud GPU endpoint
- python bench_endpoint.py --image shot.png \
- --url https://api.runpod.ai/v2/<ENDPOINT_ID>/runsync \
- --api-key "$RUNPOD_API_KEY"
-
-Only uses the Python standard library (plus optional Pillow / sips for
-compression), so it runs with the system python3.
-"""
-
-import argparse
-import base64
-import json
-import os
-import statistics
-import subprocess
-import sys
-import tempfile
-import time
-import urllib.error
-import urllib.request
-
-
-def compress_image(path, max_dim, jpeg_quality):
- """Downscale to `max_dim` (longest side) and JPEG-encode, mirroring the
- app's ScreenshotCompression (maxPixelDimension=896, jpegQuality=0.48).
-
- Returns (jpeg_bytes, (width, height), method). Prefers Pillow; falls back
- to macOS `sips` so no install is needed on a Mac.
- """
- quality_pct = max(1, min(100, round(jpeg_quality * 100)))
-
- try:
- from PIL import Image # type: ignore
- except ImportError:
- Image = None
-
- if Image is not None:
- import io
-
- img = Image.open(path).convert("RGB")
- w, h = img.size
- scale = min(1.0, max_dim / max(w, h))
- if scale < 1.0:
- img = img.resize((max(1, round(w * scale)), max(1, round(h * scale))), Image.LANCZOS)
- buf = io.BytesIO()
- img.save(buf, format="JPEG", quality=quality_pct)
- return buf.getvalue(), img.size, "Pillow"
-
- # macOS fallback: sips uses the same ImageIO pipeline the app does.
- with tempfile.NamedTemporaryFile(suffix=".jpg", delete=False) as tf:
- out_path = tf.name
- try:
- subprocess.run(
- ["sips", "-Z", str(max_dim), "-s", "format", "jpeg",
- "-s", "formatOptions", str(quality_pct), path, "--out", out_path],
- check=True, capture_output=True,
- )
- with open(out_path, "rb") as f:
- data = f.read()
- dims = subprocess.run(
- ["sips", "-g", "pixelWidth", "-g", "pixelHeight", out_path],
- check=True, capture_output=True, text=True,
- ).stdout
- w = h = 0
- for line in dims.splitlines():
- line = line.strip()
- if line.startswith("pixelWidth:"):
- w = int(line.split(":")[1])
- elif line.startswith("pixelHeight:"):
- h = int(line.split(":")[1])
- return data, (w, h), "sips"
- except FileNotFoundError:
- raise SystemExit(
- "Compression needs Pillow or macOS `sips`; install Pillow "
- "(pip install pillow) or rerun with --no-compress."
- )
- finally:
- try:
- os.unlink(out_path)
- except OSError:
- pass
-
-
-def save_outputs(output, out_dir, stem):
- """Persist the structured elements JSON for the run.
-
- The worker also returns a numbered SOM image, but we don't save it — the
- overlay we keep is `<stem>.content.png` from render_content_overlay.py.
-
- Returns the list of written paths.
- """
- if not isinstance(output, dict):
- return []
- os.makedirs(out_dir, exist_ok=True)
- saved = []
-
- json_path = os.path.join(out_dir, f"{stem}.elements.json")
- with open(json_path, "w") as f:
- json.dump(
- {
- "image_size": output.get("image_size"),
- "label_coordinates": output.get("label_coordinates"),
- "parsed_content_list": output.get("parsed_content_list"),
- },
- f,
- indent=2,
- )
- saved.append(json_path)
- return saved
-
-
-def call_once(url, payload, api_key, timeout):
- data = json.dumps(payload).encode("utf-8")
- headers = {"Content-Type": "application/json"}
- if api_key:
- headers["Authorization"] = f"Bearer {api_key}"
- req = urllib.request.Request(url, data=data, headers=headers, method="POST")
- start = time.perf_counter()
- with urllib.request.urlopen(req, timeout=timeout) as resp:
- body = json.loads(resp.read().decode("utf-8"))
- total = time.perf_counter() - start
-
- # RunPod (cloud and local API server) wraps handler output under "output"
- # and reports queue/compute timing as top-level ms fields.
- output = body.get("output", body)
- status = body.get("status")
- delay_ms = body.get("delayTime")
- exec_ms = body.get("executionTime")
- queue_s = delay_ms / 1000.0 if isinstance(delay_ms, (int, float)) else None
- compute_s = exec_ms / 1000.0 if isinstance(exec_ms, (int, float)) else None
- transport_s = None
- if queue_s is not None and compute_s is not None:
- transport_s = max(0.0, total - queue_s - compute_s)
- return {
- "total": total,
- "queue": queue_s,
- "compute": compute_s,
- "transport": transport_s,
- "status": status,
- "output": output,
- }
-
-
-BENCH_DIR = os.path.dirname(os.path.abspath(__file__))
-RESULTS_ROOT = os.path.join(BENCH_DIR, "results")
-
-
-def fmt(value):
- return f"{value:6.2f}s" if isinstance(value, float) else " n/a"
-
-
-def median_opt(values):
- present = [v for v in values if isinstance(v, float)]
- return statistics.median(present) if present else None
-
-
-def main():
- ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
- ap.add_argument("--image", required=True, help="Path to a PNG/JPEG screenshot.")
- ap.add_argument("--url", default="http://localhost:8000/runsync", help="Full /runsync URL.")
- ap.add_argument("--api-key", default=None, help="Bearer token (RunPod Cloud only).")
- ap.add_argument("--runs", type=int, default=4, help="Total requests (first is the cold run).")
- ap.add_argument("--box-threshold", type=float, default=0.05)
- ap.add_argument("--iou-threshold", type=float, default=0.1)
- ap.add_argument("--imgsz", type=int, default=640)
- ap.add_argument("--timeout", type=float, default=600.0, help="Per-request timeout (s).")
- ap.add_argument("--no-compress", action="store_true",
- help="Upload the raw file instead of the app-style compressed JPEG.")
- ap.add_argument("--max-dim", type=int, default=896,
- help="Longest-side pixel cap for compression (app uses 896).")
- ap.add_argument("--jpeg-quality", type=float, default=0.48,
- help="JPEG quality 0-1 for compression (app uses 0.48).")
- ap.add_argument("--out-dir", default=None,
- help="Where to save results (default: vision/bench/results/<image>-<timestamp>/).")
- ap.add_argument("--no-save", action="store_true",
- help="Skip saving the overlay, elements JSON, and summary.")
- args = ap.parse_args()
-
- raw_bytes = os.path.getsize(args.image)
-
- # Compress once up front (the app does this per screenshot; we measure it as
- # its own line and reuse the bytes so the per-run compress time is the cost
- # of preparing one screenshot, not re-reading the file each loop).
- if args.no_compress:
- with open(args.image, "rb") as f:
- image_data = f.read()
- compress_s = 0.0
- prep_note = "raw upload (no compression)"
- else:
- t0 = time.perf_counter()
- image_data, dims, method = compress_image(args.image, args.max_dim, args.jpeg_quality)
- compress_s = time.perf_counter() - t0
- prep_note = (
- f"compressed via {method} -> {dims[0]}x{dims[1]} JPEG q{round(args.jpeg_quality * 100)} "
- f"({len(image_data) // 1024} KB, {100 * len(image_data) / raw_bytes:.1f}% of raw)"
- )
-
- image_b64 = base64.b64encode(image_data).decode("ascii")
- payload = {
- "input": {
- "image": image_b64,
- "box_threshold": args.box_threshold,
- "iou_threshold": args.iou_threshold,
- "imgsz": args.imgsz,
- }
- }
-
- # Buffer the console report so we can both print it and write it to disk.
- lines = []
-
- def emit(line=""):
- print(line)
- lines.append(line)
-
- emit(f"Endpoint: {args.url}")
- emit(f"Image: {args.image} (raw {raw_bytes // 1024} KB)")
- emit(f"Payload: {prep_note}")
- emit(f" {len(image_b64) // 1024} KB base64 uploaded per run")
- emit(f"Runs: {args.runs} (run 1 is cold)")
- emit(f"Compress: {compress_s * 1000:.0f} ms (local, once)")
- emit()
-
- header = f" {'run':<14} {'total':>7} {'queue':>7} {'compute':>7} {'transport':>10} status / elements"
- emit(header)
- emit(f" {'-' * (len(header) - 2)}")
-
- runs = []
- rows = []
- for i in range(1, args.runs + 1):
- try:
- r = call_once(args.url, payload, args.api_key, args.timeout)
- except urllib.error.URLError as e:
- print(f" run {i}: request failed: {e}", file=sys.stderr)
- sys.exit(1)
- output = r["output"]
- n_elements = len(output.get("parsed_content_list", [])) if isinstance(output, dict) else 0
- label = "cold" if i == 1 else "warm"
- emit(
- f" run {i} ({label}){'':<{4 if i < 10 else 3}}"
- f"{fmt(r['total'])} {fmt(r['queue'])} {fmt(r['compute'])} {fmt(r['transport']):>10}"
- f" {r['status']} / {n_elements} elements"
- )
- runs.append(r)
- rows.append({
- "run": i, "label": label, "total_s": r["total"], "queue_s": r["queue"],
- "compute_s": r["compute"], "transport_s": r["transport"],
- "status": r["status"], "elements": n_elements,
- })
-
- warm = runs[1:] or runs
- warm_median = {
- "total_s": median_opt([r["total"] for r in warm]),
- "queue_s": median_opt([r["queue"] for r in warm]),
- "compute_s": median_opt([r["compute"] for r in warm]),
- "transport_s": median_opt([r["transport"] for r in warm]),
- }
- emit()
- emit("Summary (warm median)")
- emit(f" total: {fmt(warm_median['total_s'])}")
- emit(f" queue: {fmt(warm_median['queue_s'])}")
- emit(f" compute: {fmt(warm_median['compute_s'])} <- actual model work")
- emit(f" transport: {fmt(warm_median['transport_s'])} <- upload + download + network")
- emit(f" compress: {compress_s * 1000:6.0f} ms (local, per screenshot)")
- emit()
- emit(f" cold total: {fmt(runs[0]['total'])}")
-
- if args.no_save:
- return
-
- stem = os.path.splitext(os.path.basename(args.image))[0]
- out_dir = args.out_dir or os.path.join(RESULTS_ROOT, f"{stem}-{time.strftime('%Y%m%d-%H%M%S')}")
- os.makedirs(out_dir, exist_ok=True)
-
- saved = save_outputs(runs[-1]["output"], out_dir, stem)
-
- summary = {
- "endpoint": args.url,
- "image": os.path.abspath(args.image),
- "raw_kb": raw_bytes // 1024,
- "payload": prep_note,
- "payload_b64_kb": len(image_b64) // 1024,
- "compress_ms": round(compress_s * 1000),
- "runs": rows,
- "warm_median": warm_median,
- "cold_total_s": runs[0]["total"],
- }
- summary_json = os.path.join(out_dir, "summary.json")
- with open(summary_json, "w") as f:
- json.dump(summary, f, indent=2)
- saved.append(summary_json)
-
- summary_txt = os.path.join(out_dir, "summary.txt")
- with open(summary_txt, "w") as f:
- f.write("\n".join(lines) + "\n")
- saved.append(summary_txt)
-
- emit()
- emit(f"Saved to {out_dir}")
- for path in saved:
- print(f" {os.path.basename(path)}")
-
-
-if __name__ == "__main__":
- main()
diff --git a/vision/bench/latency-report.md b/vision/bench/latency-report.md
deleted file mode 100644
index 9e9990c0..00000000
--- a/vision/bench/latency-report.md
+++ /dev/null
@@ -1,82 +0,0 @@
-# Vision parsing: latency & cost notes
-
-How the screenshot parser performs and why it's configured this way. It runs icon
-detection, OCR, and captioning on a serverless GPU; the site calls it at
-`POST /api/inference/vision`.
-
-```
-Client Queue Worker (GPU)
- | | |
- | compress -> 1568px JPEG (~270KB), ~30ms |
- | | |
- |--------- upload ----------->| |
- | |------ dispatch ------->|
- | | | parse ~0.2s
- | | | (16ms detect + OCR + caps)
- |<-------- elements ----------+------------------------|
- |
- v
- what we control: ~0.2s on the GPU. upload + queue depend on the connection.
-```
-
-## Compress before upload
-
-We downscale each screenshot to 1568px and JPEG-encode it (~270KB) before
-sending. This is the biggest win, because the model is fast: ~0.2s to parse, of
-which only 16ms is detection. Most of the wait is transferring the image. A raw
-screenshot is several megabytes, so the upload and queue dominate — and those
-depend on the caller's connection, not us. At ~270KB the upload shrinks
-accordingly, with enough detail left to parse.
-
-## Resolution is for detection, not speed
-
-We send 1568px rather than smaller because OCR reads the raw pixels. Below ~900px,
-dense screens degrade — a file list blurs and adjacent rows merge into one box. At
-1568px they stay separate. The larger image costs a bit more to process, but the
-parse stays around 0.2s, so it's worth it.
-
-## Overlap threshold (iou 0.7)
-
-IoU (intersection-over-union) measures how much two boxes overlap: 0 is none, 1 is
-identical. After detection, a cleanup pass discards a box when it overlaps a
-neighbor by more than this threshold, treating it as a duplicate. At 0.1 it was
-too aggressive — boxes that barely touched were removed, so packed elements like
-list rows and toolbar icons collapsed into one. At 0.7, only near-identical boxes
-merge, and dense neighbors stay separate.
-
-## Endpoint config in code
-
-The scaling settings live in `vision/deploy/endpoint.json` and are pushed to the
-live endpoint through the host's API; the image build never reads them.
-
-- **Fast cold-boot (snapshots).** A cold start is ~2s, so scaling up from zero
- workers is cheap.
-- **Idle timeout 10s.** A worker is billed while it idles after a request. At 60s
- that idle was most of the cost of a one-off request; 10s brings it to ~$0.003.
-- **0 to 5 workers, queue-delay scaling.** Parallelism comes from adding workers
- (one GPU, one request each), not from loading more onto a single GPU.
-
-## Measure compute and transport separately
-
-The bench (`vision/bench/bench_endpoint.py`) splits each run into queue, compute,
-and transport using the host's own timings. This keeps the measurements honest: a
-worker log reporting "15ms" and a client stopwatch reading several seconds are
-both correct — they measure different things. Separating them shows the pipe, not
-the model, is the slow part.
-
-## Request timing
-
-Warm, the model parses in ~0.2s and the worker finishes in under a second — that's
-the part we control. Upload, queue, and the response depend on the caller's
-connection, so total wall-clock varies. The first request after a scale-up is
-slower: the model warms up (~0.5s parse) on top of a ~2s boot.
-
-| Stage (what we control) | Warm |
-| --- | --- |
-| Compress (local) | ~30ms |
-| Detection inference | ~16ms |
-| Parse (detection + OCR + captions) | ~0.2s |
-| Worker execution | ~0.7s |
-| Cold boot (snapshot) | ~2s |
-| Cost / isolated request | ~$0.003 |
-| Upload + queue + response | connection-bound |
diff --git a/vision/bench/render_content_overlay.py b/vision/bench/render_content_overlay.py
deleted file mode 100644
index ac275f44..00000000
--- a/vision/bench/render_content_overlay.py
+++ /dev/null
@@ -1,104 +0,0 @@
-#!/usr/bin/env python3
-"""Draw each parsed element's content onto the screenshot.
-
-Every box is outlined and labeled with "AI <content>" — the content the model
-pulled out of it (OCR text for text regions, Florence-2 captions for icons).
-Boxes are colored from a per-index palette so adjacent ones read apart, the same
-color scheme the worker's numbered output uses; the label is white/black text on
-a chip in that color.
-
-Needs Pillow (in vision/bench/.venv). Run it on a result dir from
-bench_endpoint.py:
-
- vision/bench/.venv/bin/python vision/bench/render_content_overlay.py \
- --result-dir vision/bench/results/desktop --image ~/Desktop/desktop.png
-
-Writes `<stem>.content.png` next to the elements JSON.
-"""
-
-import argparse
-import json
-import os
-
-from PIL import Image, ImageDraw, ImageFont
-
-# Distinct per-index palette (Apple system colors) cycled by element index, so
-# adjacent boxes read apart.
-PALETTE = [
- (255, 59, 48), (255, 149, 0), (255, 204, 0), (52, 199, 89),
- (0, 199, 190), (48, 176, 199), (0, 122, 255), (88, 86, 214),
- (175, 82, 222), (255, 45, 85), (162, 132, 94), (142, 142, 147),
-]
-
-
-def color_for(index):
- return PALETTE[index % len(PALETTE)]
-
-
-def text_on(color):
- # Black on light chips, white on dark — keep the label legible.
- r, g, b = color
- return (0, 0, 0) if (0.299 * r + 0.587 * g + 0.114 * b) > 150 else (255, 255, 255)
-
-
-def load_font(size):
- for path in (
- "/System/Library/Fonts/SFNS.ttf",
- "/System/Library/Fonts/Supplemental/Arial.ttf",
- "/Library/Fonts/Arial.ttf",
- ):
- try:
- return ImageFont.truetype(path, size)
- except OSError:
- continue
- return ImageFont.load_default()
-
-
-def render(image_path, elements_path, out_path):
- elements = json.load(open(elements_path))["parsed_content_list"]
- img = Image.open(image_path).convert("RGB")
- W, H = img.size
- draw = ImageDraw.Draw(img, "RGBA")
- font = load_font(max(11, W // 130))
- line_w = max(2, round(W / 1100))
-
- drawn = 0
- for index, el in enumerate(elements):
- x1, y1, x2, y2 = el["bbox"]
- if x2 <= x1 or y2 <= y1:
- continue
- box = (x1 * W, y1 * H, x2 * W, y2 * H)
- color = color_for(index)
- draw.rectangle(box, outline=color + (255,), width=line_w)
-
- # App-style label: "AI <content>" on a chip in this box's color.
- content = (el.get("content") or "").strip() or (el.get("type") or "element")
- label = f"AI {content}"
- tb = draw.textbbox((0, 0), label, font=font)
- tw, th = tb[2] - tb[0], tb[3] - tb[1]
- lx = box[0]
- ly = box[1] - th - 5 if box[1] - th - 5 >= 0 else box[3] + 1
- draw.rectangle((lx, ly, lx + tw + 5, ly + th + 5), fill=color + (255,))
- draw.text((lx + 3, ly + 2), label, fill=text_on(color), font=font)
- drawn += 1
-
- img.save(out_path)
- return len(elements), drawn
-
-
-def main():
- ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
- ap.add_argument("--result-dir", required=True, help="A bench result dir (holds <stem>.elements.json).")
- ap.add_argument("--image", required=True, help="Source screenshot to draw on.")
- ap.add_argument("--out", default=None, help="Output path (default: <stem>.content.png in the result dir).")
- args = ap.parse_args()
-
- stem = os.path.splitext(os.path.basename(args.image))[0]
- elements_path = os.path.join(args.result_dir, f"{stem}.elements.json")
- out_path = args.out or os.path.join(args.result_dir, f"{stem}.content.png")
- total, drawn = render(args.image, elements_path, out_path)
- print(f"{stem}: {total} boxes, {drawn} labels -> {out_path}")
-
-
-if __name__ == "__main__":
- main()
diff --git a/vision/bench/results/apple-music/apple-music.content.png b/vision/bench/results/apple-music/apple-music.content.png
deleted file mode 100644
index 05948051..00000000
Binary files a/vision/bench/results/apple-music/apple-music.content.png and /dev/null differ
diff --git a/vision/bench/results/apple-music/apple-music.elements.json b/vision/bench/results/apple-music/apple-music.elements.json
deleted file mode 100644
index 67eb8bce..00000000
--- a/vision/bench/results/apple-music/apple-music.elements.json
+++ /dev/null
@@ -1,1504 +0,0 @@
-{
- "image_size": {
- "height": 1013,
- "width": 1568
- },
- "label_coordinates": {
- "0": [
- 0.14221938775510204,
- 0.12339585389930899,
- 0.08418367346938775,
- 0.019743336623889437
- ],
- "1": [
- 0.1434948979591837,
- 0.5083909783170348,
- 0.0778061224489796,
- 0.017769002961500493
- ],
- "10": [
- 0.28380103987090444,
- 0.7532082922013821,
- 0.06377549074134048,
- 0.013820335636722606
- ],
- "11": [
- 0.42410714285714285,
- 0.7532082922013821,
- 0.030612244897959183,
- 0.013820335636722606
- ],
- "12": [
- 0.5644132653061225,
- 0.7532082922013821,
- 0.0778061224489796,
- 0.013820335636722606
- ],
- "13": [
- 0.7059948979591837,
- 0.7532082922013821,
- 0.016581632653061226,
- 0.013820335636722606
- ],
- "14": [
- 0.17283163265306123,
- 0.8144126357354393,
- 0.02040816326530612,
- 0.013820335636722606
- ],
- "15": [
- 0.4292091836734694,
- 0.857847976307996,
- 0.047193877551020405,
- 0.019743336623889437
- ],
- "16": [
- 0.30612244897959184,
- 0.8687067511992534,
- 0.08801020408163265,
- 0.027640671273445213
- ],
- "17": [
- 0.32908163265306123,
- 0.8943731490621916,
- 0.042091836734693876,
- 0.02566633761105627
- ],
- "18": [
- 0.02614795918367347,
- 0.9703849950641659,
- 0.03443877551020408,
- 0.013820335636722606
- ],
- "19": [
- 0.14183751904234596,
- 0.14894129610956008,
- 0.15995785654807576,
- 0.33239215877190276
- ],
- "2": [
- 0.14221938775510204,
- 0.7393878963127777,
- 0.0625,
- 0.013820335636722606
- ],
- "20": [
- 0.8167440453354193,
- 0.14819442884590173,
- 0.16284428810586735,
- 0.33282434551995926
- ],
- "21": [
- 0.8430505869339924,
- 0.5306580855133267,
- 0.13712606624681123,
- 0.2416236628902394
- ],
- "22": [
- 0.704118689712213,
- 0.5320247186960143,
- 0.13410358039700254,
- 0.20642210347513573
- ],
- "23": [
- 0.42377152734873247,
- 0.5319453067158193,
- 0.13290506479691486,
- 0.20582801992071817
- ],
- "24": [
- 0.4799522088498485,
- 0.15213842664984112,
- 0.16026866679288904,
- 0.32772444548988156
- ],
- "25": [
- 0.14095741388749103,
- 0.5317433424080701,
- 0.1372789363471829,
- 0.20762810514097976
- ],
- "26": [
- 0.5635659743328484,
- 0.5313074802952246,
- 0.13450953425193318,
- 0.21719996008915352
- ],
- "27": [
- 0.281976115946867,
- 0.5304331049867349,
- 0.1346690119529257,
- 0.2095059152887463
- ],
- "28": [
- 0.14060804795245735,
- 0.8552522047368584,
- 0.13622377356704402,
- 0.14474779526314166
- ],
- "29": [
- 0.31006336212158203,
- 0.15128062060884284,
- 0.16097183616793886,
- 0.32936974362583293
- ],
- "3": [
- 0.28380102040816324,
- 0.7393878963127777,
- 0.016581632653061226,
- 0.013820335636722606
- ],
- "30": [
- 0.009149716824901347,
- 0.10411753376756464,
- 0.10761482861577248,
- 0.030431922543554882
- ],
- "31": [
- 0.6483896995077327,
- 0.1507615506472498,
- 0.16136807811503506,
- 0.33147229329266414
- ],
- "32": [
- 0.009312843789859694,
- 0.1330338352877437,
- 0.1079207537125568,
- 0.03006427309237722
- ],
- "33": [
- 0.007740059677435428,
- 0.2471709632685248,
- 0.10994067484018755,
- 0.030675558559044917
- ],
- "34": [
- 0.008573600224086217,
- 0.2762543055994725,
- 0.11070562868702169,
- 0.03018197514344768
- ],
- "35": [
- 0.007900276962591678,
- 0.3048856684391967,
- 0.11131019981539979,
- 0.03189391186065523
- ],
- "36": [
- 0.008019194311025192,
- 0.3605879908841313,
- 0.11091975776516662,
- 0.03183739559553924
- ],
- "37": [
- 0.008458113183780593,
- 0.1619358194545132,
- 0.10955979872723015,
- 0.029356584614850693
- ],
- "38": [
- 0.007909234689206493,
- 0.39065630085258823,
- 0.11125091630585339,
- 0.030395266804972853
- ],
- "39": [
- 0.008940302595800283,
- 0.21784209451929604,
- 0.11010670175357741,
- 0.03126271314705701
- ],
- "4": [
- 0.3080357142857143,
- 0.7393878963127777,
- 0.021683673469387755,
- 0.013820335636722606
- ],
- "40": [
- 0.008907459220107721,
- 0.07095470221350954,
- 0.10795414204500159,
- 0.03451877002772551
- ],
- "41": [
- 0.107559719864203,
- 0,
- 0.03195032781484176,
- 0.02860153627536845
- ],
- "42": [
- 0.167897029798858,
- 0,
- 0.042601955180265466,
- 0.029035005258784243
- ],
- "43": [
- 0.13904671766320054,
- 0,
- 0.029634310274708028,
- 0.028317130447493136
- ],
- "44": [
- 0.2496296045731525,
- 0,
- 0.040391415965800384,
- 0.027754257367841113
- ],
- "45": [
- 0.28997355091328525,
- 0,
- 0.028878698543626433,
- 0.028426444306896094
- ],
- "46": [
- 0.2094701844818738,
- 0,
- 0.04161441569425622,
- 0.02774360031625008
- ],
- "47": [
- 0.025005564397695114,
- 0,
- 0.03440512443075375,
- 0.030921830594363122
- ],
- "48": [
- 0.14344774946874503,
- 0.06761826179105226,
- 0.058272624502376635,
- 0.03686308037022612
- ],
- "49": [
- 0.009233036819769412,
- 0.19172579585505614,
- 0.1095822879246303,
- 0.030237075489341684
- ],
- "5": [
- 0.42410714285714285,
- 0.7393878963127777,
- 0.039540816326530615,
- 0.013820335636722606
- ],
- "50": [
- 0.17220602230149873,
- 0.8254910287358095,
- 0.0661502371028978,
- 0.020951025342423494
- ],
- "51": [
- 0.8463800391372369,
- 0.8554713408309168,
- 0.13260136818399235,
- 0.14452871942096496
- ],
- "52": [
- 0.7028292441854671,
- 0,
- 0.020341367137675384,
- 0.02867208369745554
- ],
- "53": [
- 0.7245115941884567,
- 0,
- 0.019992594816246812,
- 0.02900038678695513
- ],
- "54": [
- 0.7693601335797992,
- 0,
- 0.017572986836336096,
- 0.030056395158833602
- ],
- "55": [
- 0.08404666550305424,
- 0,
- 0.025075893012844786,
- 0.028722923105584434
- ],
- "56": [
- 0.745484955456792,
- 0,
- 0.021636184380978953,
- 0.02964704010823166
- ],
- "57": [
- 0.7881063344527264,
- 0,
- 0.019009337133290817,
- 0.030066357430912783
- ],
- "58": [
- 0.672830309186663,
- 0.942431978035538,
- 0.02099048848054847,
- 0.03457819344613771
- ],
- "59": [
- 0.8080530750508211,
- 0,
- 0.022101266043526784,
- 0.03052440729922627
- ],
- "6": [
- 0.5644133042316047,
- 0.7393878963127777,
- 0.029336734693877552,
- 0.013820335636722606
- ],
- "60": [
- 0.694572215177575,
- 0.9429018222097113,
- 0.01925425626793686,
- 0.03460120966498026
- ],
- "61": [
- 0.00503803090173371,
- 0,
- 0.020137560610868493,
- 0.030441775609098325
- ],
- "62": [
- 0.7362642872090243,
- 0.9377498645537697,
- 0.02350772157007334,
- 0.043434738053738896
- ],
- "63": [
- 0.563537208401427,
- 0.8541594163568916,
- 0.13450334510024714,
- 0.10500155690862537
- ],
- "64": [
- 0.005953754697527204,
- 0.03512449725829837,
- 0.045507591597887934,
- 0.0357949576043659
- ],
- "65": [
- 0.8999808953732861,
- 0,
- 0.021660629583864797,
- 0.031095184672620194
- ],
- "66": [
- 0.716209956577846,
- 0.9380110564613154,
- 0.01859314587651467,
- 0.043392923247778874
- ],
- "67": [
- 0.8757180194465481,
- 0,
- 0.02277358697385204,
- 0.031544260776290065
- ],
- "68": [
- 0.008457466047637316,
- 0.3359533763708508,
- 0.11122495300915776,
- 0.0287694300268386
- ],
- "69": [
- 0.8302218378806601,
- 0,
- 0.03433196398676658,
- 0.031070673453960813
- ],
- "7": [
- 0.6179846938775511,
- 0.7393878963127777,
- 0.02040816326530612,
- 0.013820335636722606
- ],
- "70": [
- 0.3823133585404377,
- 0.9347851707104516,
- 0.03185630331234056,
- 0.06077257854917325
- ],
- "71": [
- 0.5001729459178691,
- 0.9794386838289734,
- 0.06503155766701212,
- 0.020561316171026654
- ],
- "72": [
- 0.9879422869001117,
- 0.3785668210239542,
- 0.012057713099888392,
- 0.1005117071816387
- ],
- "73": [
- 0.35962132045200895,
- 0.9363982943415289,
- 0.024109665228396047,
- 0.04904599580454097
- ],
- "74": [
- 0.9894387381417411,
- 0.531246324635211,
- 0.010561261858258928,
- 0.21492930767213722
- ],
- "75": [
- 0.14194276381512078,
- 0.8092999623064228,
- 0.027090851141481985,
- 0.04013341694687809
- ],
- "76": [
- 0.6414754439373406,
- 0.9452428488246545,
- 0.020481109619140625,
- 0.039769013565831686
- ],
- "77": [
- 1.9493455789527114e-05,
- 0,
- 0.005365241547020114,
- 0.02940729598773079
- ],
- "78": [
- 0.9175917956293845,
- 0,
- 0.07953674939213967,
- 0.030835489647762677
- ],
- "79": [
- 0.9881911764339525,
- 0.8542509389653258,
- 0.011808745715082909,
- 0.14574906103467425
- ],
- "8": [
- 0.7059948979591837,
- 0.7393878963127777,
- 0.03571428571428571,
- 0.013820335636722606
- ],
- "80": [
- 0.4142061739551778,
- 0.9348648236981737,
- 0.03096023870974171,
- 0.04872991443268756
- ],
- "81": [
- 0.006333942924227033,
- 0.9630354102719336,
- 0.01801286424909319,
- 0.029041407021224087
- ],
- "82": [
- 0.5996757429473254,
- 0.9764751347714091,
- 0.0963304869982661,
- 0.02352486522859082
- ],
- "9": [
- 0.16262755102040816,
- 0.7532082922013821,
- 0.025510204081632654,
- 0.013820335636722606
- ]
- },
- "parsed_content_list": [
- {
- "bbox": [
- 0.1422193944454193,
- 0.12339585274457932,
- 0.22640305757522583,
- 0.14313918352127075
- ],
- "content": "Top Picks for You",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.14349490404129028,
- 0.5083909034729004,
- 0.22130101919174194,
- 0.526159942150116
- ],
- "content": "Recently Played",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.1422193944454193,
- 0.7393879294395447,
- 0.2047193944454193,
- 0.7532082796096802
- ],
- "content": "David Le's Station",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.28380101919174194,
- 0.7393879294395447,
- 0.3003826439380646,
- 0.7532082796096802
- ],
- "content": "New",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.3080357015132904,
- 0.7393879294395447,
- 0.3297193944454193,
- 0.7532082796096802
- ],
- "content": "K-Pop",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.4241071343421936,
- 0.7393879294395447,
- 0.4636479616165161,
- 0.7532082796096802
- ],
- "content": "Parachutes",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5644132494926453,
- 0.7393879294395447,
- 0.59375,
- 0.7532082796096802
- ],
- "content": "Inspired",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.6179847121238708,
- 0.7393879294395447,
- 0.6383928656578064,
- 0.7532082796096802
- ],
- "content": "Punk",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.7059949040412903,
- 0.7393879294395447,
- 0.7417091727256775,
- 0.7532082796096802
- ],
- "content": "Discovery",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.16262754797935486,
- 0.7532082796096802,
- 0.18813775479793549,
- 0.7670286297798157
- ],
- "content": "Station",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.28380101919174194,
- 0.7532082796096802,
- 0.3475765287876129,
- 0.7670286297798157
- ],
- "content": "Apple Music K-Pop",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.4241071343421936,
- 0.7532082796096802,
- 0.4547193944454193,
- 0.7670286297798157
- ],
- "content": "Coldplay",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5644132494926453,
- 0.7532082796096802,
- 0.6422193646430969,
- 0.7670286297798157
- ],
- "content": "Apple Music Electronic",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.7059949040412903,
- 0.7532082796096802,
- 0.7225765585899353,
- 0.7670286297798157
- ],
- "content": "Daft",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.17283163964748383,
- 0.8144126534461975,
- 0.19323979318141937,
- 0.8282329440116882
- ],
- "content": "More",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.4292091727256775,
- 0.8578479886054993,
- 0.47640305757522583,
- 0.8775913119316101
- ],
- "content": "Ihe verve",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.30612245202064514,
- 0.8687068223953247,
- 0.3941326439380646,
- 0.8963474631309509
- ],
- "content": "COOLNEW",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.32908162474632263,
- 0.8943731784820557,
- 0.3711734712123871,
- 0.9200394749641418
- ],
- "content": "MOM",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.026147959753870964,
- 0.9703850150108337,
- 0.06058673560619354,
- 0.9842053055763245
- ],
- "content": "David Le",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.14183752238750458,
- 0.1489412933588028,
- 0.3017953932285309,
- 0.481333464384079
- ],
- "content": "Rold Dinner Nlall ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.8167440295219421,
- 0.14819443225860596,
- 0.9795883297920227,
- 0.4810188114643097
- ],
- "content": "Music Energy moving' nonetol Hlah-voltogo ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.843050479888916,
- 0.5306580662727356,
- 0.9801766276359558,
- 0.7722817659378052
- ],
- "content": "Music SUNNERTIME Marshmello'$ Summer Sounds Apple Music Summertime Sounds ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.7041186690330505,
- 0.5320247411727905,
- 0.838222324848175,
- 0.7384468913078308
- ],
- "content": "dopt pum ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.42377153038978577,
- 0.5319453477859497,
- 0.5566765666007996,
- 0.7377733588218689
- ],
- "content": "COLDMLAY ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.47995221614837646,
- 0.15213841199874878,
- 0.6402208805084229,
- 0.47986283898353577
- ],
- "content": "BITch @ ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.14095741510391235,
- 0.5317433476448059,
- 0.27823635935783386,
- 0.7393714785575867
- ],
- "content": "Music ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.5635659694671631,
- 0.5313075184822083,
- 0.698075532913208,
- 0.7485074996948242
- ],
- "content": "#Music Inspired By ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.2819761037826538,
- 0.5304331183433533,
- 0.4166451394557953,
- 0.7399389743804932
- ],
- "content": "Music ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.1406080573797226,
- 0.8552521467208862,
- 0.27683183550834656,
- 1
- ],
- "content": "STEREOPHONICS Vou Gotta THERE COME BAck ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.3100633919239044,
- 0.15128059685230255,
- 0.4710352420806885,
- 0.4806503355503082
- ],
- "content": "EMusic Vov Davld Lo's Statlon ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.009149717167019844,
- 0.10411753505468369,
- 0.11676454544067383,
- 0.13454945385456085
- ],
- "content": "Home ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.6483897566795349,
- 0.15076154470443726,
- 0.8097578287124634,
- 0.48223382234573364
- ],
- "content": "#Music plobor Biobar Similar Artists Station ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.009312843903899193,
- 0.13303382694721222,
- 0.11723359674215317,
- 0.16309811174869537
- ],
- "content": "New ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.00774005614221096,
- 0.24717096984386444,
- 0.11768073588609695,
- 0.2778465151786804
- ],
- "content": "Artists ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.008573602885007858,
- 0.2762543261051178,
- 0.11927922815084457,
- 0.30643630027770996
- ],
- "content": "Albums ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.00790027342736721,
- 0.3048856556415558,
- 0.11921047419309616,
- 0.33677956461906433
- ],
- "content": "Songs ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.00801919586956501,
- 0.3605879843235016,
- 0.11893895268440247,
- 0.3924253582954407
- ],
- "content": "Playlists ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.008458113297820091,
- 0.16193580627441406,
- 0.11801792681217194,
- 0.1912923902273178
- ],
- "content": "Radio ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.007909237407147884,
- 0.39065632224082947,
- 0.11916015297174454,
- 0.421051561832428
- ],
- "content": "Favorite Songs ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.008940302766859531,
- 0.21784208714962006,
- 0.11904700100421906,
- 0.2491048127412796
- ],
- "content": "Recently Added ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.008907460607588291,
- 0.07095470279455185,
- 0.11686160415410995,
- 0.10547347366809845
- ],
- "content": "Search ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.10755971819162369,
- 0,
- 0.13951005041599274,
- 0.02860153652727604
- ],
- "content": "Song ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.1678970456123352,
- 0,
- 0.21049900352954865,
- 0.02903500571846962
- ],
- "content": "Controls ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.13904671370983124,
- 0,
- 0.16868102550506592,
- 0.028317131102085114
- ],
- "content": "Vicw ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.24962960183620453,
- 0,
- 0.29002103209495544,
- 0.02775425650179386
- ],
- "content": "Window ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.28997355699539185,
- 0,
- 0.3188522458076477,
- 0.02842644415795803
- ],
- "content": "Help ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.20947016775608063,
- 0,
- 0.2510845959186554,
- 0.02774360030889511
- ],
- "content": "Account ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.025005564093589783,
- 0,
- 0.059410687536001205,
- 0.030921829864382744
- ],
- "content": "Music ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.14344775676727295,
- 0.06761825829744339,
- 0.20172038674354553,
- 0.10448134690523148
- ],
- "content": "Home ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.009233033284544945,
- 0.19172579050064087,
- 0.11881532520055771,
- 0.22196286916732788
- ],
- "content": "Library ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.17220602929592133,
- 0.8254910111427307,
- 0.23835626244544983,
- 0.8464420437812805
- ],
- "content": "Parachutes ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.8463800549507141,
- 0.8554713129997253,
- 0.9789814352989197,
- 1
- ],
- "content": "A night time or night-time-related function.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.7028293013572693,
- 0,
- 0.7231706380844116,
- 0.02867208421230316
- ],
- "content": "Toggleoggleoggle",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.7245116233825684,
- 0,
- 0.7445042133331299,
- 0.029000386595726013
- ],
- "content": "Toggleoggleoggle",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.7693601250648499,
- 0,
- 0.7869331240653992,
- 0.030056394636631012
- ],
- "content": "Navigator",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.08404666185379028,
- 0,
- 0.10912255942821503,
- 0.02872292324900627
- ],
- "content": "Edit",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.7454850077629089,
- 0,
- 0.7671211957931519,
- 0.029647039249539375
- ],
- "content": "Pill",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.7881062626838684,
- 0,
- 0.8071156740188599,
- 0.03006635792553425
- ],
- "content": "Navigation Bar",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.6728302836418152,
- 0.9424319863319397,
- 0.6938207149505615,
- 0.9770101308822632
- ],
- "content": "Comment",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.8080531358718872,
- 0,
- 0.8301543593406677,
- 0.03052440658211708
- ],
- "content": "Text formatting.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.6945722103118896,
- 0.9429019093513489,
- 0.7138264775276184,
- 0.9775030612945557
- ],
- "content": "Bullets",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.005038030911237001,
- 0,
- 0.025175590068101883,
- 0.030441775918006897
- ],
- "content": "Apple",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.7362642884254456,
- 0.9377498626708984,
- 0.759772002696991,
- 0.9811846017837524
- ],
- "content": "Mute",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.5635371804237366,
- 0.854159414768219,
- 0.6980405449867249,
- 0.9591609239578247
- ],
- "content": "a calendar or a calendar application.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.005953754298388958,
- 0.03512449935078621,
- 0.051461346447467804,
- 0.0709194540977478
- ],
- "content": "More options",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.899980902671814,
- 0,
- 0.92164146900177,
- 0.03109518438577652
- ],
- "content": "More Options",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.7162100076675415,
- 0.9380109906196594,
- 0.7348030805587769,
- 0.9814039468765259
- ],
- "content": "Podcast",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.8757179975509644,
- 0,
- 0.898491621017456,
- 0.03154426068067551
- ],
- "content": "WiFi connectivity",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.008457468822598457,
- 0.33595341444015503,
- 0.1196824163198471,
- 0.3647228181362152
- ],
- "content": "View as full screen",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.8302218317985535,
- 0,
- 0.8645538091659546,
- 0.03107067383825779
- ],
- "content": "Shuffle or randomize.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.38231340050697327,
- 0.9347851276397705,
- 0.41416966915130615,
- 0.9955577254295349
- ],
- "content": "Play video.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.5001729726791382,
- 0.9794386625289917,
- 0.5652045607566833,
- 1
- ],
- "content": "A filter or filter element.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.9879422187805176,
- 0.37856680154800415,
- 1,
- 0.4790785312652588
- ],
- "content": "System",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.3596213459968567,
- 0.9363983273506165,
- 0.3837309777736664,
- 0.9854443073272705
- ],
- "content": "a code block or command line.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.9894387125968933,
- 0.5312463045120239,
- 1,
- 0.7461756467819214
- ],
- "content": "a button to close a window or dialog box.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.14194276928901672,
- 0.8092999458312988,
- 0.16903361678123474,
- 0.8494333624839783
- ],
- "content": "GoldGold",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.6414754390716553,
- 0.9452428221702576,
- 0.6619565486907959,
- 0.9850118160247803
- ],
- "content": "a person.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 1.9493325453368016e-05,
- 0,
- 0.005384734831750393,
- 0.02940729632973671
- ],
- "content": "a field or field with trees.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.9175918102264404,
- 0,
- 0.9971286058425903,
- 0.030835488811135292
- ],
- "content": "Fullscreen",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.9881911873817444,
- 0.8542509078979492,
- 1,
- 1
- ],
- "content": "a file.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.4142061769962311,
- 0.9348648190498352,
- 0.4451664388179779,
- 0.9835947155952454
- ],
- "content": "Play",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.006333942525088787,
- 0.9630354046821594,
- 0.02434680610895157,
- 0.9920767545700073
- ],
- "content": "DL (Ctrl+F4)",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.5996757745742798,
- 0.9764751195907593,
- 0.6960062384605408,
- 1
- ],
- "content": "a paintbrush tool.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- }
- ]
-}
\ No newline at end of file
diff --git a/vision/bench/results/apple-music/summary.json b/vision/bench/results/apple-music/summary.json
deleted file mode 100644
index 3dc86aa7..00000000
--- a/vision/bench/results/apple-music/summary.json
+++ /dev/null
@@ -1,27 +0,0 @@
-{
- "endpoint": "https://api.runpod.ai/v2/tjeid8c15y30sl/runsync",
- "image": "/Users/davidle/Desktop/apple-music.png",
- "raw_kb": 5623,
- "payload": "compressed via sips -> 1568x1013 JPEG q80 (284 KB, 5.1% of raw)",
- "payload_b64_kb": 379,
- "compress_ms": 80,
- "runs": [
- {
- "run": 1,
- "label": "cold",
- "total_s": 3.814078041992616,
- "queue_s": 0.104,
- "compute_s": 1.701,
- "transport_s": 2.0090780419926157,
- "status": "COMPLETED",
- "elements": 83
- }
- ],
- "warm_median": {
- "total_s": 3.814078041992616,
- "queue_s": 0.104,
- "compute_s": 1.701,
- "transport_s": 2.0090780419926157
- },
- "cold_total_s": 3.814078041992616
-}
\ No newline at end of file
diff --git a/vision/bench/results/apple-music/summary.txt b/vision/bench/results/apple-music/summary.txt
deleted file mode 100644
index 1022ac67..00000000
--- a/vision/bench/results/apple-music/summary.txt
+++ /dev/null
@@ -1,19 +0,0 @@
-Endpoint: https://api.runpod.ai/v2/tjeid8c15y30sl/runsync
-Image: /Users/davidle/Desktop/apple-music.png (raw 5623 KB)
-Payload: compressed via sips -> 1568x1013 JPEG q80 (284 KB, 5.1% of raw)
- 379 KB base64 uploaded per run
-Runs: 1 (run 1 is cold)
-Compress: 80 ms (local, once)
-
- run total queue compute transport status / elements
- --------------------------------------------------------------------
- run 1 (cold) 3.81s 0.10s 1.70s 2.01s COMPLETED / 83 elements
-
-Summary (warm median)
- total: 3.81s
- queue: 0.10s
- compute: 1.70s <- actual model work
- transport: 2.01s <- upload + download + network
- compress: 80 ms (local, per screenshot)
-
- cold total: 3.81s
diff --git a/vision/bench/results/desktop/desktop.content.png b/vision/bench/results/desktop/desktop.content.png
deleted file mode 100644
index 1adb6387..00000000
Binary files a/vision/bench/results/desktop/desktop.content.png and /dev/null differ
diff --git a/vision/bench/results/desktop/desktop.elements.json b/vision/bench/results/desktop/desktop.elements.json
deleted file mode 100644
index 142fa906..00000000
--- a/vision/bench/results/desktop/desktop.elements.json
+++ /dev/null
@@ -1,2098 +0,0 @@
-{
- "image_size": {
- "height": 1013,
- "width": 1568
- },
- "label_coordinates": {
- "0": [
- 0.16645408163265307,
- 0.008884501480750246,
- 0.03188775510204082,
- 0.013820335636722606
- ],
- "1": [
- 0.2072704081632653,
- 0.008884501480750246,
- 0.01913265306122449,
- 0.013820335636722606
- ],
- "10": [
- 0.32716836734693877,
- 0.20039488179544823,
- 0.042091836734693876,
- 0.013820335636722606
- ],
- "100": [
- 0.22385782125044842,
- 0.11478447419869818,
- 0.01147036649742905,
- 0.02124656082258761
- ],
- "101": [
- 0.004656125088127292,
- 0,
- 0.020249247550964355,
- 0.029387378975057648
- ],
- "102": [
- 0.42393773915816324,
- 0.8662648629341991,
- 0.08982553287428252,
- 0.11669596514838351
- ],
- "103": [
- 0.3481409890311105,
- 0.8697112103205207,
- 0.06304410039162149,
- 0.13028878967947927
- ],
- "104": [
- 1.634748614564234e-05,
- 0,
- 0.005537989796424399,
- 0.027003112220575873
- ],
- "105": [
- 0.30703900784862287,
- 0.21491838701856336,
- 0.019047990137216995,
- 0.017044172823605628
- ],
- "106": [
- 0.19877665383475168,
- 0.72161865234375,
- 0.019423153935646524,
- 0.029253855156404246
- ],
- "107": [
- 0.3071632190626495,
- 0.18015496422484267,
- 0.007658471866529815,
- 0.014658409186435711
- ],
- "108": [
- 0.9907731036750638,
- 0.06438877599025172,
- 0.009226896324936224,
- 0.08521574270807626
- ],
- "109": [
- 0.31483401084432794,
- 0.1798097661311158,
- 0.010988936132314255,
- 0.015492415734128209
- ],
- "11": [
- 0.5095663265306123,
- 0.19842053307008883,
- 0.04846938775510204,
- 0.01579466929911155
- ],
- "110": [
- 0.3049705661073023,
- 0.3028928675750401,
- 0.021571451303910236,
- 0.017682240251727543
- ],
- "111": [
- 0.20209224856629662,
- 0.5858738740128332,
- 0.01156970432826451,
- 0.01666205538931392
- ],
- "112": [
- 0.9004359342613999,
- 0,
- 0.021519407934072067,
- 0.032589651636874925
- ],
- "113": [
- 0.8284638852489238,
- 0,
- 0.027832809759646047,
- 0.03344045716235809
- ],
- "114": [
- 0.3049724345304528,
- 0.28663263198536215,
- 0.021408042129205197,
- 0.016820096075240622
- ],
- "115": [
- 0.6563012259347099,
- 0.2513957047156497,
- 0.010477494220344387,
- 0.015811178314721126
- ],
- "12": [
- 0.5669642467888034,
- 0.19842053307008883,
- 0.03316334315708706,
- 0.01579466929911155
- ],
- "13": [
- 0.1996173469387755,
- 0.22211253701875616,
- 0.03316326530612245,
- 0.013820335636722606
- ],
- "14": [
- 0.32716836734693877,
- 0.21816385463100785,
- 0.02040816326530612,
- 0.013820335636722606
- ],
- "15": [
- 0.5235969387755102,
- 0.21618953603158933,
- 0.03571428571428571,
- 0.01579466929911155
- ],
- "16": [
- 0.5669642857142857,
- 0.21618953603158933,
- 0.03188775510204082,
- 0.01579466929911155
- ],
- "17": [
- 0.6715561224489796,
- 0.21816385463100785,
- 0.02423469387755102,
- 0.013820335636722606
- ],
- "18": [
- 0.32716836734693877,
- 0.23593285759250832,
- 0.02423469387755102,
- 0.013820335636722606
- ],
- "19": [
- 0.5095662876051299,
- 0.23593285759250832,
- 0.017857142857142856,
- 0.013820335636722606
- ],
- "2": [
- 0.9610969387755102,
- 0.006910167818361303,
- 0.030612244897959183,
- 0.01579466929911155
- ],
- "20": [
- 0.5338010204081632,
- 0.23395853899308983,
- 0.021683673469387755,
- 0.01579466929911155
- ],
- "21": [
- 0.5631377551020408,
- 0.23593285759250832,
- 0.03571428571428571,
- 0.013820335636722606
- ],
- "22": [
- 0.670280612244898,
- 0.23593285759250832,
- 0.025510204081632654,
- 0.013820335636722606
- ],
- "23": [
- 0.32716836734693877,
- 0.25370187561697927,
- 0.01913265306122449,
- 0.013820335636722606
- ],
- "24": [
- 0.5095662876051299,
- 0.25370187561697927,
- 0.017857142857142856,
- 0.013820335636722606
- ],
- "25": [
- 0.5389030612244898,
- 0.2517275720805312,
- 0.02040816326530612,
- 0.01579466929911155
- ],
- "26": [
- 0.5682397959183674,
- 0.2517275720805312,
- 0.030612244897959183,
- 0.01579466929911155
- ],
- "27": [
- 0.6715561224489796,
- 0.25370187561697927,
- 0.02423469387755102,
- 0.013820335636722606
- ],
- "28": [
- 0.32716836734693877,
- 0.2714708785784798,
- 0.021683673469387755,
- 0.013820335636722606
- ],
- "29": [
- 0.5095662876051299,
- 0.2694965449160908,
- 0.08035714285714286,
- 0.01579466929911155
- ],
- "3": [
- 0.3539540816326531,
- 0.1194471865745311,
- 0.036989795918367346,
- 0.01579466929911155
- ],
- "30": [
- 0.670280612244898,
- 0.2714708785784798,
- 0.025510204081632654,
- 0.013820335636722606
- ],
- "31": [
- 0.32716836734693877,
- 0.28923985141403935,
- 0.05229591836734694,
- 0.013820335636722606
- ],
- "32": [
- 0.5338010204081632,
- 0.28726551775165043,
- 0.02040816326530612,
- 0.01579466929911155
- ],
- "33": [
- 0.5618622448979592,
- 0.28923985141403935,
- 0.03188775510204082,
- 0.013820335636722606
- ],
- "34": [
- 0.6383928960683395,
- 0.28923985141403935,
- 0.028061185564313616,
- 0.013820335636722606
- ],
- "35": [
- 0.6715561224489796,
- 0.28825268458284486,
- 0.04081632653061224,
- 0.01678183613030602
- ],
- "36": [
- 0.32716836734693877,
- 0.3070088845014807,
- 0.026785714285714284,
- 0.013820335636722606
- ],
- "37": [
- 0.5108418756601761,
- 0.3070088845014807,
- 0.05357142857142857,
- 0.013820335636722606
- ],
- "38": [
- 0.6383928960683395,
- 0.3070088845014807,
- 0.028061185564313616,
- 0.013820335636722606
- ],
- "39": [
- 0.6715561224489796,
- 0.30602174779622715,
- 0.04081632653061224,
- 0.01678183613030602
- ],
- "4": [
- 0.8450255102040817,
- 0.12142152023692004,
- 0.026785714285714284,
- 0.013820335636722606
- ],
- "40": [
- 0.32716836734693877,
- 0.32477788746298125,
- 0.047193877551020405,
- 0.01579466929911155
- ],
- "41": [
- 0.5108418367346939,
- 0.32477788746298125,
- 0.0625,
- 0.013820335636722606
- ],
- "42": [
- 0.6511479591836735,
- 0.32477788746298125,
- 0.015306122448979591,
- 0.013820335636722606
- ],
- "43": [
- 0.6715561224489796,
- 0.32379072063178677,
- 0.04081632653061224,
- 0.01678183613030602
- ],
- "44": [
- 0.19961732747603436,
- 0.36426456071076013,
- 0.03443877551020408,
- 0.013820335636722606
- ],
- "45": [
- 0.21747450925865952,
- 0.587364204308829,
- 0.01913265306122449,
- 0.013820335636722606
- ],
- "46": [
- 0.21747448979591838,
- 0.7295162882527148,
- 0.025510204081632654,
- 0.013820335636722606
- ],
- "47": [
- 0.22895408163265307,
- 0.7857847976307996,
- 0.02295918367346939,
- 0.017769002961500493
- ],
- "48": [
- 0.9406683785574776,
- 0.1574006000605411,
- 0.05933162144252232,
- 0.09574527119130985
- ],
- "49": [
- 0.19665089432074098,
- 0.46297265793705267,
- 0.10048212323869977,
- 0.031182156381108096
- ],
- "5": [
- 0.5261479591836735,
- 0.1549851924975321,
- 0.030612244897959183,
- 0.011846001974333662
- ],
- "50": [
- 0.1955022228007414,
- 0.435280411052798,
- 0.10170901551538584,
- 0.02992753144666214
- ],
- "51": [
- 0.9440059272610412,
- 0.2516214685167047,
- 0.05494378537547832,
- 0.07825646960605874
- ],
- "52": [
- 0.19631963846634845,
- 0.262328258036155,
- 0.10027445579061703,
- 0.03164236025419546
- ],
- "53": [
- 0.9433964320591518,
- 0.06397993684638759,
- 0.05150028150908801,
- 0.08989070127888234
- ],
- "54": [
- 0.19505921188665895,
- 0.14547181200063242,
- 0.10423819872797752,
- 0.03310699312260921
- ],
- "55": [
- 0.1943054588473573,
- 0.1761457136329282,
- 0.10501145343391263,
- 0.032742408985917446
- ],
- "56": [
- 0.1967579783225546,
- 0.49233120073806147,
- 0.10099858653788664,
- 0.03059337499228776
- ],
- "57": [
- 0.19694378911232463,
- 0.2906568250439598,
- 0.10123521454480229,
- 0.0318530610848038
- ],
- "58": [
- 0.19647999199069277,
- 0.520804251758391,
- 0.10188904587103396,
- 0.03160476307996051
- ],
- "59": [
- 0.8341969859843351,
- 0.15678737147068578,
- 0.048441361407844385,
- 0.08763609092539178
- ],
- "6": [
- 0.32716836734693877,
- 0.18262586377097728,
- 0.047193877551020405,
- 0.013820335636722606
- ],
- "60": [
- 0.1962526477113062,
- 0.37738971013542694,
- 0.10097655471490354,
- 0.03141484914856861
- ],
- "61": [
- 0.896812594666773,
- 0.24862232565997502,
- 0.04522051130022321,
- 0.08362043857103744
- ],
- "62": [
- 0.8273292074398119,
- 0.24448163558794267,
- 0.05978019870057398,
- 0.08166053523433645
- ],
- "63": [
- 0.1956330902722417,
- 0.3204373720250031,
- 0.10263458563356984,
- 0.03450372212024926
- ],
- "64": [
- 0.1964112106634646,
- 0.5522304282607354,
- 0.10067978683783084,
- 0.03173743772365498
- ],
- "65": [
- 0.19596033680195712,
- 0.22564805719012215,
- 0.1013591922059351,
- 0.0387003861904615
- ],
- "66": [
- 0.19639256535744182,
- 0.6080920566811142,
- 0.10177985989317602,
- 0.030219572317682627
- ],
- "67": [
- 0.1977064171615912,
- 0.6353441635689165,
- 0.10048730032784599,
- 0.030556139329343535
- ],
- "68": [
- 0.19772050818618464,
- 0.692065465838629,
- 0.10093381453533562,
- 0.03063109267028628
- ],
- "69": [
- 0.08509613543140646,
- 0,
- 0.05433277207977918,
- 0.026365656725628354
- ],
- "7": [
- 0.5248723711286273,
- 0.18065153010858837,
- 0.03061232274892379,
- 0.01579466929911155
- ],
- "70": [
- 0.6664076046067842,
- 0.1945374348556654,
- 0.07306982546436544,
- 0.0207141450691788
- ],
- "71": [
- 0.026129272519325723,
- 0,
- 0.034505223741336744,
- 0.028394224613110617
- ],
- "72": [
- 0.19764631621691645,
- 0.6627300898861673,
- 0.10046102562729194,
- 0.03194928334001727
- ],
- "73": [
- 0.19830639508305764,
- 0.7498207506516843,
- 0.10091326188068,
- 0.03139520703510612
- ],
- "74": [
- 0.6906682228555485,
- 0.1070857297515116,
- 0.023547581263950894,
- 0.036137557335690705
- ],
- "75": [
- 0.5388329175053811,
- 0.1074868792489627,
- 0.030413802789182078,
- 0.035313063053835296
- ],
- "76": [
- 0.5759517903230629,
- 0.10723928167224518,
- 0.031681216492944836,
- 0.034943606046204036
- ],
- "77": [
- 0.6118539693404217,
- 0.10781659260907037,
- 0.021017269212372448,
- 0.03430455199351786
- ],
- "78": [
- 0.7159804519341917,
- 0.10551928626584865,
- 0.025740487234933034,
- 0.038303537171072925
- ],
- "79": [
- 0.8967944553920201,
- 0.15759682537655015,
- 0.04600447051379145,
- 0.08880151294885241
- ],
- "8": [
- 0.5631377940275231,
- 0.18065153010858837,
- 0.03188775510204082,
- 0.01579466929911155
- ],
- "80": [
- 0.7023333335409359,
- 0,
- 0.02112392503388074,
- 0.03189717299375694
- ],
- "81": [
- 0.19541212977195271,
- 0.4050584286598439,
- 0.10181115598094706,
- 0.03212967747408687
- ],
- "82": [
- 0.7245797916334502,
- 0,
- 0.020404115015146683,
- 0.0315565954662146
- ],
- "83": [
- 0.8871805619220344,
- 0.0648645323803253,
- 0.05715054881816008,
- 0.09300368253476293
- ],
- "84": [
- 0.6329647764867666,
- 0.10747991262513111,
- 0.02267899805185746,
- 0.034534932595014806
- ],
- "85": [
- 0.7453443566147162,
- 0,
- 0.02157561633051658,
- 0.03159113862319148
- ],
- "86": [
- 0.6554354453573421,
- 0.10746015753939027,
- 0.02095078448860013,
- 0.03448327413786787
- ],
- "87": [
- 0.7880228003677057,
- 0,
- 0.019890921456473216,
- 0.031006380930351716
- ],
- "88": [
- 0.8085945285096461,
- 0,
- 0.022652762276785716,
- 0.030589115937393958
- ],
- "89": [
- 0.30504246147311465,
- 0.10853877166416662,
- 0.02177892412458147,
- 0.033720319621772424
- ],
- "9": [
- 0.6709183673469388,
- 0.17966436327739388,
- 0.024872448979591837,
- 0.017769002961500493
- ],
- "90": [
- 0.7682951323840083,
- 0,
- 0.0194583814971301,
- 0.0315437881755923
- ],
- "91": [
- 0.326895149386659,
- 0.10844398039114557,
- 0.02117818715621014,
- 0.033587208151935
- ],
- "92": [
- 0.19412784187161192,
- 0.11077265537032252,
- 0.024477588887117346,
- 0.02993939353588891
- ],
- "93": [
- 0.3054381973889409,
- 0.2677521023086593,
- 0.02054475278270488,
- 0.018165460343657455
- ],
- "94": [
- 0.8774976146464445,
- 0,
- 0.02165424580476722,
- 0.033498890190463426
- ],
- "95": [
- 0.30547344441316565,
- 0.3217558940800839,
- 0.020611665686782524,
- 0.020220591779522457
- ],
- "96": [
- 0.306397262884646,
- 0.25042429375154246,
- 0.01946889137735172,
- 0.016525946388357603
- ],
- "97": [
- 0.30680535764110334,
- 0.1986621431160538,
- 0.018372555168307558,
- 0.015378509551610317
- ],
- "98": [
- 0.3066998695840641,
- 0.23298169029665133,
- 0.019316926294443558,
- 0.01708333654676703
- ],
- "99": [
- 0.6553910703075175,
- 0.19703781263496423,
- 0.015704563685825894,
- 0.017362061751912634
- ]
- },
- "parsed_content_list": [
- {
- "bbox": [
- 0.16645407676696777,
- 0.00888450164347887,
- 0.19834183156490326,
- 0.02270483784377575
- ],
- "content": "Window",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.20727041363716125,
- 0.00888450164347887,
- 0.22640305757522583,
- 0.02270483784377575
- ],
- "content": "Help",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.9610969424247742,
- 0.006910167634487152,
- 0.9917091727256775,
- 0.02270483784377575
- ],
- "content": "4;05PM",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.3539540767669678,
- 0.11944718658924103,
- 0.39094388484954834,
- 0.13524185121059418
- ],
- "content": "Desktop",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.8450255393981934,
- 0.12142152339220047,
- 0.8718112111091614,
- 0.13524185121059418
- ],
- "content": "spotify",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5261479616165161,
- 0.1549851894378662,
- 0.5567601919174194,
- 0.16683119535446167
- ],
- "content": "Modified",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.32716837525367737,
- 0.18262585997581482,
- 0.3743622303009033,
- 0.19644619524478912
- ],
- "content": "torm options",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5248724222183228,
- 0.18065153062343597,
- 0.5554847121238708,
- 0.19644619524478912
- ],
- "content": "5, 2026",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5631377696990967,
- 0.18065153062343597,
- 0.5950255393981934,
- 0.19644619524478912
- ],
- "content": "2.43PM",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.670918345451355,
- 0.17966435849666595,
- 0.6957908272743225,
- 0.19743336737155914
- ],
- "content": "Folder",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.32716837525367737,
- 0.200394868850708,
- 0.36926019191741943,
- 0.2142152041196823
- ],
- "content": "rough-cuts",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5095663070678711,
- 0.19842053949832916,
- 0.5580357313156128,
- 0.2142152041196823
- ],
- "content": "Feb 16, 2026",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5669642686843872,
- 0.19842053949832916,
- 0.6001275777816772,
- 0.2142152041196823
- ],
- "content": "10:12PM",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.19961734116077423,
- 0.2221125364303589,
- 0.2327806055545807,
- 0.2359328716993332
- ],
- "content": "Favorites",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.32716837525367737,
- 0.21816386282444,
- 0.3475765287876129,
- 0.2319841980934143
- ],
- "content": "temp",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5235969424247742,
- 0.21618953347206116,
- 0.5593112111091614,
- 0.2319841980934143
- ],
- "content": "20, 2026",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5669642686843872,
- 0.21618953347206116,
- 0.5988520383834839,
- 0.2319841980934143
- ],
- "content": "4*43PM",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.6715561151504517,
- 0.21816386282444,
- 0.6957908272743225,
- 0.2319841980934143
- ],
- "content": "Folder",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.32716837525367737,
- 0.2359328716993332,
- 0.35140305757522583,
- 0.2497532069683075
- ],
- "content": "anime",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5095663070678711,
- 0.2359328716993332,
- 0.5274234414100647,
- 0.2497532069683075
- ],
- "content": "May",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5338010191917419,
- 0.23395854234695435,
- 0.5554847121238708,
- 0.2497532069683075
- ],
- "content": "2026",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5631377696990967,
- 0.2359328716993332,
- 0.5988520383834839,
- 0.2497532069683075
- ],
- "content": "10:53AM",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.6702806353569031,
- 0.2359328716993332,
- 0.6957908272743225,
- 0.2497532069683075
- ],
- "content": "Folder",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.32716837525367737,
- 0.2537018656730652,
- 0.34630101919174194,
- 0.2675222158432007
- ],
- "content": "saas",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5095663070678711,
- 0.2537018656730652,
- 0.5274234414100647,
- 0.2675222158432007
- ],
- "content": "May",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5389030575752258,
- 0.25172755122184753,
- 0.5593112111091614,
- 0.2675222158432007
- ],
- "content": "2026",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5682398080825806,
- 0.25172755122184753,
- 0.5988520383834839,
- 0.2675222158432007
- ],
- "content": "6*35PM",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.6715561151504517,
- 0.2537018656730652,
- 0.6957908272743225,
- 0.2675222158432007
- ],
- "content": "Folder",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.32716837525367737,
- 0.2714708745479584,
- 0.3488520383834839,
- 0.28529122471809387
- ],
- "content": "other",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5095663070678711,
- 0.26949653029441833,
- 0.5899234414100647,
- 0.28529122471809387
- ],
- "content": "May 29, 2026 at 11:50",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.6702806353569031,
- 0.2714708745479584,
- 0.6957908272743225,
- 0.28529122471809387
- ],
- "content": "Folder",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.32716837525367737,
- 0.28923988342285156,
- 0.3794642984867096,
- 0.3030602037906647
- ],
- "content": "donkey-vision",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5338010191917419,
- 0.2872655391693115,
- 0.5542091727256775,
- 0.3030602037906647
- ],
- "content": "2026",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5618622303009033,
- 0.28923988342285156,
- 0.59375,
- 0.3030602037906647
- ],
- "content": "2.47PM",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.6383928656578064,
- 0.28923988342285156,
- 0.6664540767669678,
- 0.3030602037906647
- ],
- "content": "3,6 MB",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.6715561151504517,
- 0.28825271129608154,
- 0.7123724222183228,
- 0.3050345480442047
- ],
- "content": "PNG image",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.32716837525367737,
- 0.30700889229774475,
- 0.3539540767669678,
- 0.32082921266555786
- ],
- "content": "spotify",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5108418464660645,
- 0.30700889229774475,
- 0.5644132494926453,
- 0.32082921266555786
- ],
- "content": "Today at 10.04",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.6383928656578064,
- 0.30700889229774475,
- 0.6664540767669678,
- 0.32082921266555786
- ],
- "content": "5.4 MB",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.6715561151504517,
- 0.30602172017097473,
- 0.7123724222183228,
- 0.3228035569190979
- ],
- "content": "PNG image",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.32716837525367737,
- 0.32477790117263794,
- 0.3743622303009033,
- 0.3405725657939911
- ],
- "content": "apple-music",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5108418464660645,
- 0.32477790117263794,
- 0.5733418464660645,
- 0.33859822154045105
- ],
- "content": "Today at 3*43PM",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.6511479616165161,
- 0.32477790117263794,
- 0.6664540767669678,
- 0.33859822154045105
- ],
- "content": "MB",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.6715561151504517,
- 0.3237907290458679,
- 0.7123724222183228,
- 0.3405725657939911
- ],
- "content": "PNG image",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.19961734116077423,
- 0.3642645478248596,
- 0.23405611515045166,
- 0.3780848979949951
- ],
- "content": "Locatlong",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.21747449040412903,
- 0.5873642563819885,
- 0.2366071492433548,
- 0.601184606552124
- ],
- "content": "Blue",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.21747449040412903,
- 0.7295162677764893,
- 0.24298469722270966,
- 0.7433366179466248
- ],
- "content": "Yellow",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.22895407676696777,
- 0.7857847809791565,
- 0.25191327929496765,
- 0.8035538196563721
- ],
- "content": "Tags ..",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.9406684041023254,
- 0.15740059316158295,
- 1,
- 0.25314587354660034
- ],
- "content": "rough cuts ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.196650892496109,
- 0.46297264099121094,
- 0.29713302850723267,
- 0.49415481090545654
- ],
- "content": "AirDrop ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.19550220668315887,
- 0.4352804124355316,
- 0.297211229801178,
- 0.46520793437957764
- ],
- "content": "dle ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.9440059065818787,
- 0.25162145495414734,
- 0.9989497065544128,
- 0.32987794280052185
- ],
- "content": "Tor options ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.19631962478160858,
- 0.26232823729515076,
- 0.2965940833091736,
- 0.29397061467170715
- ],
- "content": "Desktop ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.9433965086936951,
- 0.06397993862628937,
- 0.9948967695236206,
- 0.15387064218521118
- ],
- "content": "temp ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.19505921006202698,
- 0.14547181129455566,
- 0.29929742217063904,
- 0.17857879400253296
- ],
- "content": "Recents ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.19430546462535858,
- 0.1761457324028015,
- 0.2993169128894806,
- 0.20888812839984894
- ],
- "content": "Shared ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.196757972240448,
- 0.49233123660087585,
- 0.297756552696228,
- 0.5229246020317078
- ],
- "content": "Network ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.19694380462169647,
- 0.29065683484077454,
- 0.2981790006160736,
- 0.32250988483428955
- ],
- "content": "Documents ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.19647999107837677,
- 0.520804226398468,
- 0.29836902022361755,
- 0.5524089932441711
- ],
- "content": "Trash ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.8341969847679138,
- 0.15678736567497253,
- 0.8826382756233215,
- 0.24442346394062042
- ],
- "content": "applc ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.19625264406204224,
- 0.37738969922065735,
- 0.29722920060157776,
- 0.4088045656681061
- ],
- "content": "iCloud Drive ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.8968126177787781,
- 0.248622328042984,
- 0.9420331120491028,
- 0.3322427570819855
- ],
- "content": "other ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.8273292183876038,
- 0.2444816380739212,
- 0.8871093392372131,
- 0.32614219188690186
- ],
- "content": "donkey-vision ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.19563308358192444,
- 0.32043737173080444,
- 0.2982676923274994,
- 0.35494109988212585
- ],
- "content": "Downloads ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.19641120731830597,
- 0.5522304177284241,
- 0.29709097743034363,
- 0.5839678645133972
- ],
- "content": "Tags ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.19596035778522491,
- 0.22564806044101715,
- 0.29731953144073486,
- 0.26434844732284546
- ],
- "content": "Applications ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.19639256596565247,
- 0.6080920696258545,
- 0.2981724441051483,
- 0.6383116245269775
- ],
- "content": "Work ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.1977064162492752,
- 0.6353441476821899,
- 0.29819372296333313,
- 0.6659002900123596
- ],
- "content": "Important ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.19772052764892578,
- 0.692065417766571,
- 0.29865431785583496,
- 0.7226965427398682
- ],
- "content": "Orange ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.08509613573551178,
- 0,
- 0.13942891359329224,
- 0.026365656405687332
- ],
- "content": "Edit View ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.666407585144043,
- 0.1945374459028244,
- 0.7394773364067078,
- 0.2152515947818756
- ],
- "content": "Folder ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.02612927183508873,
- 0,
- 0.06063449755311012,
- 0.028394224122166634
- ],
- "content": "Finder ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.19764631986618042,
- 0.6627300977706909,
- 0.2981073260307312,
- 0.6946794390678406
- ],
- "content": "Red ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.1983063966035843,
- 0.7498207688331604,
- 0.29921966791152954,
- 0.7812159657478333
- ],
- "content": "Purple ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.6906682252883911,
- 0.10708573460578918,
- 0.7142157554626465,
- 0.14322328567504883
- ],
- "content": "a dropdown menu.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.5388329029083252,
- 0.10748688876628876,
- 0.5692467093467712,
- 0.14279994368553162
- ],
- "content": "Decrease",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.5759518146514893,
- 0.10723928362131119,
- 0.6076329946517944,
- 0.14218288660049438
- ],
- "content": "a search engine or more options.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.6118539571762085,
- 0.10781659185886383,
- 0.6328712105751038,
- 0.14212113618850708
- ],
- "content": "Share",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.7159804701805115,
- 0.10551928728818893,
- 0.7417209148406982,
- 0.1438228189945221
- ],
- "content": "Search",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.8967944383621216,
- 0.15759682655334473,
- 0.942798912525177,
- 0.24639832973480225
- ],
- "content": "Save",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.702333390712738,
- 0,
- 0.7234572768211365,
- 0.03189717233181
- ],
- "content": "Toggleoggleoggle",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.19541214406490326,
- 0.40505844354629517,
- 0.297223299741745,
- 0.43718811869621277
- ],
- "content": "View as full",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.7245798707008362,
- 0,
- 0.7449839115142822,
- 0.031556595116853714
- ],
- "content": "Toggleoggleoggle",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.887180507183075,
- 0.06486453115940094,
- 0.9443311095237732,
- 0.15786822140216827
- ],
- "content": "online",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.6329647898674011,
- 0.10747992247343063,
- 0.6556437611579895,
- 0.14201484620571136
- ],
- "content": "Toggle Pane",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.745344340801239,
- 0,
- 0.7669199705123901,
- 0.03159113973379135
- ],
- "content": "Pill",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.6554355025291443,
- 0.107460156083107,
- 0.6763862371444702,
- 0.14194342494010925
- ],
- "content": "More options",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.7880228161811829,
- 0,
- 0.8079137802124023,
- 0.031006380915641785
- ],
- "content": "Navigation",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.8085945844650269,
- 0,
- 0.8312472701072693,
- 0.030589116737246513
- ],
- "content": "A text input or translation option.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.30504247546195984,
- 0.10853876918554306,
- 0.32682138681411743,
- 0.14225909113883972
- ],
- "content": "Back",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.7682951092720032,
- 0,
- 0.7877535223960876,
- 0.0315437875688076
- ],
- "content": "Navigator",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.3268951177597046,
- 0.10844399034976959,
- 0.3480733335018158,
- 0.14203119277954102
- ],
- "content": "Next",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.19412782788276672,
- 0.11077264696359634,
- 0.2186054140329361,
- 0.14071203768253326
- ],
- "content": "Color picker",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.30543819069862366,
- 0.26775214076042175,
- 0.3259829580783844,
- 0.28591758012771606
- ],
- "content": "Undo (Ctrl+T)",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.8774975538253784,
- 0,
- 0.8991518616676331,
- 0.033498890697956085
- ],
- "content": "Wi-Fi connectivity.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.3054734468460083,
- 0.32175588607788086,
- 0.3260851204395294,
- 0.34197649359703064
- ],
- "content": "Image",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.3063972592353821,
- 0.2504242956638336,
- 0.325866162776947,
- 0.26695024967193604
- ],
- "content": "Undo (Ctrl+F)",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.3068053722381592,
- 0.19866213202476501,
- 0.325177937746048,
- 0.21404065191745758
- ],
- "content": "System",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.30669984221458435,
- 0.23298168182373047,
- 0.32601678371429443,
- 0.25006502866744995
- ],
- "content": "Undo (Ctrl+F)",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.6553910970687866,
- 0.19703781604766846,
- 0.671095609664917,
- 0.21439987421035767
- ],
- "content": "More options",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.2238578200340271,
- 0.11478447169065475,
- 0.23532818257808685,
- 0.13603103160858154
- ],
- "content": "Green",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.004656125791370869,
- 0,
- 0.02490537241101265,
- 0.029387379065155983
- ],
- "content": "Apple's logo.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.42393770813941956,
- 0.866264820098877,
- 0.5137632489204407,
- 0.9829608201980591
- ],
- "content": "Slide",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.34814098477363586,
- 0.8697112202644348,
- 0.41118505597114563,
- 1
- ],
- "content": "Lavender",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 1.6347690689144656e-05,
- 0,
- 0.005554337054491043,
- 0.027003111317753792
- ],
- "content": "a field or field.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.3070390224456787,
- 0.21491838991641998,
- 0.32608702778816223,
- 0.23196256160736084
- ],
- "content": "Undo",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.19877664744853973,
- 0.72161865234375,
- 0.21819980442523956,
- 0.7508724927902222
- ],
- "content": "Yellow",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.30716320872306824,
- 0.1801549643278122,
- 0.31482169032096863,
- 0.19481338560581207
- ],
- "content": "Next",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.99077308177948,
- 0.06438878178596497,
- 1,
- 0.14960451424121857
- ],
- "content": "a user profile or account.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.31483399868011475,
- 0.17980976402759552,
- 0.32582294940948486,
- 0.19530218839645386
- ],
- "content": "a folder.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.30497056245803833,
- 0.30289286375045776,
- 0.3265420198440552,
- 0.32057511806488037
- ],
- "content": "Photo",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.202092245221138,
- 0.5858739018440247,
- 0.21366193890571594,
- 0.6025359630584717
- ],
- "content": "Blue",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.9004359245300293,
- 0,
- 0.9219552874565125,
- 0.03258965164422989
- ],
- "content": "View as full screen",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.8284639120101929,
- 0,
- 0.856296718120575,
- 0.03344045579433441
- ],
- "content": "a secure or secure connection.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.30497244000434875,
- 0.28663262724876404,
- 0.3263804614543915,
- 0.303452730178833
- ],
- "content": "Toggleoggleoggle",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.6563012003898621,
- 0.25139570236206055,
- 0.6667787432670593,
- 0.2672068774700165
- ],
- "content": "More options",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- }
- ]
-}
\ No newline at end of file
diff --git a/vision/bench/results/desktop/summary.json b/vision/bench/results/desktop/summary.json
deleted file mode 100644
index f778372e..00000000
--- a/vision/bench/results/desktop/summary.json
+++ /dev/null
@@ -1,27 +0,0 @@
-{
- "endpoint": "https://api.runpod.ai/v2/tjeid8c15y30sl/runsync",
- "image": "/Users/davidle/Desktop/desktop.png",
- "raw_kb": 7476,
- "payload": "compressed via sips -> 1568x1013 JPEG q80 (272 KB, 3.6% of raw)",
- "payload_b64_kb": 362,
- "compress_ms": 82,
- "runs": [
- {
- "run": 1,
- "label": "cold",
- "total_s": 3.867321832993184,
- "queue_s": 0.102,
- "compute_s": 1.587,
- "transport_s": 2.178321832993184,
- "status": "COMPLETED",
- "elements": 116
- }
- ],
- "warm_median": {
- "total_s": 3.867321832993184,
- "queue_s": 0.102,
- "compute_s": 1.587,
- "transport_s": 2.178321832993184
- },
- "cold_total_s": 3.867321832993184
-}
\ No newline at end of file
diff --git a/vision/bench/results/desktop/summary.txt b/vision/bench/results/desktop/summary.txt
deleted file mode 100644
index 4c300c10..00000000
--- a/vision/bench/results/desktop/summary.txt
+++ /dev/null
@@ -1,19 +0,0 @@
-Endpoint: https://api.runpod.ai/v2/tjeid8c15y30sl/runsync
-Image: /Users/davidle/Desktop/desktop.png (raw 7476 KB)
-Payload: compressed via sips -> 1568x1013 JPEG q80 (272 KB, 3.6% of raw)
- 362 KB base64 uploaded per run
-Runs: 1 (run 1 is cold)
-Compress: 82 ms (local, once)
-
- run total queue compute transport status / elements
- --------------------------------------------------------------------
- run 1 (cold) 3.87s 0.10s 1.59s 2.18s COMPLETED / 116 elements
-
-Summary (warm median)
- total: 3.87s
- queue: 0.10s
- compute: 1.59s <- actual model work
- transport: 2.18s <- upload + download + network
- compress: 82 ms (local, per screenshot)
-
- cold total: 3.87s
diff --git a/vision/bench/results/spotify/spotify.content.png b/vision/bench/results/spotify/spotify.content.png
deleted file mode 100644
index 3970d0ba..00000000
Binary files a/vision/bench/results/spotify/spotify.content.png and /dev/null differ
diff --git a/vision/bench/results/spotify/spotify.elements.json b/vision/bench/results/spotify/spotify.elements.json
deleted file mode 100644
index b31800ef..00000000
--- a/vision/bench/results/spotify/spotify.elements.json
+++ /dev/null
@@ -1,1954 +0,0 @@
-{
- "image_size": {
- "height": 1013,
- "width": 1568
- },
- "label_coordinates": {
- "0": [
- 0.0695153012567637,
- 0.008884501480750246,
- 0.014030617110583246,
- 0.013820335636722606
- ],
- "1": [
- 0.14859695823825136,
- 0.008884501480750246,
- 0.03443877551020408,
- 0.013820335636722606
- ],
- "10": [
- 0.18941326530612246,
- 0.192497532082922,
- 0.03188775510204082,
- 0.01579466929911155
- ],
- "100": [
- 0.010349132576767279,
- 0.22549643332960576,
- 0.03324799148403868,
- 0.04878754535761661
- ],
- "101": [
- 0.9776452512157207,
- 0.9448951954667448,
- 0.01718092937858737,
- 0.03014931862351925
- ],
- "102": [
- 0.9686181204659599,
- 0.514980726628054,
- 0.01834262147241709,
- 0.030895598431330207
- ],
- "103": [
- 2.5892136048297494e-05,
- 0,
- 0.005452852468101346,
- 0.02969358092116933
- ],
- "104": [
- 0.2734973479290398,
- 0.3128105984506108,
- 0.026398756066147163,
- 0.0349672398468349
- ],
- "105": [
- 0.21768822961924028,
- 0.09972180501141412,
- 0.023079346637336576,
- 0.0381462948225799
- ],
- "106": [
- 0.9318671713069994,
- 0.945142951204652,
- 0.023810094716597577,
- 0.02864157553368707
- ],
- "107": [
- 0.0070346253258841375,
- 0.9328103550330084,
- 0.035654591054332496,
- 0.054006533232045906
- ],
- "11": [
- 0.5121173469387755,
- 0.18460019743336623,
- 0.21683673469387754,
- 0.04146100691016782
- ],
- "12": [
- 0.056760204081632654,
- 0.25370184549103836,
- 0.029336744425248126,
- 0.01579466929911155
- ],
- "13": [
- 0.0899234693877551,
- 0.25370184549103836,
- 0.03316326530612245,
- 0.01579466929911155
- ],
- "14": [
- 0.5223214285714286,
- 0.2596248766041461,
- 0.04336734693877551,
- 0.01579466929911155
- ],
- "15": [
- 0.04783163265306122,
- 0.3109575518262586,
- 0.02423469387755102,
- 0.013820335636722606
- ],
- "16": [
- 0.3086734888504963,
- 0.32181638696939785,
- 0.024872468442332988,
- 0.017769002961500493
- ],
- "17": [
- 0.35012755102040816,
- 0.3228035538005923,
- 0.03571428571428571,
- 0.01579466929911155
- ],
- "18": [
- 0.6881378329530055,
- 0.4629812739561482,
- 0.03443877551020408,
- 0.013820335636722606
- ],
- "19": [
- 0.7602040816326531,
- 0.5093780848963475,
- 0.07270408163265306,
- 0.027640671273445213
- ],
- "2": [
- 0.19196428571428573,
- 0.008884501480750246,
- 0.030612244897959183,
- 0.013820335636722606
- ],
- "20": [
- 0.7608418367346939,
- 0.5380059230009872,
- 0.039540816326530615,
- 0.01579466929911155
- ],
- "21": [
- 0.38711734693877553,
- 0.5893385982230998,
- 0.03188775510204082,
- 0.01579466929911155
- ],
- "22": [
- 0.7697703303123007,
- 0.5932872655478776,
- 0.07142857142857142,
- 0.017769002961500493
- ],
- "23": [
- 0.38711734693877553,
- 0.6051332675222113,
- 0.025510204081632654,
- 0.013820335636722606
- ],
- "24": [
- 0.3781887755102041,
- 0.6446200010218719,
- 0.07015306122448979,
- 0.01579466929911155
- ],
- "25": [
- 0.37691330423160474,
- 0.6623889437314906,
- 0.02423469387755102,
- 0.013820335636722606
- ],
- "26": [
- 0.4190051020408163,
- 0.6623889437314906,
- 0.042091836734693876,
- 0.013820335636722606
- ],
- "27": [
- 0.27359695823825136,
- 0.7058242843040474,
- 0.021683673469387755,
- 0.013820335636722606
- ],
- "28": [
- 0.27295916421072824,
- 0.7226061204343535,
- 0.05994899905457789,
- 0.023692003948667325
- ],
- "29": [
- 0.6881378329530055,
- 0.7334650158293744,
- 0.03443877551020408,
- 0.01579466929911155
- ],
- "3": [
- 0.23150508257807517,
- 0.008884501480750246,
- 0.019132691986706793,
- 0.013820335636722606
- ],
- "30": [
- 0.7697704081632653,
- 0.8262586377097729,
- 0.04081632653061224,
- 0.017769002961500493
- ],
- "31": [
- 0.7684948979591837,
- 0.8538993089832182,
- 0.12627551020408162,
- 0.019743336623889437
- ],
- "32": [
- 0.4840561224489796,
- 0.8815399802566634,
- 0.021683673469387755,
- 0.013820335636722606
- ],
- "33": [
- 0.5408163265306123,
- 0.8746298124383021,
- 0.026785714285714284,
- 0.023692003948667325
- ],
- "34": [
- 0.5854591836734694,
- 0.87857847976308,
- 0.0389030612244898,
- 0.019743336623889437
- ],
- "35": [
- 0.6894131874551579,
- 0.8815399802566634,
- 0.02295918367346939,
- 0.013820335636722606
- ],
- "36": [
- 0.7697704081632653,
- 0.8914116485686081,
- 0.03571428571428571,
- 0.01579466929911155
- ],
- "37": [
- 0.8169642857142857,
- 0.8933859822309971,
- 0.016581632653061226,
- 0.013820335636722606
- ],
- "38": [
- 0.8552295918367347,
- 0.8933859822309971,
- 0.06887755102040816,
- 0.013820335636722606
- ],
- "39": [
- 0.9304846938775511,
- 0.8914116485686081,
- 0.04081632653061224,
- 0.01579466929911155
- ],
- "4": [
- 0.9559948979591837,
- 0.008884501480750246,
- 0.036989795918367346,
- 0.013820335636722606
- ],
- "40": [
- 0.7697704081632653,
- 0.9062190907846434,
- 0.20408163265306123,
- 0.018756169792694965
- ],
- "41": [
- 0.4419642857142857,
- 0.9308982615645052,
- 0.11734693877551021,
- 0.035538005923000986
- ],
- "42": [
- 0.3156887949729452,
- 0.9723593287265548,
- 0.016581613190320074,
- 0.013820335636722606
- ],
- "43": [
- 0.39987244897959184,
- 0.6594274432379073,
- 0.01913265306122449,
- 0.01678183613030602
- ],
- "44": [
- 0.58006138704261,
- 0.4934063353166646,
- 0.09696150799186862,
- 0.20261972771933612
- ],
- "45": [
- 0.4783496078179807,
- 0.49486916062862474,
- 0.09353917958785077,
- 0.2038884214681793
- ],
- "46": [
- 0.3751486564169125,
- 0.028989782455760657,
- 0.2846154582743742,
- 0.05785660211802235
- ],
- "47": [
- 0.012596755611653231,
- 0.1447105746631679,
- 0.04683203599890884,
- 0.03206605148692004
- ],
- "48": [
- 0.6834940229143415,
- 0.4927587179652795,
- 0.0686746714066486,
- 0.20419697135480935
- ],
- "49": [
- 0.42255070744728557,
- 0.3606170322911525,
- 0.15194312893614478,
- 0.046830835370650296
- ],
- "5": [
- 0.01594387998386305,
- 0.10760119213168266,
- 0.05357142857142857,
- 0.019743329092404213
- ],
- "50": [
- 0.27106018455661074,
- 0.36041600188757095,
- 0.15268113661785515,
- 0.048123901005676206
- ],
- "51": [
- 0.27404166241081396,
- 0.4940390403273075,
- 0.09522936295489876,
- 0.19714704929664364
- ],
- "52": [
- 0.010258441068688217,
- 0.451038507459665,
- 0.22214404904112525,
- 0.05718111450556824
- ],
- "53": [
- 0.009205195368552695,
- 0.6229568586886106,
- 0.21685822156010842,
- 0.05717975883822803
- ],
- "54": [
- 0.06089101032334931,
- 0.14523588069452587,
- 0.045354497675993005,
- 0.03156237588112352
- ],
- "55": [
- 0.010520691774329361,
- 0.3979400604639684,
- 0.23084091653629224,
- 0.04781360381447433
- ],
- "56": [
- 0.009719303676060267,
- 0.33765850462541647,
- 0.23084155880675025,
- 0.0570423845477542
- ],
- "57": [
- 0.009891003978495695,
- 0.5662159143054356,
- 0.216797906525281,
- 0.05687765590294916
- ],
- "58": [
- 0.009775356370575574,
- 0.5117536960914363,
- 0.22039232448655732,
- 0.053711178507527144
- ],
- "59": [
- 0.04758900525618573,
- 0.286834144403998,
- 0.02814068113054548,
- 0.019245234317158192
- ],
- "6": [
- 0.7602040816326531,
- 0.10661401023751774,
- 0.04145408163265306,
- 0.01974334415537466
- ],
- "60": [
- 0.10757595179032306,
- 0.14555321229292634,
- 0.04127831361731704,
- 0.03079169406118892
- ],
- "61": [
- 0.27328668321881977,
- 0.7663382617765918,
- 0.0968975339617048,
- 0.15312702446322804
- ],
- "62": [
- 0.047278949192592075,
- 0.22891404056831502,
- 0.05425823951254086,
- 0.020403847877976927
- ],
- "63": [
- 0.27381169066137195,
- 0.4497022412253979,
- 0.13928121449996014,
- 0.028031043215541708
- ],
- "64": [
- 0.021747477200566506,
- 0,
- 0.04294475244016063,
- 0.030762471898522336
- ],
- "65": [
- 0.93038154134945,
- 0.8478104948173741,
- 0.04799465257294324,
- 0.03264567458970879
- ],
- "66": [
- 0.15421617274381677,
- 0.09903491849607447,
- 0.060613369455142896,
- 0.03467735298058829
- ],
- "67": [
- 0.010927823125099649,
- 0.6817393175823667,
- 0.2184713130094567,
- 0.050834874174790226
- ],
- "68": [
- 0.0466434906940071,
- 0.9401940423903319,
- 0.07385999329236088,
- 0.038094613770977295
- ],
- "69": [
- 0.08955291825897839,
- 0,
- 0.05353314535958426,
- 0.027550474206410415
- ],
- "7": [
- 0.5465561224489796,
- 0.12142152023692004,
- 0.08418367346938775,
- 0.019743336623889437
- ],
- "70": [
- 0.3760816613022162,
- 0.7656286151129073,
- 0.09396556932099011,
- 0.15398585478621668
- ],
- "71": [
- 0.3448911005136918,
- 0.03201548509513049,
- 0.03080227910255899,
- 0.05057347891714354
- ],
- "72": [
- 0.004894363636873207,
- 0.035978590277081536,
- 0.047808394140126755,
- 0.041212385051459924
- ],
- "73": [
- 0.6986004576391104,
- 0,
- 0.020401779486208547,
- 0.02902264232579012
- ],
- "74": [
- 0.7628018904705437,
- 0,
- 0.01995133380500638,
- 0.029507805540907654
- ],
- "75": [
- 0.9649586580237564,
- 0.03291509841060356,
- 0.031732987384406885,
- 0.048350365234857015
- ],
- "76": [
- 0.7195011839574698,
- 0,
- 0.0201263427734375,
- 0.029419382138642956
- ],
- "77": [
- 0.784203120640346,
- 0,
- 0.018760058344626913,
- 0.029734610569794817
- ],
- "78": [
- 0.7409005457041214,
- 0,
- 0.02084911112882653,
- 0.029456582027032368
- ],
- "79": [
- 0.9418883810237962,
- 0.035522302776310306,
- 0.021989004952566966,
- 0.04376863138814552
- ],
- "8": [
- 0.548469387755102,
- 0.1441263573543929,
- 0.05803571428571429,
- 0.01678183613030602
- ],
- "80": [
- 0.8045983606455277,
- 0,
- 0.021617188745615433,
- 0.030099550569163375
- ],
- "81": [
- 0.9196810041155133,
- 0.03726533699600506,
- 0.022023259376992985,
- 0.040672656225899245
- ],
- "82": [
- 0.8250376642966757,
- 0.9454778311636229,
- 0.017945581552933673,
- 0.029979769828171274
- ],
- "83": [
- 0.8947929070920361,
- 0,
- 0.020548606405452807,
- 0.03135072043476312
- ],
- "84": [
- 0.05306020074961137,
- 0.03727699950087341,
- 0.01825682970942283,
- 0.03905122913236326
- ],
- "85": [
- 0.8983937477578923,
- 0.9447249236488154,
- 0.031814108089524874,
- 0.02822993467731984
- ],
- "86": [
- 0.8700136457170758,
- 0,
- 0.02496711575255102,
- 0.031069679297911334
- ],
- "87": [
- 0.844688571229273,
- 0.9456241829844829,
- 0.01700389628507653,
- 0.0284405752560464
- ],
- "88": [
- 0.07209396848873216,
- 0.03687979650167934,
- 0.01847073983172981,
- 0.039172568890796594
- ],
- "89": [
- 0.4397279389050542,
- 0.9343330405895237,
- 0.01783736871213329,
- 0.03596145614819842
- ],
- "9": [
- 0.35267857142857145,
- 0.18657453109575517,
- 0.026785714285714284,
- 0.013820335636722606
- ],
- "90": [
- 0.010644483322999915,
- 0.28225623655178,
- 0.03258451636956663,
- 0.04633538414671767
- ],
- "91": [
- 0.8640503396793288,
- 0.9453993832135366,
- 0.016077781210140307,
- 0.028209931052566632
- ],
- "92": [
- 0.9579316742566167,
- 0.9451709080778011,
- 0.018238768285634567,
- 0.029882643794730997
- ],
- "93": [
- 0.8269350480060188,
- 0,
- 0.03306128054249043,
- 0.030492771295545604
- ],
- "94": [
- 0.8830389684560348,
- 0.945025399783286,
- 0.014456145617426659,
- 0.028434309060340574
- ],
- "95": [
- 0.7598882402692523,
- 0.14337289250967886,
- 0.22784579530054208,
- 0.37021901318021966
- ],
- "96": [
- 0.5119574021319954,
- 0.11800540423228498,
- 0.03202204801598374,
- 0.04860953117287759
- ],
- "97": [
- 0.011849756143531022,
- 0.1824839856523399,
- 0.02146280541711924,
- 0.034883399353328605
- ],
- "98": [
- 0.26384283571827166,
- 0.10172352315407515,
- 0.015009121019013074,
- 0.030118605226778443
- ],
- "99": [
- 0.004816442119831942,
- 0,
- 0.020643239118614976,
- 0.02995776471121996
- ]
- },
- "parsed_content_list": [
- {
- "bbox": [
- 0.06951530277729034,
- 0.00888450164347887,
- 0.08354591578245163,
- 0.02270483784377575
- ],
- "content": "File",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.14859694242477417,
- 0.00888450164347887,
- 0.1830357164144516,
- 0.02270483784377575
- ],
- "content": "Playback",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.1919642835855484,
- 0.00888450164347887,
- 0.22257652878761292,
- 0.02270483784377575
- ],
- "content": "Window",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.23150509595870972,
- 0.00888450164347887,
- 0.2506377696990967,
- 0.02270483784377575
- ],
- "content": "Help",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.9559949040412903,
- 0.00888450164347887,
- 0.9929847121238708,
- 0.02270483784377575
- ],
- "content": "10:04AM",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.015943877398967743,
- 0.10760118812322617,
- 0.06951530277729034,
- 0.1273445188999176
- ],
- "content": "Your Library",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.7602040767669678,
- 0.10661401599645615,
- 0.8016581535339355,
- 0.12635736167430878
- ],
- "content": "Coldplay",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5465561151504517,
- 0.12142152339220047,
- 0.6307398080825806,
- 0.1411648541688919
- ],
- "content": "HOT SUMMER POP",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5484693646430969,
- 0.14412635564804077,
- 0.6065050959587097,
- 0.16090819239616394
- ],
- "content": "Advertisement",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.3526785671710968,
- 0.1865745335817337,
- 0.3794642984867096,
- 0.200394868850708
- ],
- "content": "Spotify",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.18941326439380646,
- 0.19249753654003143,
- 0.22130101919174194,
- 0.20829220116138458
- ],
- "content": "Recents",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.512117326259613,
- 0.18460020422935486,
- 0.7289540767669678,
- 0.22606121003627777
- ],
- "content": "30I3 #zw4JH4I--Fot3f",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.05676020309329033,
- 0.2537018656730652,
- 0.08609694242477417,
- 0.26949653029441833
- ],
- "content": "Playlist",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.08992347121238708,
- 0.2537018656730652,
- 0.12308673560619354,
- 0.26949653029441833
- ],
- "content": "DavidLe",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5223214030265808,
- 0.2596248686313629,
- 0.5656887888908386,
- 0.27541953325271606
- ],
- "content": "Listennow",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.04783163219690323,
- 0.31095755100250244,
- 0.07206632941961288,
- 0.32477790117263794
- ],
- "content": "Artist",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.3086734712123871,
- 0.3218163847923279,
- 0.3335459232330322,
- 0.33958539366722107
- ],
- "content": "Music",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.35012754797935486,
- 0.3228035569190979,
- 0.38584184646606445,
- 0.33859822154045105
- ],
- "content": "Podcasts",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.6881377696990967,
- 0.462981253862381,
- 0.7225765585899353,
- 0.4768015742301941
- ],
- "content": "Showall",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.7602040767669678,
- 0.5093780755996704,
- 0.8329081535339355,
- 0.5370187759399414
- ],
- "content": "Angelsong",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.7608418464660645,
- 0.5380059480667114,
- 0.800382673740387,
- 0.5538005828857422
- ],
- "content": "Coldplay",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.3871173560619354,
- 0.5893386006355286,
- 0.4190050959587097,
- 0.6051332950592041
- ],
- "content": "Release",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.7697703838348389,
- 0.5932872891426086,
- 0.8411989808082581,
- 0.6110562682151794
- ],
- "content": "About the artist",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.3871173560619354,
- 0.6051332950592041,
- 0.41262754797935486,
- 0.6189535856246948
- ],
- "content": "Radar",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.3781887888908386,
- 0.6446199417114258,
- 0.44834184646606445,
- 0.6604146361351013
- ],
- "content": "Catch all the latest",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.37691327929496765,
- 0.6623889207839966,
- 0.4011479616165161,
- 0.6762092709541321
- ],
- "content": "music",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.4190050959587097,
- 0.6623889207839966,
- 0.46109694242477417,
- 0.6762092709541321
- ],
- "content": "artists YOa",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.27359694242477417,
- 0.7058242559432983,
- 0.2952806055545807,
- 0.7196446061134338
- ],
- "content": "Made",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.2729591727256775,
- 0.7226061224937439,
- 0.33290815353393555,
- 0.7462981343269348
- ],
- "content": "David Le",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.6881377696990967,
- 0.7334649562835693,
- 0.7225765585899353,
- 0.7492596507072449
- ],
- "content": "Showall",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.7697703838348389,
- 0.826258659362793,
- 0.8105867505073547,
- 0.8440276384353638
- ],
- "content": "Coldplay",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.7684949040412903,
- 0.8538993000984192,
- 0.8947703838348389,
- 0.87364262342453
- ],
- "content": "92,932,665 monthly listeners",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.48405611515045166,
- 0.8815400004386902,
- 0.5057398080825806,
- 0.8953602910041809
- ],
- "content": "Daily",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5408163070678711,
- 0.8746297955513,
- 0.5676020383834839,
- 0.898321807384491
- ],
- "content": "03",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5854591727256775,
- 0.8785784840583801,
- 0.6243622303009033,
- 0.898321807384491
- ],
- "content": "Daily Mix]",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.6894132494926453,
- 0.8815400004386902,
- 0.7123724222183228,
- 0.8953602910041809
- ],
- "content": "Daily E",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.7697703838348389,
- 0.8914116621017456,
- 0.8054847121238708,
- 0.9072062969207764
- ],
- "content": "Coldplay",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.8169642686843872,
- 0.8933860063552856,
- 0.8335459232330322,
- 0.9072062969207764
- ],
- "content": "one",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.8552296161651611,
- 0.8933860063552856,
- 0.9241071343421936,
- 0.9072062969207764
- ],
- "content": "most popular acts",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.9304847121238708,
- 0.8914116621017456,
- 0.9713010191917419,
- 0.9072062969207764
- ],
- "content": "the world;",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.7697703838348389,
- 0.9062191247940063,
- 0.9738520383834839,
- 0.9249753355979919
- ],
- "content": "having sold more than I00 million albums and releasing",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.4419642984867096,
- 0.9308983087539673,
- 0.5593112111091614,
- 0.9664363265037537
- ],
- "content": "X K 0 M Q",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.3156887888908386,
- 0.972359299659729,
- 0.33227041363716125,
- 0.9861796498298645
- ],
- "content": "0.58",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.39987245202064514,
- 0.6594274640083313,
- 0.4190050959587097,
- 0.6762092709541321
- ],
- "content": "from",
- "interactivity": false,
- "source": "box_ocr_content_ocr",
- "type": "text"
- },
- {
- "bbox": [
- 0.5800613760948181,
- 0.4934062957763672,
- 0.6770228743553162,
- 0.6960260272026062
- ],
- "content": "THISIS FREQUENCY best in Black musicl Cover: ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.47834959626197815,
- 0.4948691427707672,
- 0.5718887448310852,
- 0.6987575888633728
- ],
- "content": "Singled Out buzzing here. The hottest newa ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.3751486837863922,
- 0.02898978441953659,
- 0.6597641706466675,
- 0.08684638142585754
- ],
- "content": "doyou want to play? What ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.012596755288541317,
- 0.14471057057380676,
- 0.05942879244685173,
- 0.17677663266658783
- ],
- "content": "Playlists ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.6834940314292908,
- 0.49275872111320496,
- 0.7521687746047974,
- 0.696955680847168
- ],
- "content": "Dance Popa 7oetimjich ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.4225507080554962,
- 0.3606170415878296,
- 0.5744938254356384,
- 0.407447874546051
- ],
- "content": "Lovo Letter ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.27106019854545593,
- 0.3604159951210022,
- 0.42374131083488464,
- 0.40853989124298096
- ],
- "content": "Coldplay ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.27404168248176575,
- 0.49403905868530273,
- 0.36927103996276855,
- 0.6911860704421997
- ],
- "content": "RDA Ariana Grande, Z4eiN3530t_ ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.010258435271680355,
- 0.45103850960731506,
- 0.2324024885892868,
- 0.5082195997238159
- ],
- "content": "Cigarettes After Sex Album Cigarettes After Sex ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.009205197915434837,
- 0.6229568719863892,
- 0.22606341540813446,
- 0.6801366209983826
- ],
- "content": "Eyes Wide Album Sabrina Carpenter Open ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.060891006141901016,
- 0.14523588120937347,
- 0.1062455102801323,
- 0.1767982691526413
- ],
- "content": "Albums ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.010520695708692074,
- 0.397940069437027,
- 0.24136163294315338,
- 0.44575366377830505
- ],
- "content": "Playlist * ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.009719299152493477,
- 0.33765852451324463,
- 0.2405608594417572,
- 0.39470088481903076
- ],
- "content": "You're Love Single Mood; ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.009891008958220482,
- 0.5662159323692322,
- 0.2266889065504074,
- 0.6230935454368591
- ],
- "content": "Discover Weekly Playlist Made for DavidLe ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.009775351732969284,
- 0.5117536187171936,
- 0.2301676869392395,
- 0.5654648542404175
- ],
- "content": "Album Cigarettes After Sox Cry ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.047589004039764404,
- 0.28683412075042725,
- 0.07572968304157257,
- 0.30607935786247253
- ],
- "content": "Mood: ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.10757595300674438,
- 0.14555321633815765,
- 0.14885427057743073,
- 0.17634491622447968
- ],
- "content": "Artists ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.27328670024871826,
- 0.7663382291793823,
- 0.3701842129230499,
- 0.9194653034210205
- ],
- "content": "Daily ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.04727894812822342,
- 0.2289140373468399,
- 0.10153719037771225,
- 0.24931789934635162
- ],
- "content": "Liked Songs ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.2738116979598999,
- 0.44970226287841797,
- 0.4130929112434387,
- 0.4777332842350006
- ],
- "content": "It's New Music Fridayl ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.021747475489974022,
- 0,
- 0.06469222903251648,
- 0.03076247125864029
- ],
- "content": "Spotify ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.9303815960884094,
- 0.8478105664253235,
- 0.9783762097358704,
- 0.8804562091827393
- ],
- "content": "Follow ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.15421617031097412,
- 0.09903492778539658,
- 0.21482954919338226,
- 0.1337122768163681
- ],
- "content": "Create ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.010927819646894932,
- 0.681739330291748,
- 0.22939912974834442,
- 0.7325741648674011
- ],
- "content": "Return Album ODESZA ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.04664349555969238,
- 0.9401940107345581,
- 0.1205034852027893,
- 0.9782886505126953
- ],
- "content": "Angelsong Coldplay ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.08955291658639908,
- 0,
- 0.14308606088161469,
- 0.02755047380924225
- ],
- "content": "Edit View ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.3760816752910614,
- 0.7656286358833313,
- 0.4700472056865692,
- 0.9196145534515381
- ],
- "content": "Daily ",
- "interactivity": true,
- "source": "box_yolo_content_ocr",
- "type": "icon"
- },
- {
- "bbox": [
- 0.34489110112190247,
- 0.03201548755168915,
- 0.3756934106349945,
- 0.08258896321058273
- ],
- "content": "Home",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.00489436462521553,
- 0.035978592932224274,
- 0.052702758461236954,
- 0.07719097286462784
- ],
- "content": "Color picker",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.6986004710197449,
- 0,
- 0.7190021872520447,
- 0.029022641479969025
- ],
- "content": "Toggleoggleoggle",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.7628018856048584,
- 0,
- 0.7827531695365906,
- 0.029507804661989212
- ],
- "content": "Mute",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.964958667755127,
- 0.03291509672999382,
- 0.9966916441917419,
- 0.08126546442508698
- ],
- "content": "Currency",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.7195011973381042,
- 0,
- 0.7396275401115417,
- 0.02941938303411007
- ],
- "content": "Toggleoggleoggle",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.7842031717300415,
- 0,
- 0.8029631972312927,
- 0.02973460964858532
- ],
- "content": "Navigation Bar",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.7409006357192993,
- 0,
- 0.7617496848106384,
- 0.02945658192038536
- ],
- "content": "Panther",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.9418883919715881,
- 0.035522300750017166,
- 0.9638773798942566,
- 0.07929093390703201
- ],
- "content": "28, September, 2024",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.804598331451416,
- 0,
- 0.8262154459953308,
- 0.030099550262093544
- ],
- "content": "Text formatting options.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.9196809530258179,
- 0.037265338003635406,
- 0.941704273223877,
- 0.07793799042701721
- ],
- "content": "Notifications",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.8250376582145691,
- 0.9454777836799622,
- 0.8429831862449646,
- 0.9754576086997986
- ],
- "content": "Fill",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.894792914390564,
- 0,
- 0.9153414964675903,
- 0.031350720673799515
- ],
- "content": "More Options",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.05306020379066467,
- 0.03727699816226959,
- 0.07131703197956085,
- 0.07632823288440704
- ],
- "content": "Previous",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.8983937501907349,
- 0.944724977016449,
- 0.9302077889442444,
- 0.9729548692703247
- ],
- "content": "Line",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.8700135946273804,
- 0,
- 0.894980788230896,
- 0.0310696791857481
- ],
- "content": "Link",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.8446885943412781,
- 0.9456241726875305,
- 0.8616925477981567,
- 0.9740647077560425
- ],
- "content": "Center",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.07209397107362747,
- 0.03687979653477669,
- 0.09056470543146133,
- 0.07605236023664474
- ],
- "content": "Next",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.43972793221473694,
- 0.9343330264091492,
- 0.4575652778148651,
- 0.9702944755554199
- ],
- "content": "Shuffle",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.010644486173987389,
- 0.28225621581077576,
- 0.04322900250554085,
- 0.32859158515930176
- ],
- "content": "a person with purple hair.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.8640503883361816,
- 0.9453994035720825,
- 0.8801282048225403,
- 0.9736093282699585
- ],
- "content": "Switch between plug-in and switch between plug ins",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.9579315781593323,
- 0.9451709389686584,
- 0.9761704206466675,
- 0.9750535488128662
- ],
- "content": "Copy",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.8269351124763489,
- 0,
- 0.8599963188171387,
- 0.030492771416902542
- ],
- "content": "a power or power-up function.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.8830388784408569,
- 0.9450254440307617,
- 0.8974950909614563,
- 0.973459780216217
- ],
- "content": "Diamond",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.7598882913589478,
- 0.14337289333343506,
- 0.9877340197563171,
- 0.5135919451713562
- ],
- "content": "Siri",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.5119573473930359,
- 0.11800540238618851,
- 0.5439794063568115,
- 0.16661493480205536
- ],
- "content": "Duck",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.01184975728392601,
- 0.1824839860200882,
- 0.033312562853097916,
- 0.21736738085746765
- ],
- "content": "Find",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.2638428509235382,
- 0.10172352939844131,
- 0.2788519561290741,
- 0.13184213638305664
- ],
- "content": "A logo for the Volkswagen company.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.00481644319370389,
- 0,
- 0.02545968070626259,
- 0.029957763850688934
- ],
- "content": "Apple Store",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.010349130257964134,
- 0.22549642622470856,
- 0.04359712451696396,
- 0.2742839753627777
- ],
- "content": "Heart",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.9776452779769897,
- 0.9448952674865723,
- 0.9948262572288513,
- 0.9750445485115051
- ],
- "content": "Open Menu",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.9686180949211121,
- 0.5149807333946228,
- 0.986960768699646,
- 0.5458763241767883
- ],
- "content": "Add",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 2.5892255507642403e-05,
- 0,
- 0.0054787443950772285,
- 0.02969358116388321
- ],
- "content": "a field or field with trees.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.2734973728656769,
- 0.31281059980392456,
- 0.29989612102508545,
- 0.3477778434753418
- ],
- "content": "All",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.21768823266029358,
- 0.09972180426120758,
- 0.24076758325099945,
- 0.13786810636520386
- ],
- "content": "Decrease",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.9318671822547913,
- 0.945142924785614,
- 0.955677330493927,
- 0.9737845063209534
- ],
- "content": "Increase",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- },
- {
- "bbox": [
- 0.007034625858068466,
- 0.9328103065490723,
- 0.04268921539187431,
- 0.9868168830871582
- ],
- "content": "Microsoft Edge browser.",
- "interactivity": true,
- "source": "box_yolo_content_yolo",
- "type": "icon"
- }
- ]
-}
\ No newline at end of file
diff --git a/vision/bench/results/spotify/summary.json b/vision/bench/results/spotify/summary.json
deleted file mode 100644
index 12ee6d2f..00000000
--- a/vision/bench/results/spotify/summary.json
+++ /dev/null
@@ -1,27 +0,0 @@
-{
- "endpoint": "https://api.runpod.ai/v2/tjeid8c15y30sl/runsync",
- "image": "/Users/davidle/Desktop/spotify.png",
- "raw_kb": 5235,
- "payload": "compressed via sips -> 1568x1013 JPEG q80 (295 KB, 5.6% of raw)",
- "payload_b64_kb": 394,
- "compress_ms": 103,
- "runs": [
- {
- "run": 1,
- "label": "cold",
- "total_s": 26.81182220800838,
- "queue_s": 21.11,
- "compute_s": 3.545,
- "transport_s": 2.1568222080083803,
- "status": "COMPLETED",
- "elements": 108
- }
- ],
- "warm_median": {
- "total_s": 26.81182220800838,
- "queue_s": 21.11,
- "compute_s": 3.545,
- "transport_s": 2.1568222080083803
- },
- "cold_total_s": 26.81182220800838
-}
\ No newline at end of file
diff --git a/vision/bench/results/spotify/summary.txt b/vision/bench/results/spotify/summary.txt
deleted file mode 100644
index 6103aa66..00000000
--- a/vision/bench/results/spotify/summary.txt
+++ /dev/null
@@ -1,19 +0,0 @@
-Endpoint: https://api.runpod.ai/v2/tjeid8c15y30sl/runsync
-Image: /Users/davidle/Desktop/spotify.png (raw 5235 KB)
-Payload: compressed via sips -> 1568x1013 JPEG q80 (295 KB, 5.6% of raw)
- 394 KB base64 uploaded per run
-Runs: 1 (run 1 is cold)
-Compress: 103 ms (local, once)
-
- run total queue compute transport status / elements
- --------------------------------------------------------------------
- run 1 (cold) 26.81s 21.11s 3.54s 2.16s COMPLETED / 108 elements
-
-Summary (warm median)
- total: 26.81s
- queue: 21.11s
- compute: 3.54s <- actual model work
- transport: 2.16s <- upload + download + network
- compress: 103 ms (local, per screenshot)
-
- cold total: 26.81s
diff --git a/vision/handler.py b/vision/handler.py
deleted file mode 100644
index 17559c5b..00000000
--- a/vision/handler.py
+++ /dev/null
@@ -1,112 +0,0 @@
-"""RunPod Serverless handler for Microsoft OmniParser V2.
-
-Wraps OmniParser's screen-parsing pipeline (YOLO icon detection + OCR +
-Florence-2 captioning) behind a RunPod serverless endpoint. The handler returns
-OmniParser's *raw* output verbatim; downstream callers (e.g. the Donkey site
-adapter) own any normalization into product-specific coordinate contracts.
-
-Coordinate note: every `bbox` in `parsed_content_list` is `[x1, y1, x2, y2]`
-in ratio space (0-1), independent of image size. `label_coordinates` is
-`[x, y, w, h]` in ratio space. See OmniParser util/utils.get_som_labeled_img.
-"""
-
-import base64
-import io
-import os
-import sys
-
-import runpod
-import torch
-from PIL import Image
-
-# OmniParser resolves its weights relative to the working directory, and its
-# `util` package is only importable when its repo root is on sys.path. chdir
-# alone doesn't add the cwd to sys.path (Python seeds sys.path with the script's
-# dir, /app), so do both explicitly before importing from util.
-OMNIPARSER_ROOT = "/app/OmniParser"
-os.chdir(OMNIPARSER_ROOT)
-sys.path.insert(0, OMNIPARSER_ROOT)
-
-from util.utils import ( # noqa: E402 (import after sys.path setup, by design)
- check_ocr_box,
- get_caption_model_processor,
- get_som_labeled_img,
- get_yolo_model,
-)
-
-DEVICE = torch.device("cuda" if torch.cuda.is_available() else "cpu")
-
-# Load once at cold start so warm invocations reuse the resident models.
-yolo_model = get_yolo_model(model_path="weights/icon_detect/model.pt")
-caption_model_processor = get_caption_model_processor(
- model_name="florence2",
- model_name_or_path="weights/icon_caption_florence",
-)
-
-
-def parse_image(image, box_threshold=0.05, iou_threshold=0.7, imgsz=640):
- box_overlay_ratio = image.size[0] / 3200
- draw_bbox_config = {
- "text_scale": 0.8 * box_overlay_ratio,
- "text_thickness": max(int(2 * box_overlay_ratio), 1),
- "text_padding": max(int(3 * box_overlay_ratio), 1),
- "thickness": max(int(3 * box_overlay_ratio), 1),
- }
-
- ocr_bbox_rslt, _ = check_ocr_box(
- image,
- display_img=False,
- output_bb_format="xyxy",
- goal_filtering=None,
- easyocr_args={"paragraph": False, "text_threshold": 0.9},
- use_paddleocr=False,
- )
- text, ocr_bbox = ocr_bbox_rslt
-
- labeled_img_b64, label_coordinates, parsed_content_list = get_som_labeled_img(
- image,
- yolo_model,
- BOX_TRESHOLD=box_threshold,
- output_coord_in_ratio=True,
- ocr_bbox=ocr_bbox,
- draw_bbox_config=draw_bbox_config,
- caption_model_processor=caption_model_processor,
- ocr_text=text,
- iou_threshold=iou_threshold,
- imgsz=imgsz,
- )
-
- return {
- # SOM-annotated PNG (base64). Useful for debugging overlays.
- "labeled_image_base64": labeled_img_b64,
- # Pixel dimensions of the parsed image, so callers can convert the ratio
- # bboxes below into pixel coordinates.
- "image_size": {"width": image.size[0], "height": image.size[1]},
- # {label: [x, y, w, h]} in ratio space.
- "label_coordinates": label_coordinates,
- # [{type, bbox:[x1,y1,x2,y2] ratio, interactivity, content, source?}]
- "parsed_content_list": parsed_content_list,
- }
-
-
-def handler(event):
- inp = event["input"]
- image_b64 = inp["image"]
- box_threshold = inp.get("box_threshold", 0.05)
- # 0.7 keeps densely-packed, adjacent boxes (e.g. file-list rows) distinct;
- # a low value over-merges neighbors via NMS. See OmniParser remove_overlap_new.
- iou_threshold = inp.get("iou_threshold", 0.7)
- imgsz = inp.get("imgsz", 640)
-
- image_bytes = base64.b64decode(image_b64)
- image = Image.open(io.BytesIO(image_bytes)).convert("RGB")
-
- return parse_image(
- image,
- box_threshold=box_threshold,
- iou_threshold=iou_threshold,
- imgsz=imgsz,
- )
-
-
-runpod.serverless.start({"handler": handler})
diff --git a/vision/requirements-worker.txt b/vision/requirements-worker.txt
deleted file mode 100644
index f14f182f..00000000
--- a/vision/requirements-worker.txt
+++ /dev/null
@@ -1,6 +0,0 @@
-# Worker-only deps layered on top of OmniParser's own requirements.txt
-# (installed from the cloned repo inside the Docker build). These are also
-# pip-installed in the Dockerfile; this file documents the worker surface and
-# supports local handler testing.
-runpod
-huggingface_hub
diff --git a/vision/test_input.json b/vision/test_input.json
deleted file mode 100644
index 2c4ec5ac..00000000
--- a/vision/test_input.json
+++ /dev/null
@@ -1,8 +0,0 @@
-{
- "input": {
- "image": "REPLACE_WITH_BASE64_SCREENSHOT",
- "box_threshold": 0.05,
- "iou_threshold": 0.1,
- "imgsz": 640
- }
-}