Skip to content

V- Modal AI: Visual Video / Image Search - Flutter Video SDK

5.8 KB
Raw Blame

Search application integration

This guide uses the advanced tenant-scoped project/client and signed-image path. For an account-based app, use the complete session-bound controller/widget recipe and live SessionAsset media through UserScope.imageBytes. Do not mix raw selectors or arbitrary signed URLs into a feature relying on session isolation.

Keep the project above individual pages so navigation does not close shared network state. Retain an immutable scope in the page controller, provider, BLoC, or notifier:

final project = VModal.configure(
  projectId: 'video_search',
  apiKeyProvider: keys,
);
final scope = project.scope(
  collectionName: 'global',
  streamName: 'uploads',
);
final result = await scope.search(
  'red bicycle',
  options: const ScopedSearchOptions(
    searchSources: ['image'],
  ),
);

Use versionLancedb in ScopedSearchOptions when your application tracks a specific index version. Collection and stream cannot be overridden per search, so overlapping searches retain the organization with which they started.

Use result.videoHits as the recommended parsing path. Each VideoSearchHit exposes nullable assetId, fileName, playbackOffsetMs, distance, and previewImageUrl fields while retaining the complete service map in raw:

for (final hit in result.videoHits) {
  final seconds = hit.playbackOffsetMs == null
      ? null
      : hit.playbackOffsetMs! / 1000;
  print('${hit.assetId} ${hit.fileName} $seconds ${hit.distance}');
}

playbackOffsetMs is elapsed time from the beginning of the source video. distance is the raw lower-is-better distance, not similarity or confidence. assetId is stable only when the server supplies canonical asset_id; legacy responses legitimately expose a null identity.

When previewImageUrl is present, validate it at the application boundary and reuse it. Otherwise convert each usable typed hit into one bulk lookup record with these fields:

mode                 vid_file
group_name           selected authenticated collection
modality             image
stream_name          hit stream, or selected stream as fallback
filename             hit.fileName
ts_unix_13digits     zero-padded hit.playbackOffsetMs, when present

For backward compatibility, result.data remains unchanged and callers may still inspect raw maps. The typed model already handles the documented legacy filename, seconds, relative-millisecond, score, and preview aliases. New code should not repeat those aliases or derive a filename from item_id.

Image URL resolution remains an advanced low-level resource. When a result flow needs it, create VmodalClient first, pass it to VModal.fromClient, and use client.images while the project is alive. Obtain raw image selectors from trusted result data or a low-level collection response; never manually compose a backend project/collection name.

Send all candidate records in one request:

final candidates = exampleSearchCandidates(result, 'agroup', 'astream');
final resolved = candidates.isEmpty
    ? null
    : await client.images.getUrlBulk(
        candidates.map((candidate) => candidate.record).toList(),
      );
final images = resolved == null
    ? const <ExampleSearchImage>[]
    : exampleSearchImages(candidates, resolved);

Map each usable url_pre_signed back through input_index; do not join by response order when an index is present. Accept numeric or parseable-string indexes, reject negative/out-of-range values, keep the first usable duplicate, and sort the final cards by candidate index. A missing index may use only its bounded response-row position as a compatibility fallback.

Use a responsive GridView.builder with one page scroll owner. Show total backend matches separately from resolved image count, and keep an image load failure local to its card. A successful search may legitimately have matches but no image-backed cards because filenames or URL records were unavailable.

Presigned image URLs are temporary capabilities. Keep them in current widget state only, resolve fresh URLs for every search, and never log or persist them. Download them directly with NetworkImage/Image.network; do not attach the VModal Bearer token or identity headers to that image GET.

Capture a search generation plus the query/collection/stream snapshot before awaiting network work. After both search and bulk resolution, check mounted and that the generation/scope is still current before updating UI. Invalidate old results when the client, query, collection, stream, or upload lifecycle changes.

Render only safe SdkException.toString() output. It contains classification and status metadata, never response bodies, URLs, keys, or identities. Cancel an upload task when the user explicitly cancels it; do not close the shared application client merely because a page was disposed. If application logic inspects a structured error body, server filesystem paths are replaced with **** before the exception reaches the application. Do not render the remaining raw server detail as user-facing copy.

For independent low-level searches, use client.searches.searchBatch(...). batchSize controls the contiguous mini-batch owned by a worker, while nWorker is the maximum number of search requests in flight. Results retain the input order even when network responses complete in another order:

final cancellation = CancellationToken();
final results = await client.searches.searchBatch(
  requests,
  batchSize: 8,
  nWorker: 3,
  cancellation: cancellation,
);

Keep the token with the screen, controller, or session that owns the batch. Cancel and await active batches before clearing or replacing authentication credentials, logging out, or closing VmodalClient. The batch helper never cancels a caller-owned token after an item failure and does not retry searches.