Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

refactor: package media lambda as a verified zip

kashorinbade2203ea11
diff --git a/.gitignore b/.gitignore
index a45252b3..d2ca30d7 100644
--- a/.gitignore
+++ b/.gitignore
@@ -13,3 +13,4 @@ services/whisperx/**/__pycache__/
services/whisperx/src/*.egg-info/
services/image-opencv/.venv/
services/image-opencv/**/__pycache__/
+services/media-lambda/build/
diff --git a/services/media-lambda/Dockerfile b/services/media-lambda/Dockerfile
deleted file mode 100644
index c75f2426..00000000
--- a/services/media-lambda/Dockerfile
+++ /dev/null
@@ -1,16 +0,0 @@
-# FFmpeg is pinned by digest-bearing tag and then asserted, because a mutable
-# upstream tag must never quietly change what a Build computes.
-FROM mwader/static-ffmpeg:8.0.1 AS ffmpeg
-
-FROM public.ecr.aws/lambda/nodejs:20
-COPY --from=ffmpeg /ffmpeg /opt/bin/ffmpeg
-COPY --from=ffmpeg /ffprobe /opt/bin/ffprobe
-RUN /opt/bin/ffmpeg -version | head -1 | grep -q 'ffmpeg version 8.0.1' \
- && /opt/bin/ffprobe -version | head -1 | grep -q 'ffprobe version 8.0.1'
-
-ENV FFMPEG_PATH=/opt/bin/ffmpeg \
- FFPROBE_PATH=/opt/bin/ffprobe \
- NODE_ENV=production
-
-COPY index.mjs ${LAMBDA_TASK_ROOT}/index.mjs
-CMD ["index.handler"]
diff --git a/services/media-lambda/README.md b/services/media-lambda/README.md
index 3a7ee19b..58519078 100644
--- a/services/media-lambda/README.md
+++ b/services/media-lambda/README.md
@@ -1,101 +1,89 @@
# Narratage media function
-The five media Needs — inspect, normalize, speech-evidence projection, timeline
-audio and mux — executed in AWS Lambda instead of on the machine running the
-Build. This is deployment state, not an author-importable SVML package.
+The five media Needs — inspect, normalize, speech-evidence projection, timeline audio and mux —
+executed in AWS Lambda instead of on the machine running the Build. This is deployment state, not
+an author-importable SVML package.
-It owns no media logic. `@narratage/media-execution` holds the ffmpeg argv and
-the frame arithmetic, and the same code answers a local Build through
-`@narratage/provider-media-local`. Only two things are particular to this
-deployment: where the bytes live, and which ffmpeg binary the image carries.
+It owns no media logic. `@narratage/media-execution` holds the FFmpeg arguments and frame/sample
+arithmetic, and the same code answers a local Build through `@narratage/provider-media-local`.
+This service contributes only an S3 byte gateway and one managed-runtime Lambda entry point.
-## What it requires, and why
+## Two immutable deployment objects
-**An S3 ArtifactStore, shared with the function.** A synchronous Lambda
-invocation carries a few megabytes; a programme does not fit. So media never
-travels in the payload — the request names the bucket the Artifacts already
-occupy, and the function reads and writes there using the same
-content-addressed key rule as `@narratage/artifact-store-s3`. A Provider
-configured against a different bucket fails on the first result, naming the
-Artifact it cannot read.
+The deployment is intentionally split:
-**An execution role** that can read and write the Artifact prefix in that
-bucket, and write CloudWatch Logs. Nothing else.
+1. `build.mjs` produces a small Node.js 22 function ZIP containing the handler and all JavaScript
+ dependencies;
+2. an independently published Lambda Layer supplies `/opt/bin/ffmpeg` and `/opt/bin/ffprobe`.
-## First-time AWS setup
+`publish.sh` requires an exact Layer **version** ARN. It never selects a public Layer, never creates
+an ECR repository and never invokes Docker. On the first invocation in each warm environment the
+handler runs both binaries with `-version` and refuses the fulfillment unless both report FFmpeg
+8.0.1.
-One IAM user and one bucket. Everything else — the ECR repository, the function,
-and (for the renderer) its whole stack — is created by the scripts that need it.
+The repository does not yet publish a redistributable FFmpeg Layer. That is deliberate: an exact
+binary build, its complete license surface, upstream archive digest and regional Layer versions are
+one supply-chain artifact of their own. Until that artifact is reviewed, the function packager is
+complete but a fresh account must provide its own compatible Layer.
-The policies under `iam/` are templates: replace `ACCOUNT_ID`, `REGION` and
-`BUCKET`. They are scoped to named resources rather than `*`, so a mistake
-cannot reach the rest of the account.
+AWS mounts Layer `bin/` at `/opt/bin`. The reviewed Layer must therefore be:
-```bash
-export AWS_REGION=us-east-1
-export AWS_ACCOUNT_ID=123456789012
-export NARRATAGE_MEDIA_BUCKET=your-artifact-bucket
+- Linux x86-64 and compatible with Amazon Linux 2023;
+- FFmpeg and FFprobe 8.0.1;
+- packaged with executable paths `bin/ffmpeg` and `bin/ffprobe`;
+- small enough that the uncompressed function plus all Layers remain within Lambda's quota;
+- published in the same region as the function, with an immutable version ARN; it may be a
+ separately permitted cross-account Layer.
-# 1. The bucket the Build and the function share.
-aws s3api create-bucket --bucket "$NARRATAGE_MEDIA_BUCKET" --region "$AWS_REGION"
-aws s3api put-public-access-block --bucket "$NARRATAGE_MEDIA_BUCKET" \
- --public-access-block-configuration \
- BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true
-
-# 2. The role the function runs as. It can read and write Artifacts, and say
-# what happened. Nothing else.
-sed -e "s/ACCOUNT_ID/$AWS_ACCOUNT_ID/g" -e "s/REGION/$AWS_REGION/g" \
- -e "s/BUCKET/$NARRATAGE_MEDIA_BUCKET/g" iam/execution.json > /tmp/execution.json
-aws iam create-role --role-name NarratageMediaExecution \
- --assume-role-policy-document file://iam/execution-trust.json
-aws iam put-role-policy --role-name NarratageMediaExecution \
- --policy-name NarratageMediaExecution --policy-document file:///tmp/execution.json
-
-# 3. The user that publishes. Attach iam/publisher.json with the same
-# substitutions, then configure its keys as a named profile — never in this
-# repository, and never assembled by hand in code.
-aws configure --profile narratage
-```
+## Data and authority
-Credentials reach the SDK through its default chain. That is deliberate: a
-managed runtime injects `AWS_SESSION_TOKEN`, and code that rebuilds a
-credential object from an id and a secret drops it, so every request fails
-`InvalidAccessKeyId`.
+The function and the Build share one S3 ArtifactStore. Media bytes never travel in the synchronous
+Lambda payload: requests name content-addressed objects, and results are written back under the same
+key rule as `@narratage/artifact-store-s3`.
-## Deploying it
+The execution role needs only read/write access to that Artifact prefix and CloudWatch Logs. The
+publisher needs permission to update the named ZIP function, pass that role and read the selected
+Layer version. Templates under `iam/` use `ACCOUNT_ID`, `REGION` and `BUCKET` placeholders.
-```bash
-export AWS_REGION=us-east-1
-export AWS_ACCOUNT_ID=123456789012
-export NARRATAGE_MEDIA_BUCKET=your-artifact-bucket
-./publish.sh
-```
+Credentials always reach AWS clients through the SDK default credential chain. Keys do not enter
+source, Runtime Profiles, BuildState or hand-assembled credential objects.
+
+## First-time AWS setup
-`publish.sh` bundles the handler, asserts the FFmpeg version inside the image,
-pushes to ECR under an immutable `bundle-<sha256>` tag, creates or updates the
-function, publishes a **version**, and prints the Runtime Profile entry to
-paste. Overridable: `NARRATAGE_MEDIA_REPOSITORY`, `_FUNCTION`, `_ROLE`,
-`_MEMORY` (3008), `_TIMEOUT` (900), `_EPHEMERAL` (8192).
+Create the shared Artifact bucket with public access blocked. Apply the substituted
+`iam/execution.json` to the `NarratageMediaExecution` role, using `iam/execution-trust.json` as its
+trust policy, and apply the substituted `iam/publisher.json` to the identity that publishes this
+function. The same bucket can be the S3 ArtifactStore used by the Runtime.
-The printed ARN ends in `:<version>`. The Provider refuses an unqualified ARN:
-a bare function name is whatever was deployed last, and two Builds of one Run
-Source could then execute different code while recording the same Endpoint
-identity.
+The default function name is `narratage-media-zip`. AWS cannot convert an existing image-packaged
+function to ZIP, so an old `narratage-media` image function can remain until its versioned ARN is no
+longer referenced.
-## Checking it before you depend on it
+## Build and publish
+
+Building is local, network-free and creates no cloud resource:
```bash
-pnpm narratage doctor svml.runtime.json
+pnpm build:media-lambda
```
-## Tests
+It writes ignored files under `services/media-lambda/build/`: `index.mjs`, `function.zip`, the
+reviewed deployment description and its bundle hash.
-The handler is tested against real ffmpeg over an in-memory bucket, so the
-whole chain except AWS itself runs in `pnpm test`. What that does **not** cover
-is the deployed image, the role's permissions and the account's limits; those
-are only proven by publishing once and running a Build.
+Publishing mutates AWS and therefore requires an already reviewed Layer version:
```bash
-pnpm test # includes services/media-lambda/test
-pnpm build:media-lambda # bundle only, no AWS
+export AWS_REGION=us-east-1
+export AWS_ACCOUNT_ID=123456789012
+export NARRATAGE_MEDIA_BUCKET=your-artifact-bucket
+export NARRATAGE_FFMPEG_LAYER_ARN=arn:aws:lambda:us-east-1:123456789012:layer:narratage-ffmpeg-8-0-1:1
+./publish.sh
```
+
+The script verifies region affinity, resolves the Layer before mutation, creates or updates
+the ZIP function, verifies the complete effective configuration, publishes an immutable function
+version and prints the Runtime Profile Endpoint entry. The Provider refuses an unqualified
+function ARN.
+
+No real AWS deployment is part of the normal test suite. The handler itself is exercised with real
+local FFmpeg over an in-memory bucket, including rejection of a Layer version mismatch.
diff --git a/services/media-lambda/build.mjs b/services/media-lambda/build.mjs
index 4a0245a2..a34e24a6 100755
--- a/services/media-lambda/build.mjs
+++ b/services/media-lambda/build.mjs
@@ -1,24 +1,39 @@
#!/usr/bin/env node
-// Bundles the handler to one ESM file, so the image carries no node_modules
-// and its content hash is a function of the source alone.
-import { mkdir, writeFile } from "node:fs/promises";
+// Bundles the handler into a small managed-runtime ZIP. FFmpeg is deliberately
+// absent: it is an independently versioned Lambda Layer mounted at /opt.
+import { execFile } from "node:child_process";
import { createHash } from "node:crypto";
+import { chmod, mkdir, readFile, rm, stat, utimes, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
+import { promisify } from "node:util";
import { build } from "esbuild";
const root = dirname(fileURLToPath(import.meta.url));
const out = join(root, "build");
+await rm(out, { recursive: true, force: true });
await mkdir(out, { recursive: true });
+const deployment = {
+ contract: "narratage.media-lambda-deployment@1",
+ runtime: "nodejs22.x",
+ architecture: "x86_64",
+ handler: "index.handler",
+ ffmpegVersion: "8.0.1",
+ ffmpegPath: "/opt/bin/ffmpeg",
+ ffprobePath: "/opt/bin/ffprobe",
+};
+const entry = join(out, "index.mjs");
+const archive = join(out, "function.zip");
+
const result = await build({
entryPoints: [join(root, "src/handler.ts")],
- outfile: join(out, "index.mjs"),
+ outfile: entry,
bundle: true,
platform: "node",
- target: "node20",
+ target: "node22",
format: "esm",
- // The AWS SDK is bundled rather than taken from the runtime, so the image
+ // The AWS SDK is bundled rather than taken from the runtime, so the ZIP
// pins the same client version the tests ran against.
banner: { js: "import{createRequire}from'node:module';const require=createRequire(import.meta.url);" },
metafile: true,
@@ -26,10 +41,26 @@ const result = await build({
});
if (result.errors.length > 0) process.exit(1);
-const { readFile } = await import("node:fs/promises");
-const bundle = await readFile(join(out, "index.mjs"));
-const dockerfile = await readFile(join(root, "Dockerfile"));
-const hash = createHash("sha256").update(bundle).update(dockerfile).digest("hex");
-await writeFile(join(out, "Dockerfile"), dockerfile);
+const bundle = await readFile(entry);
+const deploymentBytes = Buffer.from(`${JSON.stringify(deployment)}\n`, "utf8");
+const hash = createHash("sha256").update(bundle).update(deploymentBytes).digest("hex");
+await writeFile(join(out, "deployment.json"), deploymentBytes);
await writeFile(join(out, "bundle-hash.txt"), `${hash}\n`);
-process.stdout.write(`${JSON.stringify({ bundleHash: hash, bytes: bundle.byteLength }, null, 2)}\n`);
+
+// Strip host-specific metadata and hold the only archive member at a stable
+// timestamp. The Build identity is the source/deployment hash above; this also
+// keeps repeated ZIPs byte-stable on the reference macOS/Linux toolchain.
+await chmod(entry, 0o644);
+const epoch = new Date("1980-01-01T00:00:00.000Z");
+await utimes(entry, epoch, epoch);
+await promisify(execFile)("zip", ["-X", "-q", "-j", archive, entry], {
+ env: { ...process.env, TZ: "UTC" },
+});
+const archiveStat = await stat(archive);
+
+process.stdout.write(`${JSON.stringify({
+ bundleHash: hash,
+ bundleBytes: bundle.byteLength,
+ archiveBytes: archiveStat.size,
+ deployment,
+}, null, 2)}\n`);
diff --git a/services/media-lambda/build/Dockerfile b/services/media-lambda/build/Dockerfile
deleted file mode 100644
index c75f2426..00000000
--- a/services/media-lambda/build/Dockerfile
+++ /dev/null
@@ -1,16 +0,0 @@
-# FFmpeg is pinned by digest-bearing tag and then asserted, because a mutable
-# upstream tag must never quietly change what a Build computes.
-FROM mwader/static-ffmpeg:8.0.1 AS ffmpeg
-
-FROM public.ecr.aws/lambda/nodejs:20
-COPY --from=ffmpeg /ffmpeg /opt/bin/ffmpeg
-COPY --from=ffmpeg /ffprobe /opt/bin/ffprobe
-RUN /opt/bin/ffmpeg -version | head -1 | grep -q 'ffmpeg version 8.0.1' \
- && /opt/bin/ffprobe -version | head -1 | grep -q 'ffprobe version 8.0.1'
-
-ENV FFMPEG_PATH=/opt/bin/ffmpeg \
- FFPROBE_PATH=/opt/bin/ffprobe \
- NODE_ENV=production
-
-COPY index.mjs ${LAMBDA_TASK_ROOT}/index.mjs
-CMD ["index.handler"]
diff --git a/services/media-lambda/iam/publisher.json b/services/media-lambda/iam/publisher.json
index bbe2b732..c9c5c32e 100644
--- a/services/media-lambda/iam/publisher.json
+++ b/services/media-lambda/iam/publisher.json
@@ -2,25 +2,10 @@
"Version": "2012-10-17",
"Statement": [
{
- "Sid": "PushTheImage",
+ "Sid": "ReadThePinnedFfmpegLayer",
"Effect": "Allow",
- "Action": [
- "ecr:BatchCheckLayerAvailability",
- "ecr:CompleteLayerUpload",
- "ecr:CreateRepository",
- "ecr:DescribeImages",
- "ecr:DescribeRepositories",
- "ecr:InitiateLayerUpload",
- "ecr:PutImage",
- "ecr:UploadLayerPart"
- ],
- "Resource": "arn:aws:ecr:REGION:ACCOUNT_ID:repository/narratage-media"
- },
- {
- "Sid": "EcrLoginIsAccountWide",
- "Effect": "Allow",
- "Action": ["ecr:GetAuthorizationToken"],
- "Resource": "*"
+ "Action": ["lambda:GetLayerVersion"],
+ "Resource": "arn:aws:lambda:REGION:*:layer:*:*"
},
{
"Sid": "PublishTheFunction",
diff --git a/services/media-lambda/publish.sh b/services/media-lambda/publish.sh
index 242453ac..d84ccf43 100755
--- a/services/media-lambda/publish.sh
+++ b/services/media-lambda/publish.sh
@@ -1,72 +1,97 @@
#!/usr/bin/env bash
-# Build, push and publish one immutable version of the media function.
+# Publish one immutable managed-runtime ZIP of the media function.
#
-# Every value below is read from the environment with no default that could
-# reach someone else's account. Run it once per deployment; the ARN it prints
-# is what a Runtime Profile names.
+# FFmpeg is a separate, immutable Layer. This script never builds an image,
+# creates an ECR repository or guesses which Layer an account should trust.
set -euo pipefail
: "${AWS_REGION:?set AWS_REGION, e.g. us-east-1}"
: "${AWS_ACCOUNT_ID:?set AWS_ACCOUNT_ID, your 12-digit account id}"
: "${NARRATAGE_MEDIA_BUCKET:?set NARRATAGE_MEDIA_BUCKET, the ArtifactStore bucket the function shares}"
-REPOSITORY="${NARRATAGE_MEDIA_REPOSITORY:-narratage-media}"
-FUNCTION="${NARRATAGE_MEDIA_FUNCTION:-narratage-media}"
+: "${NARRATAGE_FFMPEG_LAYER_ARN:?set NARRATAGE_FFMPEG_LAYER_ARN to an immutable layer version ARN}"
+
+FUNCTION="${NARRATAGE_MEDIA_FUNCTION:-narratage-media-zip}"
ROLE="${NARRATAGE_MEDIA_ROLE:-NarratageMediaExecution}"
MEMORY="${NARRATAGE_MEDIA_MEMORY:-3008}"
TIMEOUT="${NARRATAGE_MEDIA_TIMEOUT:-900}"
EPHEMERAL="${NARRATAGE_MEDIA_EPHEMERAL:-8192}"
+FFMPEG_VERSION="8.0.1"
here="$(cd "$(dirname "$0")" && pwd)"
-registry="${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com"
+layer="$NARRATAGE_FFMPEG_LAYER_ARN"
+if [[ ! "$layer" =~ ^arn:aws[a-zA-Z-]*:lambda:([a-z0-9-]+):([0-9]{12}):layer:[A-Za-z0-9-_]+:([0-9]+)$ ]]; then
+ printf 'NARRATAGE_FFMPEG_LAYER_ARN must name an immutable Layer version; got %s\n' "$layer" >&2
+ exit 2
+fi
+if [ "${BASH_REMATCH[1]}" != "$AWS_REGION" ]; then
+ printf 'FFmpeg Layer belongs to region %s, not configured region %s\n' \
+ "${BASH_REMATCH[1]}" "$AWS_REGION" >&2
+ exit 2
+fi
-echo "==> bundling"
+printf 'Bundling managed-runtime function ZIP\n'
(cd "$here" && node build.mjs >/dev/null)
-hash="$(cat "$here/build/bundle-hash.txt")"
-tag="bundle-${hash}"
-echo " bundle ${hash}"
-
-echo "==> ensuring the ECR repository exists with immutable tags"
-aws ecr describe-repositories --region "$AWS_REGION" --repository-names "$REPOSITORY" >/dev/null 2>&1 \
- || aws ecr create-repository --region "$AWS_REGION" --repository-name "$REPOSITORY" \
- --image-tag-mutability IMMUTABLE --image-scanning-configuration scanOnPush=true >/dev/null
+hash="$(tr -d '\n' < "$here/build/bundle-hash.txt")"
+runtime="$(node -p "require('$here/build/deployment.json').runtime")"
+architecture="$(node -p "require('$here/build/deployment.json').architecture")"
+handler="$(node -p "require('$here/build/deployment.json').handler")"
+archive="$here/build/function.zip"
-echo "==> building and pushing ${registry}/${REPOSITORY}:${tag}"
-aws ecr get-login-password --region "$AWS_REGION" \
- | docker login --username AWS --password-stdin "$registry" >/dev/null
-# Lambda runs x86_64 here; the image must match whatever --architectures says.
-docker buildx build --platform linux/amd64 --provenance=false \
- -t "${registry}/${REPOSITORY}:${tag}" "$here/build" --push
+# Resolve the exact Layer before mutating the function. The ARN is immutable;
+# runtime validation then proves that /opt/bin really contains the promised
+# pair rather than trusting its name.
+aws lambda get-layer-version-by-arn --region "$AWS_REGION" --arn "$layer" >/dev/null
-digest="$(aws ecr describe-images --region "$AWS_REGION" --repository-name "$REPOSITORY" \
- --image-ids imageTag="$tag" --query 'imageDetails[0].imageDigest' --output text)"
-image="${registry}/${REPOSITORY}@${digest}"
-echo " image ${digest}"
+node -e 'const fs=require("node:fs");fs.writeFileSync(process.argv[1],JSON.stringify({Variables:{NARRATAGE_MEDIA_BUNDLE_HASH:process.argv[2],NARRATAGE_FFMPEG_VERSION:process.argv[3],FFMPEG_PATH:"/opt/bin/ffmpeg",FFPROBE_PATH:"/opt/bin/ffprobe"}}))' \
+ "$here/build/environment.json" "$hash" "$FFMPEG_VERSION"
-echo "==> creating or updating the function"
+printf 'Publishing bundle %s with Layer %s\n' "$hash" "$layer"
if aws lambda get-function --region "$AWS_REGION" --function-name "$FUNCTION" >/dev/null 2>&1; then
+ package_type="$(aws lambda get-function-configuration --region "$AWS_REGION" \
+ --function-name "$FUNCTION" --query PackageType --output text)"
+ if [ "$package_type" != Zip ]; then
+ printf '%s is a %s function; AWS cannot change package type. Set NARRATAGE_MEDIA_FUNCTION to a new name.\n' \
+ "$FUNCTION" "$package_type" >&2
+ exit 2
+ fi
aws lambda update-function-code --region "$AWS_REGION" --function-name "$FUNCTION" \
- --image-uri "$image" >/dev/null
- aws lambda wait function-updated --region "$AWS_REGION" --function-name "$FUNCTION"
+ --zip-file "fileb://$archive" --architectures "$architecture" >/dev/null
+ aws lambda wait function-updated-v2 --region "$AWS_REGION" --function-name "$FUNCTION"
aws lambda update-function-configuration --region "$AWS_REGION" --function-name "$FUNCTION" \
+ --role "arn:aws:iam::${AWS_ACCOUNT_ID}:role/${ROLE}" \
+ --runtime "$runtime" --handler "$handler" --layers "$layer" \
--memory-size "$MEMORY" --timeout "$TIMEOUT" \
--ephemeral-storage "Size=${EPHEMERAL}" \
- --environment "Variables={NARRATAGE_MEDIA_BUNDLE_HASH=${hash}}" >/dev/null
- aws lambda wait function-updated --region "$AWS_REGION" --function-name "$FUNCTION"
+ --environment "file://$here/build/environment.json" >/dev/null
+ aws lambda wait function-updated-v2 --region "$AWS_REGION" --function-name "$FUNCTION"
else
aws lambda create-function --region "$AWS_REGION" --function-name "$FUNCTION" \
- --package-type Image --code "ImageUri=${image}" \
+ --runtime "$runtime" --handler "$handler" \
+ --zip-file "fileb://$archive" \
--role "arn:aws:iam::${AWS_ACCOUNT_ID}:role/${ROLE}" \
- --architectures x86_64 --memory-size "$MEMORY" --timeout "$TIMEOUT" \
+ --architectures "$architecture" --layers "$layer" \
+ --memory-size "$MEMORY" --timeout "$TIMEOUT" \
--ephemeral-storage "Size=${EPHEMERAL}" \
- --environment "Variables={NARRATAGE_MEDIA_BUNDLE_HASH=${hash}}" >/dev/null
- aws lambda wait function-active --region "$AWS_REGION" --function-name "$FUNCTION"
+ --environment "file://$here/build/environment.json" >/dev/null
+ aws lambda wait function-active-v2 --region "$AWS_REGION" --function-name "$FUNCTION"
+fi
+
+read -r actual_runtime actual_handler actual_architecture actual_layer layer_count actual_hash actual_ffmpeg package_type < <(
+ aws lambda get-function-configuration --region "$AWS_REGION" --function-name "$FUNCTION" \
+ --query '[Runtime,Handler,Architectures[0],Layers[0].Arn,length(Layers),Environment.Variables.NARRATAGE_MEDIA_BUNDLE_HASH,Environment.Variables.NARRATAGE_FFMPEG_VERSION,PackageType]' \
+ --output text
+)
+if [ "$actual_runtime" != "$runtime" ] || [ "$actual_handler" != "$handler" ] \
+ || [ "$actual_architecture" != "$architecture" ] || [ "$actual_layer" != "$layer" ] \
+ || [ "$layer_count" != 1 ] || [ "$actual_hash" != "$hash" ] \
+ || [ "$actual_ffmpeg" != "$FFMPEG_VERSION" ] || [ "$package_type" != Zip ]; then
+ printf 'published function configuration does not match the reviewed deployment\n' >&2
+ exit 1
fi
-# A version, not $LATEST. The Provider refuses an unqualified ARN, because a
-# bare name is whatever was deployed last and two Builds of one Run Source
-# could then execute different code under one Endpoint identity.
+# A version, not $LATEST. The Provider refuses an unqualified ARN.
version="$(aws lambda publish-version --region "$AWS_REGION" --function-name "$FUNCTION" \
- --description "bundle ${hash}" --query 'Version' --output text)"
+ --description "bundle ${hash}; ffmpeg layer ${layer}" --query Version --output text)"
arn="arn:aws:lambda:${AWS_REGION}:${AWS_ACCOUNT_ID}:function:${FUNCTION}:${version}"
cat <<JSON
diff --git a/services/media-lambda/src/handler.ts b/services/media-lambda/src/handler.ts
index 90e098f3..3d4b20f6 100644
--- a/services/media-lambda/src/handler.ts
+++ b/services/media-lambda/src/handler.ts
@@ -9,7 +9,9 @@ import {
executeRenderTimelineAudio,
} from "@narratage/media-execution";
import type { MediaExecutionEnvironment, MediaOperationResult } from "@narratage/media-execution";
+import { execFile } from "node:child_process";
import { createHash } from "node:crypto";
+import { promisify } from "node:util";
import type { BlobRef, CanonicalValue, Digest } from "@narratage/protocol";
import {
MEDIA_LAMBDA_RESPONSE,
@@ -23,7 +25,7 @@ import type { MediaLambdaArtifactLocation, MediaLambdaOperation } from "@narrata
* It owns no media logic: `@narratage/media-execution` holds the ffmpeg argv
* and the frame arithmetic, and the same code answers a local Build. What is
* particular to this deployment is only where the bytes live and which ffmpeg
- * binary the image carries.
+ * binary its immutable Layer carries.
*/
const OPERATIONS: Record<
MediaLambdaOperation,
@@ -38,6 +40,7 @@ const OPERATIONS: Record<
const FFMPEG_PATH = process.env.FFMPEG_PATH ?? "/opt/bin/ffmpeg";
const FFPROBE_PATH = process.env.FFPROBE_PATH ?? "/opt/bin/ffprobe";
+const run = promisify(execFile);
function positiveInteger(value: string | undefined, fallback: number): number {
if (value === undefined) return fallback;
@@ -96,42 +99,89 @@ export type MediaLambdaHandlerOptions = {
readonly client?: S3ObjectClient;
readonly ffmpegPath?: string;
readonly ffprobePath?: string;
+ /**
+ * The deployed ZIP sets this to the version promised by its immutable Layer.
+ * Tests and embedded handlers may omit it when their binary identity is
+ * already controlled by the caller.
+ */
+ readonly expectedFfmpegVersion?: string;
};
+function versionMatches(line: string, tool: "ffmpeg" | "ffprobe", expected: string): boolean {
+ const escaped = expected.replace(/[.*+?^${}()|[\]\\]/gu, "\\$&");
+ return new RegExp(`^${tool} version (?:n)?${escaped}(?:[-\\s]|$)`, "u").test(line);
+}
+
+async function assertBinaryVersion(path: string, tool: "ffmpeg" | "ffprobe", expected: string): Promise<void> {
+ const { stdout } = await run(path, ["-version"], {
+ encoding: "utf8",
+ maxBuffer: 64 * 1024,
+ timeout: 10_000,
+ });
+ const firstLine = stdout.split(/\r?\n/u, 1)[0] ?? "";
+ if (!versionMatches(firstLine, tool, expected)) {
+ throw new Error(`${tool} Layer mismatch: expected ${expected}, got ${firstLine || "no version"}`);
+ }
+}
+
+async function assertBinaryPair(
+ ffmpegPath: string,
+ ffprobePath: string,
+ expected: string | undefined,
+): Promise<void> {
+ if (expected === undefined) return;
+ await Promise.all([
+ assertBinaryVersion(ffmpegPath, "ffmpeg", expected),
+ assertBinaryVersion(ffprobePath, "ffprobe", expected),
+ ]);
+}
+
/**
* The handler, with its S3 client and binaries open to substitution. The
- * deployed entry point below closes them over the image's own defaults; a test
+ * deployed entry point below closes them over the Layer's fixed paths; a test
* can supply an in-memory bucket and run the real ffmpeg.
*/
export function createMediaLambdaHandler(options: MediaLambdaHandlerOptions = {}) {
+ const ffmpegPath = options.ffmpegPath ?? FFMPEG_PATH;
+ const ffprobePath = options.ffprobePath ?? FFPROBE_PATH;
+ // One promise per warm execution environment. A missing or wrong Layer is a
+ // typed fulfillment failure, not a deployment fact silently discovered after
+ // several expensive transformations have already run.
+ let binariesReady: Promise<void> | undefined;
return async function handle(event: unknown): Promise<unknown> {
- let operation: MediaLambdaOperation = "inspect";
- try {
- const request = parseMediaLambdaRequest(event);
- operation = request.operation;
- const env: MediaExecutionEnvironment = {
- artifacts: gateway(request.artifacts, options.client ?? new AwsS3ObjectClient({})),
- ffmpegPath: options.ffmpegPath ?? FFMPEG_PATH,
- ffprobePath: options.ffprobePath ?? FFPROBE_PATH,
- processTimeoutMs: positiveInteger(process.env.SVML_MEDIA_TIMEOUT_MS, 14 * 60_000),
- maxProbeOutputBytes: positiveInteger(process.env.SVML_MEDIA_MAX_PROBE_BYTES, 256 * 1024 * 1024),
- label: "media.aws-lambda",
- };
- const result = await OPERATIONS[operation](env, request.constraints);
- return {
- contract: MEDIA_LAMBDA_RESPONSE,
- operation,
- ok: true as const,
- value: result.value,
- metadata: result.metadata,
- };
- } catch (error) {
- // Never thrown: the transport will not copy a failed function's payload
- // into its error, so a throw would reach the Build as "Unhandled".
- return failure(operation, error);
- }
+ let operation: MediaLambdaOperation = "inspect";
+ try {
+ const request = parseMediaLambdaRequest(event);
+ operation = request.operation;
+ binariesReady ??= assertBinaryPair(
+ ffmpegPath,
+ ffprobePath,
+ options.expectedFfmpegVersion ?? process.env.NARRATAGE_FFMPEG_VERSION,
+ );
+ await binariesReady;
+ const env: MediaExecutionEnvironment = {
+ artifacts: gateway(request.artifacts, options.client ?? new AwsS3ObjectClient({})),
+ ffmpegPath,
+ ffprobePath,
+ processTimeoutMs: positiveInteger(process.env.SVML_MEDIA_TIMEOUT_MS, 14 * 60_000),
+ maxProbeOutputBytes: positiveInteger(process.env.SVML_MEDIA_MAX_PROBE_BYTES, 256 * 1024 * 1024),
+ label: "media.aws-lambda",
+ };
+ const result = await OPERATIONS[operation](env, request.constraints);
+ return {
+ contract: MEDIA_LAMBDA_RESPONSE,
+ operation,
+ ok: true as const,
+ value: result.value,
+ metadata: result.metadata,
+ };
+ } catch (error) {
+ // Never thrown: the transport will not copy a failed function's payload
+ // into its error, so a throw would reach the Build as "Unhandled".
+ return failure(operation, error);
+ }
};
}
-/** The image's entry point. */
+/** The managed-runtime ZIP's entry point. */
export const handler = createMediaLambdaHandler();
diff --git a/services/media-lambda/test/handler.test.ts b/services/media-lambda/test/handler.test.ts
index ff42d6b7..17de5223 100644
--- a/services/media-lambda/test/handler.test.ts
+++ b/services/media-lambda/test/handler.test.ts
@@ -102,3 +102,30 @@ test("an envelope from another contract is refused before any media is touched",
assert.equal(reply.ok, false);
assert.match(String(reply.message), /contract must be svml\.media-lambda-request@1/u);
});
+
+test("a Layer that lies about its FFmpeg version fails before reading an Artifact",
+ { skip: (await ffmpegAvailable()) ? false : "ffmpeg and ffprobe are not on PATH" },
+ async () => {
+ const handle = createMediaLambdaHandler({
+ client: new Bucket(),
+ ffmpegPath: "ffmpeg",
+ ffprobePath: "ffprobe",
+ expectedFfmpegVersion: "0.0.0-impossible",
+ });
+ const reply = await handle({
+ contract: "svml.media-lambda-request@1",
+ operation: "inspect",
+ artifacts: { bucket: "fixture" },
+ constraints: {
+ contract: "svml.inspect-media-request@1",
+ source: {
+ kind: "blob",
+ digest: `sha256:${"0".repeat(64)}`,
+ size: 1,
+ mediaType: "audio/wav",
+ },
+ },
+ }) as Record<string, unknown>;
+ assert.equal(reply.ok, false);
+ assert.match(String(reply.message), /Layer mismatch: expected 0\.0\.0-impossible/u);
+ });