Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

1.8 KB
Raw Blame
# Use with compose.yaml, and with compose.http.yaml when serving over HTTP. The
# `laya-serve` entry repeats the CUDA build args and device reservation because it is
# a separate service: overrides for `laya` never reach it, so it would serve on CPU.
services:
  laya:
    build:
      args:
        # cu130, not cu128: compose.yaml pins TORCH_VERSION 2.14.0 and the cu128 index has no
        # 2.14.0 wheel (its newest is 2.11.0), so the documented CUDA path could not resolve its
        # own pin (#971). cu126 also carries 2.14.0; cu130 is chosen because docker.yml already
        # builds the CUDA images with it, so Compose and CI now agree.
        TORCH_INDEX: "${LAYA_TORCH_INDEX:-cu130}"
    environment:
      LAYA_DEVICE: "${LAYA_DEVICE:-cuda}"
      # The runtime reads these when it picks the autocast dtype; empty keeps the checkpoint's
      # own `amp_dtype`. README's threshold section measures what fp16 vs bf16 decides differently.
      LAYA_CUDA_AMP: "${LAYA_CUDA_AMP:-}"
      LAYA_CPU_AMP: "${LAYA_CPU_AMP:-}"
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              device_ids: ["${LAYA_GPU_ID:-0}"]
              capabilities: [gpu]

  laya-serve:
    build:
      context: .
      args:
        TORCH_INDEX: "${LAYA_TORCH_INDEX:-cu130}"
    environment:
      LAYA_DEVICE: "${LAYA_DEVICE:-cuda}"
      # The runtime reads these when it picks the autocast dtype; empty keeps the checkpoint's
      # own `amp_dtype`. README's threshold section measures what fp16 vs bf16 decides differently.
      LAYA_CUDA_AMP: "${LAYA_CUDA_AMP:-}"
      LAYA_CPU_AMP: "${LAYA_CPU_AMP:-}"
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              device_ids: ["${LAYA_GPU_ID:-0}"]
              capabilities: [gpu]