Skip to content

Free, open-source AI video generator for Mac. Run LTX-2.5 and MiniMax H3 (Hailuo) locally on Apple Silicon: text-to-video and image-to-video with sound, lip-sync, character LoRA training, songs from lyrics (YuE2) and image editing (Qwen-Image-Edit). One-click install via Pinokio. Offline: no cloud, no API key, no NVIDIA GPU.

16.2 KB
Raw Blame
#!/usr/bin/env node
/*
 * Gate for the Pinokio launcher scripts: no single DISPATCH may be long enough
 * to wedge Pinokio's shell.
 *
 * ---------------------------------------------------------------------------
 * WHY
 * ---------------------------------------------------------------------------
 * Pinokio 8.0.x hangs — one CPU core at 100%, no child process ever spawns,
 * the install dead until a force-quit — when a single command written to its
 * pty is too long.  @Morac2 bisected the ceiling in #50: a 764-char command
 * hangs, ~373 passes.  v3.6.2 adopted "no generated shell line over 350" from
 * that, and the issue was closed.
 *
 * It came back twice, from two different people:
 *   #50  @davidaircloud  "The individual lines are now short, but the joined
 *                         message generated by install.js is still 1,417
 *                         characters."  Moving that body into a .sh and
 *                         dispatching `bash ../x.sh` fixed his install.
 *   #56  @itrejomx       same stall before `uv venv`; fixed by splitting the
 *                         payload into smaller shell.run steps.
 *
 * The rule was right and the MEASUREMENT was wrong.  What Pinokio dispatches
 * is not "a line of the source" — it is one string per shell:
 *
 *   pinokiod/kernel/shells.js  launch()
 *       message is an ARRAY and no `chain` param  -> each element is launched
 *                                                    in its OWN shell (so an
 *                                                    array is already safe;
 *                                                    element size is what
 *                                                    counts)
 *       message is an ARRAY and `chain` is set    -> Shell.build() joins them
 *                                                    into ONE dispatch
 *       message is a STRING                       -> ONE dispatch, as-is
 *
 *   pinokiod/kernel/shell.js:1911
 *       this.ptyProcess.write(`${this.exec_cmd || this.cmd}${this.EOL}`)
 *       — one unchunked write, however big it is.  The same file's
 *         Shell.emit() (shell.js:665-711) chunks anything over 1024 chars into
 *         256-byte pieces with a 10 ms gap, so the kernel already knows large
 *         pty writes are unsafe; the shell.run path just doesn't do it.
 *
 * Every long payload in this repo is written `[ ... ].join("\n")` — a STRING,
 * deliberately, so an if/else runs in one shell.  Those are exactly the
 * dispatches nobody was measuring: the venv step's source lines were all under
 * 110 chars and it dispatched 1,417 in one go.
 *
 * A local pty probe (bash --noprofile --norc, Pinokio's own shell/args, one
 * unchunked os.write) completes a 1,475-byte payload fine, so this is not a
 * kernel tty limit — it is Pinokio's terminal layer, and dispatch size is the
 * only lever we have from this side.
 *
 * ---------------------------------------------------------------------------
 * THE RULE
 * ---------------------------------------------------------------------------
 * Every dispatch <= 350 chars, measured the way @Morac2 measured it: the
 * message content, before Pinokio prepends its conda activation.  Anything
 * bigger belongs in scripts/pinokio/ and becomes one short `bash ...` line.
 * (4.20: the ceiling was 500 while six dispatches sat between 350 and 498.
 * They are all gone - the worst is update.js's 313 - so the gate now holds the
 * line the field bisection actually supports.)
 *
 * ---------------------------------------------------------------------------
 * THE SECOND RULE (4.20): A FAILING STEP MUST BE ABLE TO STOP THE INSTALL
 * ---------------------------------------------------------------------------
 * Pinokio 8.2 IGNORES a shell.run step's exit code.  The kernel's default `on`
 * handlers end a script only when the step's OUTPUT matches /error:/i or
 * /errno /i (pinokiod kernel/shells.js; "error:...triton" exempt).  So an
 * `exit 1` that prints neither word marches on: a refused 75 GB H3 download
 * started anyway, a failed engine checkout went on to a 37 GB download, and a
 * failed LTX-2.5 fetch ended in "Phosphene is ready" with no weights.
 *
 * The convention (5b18448): every real failure prints ONE line
 *       FATAL error: <plain reason + what to do>
 * and every OPTIONAL failure prints neither word (post_update.sh `defang`s
 * them).  This gate holds both ends, for install.js, install_h3.js,
 * install_music.js and update.js:
 *   1. every dispatch is a benign command, an optional `|| echo` step, an
 *      inline FATAL, a `must.sh` wrapper, or a call to a scripts/pinokio script
 *      that CAN print "FATAL error:" - or a script on OPTIONAL_SCRIPTS below;
 *   2. in every FATAL-capable script, every non-zero `exit` is preceded (same
 *      line, or the 16 lines before) by a "FATAL error:" line, or says why not
 *      with a `fatal-ok:` comment;
 *   3. an optional `|| echo` message never says "error:" / "errno ";
 *   4. in post_update.sh a FATAL-capable script is either `require`d or goes
 *      through `defang` - never bare, or an optional step would end the Update.
 * test_fatal_install_paths.py runs the failure and success paths themselves.
 *
 * Run: node scripts/check_pinokio_scripts.js      exit 0 = PASS
 */
const fs = require("fs")
const path = require("path")

// The evidence, and nothing beyond it:
//   proven to HANG   764 (@Morac2, #50) and 1,417 (@davidaircloud, #50 / #56)
//   proven to WORK   ~373 (@Morac2's own bisection)
// 350 is the target the field bisection supports (skill: phosphene-release,
// gotcha 5). Every dispatch in the repo is under it.
const MAX_DISPATCH = 350

const SCRIPTS = [
  "install.js", "update.js", "install_h3.js", "install_music.js", "install_qwen.js",
  "install_sharp.js", "download_q8.js", "download_upscaler.js",
  "start.js", "reset.js", "pinokio.js",
]

// Shell.build()'s array delimiters (kernel/shell.js:932), used only when a
// `chain` param is present — which is what makes an array ONE dispatch.
function chainDelimiter(chain) {
  if (chain === "*") return " ; "
  if (chain === "|") return " || "
  return " && "
}

// What kernel/shells.js launch() actually writes to the pty, as a list.
function dispatches(params) {
  const m = params.message
  if (!m) return []
  if (typeof m === "string") return [m]
  if (Array.isArray(m)) {
    if (params.chain) {
      return [m.filter((x) => x && !/^\s+$/.test(x))
        .map(String).join(chainDelimiter(params.chain))]
    }
    return m.filter(Boolean).map(String)   // one shell per element
  }
  return [String(m)]
}

function* walk(node, trail) {
  if (Array.isArray(node)) {
    for (let i = 0; i < node.length; i++) yield* walk(node[i], trail.concat(i))
    return
  }
  if (!node || typeof node !== "object") return
  if (node.method === "shell.run" && node.params) yield { step: node, trail }
  for (const k of Object.keys(node)) {
    if (k === "method" || k === "params") continue
    yield* walk(node[k], trail.concat(k))
  }
}

const root = path.resolve(__dirname, "..")
const failures = []
let steps = 0
let count = 0
let worst = 0
let worstWhere = ""

for (const name of SCRIPTS) {
  let mod
  try {
    mod = require(path.join(root, name))
  } catch (e) {
    failures.push(`${name}: FAILED TO LOAD — ${e.message}`)
    continue
  }
  // pinokio.js exports an ASYNC menu() that needs a live kernel + info object,
  // and its entries only reference the other scripts — it dispatches nothing
  // itself. Loading it (above) is the check that applies to it.
  const trees = []
  if (mod && mod.run) trees.push(mod.run)
  for (const tree of trees) {
    for (const { step, trail } of walk(tree, [])) {
      steps++
      const list = dispatches(step.params)
      list.forEach((cmd, k) => {
        count++
        const where = `${name} [${trail.join(".")}]` +
          (list.length > 1 ? ` element ${k}` : "")
        if (cmd.length > worst) { worst = cmd.length; worstWhere = where }
        if (cmd.length > MAX_DISPATCH) {
          failures.push(`${where}: DISPATCH ${cmd.length} > ${MAX_DISPATCH}` +
            ` — move it into scripts/pinokio/ and dispatch one short line` +
            `\n      starts: ${cmd.slice(0, 72).replace(/\n/g, "\\n")}`)
        }
      })
    }
  }
}

// ===========================================================================
// THE SECOND RULE - a failing step must be able to stop the install
// ===========================================================================
// See the header. Pinokio 8.2 stops a script only on /error:/i or /errno /i in
// a step's output; the stop words below are the kernel's own.
const STOP_WORDS = /error:|errno /i
const FATAL_LANES = ["install.js", "install_h3.js", "install_music.js", "update.js"]

// Scripts a lane may call that deliberately CANNOT stop it (best-effort steps
// that defang their own output). Anything not here must print "FATAL error:".
const OPTIONAL_SCRIPTS = {
  "mflux_pack.sh": "image-engine pack: best-effort, the panel offers a one-click reinstall",
  "a2v_stems_deps.sh": "vocal separation: best-effort, the Lip-sync form offers Install",
}

// Python scripts that print their own "FATAL error:" line on failure (so the
// dispatch needs no wrapper). The gate reads the file to make sure they do.
const FATAL_PYTHON = ["scripts/pinokio/music_fetch.py"]

// Inline dispatches with no script behind them that are allowed to carry no
// FATAL of their own, and why. Everything else needs one of the forms in the
// header.
const BENIGN_INLINE = [
  [/^\s*echo\b/, "prints a line"],
  [/^\s*df\b/, "prints free space"],
  [/\bgit clone\b[^\n]*ltx-2-mlx\b/, "install.js stops on the notify (next: null) right after this clone when ltx-2-mlx/.git is missing"],
  [/\bgit clone\b[^\n]*minimax-h3-mlx\b/, "the next step's h3_enter prints FATAL when the clone did not land"],
  [/h3_roots\.sh && h3_enter && git rev-parse\b/, "prints the engine's short SHA"],
  [/install_plan\.py defer\b[^\n]*\|\| echo\b/, "queues the background extras; never fails the install"],
]

const read = (rel) => fs.readFileSync(path.join(root, rel), "utf8")
const codeLines = (text) => text.split("\n").map((t, i) => ({ n: i + 1, t }))
  .filter((l) => l.t.trim() && !l.t.trim().startsWith("#"))

const scriptInfo = {}
function infoFor(rel) {
  if (scriptInfo[rel]) return scriptInfo[rel]
  let text = ""
  try { text = read(rel) } catch (e) { return (scriptInfo[rel] = { missing: true }) }
  const code = codeLines(text)
  const fatalIdx = code.filter((l) => /\bFATAL error:/.test(l.t)).map((l) => l.n)
  const bad = []
  // Each non-zero exit is covered by ONE "FATAL error:" line - on the same line
  // or among the 16 before it - and a FATAL line covers only the next exit, so
  // a new silent `exit 1` cannot hide behind an old banner.
  let pending = null
  for (const l of code) {
    if (/\bFATAL error:/.test(l.t)) pending = l.n
    if (!/\bexit\s+(?:[1-9]|"\$)/.test(l.t)) continue
    const ok = (pending !== null && l.n - pending <= 16) ||
      text.split("\n").slice(Math.max(0, l.n - 5), l.n).some((x) => /fatal-ok:/.test(x))
    if (ok) pending = null
    else bad.push(l.n)
  }
  return (scriptInfo[rel] = { text, fatal: fatalIdx.length > 0, bad,
                              defangs: /problem -/.test(text) })
}

let fatalChecked = 0
for (const lane of FATAL_LANES) {
  const mod = require(path.join(root, lane))
  for (const { step, trail } of walk(mod.run, [])) {
    for (const cmd of dispatches(step.params)) {
      fatalChecked++
      const where = `${lane} [${trail.join(".")}]`
      const first = cmd.slice(0, 90).replace(/\n/g, "\\n")
      // an optional `|| echo` message must never carry a stop word
      for (const m of cmd.matchAll(/\|\|\s*echo\s+(['"])([\s\S]*?)\1/g)) {
        if (STOP_WORDS.test(m[2])) failures.push(`${where}: an optional "|| echo" message says "error:"/"errno " - Pinokio would end the install on a failure the step calls harmless\n      ${m[2].slice(0, 80)}`)
      }
      if (/\bmust\.sh\b/.test(cmd)) {
        const reason = (/must\.sh\s+(['"])([\s\S]*?)\1/.exec(cmd) || [])[2] || ""
        if (!reason) failures.push(`${where}: must.sh needs a quoted reason first\n      ${first}`)
        else if (STOP_WORDS.test(reason)) failures.push(`${where}: a must.sh reason must not say "error:"/"errno " (must.sh adds the FATAL line itself)\n      ${reason.slice(0, 80)}`)
        continue
      }
      if (/\bFATAL error:/.test(cmd)) continue                        // inline FATAL
      const refs = [...cmd.matchAll(/scripts\/(?:pinokio\/)?([\w.-]+\.sh)/g)].map((m) => m[0])
      const rels = [...new Set(refs.map((r) => "scripts/" + r.replace(/^scripts\//, "")))]
      if (/h3_roots\.sh/.test(cmd)) rels.push("scripts/pinokio/h3_roots.sh")
      const calls = rels.filter((r) => !/h3_roots\.sh$/.test(r))
      let ok = false
      // a python script that owns its own FATAL line (checked, not trusted)
      for (const py of FATAL_PYTHON) {
        if (cmd.includes(py)) {
          if (!/FATAL error:/.test(read(py))) failures.push(`${where}: ${py} is listed as printing "FATAL error:" but does not`)
          ok = true
        }
      }
      if (calls.length) {
        // every script it calls must be FATAL-capable or declared optional
        ok = true
        for (const rel of calls) {
          const base = path.basename(rel)
          const inf = infoFor(rel)
          if (inf.missing) { failures.push(`${where}: calls ${rel}, which does not exist`); ok = true; continue }
          if (OPTIONAL_SCRIPTS[base]) {
            if (inf.fatal) failures.push(`${where}: ${rel} is listed OPTIONAL but prints "FATAL error:" - it would stop the install`)
            if (!inf.defangs) failures.push(`${rel}: listed OPTIONAL but does not defang its output ("problem -")`)
            continue
          }
          if (!inf.fatal) { failures.push(`${where}: calls ${rel}, which has NO "FATAL error:" path - Pinokio 8.2 ignores its exit code, so a failure there would let the install march on. Print "FATAL error: <reason + what to do>", or list it in OPTIONAL_SCRIPTS with a reason`); ok = true }
        }
      }
      if (/h3_roots\.sh/.test(cmd) && /\bh3_enter\b|\bh3_fetch_weights\b/.test(cmd)) {
        const inf = infoFor("scripts/pinokio/h3_roots.sh")
        if (!/h3_enter\(\)[\s\S]*?FATAL error:/.test(inf.text) || !/h3_fetch_weights\(\)[\s\S]*?FATAL error:/.test(inf.text)) {
          failures.push(`${where}: uses h3_enter/h3_fetch_weights but scripts/pinokio/h3_roots.sh does not print FATAL error: in both`)
        }
        ok = true
      }
      if (!ok) ok = BENIGN_INLINE.some(([re]) => re.test(cmd))
      if (!ok && /\|\|\s*echo\b/.test(cmd)) ok = true                // optional: a WARN, checked above
      if (!ok) failures.push(`${where}: this dispatch has no way to stop the install - Pinokio 8.2 ignores its exit code. Wrap it: bash scripts/pinokio/must.sh '<reason + what to do>' <command>, or put it in a script that prints "FATAL error: ..."\n      starts: ${first}`)
    }
  }
}
// every FATAL-capable script: no silent non-zero exit
for (const [rel, inf] of Object.entries(scriptInfo)) {
  if (inf.missing || !inf.fatal) continue
  for (const n of inf.bad) failures.push(`${rel}:${n}: a non-zero exit with no "FATAL error:" line in the 16 lines before it (or a "fatal-ok: <why>" comment) - Pinokio 8.2 would not stop on it`)
}

// post_update.sh: a FATAL-capable script is `require`d or `defang`ed, never bare
{
  const pu = codeLines(read("scripts/post_update.sh"))
  let seen = 0
  for (const l of pu) {
    for (const m of l.t.matchAll(/scripts\/pinokio\/([\w.-]+\.sh)/g)) {
      const rel = "scripts/pinokio/" + m[1]
      const inf = infoFor(rel)
      if (inf.missing || !inf.fatal) continue
      seen++
      if (!/^\s*require\b/.test(l.t) && !/\bdefang\b/.test(l.t)) {
        failures.push(`scripts/post_update.sh:${l.n}: runs ${m[1]} (it can print "FATAL error:") neither through require nor defang - an optional step's failure would end the Update`)
      }
    }
  }
  fatalChecked += seen
}
console.log(`FATAL coverage: ${fatalChecked} calls checked across ${FATAL_LANES.length} lanes + post_update.sh`)

console.log(`${count} dispatches across ${steps} shell.run steps in ${SCRIPTS.length} scripts`)
console.log(`worst dispatch: ${worst} chars (limit ${MAX_DISPATCH})  ${worstWhere}`)
if (failures.length) {
  console.log("")
  for (const f of failures) console.log("FAIL  " + f)
  console.log(`\nRESULT: FAIL (${failures.length})`)
  process.exit(1)
}
console.log("\nRESULT: PASS")