Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

6.3 KB
Raw Blame
"""Example 03 -- reading the result of a predict() call.

A guided tour of what `predict()` returns: the three top-level keys, the `answers` dict
keyed by question id, the fields of each of the three answer types, and what `usage` means.
This is the example to come back to when writing your own glue code.
"""
import json

from _common import banner, device_line, heading, load

banner("03", "Reading the result", """
    Every `predict()` call returns one plain dict with three keys:

        {"model": "laya-rl-agent",
         "answers": {<question id>: <answer>, ...},
         "usage": {"input_tokens": <batch total>, "output_tokens": 0,
                   "state_tokens": <the whole state>, "state_tokens_dropped": <worst case>,
                   "truncated": <bool>, "truncated_questions": [<question id>, ...]}}

    `answers` is keyed by the ids you chose, so glue code can index it directly instead of
    zipping a list back onto the questions. Each answer carries a `type` matching its
    question, the typed decision fields, a `confidence` and an `answer_confidence`, and an
    `action` payload.

    `usage` is the part readers misread: `input_tokens` is the whole batch, with the state
    counted once per question, so it grows with the number of questions rather than being a
    context length. `output_tokens` is always 0 -- Laya never generates text. The keys after
    them are how much of your state the model actually read: `state_tokens` is the whole
    serialized state, `state_tokens_dropped` the most of it any one question's head gave up,
    and `truncated` / `truncated_questions` name the questions that gave something up. One more
    key, `options`, appears only when the head budget was tight enough to leave some question's
    options sharing a token span -- that question can no longer tell those options apart, and
    `usage["options"]` is where the call says so.
    """)

agent = load("english")
device_line(agent)

STATE = {"from": "[email protected]", "subject": "Duplicate charge on invoice #4411",
         "body": "We were billed twice for March. Please refund the duplicate."}

QUESTIONS = {
    "department": {
        "type": "choice",
        "instructions": "Which department should handle this request?",
        "criteria": {"billing": "invoices, payments, refunds",
                     "technical": "bugs, outages, system errors",
                     "sales": "pricing, new contracts",
                     "other": "everything else"},
    },
    "urgency": {
        "type": "score",
        "instructions": "How urgent is this request?",
        "criteria": ["not urgent", "soon", "critical deadline or blocking issue"],
    },
    "churn_risk": {
        "type": "noul",
        "instructions": "Does the user threaten to cancel or leave?",
    },
}

result = agent.predict(STATE, QUESTIONS)
answers = result["answers"]

# ---------------------------------------------------------------- 1. the raw dict
heading("the raw dict for a three-question call")
print(json.dumps(result, indent=2))

# ---------------------------------------------------------------- 2. top-level keys
heading("top level")
for key, value in result.items():
    print("   %-8s %s" % (key, type(value).__name__))

# ---------------------------------------------------------------- 3. parsed fields
heading("the same answers, parsed into a table")
print("   %-12s %-7s %-34s %-11s %-11s %s" % ("question id", "type", "primary answer",
                                             "confidence", "answer_conf", "action"))
print("   " + "-" * 89)
for qid, a in answers.items():
    if a["type"] == "choice":
        primary = "%s (p=%.3f)" % (a["choice"], a["probabilities"][a["choice"]])
    elif a["type"] == "score":
        primary = "%.3f of %d  (expected, 0-based)" % (a["score"], len(a["legend"]) - 1)
    else:
        primary = "%.4f -> %s" % (a["noul"], "true" if a["noul"] > 0.5 else "false")
    print("   %-12s %-7s %-34s %-11.3f %-11.3f %.3f"
          % (qid, a["type"], primary, a["confidence"], a["answer_confidence"],
             a["action"]["act_probability"]))

heading("the fields each type carries")
print("   choice : type, choice, probabilities {label: p},")
print("            confidence, answer_confidence, action")
print("   score  : type, score, legend {index: text}, probabilities {index: p},")
print("            confidence, answer_confidence, action")
print("            (indexes and legend keys are strings: \"0\", \"1\", ...)")
print("   noul   : type, noul (P(true)), confidence, answer_confidence, action")
print("   `confidence` is not one formula: it is 1 minus normalised entropy on a `choice` or")
print("   `score` answer, but max(p[1], 1 - p[1]) -- the probability of the more likely side --")
print("   on a `noul`. The scales are not comparable: p(true)=0.600 reads 0.600 as a `noul` and")
print("   would read 0.029 as entropy, while 0.900 reads 0.900 against 0.531. `answer_confidence`")
print("   is max(p), the probability mass on the answer being reported -- that same quantity on")
print("   every question type, and the field to gate on (example 18 routes at a threshold on it).")
print("   A calibration payload that carries `binning_map` remaps `answer_confidence` and leaves")
print("   `confidence` alone.")
print("   every answer also carries action['act_probability'], a separate two-way head over")
print("   the same encoder (see agent.cfg['act_costs'] / ['cost_wrong_act']). The typed")
print("   fields above are the decision; action is an auxiliary signal your policy may use.")

# ---------------------------------------------------------------- 4. usage
heading("usage: a batch total, not a context length")
one_question = {"churn_risk": QUESTIONS["churn_risk"]}
single = agent.predict(STATE, one_question)["usage"]
print("   all three questions : input=%d  output=%d"
      % (result["usage"]["input_tokens"], result["usage"]["output_tokens"]))
print("   churn_risk alone    : input=%d  output=%d"
      % (single["input_tokens"], single["output_tokens"]))
print("   The state is encoded once per question, so the batch total is the sum over the")
print("   questions in the call -- not the length of a shared context.")

print("""
   Two keys you supply decide the shape of the output: the state and the questions dict.
   The ids in that dict become the keys of `answers`, and nothing else is needed to consume
   the result -- there is no text, so there is nothing to parse.
   """)