Skip to content

World's first open-source enterprise world model.

A statement reads on its own and a thing is named by a term (#743)

WaylandYang8e1271a77ed8
diff --git a/crates/utopia-extract/src/open.rs b/crates/utopia-extract/src/open.rs
index a04a0302..895c0f12 100644
--- a/crates/utopia-extract/src/open.rs
+++ b/crates/utopia-extract/src/open.rs
@@ -110,21 +110,35 @@ Output one JSON object and nothing else, shaped like this:\n\
\"n\": [[\"name as listed in e\", \"another name\", \"the sentence that uses it, verbatim\"]]}\n\
\n\
1. \"e\" lists the things the passage talks about, one entry each: [name, kind, named]. Every \
-thing a statement refers to is listed here once, under exactly the name the statements use. A \
-thing the passage names (people, organizations, products, places, documents, laws, events) has \
-named = 1 and is named as the passage writes it, without citation words: \"the Harbor Treaty \
-signed on May 3, 1998\" is named \"Harbor Treaty\". A thing the passage only describes has \
-named = 0 and is named by the words that say what it is (\"the northern wing\", \"farmland\", \
-\"beekeepers\"). kind is what the thing is, in two or three words, as the passage says it. A \
-name never carries a figure: \"about 40 hectares of farmland\" is the thing \"farmland\" with a \
-statement \"area\" = \"about 40 hectares\". List each thing once.\n\
+thing a statement refers to is listed here once, under exactly the name the statements use. \
+named = 1 for a proper name or a fixed term: the name of one particular person, organization, \
+product, place, document, law or event, or a term that means the same thing in any document (a \
+disease, a drug, an industry, a product category, an indicator), written as the passage writes \
+it, without citation words (\"the Harbor Treaty signed on May 3, 1998\" is named \"Harbor \
+Treaty\"). named = 0 for a role or a generic phrase whose referent this passage decides, however \
+particular it is here: \"the company\", \"the enterprises visited\", \"patients\", \"各部门\", \
+\"the northern wing\" are described things, named by the words that say what they are. kind is \
+what the thing is, in two or three words, as the passage says it. A name never carries a figure: \"about 40 hectares of farmland\" is the thing \
+\"farmland\" with a statement \"area\" = \"about 40 hectares\". List each thing once.\n\
2. \"s\" lists the statements, one entry each: [subject, phrase, object, value, qualifiers, \
when, ended, quote]. subject and object are names, written exactly as listed in \"e\" or in \
the list of things already recorded, with the same spelling every time. Exactly one of object \
and value is set; the other is null.\n\
- A statement with an object links two things. phrase is how the passage says the link, as a \
-short lowercase phrase (\"acquired\", \"was designed by\", \"houses\", \"is the captain of\", \
-\"修建\"); do not translate it into any vocabulary of your own.\n\
+ A statement with an object links two things. phrase is how the passage says the link, \
+lowercase, in the passage's own words; do not translate it into any vocabulary of your own. It \
+is the verb with the words that belong to it, so that subject —phrase→ object reads as a \
+sentence on its own (\"acquired\", \"was designed by\", \"is the captain of\", \"went into \
+partnership with\", \"细化解读\"), never a bare verb cut from a longer verb phrase (\"went\", \
+\"细化\"). Several verbs sharing one object make one statement whose phrase carries them all \
+(\"designed and built\"); one verb with several objects makes one statement per object. The \
+object is what the verb acts on, and a described thing that is the subject or the object is \
+listed like any other (\"the coating should not be used on surfaces exposed to seawater\" gives \
+coating —should not be used on→ surfaces exposed to seawater, a described thing); words that say \
+where, how, why, with what or for whom go in qualifiers under the passage's own role word. A \
+named thing mentioned in a qualifier is still listed and the qualifier names it; a generic phrase \
+that appears in a qualifier alone is words there, not a listed thing: \"各部门要走进学校、社区等\
+基层单位开展宣传活动\" gives 各部门 —开展→ 宣传活动 with {\"走进\": \"学校、社区等基层单位\"}, \
+and neither 学校 nor 社区 is listed.\n\
A statement with a value is what the passage says about one thing by itself: a figure with \
its units, a percentage, an amount, a count, a title, a status. phrase is how the passage says \
what is measured or stated (\"area\", \"was completed\", \"joined\", \"占地面积\"); value is the \
diff --git a/scripts/bench/judge_open.mjs b/scripts/bench/judge_open.mjs
index 436113c3..37e4b4b6 100644
--- a/scripts/bench/judge_open.mjs
+++ b/scripts/bench/judge_open.mjs
@@ -70,7 +70,8 @@ For each, first copy the words of the document that bear on it ("evidence", "" i
- "stated": the document states this, or a careful reader takes it directly from the document, and the statement describes it correctly;
- "misworded": the document does state a relationship between this subject and this object or value, but the statement describes it wrongly (the wrong phrase, the wrong direction, a qualifier or time that belongs to something else);
- "not_stated": the document does not state any such relationship between this subject and this object or value (it is invented, the object belongs to something else, or it goes beyond what the document says).
-Judge only from the document. Output one JSON object: {"results":[{"i":0,"evidence":"...","verdict":"stated|misworded|not_stated"}]}`;
+Then, separately, answer "alone": true when "subject —phrase→ object" reads as a complete proposition by itself, without the document; false when a reader could not tell what is being said because the phrase is a bare verb cut from a longer verb phrase, or the object or subject is a fragment of a longer noun phrase.
+Judge only from the document. Output one JSON object: {"results":[{"i":0,"evidence":"...","verdict":"stated|misworded|not_stated","alone":true}]}`;
// 取样:现行的开放陈述,连它的证据块(原文)、限定与时间词
function loadStatements(kb, sample, seed) {
@@ -128,7 +129,14 @@ async function judgeAll(kb, items) {
for (const r of res.results || []) {
const it = part[r.i];
if (!it) continue;
- out.push({ ...it, statement: render(it), evidence: r.evidence || "", verdict: ["stated", "misworded", "not_stated"].includes(r.verdict) ? r.verdict : "unjudged" });
+ out.push({
+ ...it,
+ statement: render(it),
+ evidence: r.evidence || "",
+ verdict: ["stated", "misworded", "not_stated"].includes(r.verdict) ? r.verdict : "unjudged",
+ // 读不读得通:单看「主语 —短语→ 宾语」能不能明白在说什么(ReVerb 说的 uninformative 那一类)
+ alone: typeof r.alone === "boolean" ? r.alone : null,
+ });
}
console.error(`${stamp()} 判了 ${out.length}/${items.length}`);
}
@@ -142,11 +150,17 @@ function report(judged, total) {
const pct = (a) => `${a}/${n}(${n ? ((100 * a) / n).toFixed(1) : 0}%)`;
console.log(`\n开放陈述 ${total} 条,判了 ${n} 条:stated ${pct(count("stated"))},misworded ${pct(count("misworded"))},not_stated ${pct(count("not_stated"))},未判 ${count("unjudged")}`);
console.log(`0044 的门槛:not_stated ≤ 2%${count("not_stated") / (n || 1) <= 0.02 ? ",过" : ",没过"}`);
+ const fragments = judged.filter((j) => j.alone === false);
+ console.log(`单看读不通(alone = false):${pct(fragments.length)}`);
const bad = judged.filter((j) => j.verdict !== "stated").slice(0, 20);
if (bad.length) {
console.log("\n判不对的(样本):");
for (const j of bad) console.log(` [${j.verdict}] ${j.statement}\n 证据:${j.evidence.slice(0, 160)}`);
}
+ if (fragments.length) {
+ console.log("\n读不通的(样本):");
+ for (const j of fragments.slice(0, 20)) console.log(` ${j.statement}\n 证据:${j.evidence.slice(0, 160)}`);
+ }
}
if (args.calibrate) {