An AI-powered search engine with a generative UI
TypeScript generative-aigenerative-uinextjsollamareactsearxngshadcn-uitailwindcsstavilytypescriptupstashvercel-ai-sdk
fix(prompt): stop appending related questions to single-fact answers (#977)
Yoshiki Miurafdfc1759c5d5
diff --git a/evals/thresholds.json b/evals/thresholds.jsonindex 1609503c..dd59e6aa 100644--- a/evals/thresholds.json+++ b/evals/thresholds.json@@ -12,7 +12,7 @@},"falsePositiveRate": {"direction": "max",- "value": null,+ "value": 0.2,"minSamples": 50},"wellFormedRate": {diff --git a/lib/render/__tests__/prompt.test.ts b/lib/render/__tests__/prompt.test.tsindex aed522dc..a20e4ded 100644--- a/lib/render/__tests__/prompt.test.ts+++ b/lib/render/__tests__/prompt.test.ts@@ -16,15 +16,16 @@ describe('render prompts', () => {const prompt = getRelatedQuestionsSpecPrompt()expect(prompt).toContain('expected on every substantive answer')+ expect(prompt).toContain('Decide from the answer you just wrote')expect(prompt).toContain(- 'Include the spec block whenever the answer contains substantive information'+ 'Include the spec block whenever that answer compares options')expect(prompt).toContain('SKIP the spec block entirely (output nothing) only in these cases')expect(prompt).toContain('Greetings, small talk, or thanks')expect(prompt).toContain(- 'Trivial one-off lookups with no natural next step'+ 'The answer delivers a single fact, value, date, quantity, or yes/no')expect(prompt).toContain('Meta/operational replies')expect(prompt).toContain('Cases where you could not answer')diff --git a/lib/render/prompt.ts b/lib/render/prompt.tsindex 46950267..86c75b20 100644--- a/lib/render/prompt.ts+++ b/lib/render/prompt.ts@@ -1,7 +1,7 @@/*** Returns the prompt section that instructs the LLM to output* related questions as a ```spec fenced block at the end of its response,- * but only when follow-ups add value (skipped for greetings, trivial+ * but only when follow-ups add value (skipped for greetings, single-fact* lookups, and non-answers).*/export function getRelatedQuestionsSpecPrompt(): string {@@ -9,11 +9,11 @@ export function getRelatedQuestionsSpecPrompt(): string {RELATED QUESTIONS:After your conclusion, generate exactly 3 follow-up questions in a \`\`\`spec fenced code block. This is expected on every substantive answer; only the narrow skip cases below are exempt.-Include the spec block whenever the answer contains substantive information, analysis, comparison, or recommendations. Anchor each follow-up to concrete details from THIS answer.+Decide from the answer you just wrote, not from what the reader might go on to wonder. Include the spec block whenever that answer compares options, recommends or advises, explains a cause or a mechanism, lays out several distinct points, or walks through steps. Anchor each follow-up to concrete details from THIS answer.SKIP the spec block entirely (output nothing) only in these cases:- Greetings, small talk, or thanks ("hi", "thanks", "how are you").-- Trivial one-off lookups with no natural next step (simple math, a single fact, a yes/no).+- The answer delivers a single fact, value, date, quantity, or yes/no, even when interesting follow-ups do exist. A lookup does not stop being a lookup because the topic is rich.- Meta/operational replies, acknowledgements, or task-completion notices.- Cases where you could not answer (refusal, clarification request, or empty result).- Short answers where suggested next questions would be generic or forced.