Turn Claude Code into a full game dev studio — 49 AI agents, 72 workflow skills, and a complete coordination system mirroring real studio hierarchy.
Agent Test Spec: analytics-engineer
Agent Summary
- Domain: Telemetry event taxonomy and naming (
[category].[action].[detail]), funnel definitions, A/B test framework design (hypothesis, segmentation, variant assignment, success metrics, minimum sample size, minimum run duration, mutual exclusion between concurrent tests), dashboard specification, privacy compliance of data collection, data-informed design recommendations - Does NOT own: Implementing tracking in game code (programmers — it writes specs for them), design decisions (game-designer decides; data informs), economy design decisions (economy-designer), live ops event design (live-ops-designer)
- Gate IDs: None; reports to technical-director for system design and producer for insights
Static Assertions (Structural)
-
description:field is present and domain-specific (references telemetry, A/B testing, event tracking, analytics) -
tools:readsRead, Glob, Grep, Write, Edit, Bash, WebSearch— Bash serves the data-pipeline workdescription:names; tracking in game code is excluded by the body's "What This Agent Must NOT Do" list, not by the tool list - Model tier is
inherit— frontmattermodel:reads exactlyinherit(tiers:.claude/docs/model-tiers.md) - Agent definition does not claim authority over game implementation, economy design, or live ops scheduling
Test Cases
Case 1: In-domain request — tutorial event tracking design
Input: "Design the analytics event tracking for our tutorial. We want to know where players drop off and which steps they complete." Expected behavior:
- Produces an event taxonomy for the tutorial: events for step start and step completion plus a drop-off event when a player exits mid-step, each with its properties and the exact condition that fires it
- Names events in its Event Naming Convention,
[category].[action].[detail]— e.g.tutorial.step.started,tutorial.step.completed,tutorial.step.abandoned - Gives every event a documented purpose ("Every event must have a documented purpose")
- Defines the tutorial as an onboarding funnel with the event that marks each funnel step (Funnel Analysis Design)
- Collects no personally identifiable information without an explicit requirement, and notes the opt-out path (Privacy Compliance)
- Does NOT produce implementation code — the output is a spec for programmers
Case 2: Out-of-domain request — implement the event tracking in code
Input: "Now that the event schema is designed, write the GDScript code to fire these events in our Godot tutorial scene." Expected behavior:
- Does not produce GDScript or any implementation code — "Implement tracking in game code (write specs for programmers)" is on its Must-NOT list
- Says implementation belongs to the programmers (e.g.
gameplay-programmer) and that its part is the specification - Produces the integration spec the programmer needs: event names, properties, when each event fires, and any privacy or opt-out constraint — the spec is its deliverable here, not an optional extra
- Asks before writing the spec to a file
Case 3: Domain boundary — A/B test design for a UI change
Input: "We want to A/B test two versions of our HUD: the current version and a minimal version with only a health bar. Design the test." Expected behavior:
- Produces a test design covering every element its A/B Test Framework names, none skipped:
- Hypothesis: what the minimal HUD is expected to change and why (e.g. it raises session length by reducing UI cognitive load)
- Segmentation: which players are eligible and how they are split
- Variant assignment: assigned per player ID, not per session, so no player sees both HUDs
- Success metrics: a primary metric (e.g. average session length) plus secondary metrics (e.g. tutorial completion rate, Day 1 retention)
- Minimum sample size: an estimate from the expected effect size — or, where baseline data is missing, a statement that the calculation needs it — never an omitted field
- Minimum run duration: e.g. at least two weeks, to capture weekly play patterns
- Output is structured as a formal test design, not a bullet list of ideas
- Does not decide in advance which HUD ships — results go to game-designer ("data informs, designers decide")
Case 4: Conflict — test result vs. design intent
Input: "The HUD test finished: the minimal HUD raised average session length by 8%. Our game-designer still wants the full HUD because playtesters kept missing the ammo counter. Just ship the minimal HUD — the data is clear." Expected behavior:
- Does not make the call on data alone — "Make game design decisions based solely on data" and "Override design intuition with data" are both on its Must-NOT list
- Presents both the result and the design concern to
game-designer, who decides ("present both to game-designer") - States what the data does and does not show (session length rose; nothing measured whether players lost information they needed) and may propose a follow-up test, without presenting it as the decision
- Does NOT tell the team to ship the minimal HUD on its own authority
Case 5: Context pass — new events consistent with existing schema
Input context: Existing event schema uses the naming convention: [domain]_[object]_[action] in snake_case. Example events: combat_enemy_killed, inventory_item_equipped, tutorial_step_completed.
Input: "Design event tracking for our new crafting system: players gather materials, open the crafting menu, and craft items."
Expected behavior:
- Notices that the supplied schema differs from its own
[category].[action].[detail]convention and says so — its Collaboration Protocol has it "flag conflicts with existing documents rather than resolving them silently" and "call out any departure from the governing document explicitly" - Either asks which convention governs, or follows the existing schema (
crafting_material_gathered,crafting_menu_opened,crafting_item_crafted) and states that it did - Does NOT silently emit dot-notation names (e.g.
crafting.material.gathered) into the snake_case schema, and never mixes the two conventions in one output - Gives each new event a documented purpose and its domain-specific properties (e.g. material type, item id)
Case 6: Conflict — overlapping A/B tests
Input: "We have two A/B tests running simultaneously: Test A (HUD variants) affects all players, and Test B (tutorial variants) also affects all players." Expected behavior:
- Flags the overlap as a mutual exclusion violation: a player in both tests sees a HUD variant and a tutorial variant at once, so neither test can attribute an outcome difference to its own variable
- Proposes resolution options: (a) run the tests one after the other, (b) split the population into exclusive segments (e.g. 50% in Test A, 50% in Test B, none in both), or (c) a factorial design if the interaction itself matters (needs a larger sample)
- Does NOT recommend continuing both tests on overlapping populations
Protocol Compliance
- Stays within declared domain (event taxonomy, funnels, A/B test design, dashboards, privacy)
- Redirects implementation requests to programmers with an integration spec, not code
- Produces A/B test designs covering hypothesis, segmentation, variant assignment, success metrics, minimum sample size and minimum run duration — never partial
- Flags concurrent tests on overlapping players as a mutual exclusion violation and proposes exclusive segments or sequencing
- Presents data and design intent to game-designer rather than deciding on data alone
- Flags a mismatch between a supplied naming convention and its own; never silently emits or mixes conventions
Coverage Notes
- Case 3 (A/B test design completeness) is a quality gate — an incomplete test design wastes experiment budget
- Case 6 (mutual exclusion) is a data integrity test — overlapping tests produce unusable results; this must be caught
- Case 4 (data vs. design intent) is a coordination test — verify the agent leaves the call to game-designer
- Case 5 is the most important context-awareness test; naming convention drift across schemas causes dashboard breakage
- No automated runner; review manually or via
/skill-test