:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
import json
from typing import Any, Literal, Self, cast
import jsonpatch
from jsonpointer import JsonPointerException
from langchain_core.runnables import RunnableConfig
from pydantic import BaseModel, Field, ValidationError
from posthog.schema import (
ArtifactContentType,
ArtifactSource,
AssistantTool,
AssistantToolCallMessage,
QuerySchemaRoot,
VisualizationArtifactContent,
)
from posthog.models import Team, User
from posthog.sync import database_sync_to_async
from products.product_analytics.backend.facade.api import save_saved_insight_query, saved_insight_for_update
from products.product_analytics.backend.facade.contracts import SavedInsightDefinition
from ee.hogai.artifacts.utils import is_visualization_artifact_message
from ee.hogai.chat_agent.insights_graph.graph import InsightsGraph
from ee.hogai.chat_agent.schema_generator.nodes import SchemaGenerationException
from ee.hogai.context.context import AssistantContextManager
from ee.hogai.tool import MaxTool, ToolMessagesArtifact
from ee.hogai.tool_errors import MaxToolRetryableError
from ee.hogai.utils.prompt import format_prompt_string
from ee.hogai.utils.types.base import ArtifactRefMessage, AssistantNodeName, AssistantState, NodePath
INSIGHT_TOOL_PROMPT = """
Use this tool to generate an insight from a structured plan. It will return a visualization that the user can analyze and a textual representation for your analysis. Without `insight_id`, these visualizations are transient and are not saved to the project. To save an insight permanently, users should click the open insight icon below the chart in the conversation.
A chart generated in this conversation can have an artifact or visualization ID without being saved. Never pass those IDs as `insight_id`. To revise an unsaved chart, omit both `insight_id` and `query_patch`, and provide a revised structured plan based on that chart and the requested changes. Only use `insight_id` after confirming the saved insight in this project through search or read_data.
To update an existing saved insight, first read its saved query with read_data, then provide its short ID as `insight_id` and `query_patch` as a JSON-encoded array of JSON Patch operations against that full query. This saves the same insight and affects every dashboard referencing it. Use narrow paths for only the changes in the current request. Do not apply remembered preferences, defaults, or other improvements to unrelated settings. For example, [{"op":"replace","path":"/source/breakdownFilter/breakdown","value":"name"}] changes a legacy breakdown property; for a breakdowns list, use /source/breakdownFilter/breakdowns/0/property. Use add to set an optional field, remove to delete a field, and move with from/path to reorder a series without rebuilding it. Parent objects must exist. Never regenerate the whole query or replace a whole filter object to change one setting. Use the saved query, not dashboard overrides. Do not create a replacement insight or use upsert_dashboard for insight edits. The existing name and description are preserved. query_description and insight_type are used only for generation, not saved edits.
When editing the currently open insight editor, omit `insight_id` to apply a draft to the editor, including its unsaved changes. Supply `insight_id` only when the user asks to save directly to that insight.
This tool can also be used to edit the visualization the user is currently viewing on the insight page. In that case, you need to generate a new plan based on the schema of the existing insight.
The tool only generates a single visualization per call. If the user asks for multiple visualizations, you need to decompose a query into multiple subqueries and call the tool for each subquery.
Follow these guidelines when retrieving data:
- If the same insight is already in the conversation history, reuse the retrieved data only when this does not violate the <data_analysis_guidelines> section (i.e. only when a presence-check, count, or sort on existing columns is enough).
- If analysis results have been provided, use them to answer the user's question. The user can already see the analysis results as a chart - you don't need to repeat the table with results nor explain each data point.
- If the retrieved data and any data earlier in the conversations allow for conclusions, answer the user's question and provide actionable feedback.
- If there is a potential data issue, retrieve a different new analysis instead of giving a subpar summary. Note: empty data is NOT a potential data issue.
- If the query cannot be answered with a UI-built insight type - trends, funnels, retention - choose the SQL type to answer the question (e.g. for listing events or aggregating in ways that aren't supported in trends/funnels/retention).
- Paths insights (user journey/path visualizations, including wildcard groups and other Advanced paths controls) cannot be created or edited with this tool, and you should not attempt to edit a paths insight as a trends/funnel/retention insight. If a user asks to add wildcard groups - whether in path groupings or in exclusions - to a paths insight, know that wildcard groups are part of "Advanced paths", a paid feature. If the user is on a free plan this control is disabled in the UI, so tell them it requires upgrading to a paid plan rather than suggesting workarounds.
Remember: do NOT retrieve data for the same query more than 3 times in a row.
Important: If the user request is about analysis of entities that are not collected data (events, properties, etc) like data warehouse entities, use SQL.
CRITICAL: When planning an insight, be minimalist. Only include filters, breakdowns, and settings that are essential to answer the user's specific question. Default settings are usually sufficient unless the user explicitly requests customization.
# Selecting a visualization type
## Trends
A trends insight visualizes events over time using time series. They're useful for finding patterns in historical data.
The trends insights have the following features:
- The insight can show multiple trends in one request.
- Custom formulas can calculate derived metrics, like `A/B*100` to calculate a ratio.
- Filter and break down data using multiple properties.
- Compare with the previous period and sample data.
- Apply various aggregation types, like sum, average, etc., and chart types.
- And more.
Examples of use cases include:
- How the product's most important metrics change over time.
- Long-term patterns, or cycles in product's usage.
- The usage of different features side-by-side.
- How the properties of events vary using aggregation (sum, average, etc).
- Users can also visualize the same data points in a variety of ways.
## Funnel
A funnel insight visualizes a sequence of events that users go through in a product. They use percentages as the primary aggregation type. Funnels REQUIRE AT LEAST TWO series (events or actions), so the conversation history should mention at least two events.
The funnel insights have the following features:
- Various visualization types (steps, time-to-convert, historical trends).
- Filter data and apply exclusion steps (events only, not actions).
- Break down data using a single property.
- Specify conversion windows (default 14 days), step order (strict/ordered/unordered), and attribution settings.
- Aggregate by users, sessions, or specific group types.
- Sample data.
- Track first-time conversions with special math aggregations.
- And more.
Examples of use cases include:
- Conversion rates between steps.
- Drop off steps (which step loses most users).
- Steps with the highest friction and time to convert.
- If product changes are improving their funnel over time.
- Average/median/histogram of time to convert.
- Conversion trends over time (using trends visualization type).
- First-time user conversions (using first_time_for_user math).
## Retention
A retention insight visualizes how many users return to the product after performing some action. They're useful for understanding user engagement and retention.
The retention insights have the following features: filter data, sample data, and more.
Examples of use cases include:
- How many users come back and perform an action after their first visit.
- How many users come back to perform action X after performing action Y.
- How often users return to use a specific feature.
# Data narrowing
<property_filters>
Use property filters to provide a narrowed results. Only include property filters when they are essential to directly answer the user's question. Avoid adding them if the question can be addressed without additional segmentation and always use the minimum set of property filters needed to answer the question. Properties have one of the four types: String, Numeric, Boolean, and DateTime.
IMPORTANT: Do not check if a property is set unless the user explicitly asks for it.
When using a property filter, you must:
- **Prioritize properties directly related to the context or objective of the user's query.** Avoid using properties for identification like IDs because neither the user nor you can retrieve the data. Instead, prioritize filtering based on general properties like `paidCustomer` or `icp_score`.
- **Ensure that you find both the property group and name.** Property groups must be one of the following: event, person, session{{#groups}}, {{.}}{{/groups}}.
- After selecting a property, **validate that the property value accurately reflects the intended criteria**.
- **Find the suitable operator for type** (e.g., `contains`, `is set`). The operators are listed below.
- If the operator requires a value, use the tool to find the property values. Verify that you can answer the question with given property values. If you can't, try to find a different property or event.
- You set logical operators to combine multiple properties of a single series: AND or OR.
Infer the property groups from the user's request. If your first guess doesn't yield any results, try to adjust the property group. You must make sure that the property name matches the lookup value, e.g. if the user asks to find data about organizations with the name "ACME", you must look for the property like "organization name."
Supported operators for the String type are:
- equals (exact)
- doesn't equal (is_not)
- contains (icontains)
- doesn't contain (not_icontains)
- matches regex (regex)
- doesn't match regex (not_regex)
- is set
- is not set
Supported operators for the Numeric type are:
- equals (exact)
- doesn't equal (is_not)
- greater than (gt)
- less than (lt)
- is set
- is not set
Supported operators for the DateTime type are:
- equals (is_date_exact)
- doesn't equal (is_not for existence check)
- before (is_date_before)
- after (is_date_after)
- is set
- is not set
Supported operators for the Boolean type are:
- equals
- doesn't equal
- is set
- is not set
All operators take a single value except for `equals` and `doesn't equal` which can take one or more values (as an array).
</property_filters>
<time_period_and_property_filters>
You must not filter events by time, so you must not look for time-related properties. Do not verify whether events have a property indicating capture time as they always have, but it's unavailable to you. Instead, include time periods in the insight plan in the `Time period` section. If the question doesn't mention time, use `last 30 days` as a default time period.
Examples:
- If the user asks you "find events that happened between March 1st, 2025, and 2025-03-07", you must include `Time period: from 2025-03-01 to 2025-03-07` in the insight plan.
- If the user asks you "find events for the last month", you must include `Time period: from last month` in the insight plan.
</time_period_and_property_filters>
# Trends guidelines
<general_knowledge>
Trends insights enable users to plot data from people, events, and properties however they want. They're useful for finding patterns in data, as well as monitoring users' product to ensure everything is running smoothly. Users can use multiple independent series in a single query to see trends. They can also use a formula to calculate a metric. Each series has its own set of property filters, so you must define them for each series. Trends insights do not require breakdowns or filters by default.
</general_knowledge>
<aggregation>
**Determine the math aggregation** the user is asking for, such as totals, averages, ratios, or custom formulas. If not specified, choose a reasonable default based on the event type (e.g., total count). By default, the total count should be used. You can aggregate data by events, event's property values,{{#groups}} {{.}}s,{{/groups}} or users. If you're aggregating by users or groups, there's no need to check for their existence, as events without required associations will automatically be filtered out.
Available math aggregations types for the event count are:
- total count
- average
- minimum
- maximum
- median
- 90th percentile
- 95th percentile
- 99th percentile
- unique users
- unique sessions
- weekly active users
- daily active users
- first time for a user
{{#groups}}
- unique {{.}}s (requires `math_group_type_index` to be set to the group type index from the group mapping)
{{/groups}}
Available math aggregation types for event's property values are:
- average
- sum
- minimum
- maximum
- median
- 90th percentile
- 95th percentile
- 99th percentile
Available math aggregation types counting number of events completed per user (intensity of usage) are:
- average
- minimum
- maximum
- median
- 90th percentile
- 95th percentile
- 99th percentile
Examples of using aggregation types:
- `unique users` to find how many distinct users have logged the event per a day.
- `average` by the `$session_diration` property to find out what was the average session duration of an event.
- `99th percentile by users` to find out what was the 99th percentile of the event count by users.
</aggregation>
<math_formulas>
If the math aggregation is more complex or not listed above, use custom formulas to perform mathematical operations like calculating percentages or metrics. If you use a formula, you must use the following syntax: `A/B`, where `A` and `B` are the names of the series. You can combine math aggregations and formulas.
When using a formula, you must:
- Identify and specify **all** events and actions needed to solve the formula.
- Carefully review the list of available events and actions to find appropriate entities for each part of the formula.
- Ensure that you find events and actions corresponding to both the numerator and denominator in ratio calculations.
Examples of using math formulas:
- If you want to calculate the percentage of users who have completed onboarding, you need to find and use events or actions similar to `$identify` and `onboarding complete`, so the formula will be `A / B * 100`, where `A` is `onboarding complete` (unique users) and `B` is `$identify` (unique users).
- To calculate conversion rate: `A / B * 100` where A is conversions and B is total events
- To calculate average value: `A / B` where A is sum of property and B is count
</math_formulas>
<time_interval>
Specify the time interval (group by's by time) in the `Time interval` section on the plan. Available intervals are: `hour`, `day`, `week`, `month`.
Unless the user has specified otherwise, use the following default interval:
- If the time period is less than two days, use the `hour` interval.
- If the time period is less than a month, use the `day` interval.
- If the time period is less than three months, use the `week` interval.
- Otherwise, use the `month` interval.
</time_interval>
<breakdowns>
Breakdowns are used to segment data by property values of maximum three properties. They divide all defined trends series to multiple subseries based on the values of the property. Include breakdowns **only when they are essential to directly answer the user's question**. You must not add breakdowns if the question can be addressed without additional segmentation. Always use the minimum set of breakdowns needed to answer the question.
When using breakdowns, you must:
- **Identify the property group** and name for each breakdown.
- **Provide the property name** for each breakdown.
- **Validate that the property value accurately reflects the intended criteria**.
Examples of using breakdowns:
- page views trend by country: you need to find a property such as `$geoip_country_code` and set it as a breakdown.
- number of users who have completed onboarding by an organization: you need to find a property such as `organization name` and set it as a breakdown.
</breakdowns>
<plan_example>
Series:
- series 1: event name
- math operation: total
- custom name: (optional) custom display name for this series
- property filter 1:
- entity
- property name
- property type
- operator
- property value
- property filter 2... Repeat for each property filter.
- series 2: action name
- action id: `numeric id`
- math operation: average by `property name`.
- custom name: (optional) custom display name for this series
- property filter 1:
- entity
- property name
- property type
- operator
- property value
- property filter 2... Repeat for each property filter.
- Repeat for each event.
(if a formula is used)
Formula:
`A/B`, where `A` is the first event and `B` is the second event.
(if a breakdown is used)
Breakdown by:
- breakdown 1:
- entity
- property name
- Repeat for each breakdown.
(if comparing to previous period is needed)
Compare to previous period: yes/no
Compare to: (optional) specific relative date like `-1y`, `-14d`, `-30h`
(if a time period or interval is explicitly mentioned)
Time period: from and/or to dates or durations. For example: `last 1 week`, `last 12 days`, `from 2025-01-15 to 2025-01-20`, `2025-01-15`, from `last month` to `2024-11-15`.
Time interval: hour/day/week/month/year
(optional visualization settings)
Display type: (ActionsLineGraph/ActionsBar/ActionsAreaGraph/ActionsLineGraphCumulative/BoldNumber/ActionsBarValue/ActionsPie/ActionsTable/WorldMap)
Show legend: yes/no
Show values on series: yes/no
Y-axis scale: linear/log10
Axis format: numeric/duration/duration_ms/duration_ns/percentage/percentage_scaled/currency
Axis prefix: (e.g., "$")
Axis postfix: (e.g., " clicks")
Decimal places: (number)
</plan_example>
# Funnel guidelines
<general_knowledge>
Funnel insights help stakeholders understand user behavior as users navigate through a product. A funnel consists of a sequence of at least two events or actions, where some users progress to the next step while others drop off. Funnels are perfect for finding conversion rates, average and median conversion time, conversion trends, and distribution of conversion time.
</general_knowledge>
<exclusion_steps>
Users may want to use exclusion events to filter out conversions in which a particular event occurred between specific steps. These events must not be included in the main sequence. You must include start and end indexes (0-based) for each exclusion where the minimum start index is 0 (first step) and the maximum end index is the number of steps minus one. Exclusion events cannot be actions, only events.
IMPORTANT: Exclusion steps filter out conversions where the exclusion event occurred BETWEEN the specified steps. This does NOT exclude users who completed the event before the funnel started or after it ended.
For example, there is a sequence with three steps: sign up (step 0), finish onboarding (step 1), purchase (step 2). If the user wants to exclude all conversions in which users navigated away between sign up and finishing onboarding, the exclusion step will be:
```
Exclusions:
- $pageleave
- start index: 0 (sign up)
- end index: 1 (finish onboarding)
```
</exclusion_steps>
<breakdown>
A breakdown is used to segment data by a single property value. They divide all defined funnel series into multiple subseries based on the values of the property. Include a breakdown **only when it is essential to directly answer the user's question**. You must not add a breakdown if the question can be addressed without additional segmentation.
When using breakdowns, you must:
- **Identify the property group** and name for a breakdown.
- **Provide the property name** for a breakdown.
- **Validate that the property value accurately reflects the intended criteria**.
Examples of using a breakdown:
- page views to sign up funnel by country: you need to find a property such as `$geoip_country_code` and set it as a breakdown.
- conversion rate of users who have completed onboarding after signing up by an organization: you need to find a property such as `organization name` and set it as a breakdown.
</breakdown>
<reminders>
- You MUST ALWAYS use AT LEAST TWO series (events or actions) in the funnel plan.
</reminders>
<plan_example>'
Sequence:
1. event: event name 1
- custom name: (optional) custom display name for this step
- math operation: (optional) first_time_for_user or first_time_for_user_with_filters
- property filter 1:
- entity
- property name
- property type
- operator
- property value
- property filter 2... Repeat for each property filter.
2. action: action name 2
- action id: `numeric id`
- custom name: (optional) custom display name for this step
- math operation: (optional) first_time_for_user or first_time_for_user_with_filters
- property filter 1:
- entity
- property name
- property type
- operator
- property value
- property filter 2... Repeat for each property filter.
3. Repeat for each event or action...
(if exclusion steps are used)
Exclusions:
- exclusion event name 1
- start index: 1
- end index: 2
- exclusion event name 2... Repeat for each exclusion...
(if a breakdown is used)
Breakdown by:
- entity
- property name
(if aggregating by groups instead of users)
Aggregate by: group type index from group mapping
(if a time period is explicitly mentioned)
Time period: from and/or to dates or durations. For example: `last 1 week`, `last 12 days`, `from 2025-01-15 to 2025-01-20`, `2025-01-15`, from `last month` to `2024-11-15`.
(optional funnel settings)
Visualization type: steps/time_to_convert/trends
Conversion window: number and unit (e.g., 14 days, 1 hour)
Step order: strict/ordered/unordered
Step reference: total/previous (for conversion percentages)
Layout: vertical/horizontal
Bin count: (only for time_to_convert, number of histogram bins)
</plan_example>
# Retention guidelines
<general_knowledge>
Retention is a type of insight that shows you how many users return during subsequent periods.
They're useful for answering questions like:
- Are new sign ups coming back to use your product after trying it?
- Have recent changes improved retention?
</general_knowledge>
<retention_plan>
Plans of retention insights must always have two events or actions:
- The activation event – an event or action that determines if the user is a part of a cohort (when they "start").
- The retention event – an event or action that determines whether a user has been retained (when they "return").
For activation and retention events, use the `$pageview` event by default or the equivalent for mobile apps `$screen`. Avoid infrequent or inconsistent events like `signed in` unless asked explicitly, as they skew the data.
The activation and retention events can be the same (e.g., both `$pageview` to see if users who viewed pages come back to view pages again) or different (e.g., activation is `signed up` and retention is `completed purchase` to see if sign-ups convert to purchases over time).
</retention_plan>
<plan_example>
Activation:
(if an event is used)
- event: chosen event name
(or if an action is used)
- action id: `numeric id`
- action name: action name
Retention:
- event: chosen event name (can be the same as activation event, or different)
(or if an action is used)
- action id: `numeric id`
- action name: action name
(if filters are used)
Filters: - property filter 1: - entity - property name - property type - operator - property value - property filter 2... Repeat for each property filter.
(if a time period is explicitly mentioned)
Time period: from and/or to dates or durations. For example: `last 1 week`, `last 12 days`, `from 2025-01-15 to 2025-01-20`, `2025-01-15`, from `last month` to `2024-11-15`.
</plan_example>
# Reminders
- Ensure that any properties included are directly relevant to the context and objectives of the user's question. Avoid unnecessary or unrelated details.
- Avoid overcomplicating the response with excessive property filters. Focus on the simplest solution that effectively answers the user's question.
- When using group aggregations (unique groups), always set `math_group_type_index` to the appropriate group type index from the group mapping.
- Custom names for series or steps are optional and should only be used when the user explicitly wants to rename them or when the default name would be unclear.
- Visualization settings (display type, axis format, etc.) should only be specified when explicitly requested or when they significantly improve the answer to the user's question.
- The default funnel step order is `ordered` (events in sequence but with other events allowed in between). Use `strict` when events must happen consecutively with no events in between. Use `unordered` when order doesn't matter.
- Exclusion events in funnels only exclude conversions where the event happened between the specified steps, not before or after the funnel.
# Summary
1. Select an insight type
2. Follow the guidelines and create a structured plan
3. Pass the structured plan and insight type to this tool.
""".strip()
INSIGHT_TOOL_CONTEXT_PROMPT_TEMPLATE = """
The user is currently editing an insight (aka query). Here is that insight's current definition, which can be edited using the `create_insight` tool:
```json
{current_query}
```
<system_reminder>
Do not remove any fields from the current insight definition. Do not change any other fields than the ones the user asked for. Keep the rest as is.
</system_reminder>
""".strip()
INSIGHT_TOOL_FAILURE_SYSTEM_REMINDER_PROMPT = """
<system_reminder>
Inform the user that you've encountered an error during the creation of the insight. Afterwards, try to generate a new insight with a different query.
Terminate if the error persists.
</system_reminder>
""".strip()
INSIGHT_TOOL_HANDLED_FAILURE_PROMPT = """
The agent has encountered the error while creating an insight.
Generated output:
```
{{{output}}}
```
Error message:
```
{{{error_message}}}
```
{{{system_reminder}}}
""".strip()
INSIGHT_TOOL_UNHANDLED_FAILURE_PROMPT = """
The agent has encountered an unknown error while creating an insight.
{{{system_reminder}}}
""".strip()
InsightType = Literal["trends", "funnel", "retention", "sql"]
class CreateInsightToolArgs(BaseModel):
insight_id: str | None = Field(
default=None,
description="Short ID of a saved insight in this project, confirmed through search or read_data. Never use an unsaved chart's artifact or visualization ID. Omit to create or revise an unsaved chart, or edit the open editor's draft.",
)
query_patch: str | None = Field(
default=None,
description="Required for saved edits: JSON-encoded JSON Patch array against the full saved query. Use add, replace, remove, move or test with precise paths. Read the saved query first. Omit for generation.",
)
expected_query: dict[str, Any] | None = Field(default=None, exclude=True)
query_description: str = Field(description="A plan of the query to generate based on the template.")
insight_type: InsightType = Field(description="The type of insight to generate.")
viz_title: str = Field(
description="Short, concise name of the insight (2-7 words) that will be displayed as a header in the insight visualization."
)
viz_description: str = Field(
description="Short, concise summary of the insight (1 sentence) that will be displayed as a description in the insight visualization."
)
class CreateInsightTool(MaxTool):
name: Literal["create_insight"] = "create_insight"
args_schema: type[BaseModel] = CreateInsightToolArgs
context_prompt_template: str = INSIGHT_TOOL_CONTEXT_PROMPT_TEMPLATE
def get_required_resource_access(self):
"""Creating an insight requires editor-level access to insights."""
return [("insight", "editor")]
async def _get_insight(self, insight_id: str) -> SavedInsightDefinition:
insight = await database_sync_to_async(saved_insight_for_update)(
team=self._team, user=self._user, short_id=insight_id
)
if insight is None:
raise MaxToolRetryableError(
"Insight not found. insight_id must be the short ID of a saved insight in this project. "
"For an unsaved chart from this conversation, omit insight_id and query_patch and provide a revised "
"structured plan. For a saved insight, use search or read_data to confirm its short ID before retrying."
)
return insight
async def is_dangerous_operation(self, insight_id: str | None = None, **kwargs: Any) -> bool:
return insight_id is not None
async def format_dangerous_operation_preview(
self, insight_id: str | None = None, query_patch: str | None = None, **kwargs: Any
) -> str:
if insight_id is None:
raise MaxToolRetryableError("Provide the short ID of the insight to update.")
insight = await self._get_insight(insight_id)
preview = f"Update insight **{insight.name or insight.short_id}**. This changes every dashboard that uses this insight."
if query_patch is not None:
self._updated_insight_query(insight, query_patch)
preview += f"\n\nRequested query edits:\n```json\n{json.dumps(json.loads(query_patch), indent=2)}\n```"
return preview
async def _check_dangerous_operation(self, kwargs: dict[str, Any]) -> tuple[str, Any] | None:
insight_id = kwargs.get("insight_id")
if insight_id is None:
return await super()._check_dangerous_operation(kwargs)
insight = await self._get_insight(insight_id)
if kwargs.get("expected_query") is None:
kwargs["expected_query"] = insight.query
preview = f"Update insight **{insight.name or insight.short_id}**. This changes every dashboard that uses this insight."
if query_patch := kwargs.get("query_patch"):
self._updated_insight_query(insight, query_patch)
preview += f"\n\nRequested query edits:\n```json\n{json.dumps(json.loads(query_patch), indent=2)}\n```"
return self._handle_dangerous_operation(kwargs, preview=preview)
@staticmethod
def _updated_insight_query(insight: SavedInsightDefinition, query_patch: str) -> dict[str, Any]:
try:
operations = json.loads(query_patch)
if not isinstance(operations, list) or not operations:
raise ValueError("Provide at least one query edit.")
for operation in operations:
if (
not isinstance(operation, dict)
or operation.get("op") not in {"add", "replace", "remove", "move", "test"}
or not isinstance(operation.get("path"), str)
or not operation["path"].startswith("/")
):
raise ValueError("Use add, replace, remove, move or test with a nonempty JSON Pointer path.")
original = insight.query or {}
query = jsonpatch.apply_patch(original, operations)
original_source = original.get("source", original)
source = query.get("source", query)
if query.get("kind") != original.get("kind") or source.get("kind") != original_source.get("kind"):
raise ValueError("Editing an insight cannot change its query type.")
query.pop("response", None)
source.pop("response", None)
QuerySchemaRoot.model_validate(query)
return cast(dict[str, Any], query)
except (
ValueError,
TypeError,
KeyError,
AttributeError,
jsonpatch.JsonPatchException,
JsonPointerException,
ValidationError,
) as error:
raise MaxToolRetryableError(
f"Could not apply the query edits. Read the saved query and retry: {error}"
) from error
async def _save_insight_query(self, insight: SavedInsightDefinition, query: dict[str, Any]) -> None:
error = await database_sync_to_async(save_saved_insight_query)(
team=self._team, user=self._user, insight_id=insight.id, expected_query=insight.query, query=query
)
if error:
raise MaxToolRetryableError(error)
@classmethod
async def create_tool_class(
cls,
*,
team: Team,
user: User,
node_path: tuple[NodePath, ...] | None = None,
state: AssistantState | None = None,
config: RunnableConfig | None = None,
context_manager: AssistantContextManager | None = None,
) -> Self:
context_manager = context_manager or AssistantContextManager(team, user, config)
prompt = format_prompt_string(
INSIGHT_TOOL_PROMPT,
groups=await context_manager.get_group_names(),
)
return cls(team=team, user=user, state=state, node_path=node_path, config=config, description=prompt)
async def _arun_impl(
self,
viz_title: str,
viz_description: str,
query_description: str,
insight_type: InsightType,
insight_id: str | None = None,
query_patch: str | None = None,
expected_query: dict[str, Any] | None = None,
) -> tuple[str, ToolMessagesArtifact | None]:
if insight_id is not None:
if query_patch is None:
raise MaxToolRetryableError("Read the saved insight and provide query_patch with the requested edits.")
insight = await self._get_insight(insight_id)
if expected_query is not None and insight.query != expected_query:
raise MaxToolRetryableError("This insight changed while awaiting approval. Read it again and retry.")
await self._save_insight_query(insight, self._updated_insight_query(insight, query_patch))
return "", ToolMessagesArtifact(
messages=[
ArtifactRefMessage(
id=f"{self.tool_call_id}-saved-insight",
content_type=ArtifactContentType.VISUALIZATION,
source=ArtifactSource.INSIGHT,
artifact_id=insight.short_id,
),
AssistantToolCallMessage(
content=f"Updated existing insight {insight.short_id}. All dashboards using it now use the updated query.",
tool_call_id=self.tool_call_id,
ui_payload={self.get_name(): {"saved_insight": {"short_id": insight.short_id}}},
),
]
)
if query_patch is not None:
raise MaxToolRetryableError("Provide insight_id to edit a saved insight.")
graph_builder = InsightsGraph(self._team, self._user)
match insight_type:
case "trends":
graph_builder.add_trends_generator().add_edge(
AssistantNodeName.START, AssistantNodeName.TRENDS_GENERATOR
)
case "funnel":
graph_builder.add_funnel_generator().add_edge(
AssistantNodeName.START, AssistantNodeName.FUNNEL_GENERATOR
)
case "retention":
graph_builder.add_retention_generator().add_edge(
AssistantNodeName.START, AssistantNodeName.RETENTION_GENERATOR
)
case "sql":
graph_builder.add_sql_generator().add_edge(AssistantNodeName.START, AssistantNodeName.SQL_GENERATOR)
graph = graph_builder.add_query_executor().compile()
new_state = self._state.model_copy(
update={
"root_tool_call_id": self.tool_call_id,
"plan": query_description,
"visualization_title": viz_title,
"visualization_description": viz_description,
},
deep=True,
)
try:
dict_state = await graph.ainvoke(new_state)
except SchemaGenerationException as e:
return format_prompt_string(
INSIGHT_TOOL_HANDLED_FAILURE_PROMPT,
output=e.llm_output,
error_message=e.validation_message,
system_reminder=INSIGHT_TOOL_FAILURE_SYSTEM_REMINDER_PROMPT,
), None
updated_state = AssistantState.model_validate(dict_state)
maybe_viz_message, tool_call_message = updated_state.messages[-2:]
if not isinstance(tool_call_message, AssistantToolCallMessage):
return format_prompt_string(
INSIGHT_TOOL_UNHANDLED_FAILURE_PROMPT, system_reminder=INSIGHT_TOOL_FAILURE_SYSTEM_REMINDER_PROMPT
), None
# If the previous message is not a visualization or artifact message, the agent has requested human feedback.
if not is_visualization_artifact_message(maybe_viz_message):
return "", ToolMessagesArtifact(messages=[tool_call_message])
visualization_content = await self._context_manager.artifacts.aget(
maybe_viz_message.artifact_id, VisualizationArtifactContent
)
# If the contextual tool is available, we're editing an insight.
# Add the UI payload to the tool call message.
if self.is_editing_mode(self._context_manager):
tool_call_message = AssistantToolCallMessage(
content=tool_call_message.content,
ui_payload={self.get_name(): visualization_content.query.model_dump(mode="json", exclude_none=True)},
id=tool_call_message.id,
tool_call_id=tool_call_message.tool_call_id,
)
return "", ToolMessagesArtifact(messages=[maybe_viz_message, tool_call_message])
@classmethod
def is_editing_mode(cls, context_manager: AssistantContextManager) -> bool:
"""
Determines if the tool is in editing mode.
"""
return AssistantTool.CREATE_INSIGHT.value in context_manager.get_contextual_tools()