Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

fix(bi): honor column groups and measure shelves in tables (#125707)

Marius Andra83d19138e75b
diff --git a/docs/published/handbook/engineering/data-warehouse.md b/docs/published/handbook/engineering/data-warehouse.md
index bd86f81eb878..3467e59677b2 100644
--- a/docs/published/handbook/engineering/data-warehouse.md
+++ b/docs/published/handbook/engineering/data-warehouse.md
@@ -129,7 +129,7 @@ The **Marks** card offers table calculations per measure: percent of total, runn
**Analysis → Missing dates** defaults to **Observed points only**. **Show gaps** creates empty buckets that occupy a point in the window but do not contribute to an average. **Fill with zero** makes those buckets contribute zero. Filling runs before calculations, separately for each series and comparison period, across the selected date range. It requires one bucketed date dimension matching the worksheet date column and a bounded range. It is available for project and warehouse tables in tables and time-series charts; direct connections do not support it. Saved worksheets and dashboard date overrides retain these semantics.
-**Table** automatically uses a pivot layout when both **Rows** and **Columns** contain dimensions. Row members run down the table; column members run across it, with every measure beneath each column header. Move all dimensions onto one shelf to return to a flat table. Period comparisons keep their comparison table layout. Direct database connections keep the flat table layout. Large pivots virtualize rows and columns while retaining grouped headers and the sticky row labels. Pivots support up to 10,000 measure columns; wider results ask you to reduce column dimensions or use Top N. Text-valued custom aggregates retain their values. The explicit **Pivot table** chart uses the same pivot layout. Expand or collapse a row or column header to move between hierarchy levels. Collapsed values are reaggregated from source rows, including averages and distinct counts. **Subtotals** also shows the intermediate aggregates while expanded. Row and column grand totals retain their separate controls, and total cells keep drill-down scoped to the visible dimensions. Expansion is local to the view; the worksheet's shelves and settings remain unchanged. Tracking records hierarchy expansion with axis and depth, and adds missing-date and full-window settings to the existing worksheet events.
+**Table** automatically uses a pivot layout when **Columns** contains dimensions, including when **Rows** contains only measures. Row members run down the table; column members run across it. Measures placed on **Rows** appear vertically, and measures placed on **Columns** appear beneath each column header. Measures split between both shelves appear as columns with guidance to place them together. Move all dimensions onto **Rows** to return to a flat table. Period comparisons keep their comparison table layout. Direct database connections keep the flat table layout. Large pivots virtualize rows and columns while retaining grouped headers and the sticky row labels. Pivots support up to 10,000 measure columns and 100,000 displayed rows, including measures on Rows. Larger results ask you to reduce dimensions or measures, or use Top N. Text-valued custom aggregates retain their values. The explicit **Pivot table** chart uses the same pivot layout. Expand or collapse a row or column header to move between hierarchy levels. Collapsed values are reaggregated from source rows, including averages and distinct counts. **Subtotals** also shows the intermediate aggregates while expanded. Row and column grand totals retain their separate controls, and total cells keep drill-down scoped to the visible dimensions. Expansion is local to the view; the worksheet's shelves and settings remain unchanged. Tracking records hierarchy expansion with axis and depth, and adds missing-date and full-window settings to the existing worksheet events.
**Hierarchies** in the data pane names an ordered set of dimensions, such as Country → State → City. Create a hierarchy with 2–10 levels, then start it on rows or columns. The level list marks the current field and shows the next levels. Selecting a result offers **Drill to** the next level in the current editor worksheet, retaining parent filters, date range, measures, variables, and formatting. The path above the result offers **Drill up**, removing only filters introduced by that drill. Navigation participates in undo/redo and respects Auto update. Changed or removed hierarchy levels disable an incompatible return path. Outside the editor, the action explicitly opens a new worksheet. Leaf values and levels already on a shelf do not offer another drill action; nested pivot expansion remains available for those shelves. Definitions stay with the saved worksheet and survive **Clear sheet**. **Import from worksheet** copies a hierarchy from an accessible saved worksheet using the same table and connection. The copy is independent: edits do not change the source worksheet or create global fields. Deleting a local calculated field removes its uses from hierarchy levels, dropping definitions with fewer than two remaining levels.
diff --git a/frontend/snapshots.yml b/frontend/snapshots.yml
index 63c544190b10..d3f5167edbc4 100644
--- a/frontend/snapshots.yml
+++ b/frontend/snapshots.yml
@@ -6747,13 +6747,13 @@ snapshots:
products-business-intelligence-metric-snapshot-context--proposed--light:
hash: v1.k794b7964.f90e05112efecb61d341c0c1b795fac5fb8b633feb0dee221ec6a0f2c1e1fd6f.QHGWgQX5pi-FN6VMPjdDfAFgbfq7cbptsTiFfzC_09E
products-business-intelligence-pivot-table--conditional-formatting--dark:
- hash: v1.k794b7964.8647a315842a7e58e16b9597686a037f51fca27b39cc31a86364531342f0de2d.KI6_i9LRgw64b1jPz_P-_1ghk-QlxGRt8OphgQTyh6o
+ hash: v1.k794b7964.8927e708d4896acb51ff9fa5302f9dd0214abdef779694f9d1a09c352296086a.FgF9lZB76rg6TesSU387LQwIgPk-4D-nKIqm0leCqnQ
products-business-intelligence-pivot-table--conditional-formatting--light:
- hash: v1.k794b7964.3cbbadacb98d1e07b58dc31fdd88e3e69bfe7c74d8512c0cf96afa81092bd4c8.Nk8eKH-PHZdZqcSCV7DoYEjwNNX8DL9nArAgYohDblU
+ hash: v1.k794b7964.4a2fad62464021ea970c5530aca3bb1677ac156af9af38a7f6b50b6baf32ce55.KT2qaQJlZnP2vtaCnHIFMyzEwDvk9gYSjQds27pSZ7k
products-business-intelligence-pivot-table--multiple-measures-and-hierarchies--dark:
- hash: v1.k794b7964.26ebc58171eb340426f9fe220739132bca02f59b8489cb5a0b8a6411f99bb34d.oNv5X9sNKiPmhC_sE607b5P0yHAJju-hcC7higWxX9g
+ hash: v1.k794b7964.04ea7d0a1ce8ad5297116b2bb38b75133d0fdee6193c53dfdc3588be8e04da93.WCi3zbnqRQ05ZksnNUhKZMAe3zZ8E1GhTjcu0rGMp9k
products-business-intelligence-pivot-table--multiple-measures-and-hierarchies--light:
- hash: v1.k794b7964.a536391f02b293d2bdef0f6bb873b8daf6848e3c6bd2fd1b12f248f5847f2a4e.L_mE9SUEZfVSdz4aGS_wbCmwOg9CYDJr9fC5-sEH0ZU
+ hash: v1.k794b7964.c12158891f540bc42e9d49f48b9bb6c89595272cb50c0464f46a7c88ef7c7387.qqAQKnrFBm4FgDyKd3CnYcXP_wupjQvoJD3o0t3rTmM
products-business-intelligence-result-menu--comparison-period--dark:
hash: v1.k794b7964.50272d68c5a1d4568fad5027f5f071a87f5d7a3e41f4e2b0ba0c092b328acc12.mvP0bfecTXSDmdsWb-4q60r34LYfl6EW_2JrJgqmAQM
products-business-intelligence-result-menu--comparison-period--light:
@@ -10883,9 +10883,9 @@ snapshots:
scenes-app-data-warehouse-sql-editor--bi-one-click-comparison--light:
hash: v1.k794b7964.0f90c14cd12a3f4a166687b69dde3516674f4152d70b7db4c7ee5e9ec81269f1.KVtNP2Z7u-Zo5FXH_WFGbtwKRetmCGCKZ7fTBGqg7G0
scenes-app-data-warehouse-sql-editor--bi-pivot-totals--dark:
- hash: v1.k794b7964.a7f2fae9f6e58cc514dc6b4ea7070928c8045be0f863eb86496ec99975f1795a.G3UiEpCP_ElGHUVHgL2bR-eweievpW4WF2HBSyVqF_c
+ hash: v1.k794b7964.68eed15af08bc6bafbfb93a6498d1e56a1ae20266b05b2884e890594438285df.XwDJzh_0PzpOcDcifgpWI488LWBSgg6phJUEk10SQxU
scenes-app-data-warehouse-sql-editor--bi-pivot-totals--light:
- hash: v1.k794b7964.d2c7e0bf74821578bb45b1f42ce92d12096b32b1320bb84dd991f9cd6a5b6851.tz00BuUNFzAY78a1Dj2q22bk0NkjHiIg26aZg2qKSlE
+ hash: v1.k794b7964.f471b9cfe8d9c08351377dd1bf21a92d5d1cddc2df92b5959509f9b89719232a.Y3DSxUC3noFECYSVLGJRvJXVU_0jfXCT2vV4I_AtmMc
scenes-app-data-warehouse-sql-editor--bi-quick-filters--dark:
hash: v1.k794b7964.be74d6e8accc2887e029a462983d1aa4895a64488e5e77558643988d1c5383d0.XDdSVpGEvgCkmdqGEpRkzla9SsQkQSw8jV0Du7HxtJY
scenes-app-data-warehouse-sql-editor--bi-quick-filters--light:
@@ -10903,13 +10903,13 @@ snapshots:
scenes-app-data-warehouse-sql-editor--bi-source-preview-large-sample--light:
hash: v1.k794b7964.ddb42f5556e7e09b1219e54d68f83213e8737ebbde55266a4dd1427d9cab5326.S77Umy6YW9HB314IPWIm5yyYg_G0KpQKpBoG99A-iBE
scenes-app-data-warehouse-sql-editor--bi-table-analysis--dark:
- hash: v1.k794b7964.8e5e14fc64a4db188055d8c1f86db24f7141b3e9bd25bba821070231ea99f53d.nYQuEegEy1vfbcvOZBhML27nhwtWKB6YNpLr5T5ulMg
+ hash: v1.k794b7964.f5c301e216273b7a0cfd59190ca46159641bc991fab8c31ffee7a2704f898314.PpMisf1ay9ifGxpH0U8JkqEngZjzafiuUSAC8rpfDqY
scenes-app-data-warehouse-sql-editor--bi-table-analysis--light:
- hash: v1.k794b7964.222855798a3123f816a945a51c85bc3ba4654e7495bf839b40642279abfcf188.ee_DmapFr84nuFWe5fEF24YDFD2qSppU-8jwlYEc4Xk
+ hash: v1.k794b7964.20cd5d30e53afdf4eb33456f4c8ed5a60ef1ec6f3f9676a19c4e8610fda0e9fe.JOi3COiabs5Jx_fwm729UTyY4RuE9EoSsCMphr3RG94
scenes-app-data-warehouse-sql-editor--bi-wide-transposed-results--dark:
- hash: v1.k794b7964.6a420f7474558367a8c3e4ed5d3fa96f7d51edb3b4f7a6cac4e9af1b5cc0e531.Fcz42DazeV4XO1xHNi5h6puOzCUrftQY5jVIeXZut8k
+ hash: v1.k794b7964.50ee98abf001c7c2fbea46f2e8f713e5644d5935117e0e776fbb79b6ce575ccb.nI7fd5MkElCJ3yCb_vlTOn0CWiAH8LiNz2mhHpeVtnA
scenes-app-data-warehouse-sql-editor--bi-wide-transposed-results--light:
- hash: v1.k794b7964.ca232e5bde8705f79816e729e77216e37fec62eb32164484e2364620b0d86dde.8AHKAl3ynJlU82UXlaJAAgcZsHhGw5-O1ZkAC5XVugE
+ hash: v1.k794b7964.0eafe803214c84bb135a4387533442f6f8c1525c100d98bdde0fa33ce4237413.NjZStKhVDecpAV0K-LuY4T3Ja1SdDNYM8hY2hsKMWwU
scenes-app-data-warehouse-sql-editor--customer-images--dark:
hash: v1.k794b7964.10b9dd73c78556d1b3990ff4ba8d0872e17f375ed655c487b3ca9ca8af21c665.K_W25GnvkQUo791SarbtGIBFaySZ0v_Ebd4cv6sJhsQ
scenes-app-data-warehouse-sql-editor--customer-images--light:
diff --git a/frontend/src/scenes/data-warehouse/editor/SQLEditorScene.stories.tsx b/frontend/src/scenes/data-warehouse/editor/SQLEditorScene.stories.tsx
index 5665fdc9ed2c..b2e19306034e 100644
--- a/frontend/src/scenes/data-warehouse/editor/SQLEditorScene.stories.tsx
+++ b/frontend/src/scenes/data-warehouse/editor/SQLEditorScene.stories.tsx
@@ -1999,8 +1999,8 @@ export const BISourcePreviewLargeSample: Story = {
const BI_WIDE_TABLE_CONFIG: BIConfig = {
...BI_WORKSHEET_CONFIG,
chartType: ChartDisplayType.ActionsTable,
- rows: [],
- columns: [{ ...biEventsField('timestamp', 'datetime'), dateBucket: 'day' }],
+ rows: [{ ...biEventsField('timestamp', 'datetime'), dateBucket: 'day' }],
+ columns: [],
values: [{ field: biEventsField('revenue', 'float'), aggregation: 'sum', shelf: 'rows' }],
filters: [],
limit: 10000,
diff --git a/products/business_intelligence/frontend/BIPivotTable.test.tsx b/products/business_intelligence/frontend/BIPivotTable.test.tsx
index d5b19cb0334d..6f76599dd2f2 100644
--- a/products/business_intelligence/frontend/BIPivotTable.test.tsx
+++ b/products/business_intelligence/frontend/BIPivotTable.test.tsx
@@ -29,6 +29,61 @@ describe('BIPivotTable', () => {
jest.restoreAllMocks()
})
+ it.each(['rows', 'columns'] as const)(
+ 'places measures on %s without row dimensions and preserves drill-down',
+ (shelf) => {
+ jest.spyOn(HTMLElement.prototype, 'offsetHeight', 'get').mockReturnValue(400)
+ jest.spyOn(HTMLElement.prototype, 'offsetWidth', 'get').mockReturnValue(800)
+ const config: BIConfig = {
+ ...DEFAULT_BI_CONFIG,
+ chartType: ChartDisplayType.ActionsTable,
+ rows: [],
+ columns: [field('channel')],
+ values: [
+ {
+ field: field('amount'),
+ aggregation: 'sum',
+ shelf,
+ display: { label: 'Revenue' },
+ formatting: { prefix: '$' },
+ },
+ { field: field('orders'), aggregation: 'count', shelf, display: { label: 'Orders' } },
+ ],
+ }
+ const onInspect = jest.fn()
+ const { container } = render(
+ <BIPivotTable
+ config={config}
+ columns={['bi_column_channel', 'sum_amount', 'count_orders_2']}
+ results={[
+ [null, 12, 3],
+ ['Search', 8, 2],
+ ]}
+ onInspect={onInspect}
+ cellBackground={(_, column) => (column === 'sum_amount' ? '#FFADAD' : undefined)}
+ />
+ )
+ const body = container.querySelector('tbody')!
+ expect(
+ Array.from(body.querySelectorAll('tr')).map((row) =>
+ Array.from(row.querySelectorAll('td')).map((cell) => cell.textContent)
+ )
+ ).toEqual(
+ shelf === 'rows'
+ ? [
+ ['Revenue', '$12', '$8'],
+ ['Orders', '3', '2'],
+ ]
+ : [['', '$12', '3', '$8', '2']]
+ )
+ expect(
+ Array.from(body.querySelectorAll<HTMLElement>('td')).filter((cell) => cell.style.backgroundColor)
+ ).toHaveLength(2)
+ fireEvent.click(screen.getByText('$12'))
+ expect(onInspect).toHaveBeenCalledWith({ bi_column_channel: null, sum_amount: 12, count_orders_2: 3 })
+ }
+ )
+
it.each<[boolean, string | number, string | number, ColumnScalar]>([
[false, 10, 20, 'INTEGER'],
[true, 10, 20, 'INTEGER'],
@@ -88,87 +143,155 @@ describe('BIPivotTable', () => {
}
)
- it('windows a large sparse pivot on both axes and inspects the last cell', () => {
+ it.each(['rows', 'columns'] as const)(
+ 'windows a sparse pivot with measures on %s and inspects the last cell',
+ (shelf) => {
+ jest.spyOn(HTMLElement.prototype, 'offsetHeight', 'get').mockImplementation(
+ function (this: HTMLElement): number {
+ return this.tagName === 'TR' ? 36 : 180
+ }
+ )
+ jest.spyOn(HTMLElement.prototype, 'offsetWidth', 'get').mockReturnValue(800)
+ const config = {
+ ...DEFAULT_BI_CONFIG,
+ chartType: ChartDisplayType.ActionsTable,
+ rows: [field('region')],
+ columns: [field('channel')],
+ values: [
+ { field: field('amount'), aggregation: 'sum' as const, shelf },
+ {
+ field: field('event'),
+ aggregation: 'custom' as const,
+ shelf,
+ customExpression: 'argMax(event, timestamp)',
+ },
+ ],
+ }
+ const results = [
+ ...Array.from({ length: 600 }, (_, index) => [`Region ${index}`, 'Channel 0', index, 'view']),
+ ...Array.from({ length: 89 }, (_, index) => ['Region 0', `Channel ${index + 1}`, index + 1, 'view']),
+ ['Region 599', 'Channel 89', 59989, 'purchase'],
+ ]
+ const onInspect = jest.fn()
+ const { container } = render(
+ <div style={{ height: 180 }}>
+ <BIPivotTable
+ config={config}
+ columns={['bi_row_region', 'bi_column_channel', 'sum_amount', 'custom_event_2']}
+ results={results}
+ onInspect={onInspect}
+ embedded
+ />
+ </div>
+ )
+ const scroll = container.querySelector('.ScrollableShadows__inner')!
+ expect(screen.getAllByText('Region 0')[0]).toBeVisible()
+ expect(screen.queryByText('Region 599')).not.toBeInTheDocument()
+ expect(container.querySelectorAll('tbody td').length).toBeLessThan(250)
+
+ fireEvent.scroll(scroll, {
+ target: {
+ scrollLeft: (shelf === 'rows' ? 91 : 181) * 192 - 800,
+ scrollTop: 600 * (shelf === 'rows' ? 2 : 1) * 36,
+ },
+ })
+
+ expect(screen.getAllByText('Region 599')[0]).toBeVisible()
+ expect(screen.queryByText('Region 0')).not.toBeInTheDocument()
+ expect(screen.getByText('59989')).toBeVisible()
+ const lastCell = screen.getByText('purchase')
+ fireEvent.click(lastCell)
+ expect(onInspect).toHaveBeenCalledWith({
+ bi_row_region: 'Region 599',
+ bi_column_channel: 'Channel 89',
+ sum_amount: 59989,
+ custom_event_2: 'purchase',
+ })
+ expect(container.querySelectorAll('tbody td').length).toBeLessThan(250)
+ }
+ )
+ it('preserves vertical scroll when collapsing and expanding a column hierarchy', () => {
jest.spyOn(HTMLElement.prototype, 'offsetHeight', 'get').mockImplementation(
function (this: HTMLElement): number {
return this.tagName === 'TR' ? 36 : 180
}
)
jest.spyOn(HTMLElement.prototype, 'offsetWidth', 'get').mockReturnValue(800)
- const config = {
+ const config: BIConfig = {
...DEFAULT_BI_CONFIG,
chartType: ChartDisplayType.ActionsTable,
rows: [field('region')],
- columns: [field('channel')],
- values: [
- { field: field('amount'), aggregation: 'sum' as const },
- { field: field('event'), aggregation: 'custom' as const, customExpression: 'argMax(event, timestamp)' },
- ],
+ columns: [field('year'), field('quarter')],
+ values: [{ field: field('amount'), aggregation: 'sum', shelf: 'rows' }],
}
- const results = [
- ...Array.from({ length: 600 }, (_, index) => [`Region ${index}`, 'Channel 0', index, 'view']),
- ...Array.from({ length: 89 }, (_, index) => ['Region 0', `Channel ${index + 1}`, index + 1, 'view']),
- ['Region 599', 'Channel 89', 59989, 'purchase'],
- ]
- const onInspect = jest.fn()
const { container } = render(
- <div style={{ height: 180 }}>
- <BIPivotTable
- config={config}
- columns={['bi_row_region', 'bi_column_channel', 'sum_amount', 'custom_event_2']}
- results={results}
- onInspect={onInspect}
- embedded
- />
- </div>
+ <BIPivotTable
+ config={config}
+ columns={['bi_row_region', 'bi_columns', 'sum_amount']}
+ results={Array.from({ length: 600 }, (_, index) => [
+ `Region ${index}`,
+ JSON.stringify(['2026', 'Q1']),
+ index,
+ ])}
+ />
)
- const scroll = container.querySelector('.ScrollableShadows__inner')!
- expect(screen.getByText('Region 0')).toBeVisible()
- expect(screen.queryByText('Region 599')).not.toBeInTheDocument()
- expect(container.querySelectorAll('tbody td').length).toBeLessThan(250)
-
- fireEvent.scroll(scroll, { target: { scrollLeft: 181 * 192 - 800, scrollTop: 600 * 36 } })
-
- expect(screen.getByText('Region 599')).toBeVisible()
- expect(screen.queryByText('Region 0')).not.toBeInTheDocument()
- expect(screen.getByText('59989')).toBeVisible()
- const lastCell = screen.getByText('purchase')
- fireEvent.click(lastCell)
- expect(onInspect).toHaveBeenCalledWith({
- bi_row_region: 'Region 599',
- bi_column_channel: 'Channel 89',
- sum_amount: 59989,
- custom_event_2: 'purchase',
- })
- expect(container.querySelectorAll('tbody td').length).toBeLessThan(250)
- })
- it('keeps the last column reachable at the width limit and rejects wider pivots', () => {
- jest.spyOn(HTMLElement.prototype, 'offsetHeight', 'get').mockReturnValue(180)
- jest.spyOn(HTMLElement.prototype, 'offsetWidth', 'get').mockReturnValue(800)
- const config: BIConfig = {
- ...DEFAULT_BI_CONFIG,
- chartType: ChartDisplayType.ActionsTable,
- rows: [field('region')],
- columns: [field('channel')],
- values: [
- { field: field('amount'), aggregation: 'sum' },
- { field: field('cost'), aggregation: 'sum' },
- ],
+ const scroll = container.querySelector<HTMLElement>('.ScrollableShadows__inner')!
+ scroll.scrollTo = (options?: ScrollToOptions | number): void => {
+ if (typeof options === 'object' && options.top !== undefined) {
+ scroll.scrollTop = options.top
+ }
+ }
+ fireEvent.scroll(scroll, { target: { scrollTop: 300 * 36 } })
+ for (const action of ['Collapse', 'Expand']) {
+ fireEvent.click(screen.getByLabelText(`${action} 2026`))
+ expect(scroll.scrollTop).toBe(300 * 36)
}
- const results = Array.from({ length: 5001 }, (_, index) => ['East', `Channel ${index}`, index, index + 10000])
- const props = { config, columns: ['bi_row_region', 'bi_column_channel', 'sum_amount', 'sum_cost_2'] }
- const { container, rerender } = render(<BIPivotTable {...props} results={results.slice(0, 5000)} />)
- const scroll = container.querySelector('.ScrollableShadows__inner')!
- fireEvent.scroll(scroll, { target: { scrollLeft: 10001 * 192 - 800 } })
- expect(screen.getByText('14999')).toBeVisible()
- expect(container.querySelectorAll('tbody td').length).toBeLessThan(30)
+ })
- rerender(<BIPivotTable {...props} results={results} />)
- expect(
- screen.getByText(
- 'This pivot has more than 10,000 measure columns. Reduce the column dimensions or use Top N.'
+ it.each(['rows', 'columns'] as const)(
+ 'keeps the last cell reachable at the %s limit and rejects larger pivots',
+ (shelf) => {
+ jest.spyOn(HTMLElement.prototype, 'offsetHeight', 'get').mockImplementation(
+ function (this: HTMLElement): number {
+ return this.tagName === 'TR' ? 36 : 180
+ }
)
- ).toBeVisible()
- expect(container.querySelector('table')).not.toBeInTheDocument()
- })
+ jest.spyOn(HTMLElement.prototype, 'offsetWidth', 'get').mockReturnValue(800)
+ const config: BIConfig = {
+ ...DEFAULT_BI_CONFIG,
+ chartType: ChartDisplayType.ActionsTable,
+ rows: [field('region')],
+ columns: [field('channel')],
+ values: [
+ { field: field('amount'), aggregation: 'sum', shelf },
+ { field: field('cost'), aggregation: 'sum', shelf },
+ ],
+ }
+ const memberLimit = shelf === 'rows' ? 50000 : 5000
+ const results = Array.from({ length: memberLimit + 1 }, (_, index) => [
+ shelf === 'rows' ? `Region ${index}` : 'East',
+ shelf === 'columns' ? `Channel ${index}` : 'Email',
+ index,
+ index + 10000,
+ ])
+ const props = { config, columns: ['bi_row_region', 'bi_column_channel', 'sum_amount', 'sum_cost_2'] }
+ const { container, rerender } = render(<BIPivotTable {...props} results={results.slice(0, memberLimit)} />)
+ const scroll = container.querySelector('.ScrollableShadows__inner')!
+ fireEvent.scroll(scroll, {
+ target: shelf === 'rows' ? { scrollTop: 100000 * 36 } : { scrollLeft: 10001 * 192 - 800 },
+ })
+ expect(screen.getByText(String(memberLimit + 9999))).toBeVisible()
+ expect(container.querySelectorAll('tbody td').length).toBeLessThan(100)
+
+ rerender(<BIPivotTable {...props} results={results} />)
+ expect(
+ screen.getByText(
+ shelf === 'rows'
+ ? 'This pivot has more than 100,000 rows. Reduce the row dimensions or measures, or use Top N.'
+ : 'This pivot has more than 10,000 measure columns. Reduce the column dimensions or use Top N.'
+ )
+ ).toBeVisible()
+ expect(container.querySelector('table')).not.toBeInTheDocument()
+ }
+ )
})
diff --git a/products/business_intelligence/frontend/BIPivotTable.tsx b/products/business_intelligence/frontend/BIPivotTable.tsx
index ea803ce07741..d58a8b7ebe75 100644
--- a/products/business_intelligence/frontend/BIPivotTable.tsx
+++ b/products/business_intelligence/frontend/BIPivotTable.tsx
@@ -12,8 +12,15 @@ import { getBIFieldPillLabel } from './biEditorTypes'
import { biPivotCellKey, biPivotKey, BIPivotMember, buildBIPivotModel, visibleBIPivotMembers } from './biPivot'
import { formatBIMeasure, getBIResultMeasures } from './biResultPresentation'
-// Virtualization still needs a full-width scroll area, which browsers cap even with only a few cells mounted.
+// Virtualization still needs a full-size scroll area, which browsers cap even with only a few cells mounted.
const MAX_PIVOT_MEASURE_COLUMNS = 10000
+const MAX_PIVOT_ROWS = 100000
+
+interface BIPivotTableRow {
+ key: string
+ member: BIPivotMember
+ measure?: ReturnType<typeof getBIResultMeasures>[number]
+}
export function BIPivotTable({
config,
@@ -41,10 +48,33 @@ export function BIPivotTable({
() => new Map(visibleBIPivotMembers(model.columns, new Set(), true).map((member) => [member.key, member])),
[model.columns]
)
- const measures = getBIResultMeasures(config, chartSettings)
+ const measures = useMemo(() => getBIResultMeasures(config, chartSettings), [config, chartSettings])
+ const measuresOnRows = config.values.every((value) => value.shelf !== 'columns')
+ const mixedMeasureShelves = !measuresOnRows && config.values.some((value) => value.shelf !== 'columns')
+ const measuresPerColumn = measuresOnRows ? 1 : measures.length
const rows = useMemo(() => visibleBIPivotMembers(model.rows, collapsedRows, true), [model.rows, collapsedRows])
const visibleColumns = visibleBIPivotMembers(model.columns, collapsedColumns, !!config.totals?.subtotals)
- if (visibleColumns.length * measures.length > MAX_PIVOT_MEASURE_COLUMNS) {
+ const exceedsRowLimit = rows.length * (measuresOnRows ? measures.length : 1) > MAX_PIVOT_ROWS
+ const exceedsColumnLimit = visibleColumns.length * measuresPerColumn > MAX_PIVOT_MEASURE_COLUMNS
+ const tableRows = useMemo<BIPivotTableRow[]>(
+ () =>
+ exceedsRowLimit || exceedsColumnLimit
+ ? []
+ : rows.flatMap((member) =>
+ measuresOnRows
+ ? measures.map((measure) => ({ key: `${member.key}:${measure.column}`, member, measure }))
+ : [{ key: member.key, member }]
+ ),
+ [rows, measures, measuresOnRows, exceedsRowLimit, exceedsColumnLimit]
+ )
+ if (exceedsRowLimit) {
+ return (
+ <LemonBanner type="info">
+ This pivot has more than 100,000 rows. Reduce the row dimensions or measures, or use Top N.
+ </LemonBanner>
+ )
+ }
+ if (exceedsColumnLimit) {
return (
<LemonBanner type="info">
This pivot has more than 10,000 measure columns. Reduce the column dimensions or use Top N.
@@ -95,12 +125,12 @@ export function BIPivotTable({
const key = biPivotKey(path)
const previous = cells[cells.length - 1]
if (previous.key === key) {
- previous.colSpan += measures.length
+ previous.colSpan += measuresPerColumn
} else {
const member = columnMembers.get(key)
cells.push({
key,
- colSpan: measures.length,
+ colSpan: measuresPerColumn,
title:
depth < column.path.length && member
? header(member, 'columns')
@@ -116,19 +146,24 @@ export function BIPivotTable({
})
return (
<div className="h-full w-full min-h-0 min-w-0 flex flex-col" data-attr="bi-pivot-table">
+ {mixedMeasureShelves && (
+ <LemonBanner type="info">
+ Measures on both shelves are shown as columns. Move all measures to Rows to show them vertically.
+ </LemonBanner>
+ )}
{formattingLoadFailed && (
<LemonBanner type="warning">
Couldn't load conditional formatting. Reload the page to try again.
</LemonBanner>
)}
- <LemonTable<BIPivotMember>
+ <LemonTable<BIPivotTableRow>
className="DataVisualizationTable"
embedded={embedded}
allowContentScroll
virtualized
- virtualizedColumnWidth={visibleColumns.length * measures.length > 50 ? 192 : undefined}
+ virtualizedColumnWidth={visibleColumns.length * measuresPerColumn > 50 ? 192 : undefined}
rowKey="key"
- dataSource={rows}
+ dataSource={tableRows}
useURLForSorting={false}
firstColumnSticky
uppercaseHeader={false}
@@ -137,25 +172,31 @@ export function BIPivotTable({
columns={[
{
key: 'rows',
- title: config.rows.map(getBIFieldPillLabel).join(' / '),
- render: (_, member) => (
+ title: [...config.rows.map(getBIFieldPillLabel), ...(measuresOnRows ? ['Measure'] : [])].join(
+ ' / '
+ ),
+ render: (_, { member, measure }) => (
<div className="flex items-center gap-1 whitespace-nowrap">
{member.path.slice(1).map((_, index) => (
<span key={index} className="w-3 shrink-0" />
))}
{header(member, 'rows')}
+ {measure && <span>{measure.label}</span>}
</div>
),
},
...visibleColumns.flatMap((column) =>
- measures.map((measure) => {
- let cachedRow: BIPivotMember | undefined
+ (measuresOnRows ? [undefined] : measures).map((columnMeasure) => {
+ let cachedRow: BIPivotTableRow | undefined
let background: string | undefined
- const getBackground = (row: BIPivotMember): string | undefined => {
+ const getBackground = (row: BIPivotTableRow): string | undefined => {
if (cachedRow !== row) {
- const record = model.cells.get(biPivotCellKey(row.path, column.path))
+ const measure = row.measure ?? columnMeasure!
+ const record = model.cells.get(biPivotCellKey(row.member.path, column.path))
const showValue =
- !row.children.length || collapsedRows.has(row.key) || config.totals?.subtotals
+ !row.member.children.length ||
+ collapsedRows.has(row.member.key) ||
+ config.totals?.subtotals
background =
record && showValue ? cellBackground?.(record, measure.column) : undefined
cachedRow = row
@@ -163,18 +204,21 @@ export function BIPivotTable({
return background
}
return {
- key: `${column.key}:${measure.column}`,
- title: measure.label,
+ key: `${column.key}:${columnMeasure?.column ?? 'values'}`,
+ title: columnMeasure?.label ?? '',
align: 'right' as const,
- style: (_: unknown, row: BIPivotMember) => ({ backgroundColor: getBackground(row) }),
- className: (_: unknown, row: BIPivotMember) => {
+ style: (_: unknown, row: BIPivotTableRow) => ({ backgroundColor: getBackground(row) }),
+ className: (_: unknown, row: BIPivotTableRow) => {
const color = getBackground(row)
return color ? getContrastingTextClass(color) : ''
},
- render: (_: unknown, row: BIPivotMember) => {
- const record = model.cells.get(biPivotCellKey(row.path, column.path))
+ render: (_: unknown, row: BIPivotTableRow) => {
+ const measure = row.measure ?? columnMeasure!
+ const record = model.cells.get(biPivotCellKey(row.member.path, column.path))
const showValue =
- !row.children.length || collapsedRows.has(row.key) || config.totals?.subtotals
+ !row.member.children.length ||
+ collapsedRows.has(row.member.key) ||
+ config.totals?.subtotals
const label = formatBIMeasure(
showValue ? record?.[measure.column] : null,
measure.settings
diff --git a/products/business_intelligence/frontend/biAnalysis.test.ts b/products/business_intelligence/frontend/biAnalysis.test.ts
index 026d40824252..b1ce3491f549 100644
--- a/products/business_intelligence/frontend/biAnalysis.test.ts
+++ b/products/business_intelligence/frontend/biAnalysis.test.ts
@@ -221,6 +221,26 @@ describe('BI analysis queries', () => {
expect(parseBIEditorState(BIEditorView.BI, worksheet)?.config.totals).toEqual(worksheet.totals)
})
+ it.each([false, true])('reaggregates column-only hierarchies with grand totals=%s', (grandTotals) => {
+ const worksheet: BIConfig = {
+ ...config,
+ chartType: ChartDisplayType.ActionsTable,
+ rows: [],
+ columns: [field('region'), field('channel')],
+ values: [{ field: field('amount', 'float'), aggregation: 'average' }],
+ totals: { rows: grandTotals },
+ }
+ const query = buildBIQuery(worksheet)!.query
+ expect(query).toContain('avg(amount) AS average_amount')
+ expect(query).toContain(
+ grandTotals
+ ? 'GROUPING SETS ((bi_column_region, bi_column_channel_2), (), (bi_column_region))'
+ : 'GROUPING SETS ((bi_column_region, bi_column_channel_2), (bi_column_region))'
+ )
+ expect(query).toContain('AS bi_columns')
+ expect(query).not.toContain('AS bi_rows')
+ })
+
it('reserves detail rows and sorts numeric dimensions before their total labels are formatted', () => {
const result = buildBIQuery({
...config,
diff --git a/products/business_intelligence/frontend/biAnalysis.ts b/products/business_intelligence/frontend/biAnalysis.ts
index e94744772a2c..7d46877247be 100644
--- a/products/business_intelligence/frontend/biAnalysis.ts
+++ b/products/business_intelligence/frontend/biAnalysis.ts
@@ -160,8 +160,8 @@ function groupingSets(config: BIConfig, input: BIAnalysisInput): string[][] {
}
if (pivot) {
// Reaggregate every hierarchy intersection: averages and distinct counts cannot be summed from leaves.
- for (let rowDepth = 1; rowDepth <= rows.length; rowDepth++) {
- for (let columnDepth = 1; columnDepth <= columns.length; columnDepth++) {
+ for (let rowDepth = Math.min(1, rows.length); rowDepth <= rows.length; rowDepth++) {
+ for (let columnDepth = Math.min(1, columns.length); columnDepth <= columns.length; columnDepth++) {
sets.push([...rows.slice(0, rowDepth), ...columns.slice(0, columnDepth)])
}
if (config.totals?.rows) {
diff --git a/products/business_intelligence/frontend/biPivot.test.ts b/products/business_intelligence/frontend/biPivot.test.ts
index c1e94fd0d9ab..e7be186d8125 100644
--- a/products/business_intelligence/frontend/biPivot.test.ts
+++ b/products/business_intelligence/frontend/biPivot.test.ts
@@ -106,12 +106,31 @@ it('pivots table members by shelf and restores flat results when dimensions retu
expect(isBIPivotTable(flat)).toBe(false)
expect(getBIResultDimensions(flat).map(({ column }) => column)).toEqual(['region', 'channel'])
expect(buildBIQuery(flat)?.query).not.toContain('bi_current')
+
+ const columnsOnly = { ...config, rows: [], columns: [region] }
+ const columnResults = [
+ ['north', 20, 5],
+ ['south', 9, 1],
+ ]
+ const horizontal = buildBIPivotModel(
+ columnsOnly,
+ [...getBIResultDimensions(columnsOnly).map(({ column }) => column), 'sum_amount', 'count_orders_2'],
+ columnResults
+ )
+ expect(isBIPivotTable(columnsOnly)).toBe(true)
+ expect(horizontal.rows.map(({ path, label }) => ({ path, label }))).toEqual([{ path: [], label: '' }])
+ expect(horizontal.columns.map(({ label }) => label)).toEqual(['north', 'south'])
+ const north = horizontal.cells.get(biPivotCellKey([], ['north']))!
+ expect(north).toMatchObject({ sum_amount: 20, count_orders_2: 5 })
+ expect(getBIDrillSelection(columnsOnly, north).filters.map(({ field, value }) => [field.name, value])).toEqual([
+ ['region', 'north'],
+ ])
+ expect(buildBIPivotModel(columnsOnly, [], []).rows).toEqual([])
})
it.each([
{ chartType: ChartDisplayType.ActionsLineGraph },
{ columns: [] },
- { rows: [] },
{ columns: [field(' ')] },
{ compareFilter: { compare: true } },
{ source: { table: 'orders', connectionId: 'example-connection' } },
diff --git a/products/business_intelligence/frontend/biPivot.ts b/products/business_intelligence/frontend/biPivot.ts
index 6ec3c64e14c8..fda0d5d17e81 100644
--- a/products/business_intelligence/frontend/biPivot.ts
+++ b/products/business_intelligence/frontend/biPivot.ts
@@ -52,10 +52,14 @@ export function buildBIPivotModel(config: BIConfig, columns: string[], results:
}
const addPath = (path: unknown[], side: 'rows' | 'columns'): void => {
if (!path.length) {
- if (!(side === 'rows' ? config.totals?.columns : config.totals?.rows) || members[side].has('[]')) {
+ const hasDimensions = (side === 'rows' ? rowCount : columnCount) > 0
+ if (
+ (hasDimensions && !(side === 'rows' ? config.totals?.columns : config.totals?.rows)) ||
+ members[side].has('[]')
+ ) {
return
}
- const total = { key: '[]', path, label: 'Total', children: [] }
+ const total = { key: '[]', path, label: hasDimensions ? 'Total' : '', children: [] }
members[side].set('[]', total)
model[side].push(total)
}
diff --git a/products/business_intelligence/frontend/biTableLayout.ts b/products/business_intelligence/frontend/biTableLayout.ts
index c6566ddd63a6..8749989e6219 100644
--- a/products/business_intelligence/frontend/biTableLayout.ts
+++ b/products/business_intelligence/frontend/biTableLayout.ts
@@ -8,7 +8,6 @@ export function isBIPivotTable(config: BIConfig): boolean {
(config.chartType === ChartDisplayType.ActionsTable &&
!config.compareFilter?.compare &&
!config.source?.connectionId &&
- config.rows.some((field) => field.expression.trim() || field.name.trim()) &&
config.columns.some((field) => field.expression.trim() || field.name.trim())))
)
}