Skip to content

A local-first personal finance app

monte carlo block bootstrap (#9101)

Michael Clarkcef7e63bf705
diff --git a/packages/desktop-client/src/components/reports/reports/monte-carlo/MonteCarlo.tsx b/packages/desktop-client/src/components/reports/reports/monte-carlo/MonteCarlo.tsx
index 3bd4e7710b..0cdcf11b2d 100644
--- a/packages/desktop-client/src/components/reports/reports/monte-carlo/MonteCarlo.tsx
+++ b/packages/desktop-client/src/components/reports/reports/monte-carlo/MonteCarlo.tsx
@@ -752,6 +752,19 @@ export function MonteCarlo() {
unusually good, so results may be optimistic for globally
diversified portfolios.
</Trans>
+ ) : config.returnModel === 'historical-block-bootstrap' ? (
+ <Trans>
+ Each scenario chains together blocks of consecutive actual US
+ market years ({{ firstYear }}&ndash;{{ lastYear }}, S&amp;P 500
+ / 10-year Treasuries / T-bills, Damodaran data): each year
+ usually carries on to the next real year, and now and then jumps
+ to a different point in history, wrapping around the end of the
+ data. Each year brings its own actual US inflation with it.
+ Crashes keep the years that followed them within a block, but
+ patterns are broken where blocks join, fees and taxes are only
+ as accurate as the rates you enter, and US history may be
+ optimistic for globally diversified portfolios.
+ </Trans>
) : (
<Trans>
Each scenario replays actual US market history ({{ firstYear }}
diff --git a/packages/desktop-client/src/components/reports/reports/monte-carlo/MonteCarloConfiguration.tsx b/packages/desktop-client/src/components/reports/reports/monte-carlo/MonteCarloConfiguration.tsx
index fadfc28424..d36cd9cbf0 100644
--- a/packages/desktop-client/src/components/reports/reports/monte-carlo/MonteCarloConfiguration.tsx
+++ b/packages/desktop-client/src/components/reports/reports/monte-carlo/MonteCarloConfiguration.tsx
@@ -28,8 +28,11 @@ import { MonteCarloPotsTableHeader } from '#components/reports/reports/monte-car
import {
createMonteCarloPot,
createMonteCarloSurplusPot,
+ DEFAULT_HISTORICAL_BLOCK_LENGTH,
getMonteCarloPotLabel,
+ MAX_HISTORICAL_BLOCK_LENGTH,
MAX_SIMULATION_COUNT,
+ MIN_HISTORICAL_BLOCK_LENGTH,
MIN_SIMULATION_COUNT,
MONTE_CARLO_DEFAULTS,
} from '#components/reports/reports/monte-carlo/monteCarloSimulation';
@@ -346,6 +349,12 @@ export function MonteCarloConfiguration({
(1928 onwards) in random order.
<br />
<br />
+ Historical, in blocks: strings together runs of
+ consecutive real years, so a crash keeps the years that
+ followed it, while every scenario is still a different mix
+ of history.
+ <br />
+ <br />
Historical sequences: replays real market history, one
scenario per starting year. Pots with a Custom allocation
always use their own return and volatility.
@@ -362,6 +371,10 @@ export function MonteCarloConfiguration({
options={[
['normal', t('Random (normal distribution)')],
['historical-bootstrap', t('Historical returns, shuffled')],
+ [
+ 'historical-block-bootstrap',
+ t('Historical returns, in blocks'),
+ ],
['historical-sequence', t('Historical sequences (replay)')],
]}
/>
@@ -395,6 +408,43 @@ export function MonteCarloConfiguration({
}
/>
</View>
+
+ {config.returnModel === 'historical-block-bootstrap' && (
+ <View style={FIELD_STYLE}>
+ <View style={FIELD_LABEL_ROW_STYLE}>
+ <Text style={FIELD_LABEL_STYLE}>
+ <Trans>Block length (years)</Trans>
+ </Text>
+ <MonteCarloHelpTooltip>
+ <Trans>
+ How many consecutive real years are replayed, on
+ average, before the scenario jumps to a different point
+ in history. Longer blocks keep more of history&apos;s
+ multi-year patterns, such as a crash and its recovery;
+ shorter blocks mix history up more.
+ <br />
+ <br />
+ Around 10 years is a common choice. 1 behaves like the
+ shuffled model.
+ </Trans>
+ </MonteCarloHelpTooltip>
+ </View>
+ <MonteCarloNumberInput
+ value={config.historicalBlockLength}
+ aria-label={t('Block length (years)')}
+ roundToInteger
+ min={MIN_HISTORICAL_BLOCK_LENGTH}
+ max={MAX_HISTORICAL_BLOCK_LENGTH}
+ step={1}
+ onCommit={newValue =>
+ onConfigChange({
+ historicalBlockLength:
+ newValue ?? DEFAULT_HISTORICAL_BLOCK_LENGTH,
+ })
+ }
+ />
+ </View>
+ )}
</View>
</View>
diff --git a/packages/desktop-client/src/components/reports/reports/monte-carlo/monteCarloSimulation.test.ts b/packages/desktop-client/src/components/reports/reports/monte-carlo/monteCarloSimulation.test.ts
index f543af510a..2e246b5713 100644
--- a/packages/desktop-client/src/components/reports/reports/monte-carlo/monteCarloSimulation.test.ts
+++ b/packages/desktop-client/src/components/reports/reports/monte-carlo/monteCarloSimulation.test.ts
@@ -102,6 +102,7 @@ function makeParams(
taxBands: [],
horizonYears: 30,
simulationCount: 1000,
+ historicalBlockLength: 10,
seed: 42,
...rest,
};
@@ -497,6 +498,165 @@ describe('runMonteCarloSimulation', () => {
expect(historical.percentileBands).toEqual(deterministic.percentileBands);
});
+ describe('block bootstrap model', () => {
+ // A 20-year history whose stock return and inflation both encode the
+ // year's position, so a captured run reveals which historical year
+ // each simulated year replayed
+ const encodedHistory = Array.from({ length: 20 }, (_, yearIndex) => ({
+ year: 2000 + yearIndex,
+ stocks: yearIndex * 0.001,
+ bonds: 0,
+ cash: 0,
+ inflation: (yearIndex + 1) * 0.0005,
+ }));
+
+ function captureHistoryIndices(
+ historicalBlockLength: number,
+ captureRunDetail: number,
+ ) {
+ const result = runMonteCarloSimulation(
+ makeParams(
+ {
+ returnModel: 'historical-block-bootstrap',
+ historicalReturns: encodedHistory,
+ historicalBlockLength,
+ inflationMean: 0.02,
+ annualWithdrawal: 0,
+ horizonYears: 100,
+ captureRunDetail,
+ },
+ { startingBalance: 1_000_000, allocationPreset: 'equity-100' },
+ ),
+ );
+ return result.runDetail!.map(row => ({
+ historyIndex: Math.round((row.potReturns[0] ?? 0) / 0.001),
+ inflation: row.inflation,
+ }));
+ }
+
+ // The share of year-to-year steps that carried on to the next
+ // historical year (wrapping), pooled over a few captured runs
+ function getContinuationShare(historicalBlockLength: number) {
+ let continued = 0;
+ let steps = 0;
+ for (let runIndex = 0; runIndex < 5; runIndex++) {
+ const years = captureHistoryIndices(historicalBlockLength, runIndex);
+ for (let position = 1; position < years.length; position++) {
+ steps++;
+ if (
+ years[position].historyIndex ===
+ (years[position - 1].historyIndex + 1) % encodedHistory.length
+ ) {
+ continued++;
+ }
+ }
+ }
+ return continued / steps;
+ }
+
+ it('continues to the next year about as often as the block length implies', () => {
+ // Length 10 continues 90% of the time (plus the odd random jump that
+ // happens to land on the next year); length 1 never continues on
+ // purpose; length 30 almost always does
+ const typical = getContinuationShare(10);
+ expect(typical).toBeGreaterThan(0.85);
+ expect(typical).toBeLessThan(0.95);
+ expect(getContinuationShare(1)).toBeLessThan(0.12);
+ expect(getContinuationShare(30)).toBeGreaterThan(0.94);
+ });
+
+ it('wraps from the last historical year back to the first', () => {
+ const years = captureHistoryIndices(30, 0);
+ const hasWrapped = years.some(
+ (year, position) =>
+ position > 0 &&
+ years[position - 1].historyIndex === encodedHistory.length - 1 &&
+ year.historyIndex === 0,
+ );
+ expect(hasWrapped).toBe(true);
+ });
+
+ it("pairs each year with its own historical year's inflation", () => {
+ for (const year of captureHistoryIndices(10, 0)) {
+ expect(year.inflation).toBeCloseTo(
+ (year.historyIndex + 1) * 0.0005,
+ 10,
+ );
+ }
+ });
+
+ it('runs the requested number of scenarios, not one per start year', () => {
+ const result = runMonteCarloSimulation(
+ makeParams({
+ returnModel: 'historical-block-bootstrap',
+ historicalReturns: encodedHistory,
+ simulationCount: 1_500,
+ }),
+ );
+ expect(result.simulationCount).toBe(1_500);
+ });
+
+ it('matches deterministic growth on a one-year history', () => {
+ // With a single historical year, continuing and jumping both land on
+ // it, so the run must match a zero-volatility 5% projection
+ const blocks = runMonteCarloSimulation(
+ makeParams(
+ {
+ returnModel: 'historical-block-bootstrap',
+ historicalReturns: [
+ {
+ year: 2000,
+ stocks: 0.05,
+ bonds: 0.02,
+ cash: 0.01,
+ inflation: 0,
+ },
+ ],
+ annualWithdrawal: 3_000,
+ horizonYears: 30,
+ },
+ { startingBalance: 100_000, allocationPreset: 'equity-100' },
+ ),
+ );
+ const deterministic = runMonteCarloSimulation(
+ makeParams(
+ { annualWithdrawal: 3_000, horizonYears: 30 },
+ {
+ startingBalance: 100_000,
+ expectedReturnMean: 0.05,
+ returnStdDev: 0,
+ },
+ ),
+ );
+ expect(blocks.percentileBands).toEqual(deterministic.percentileBands);
+ });
+
+ it('is seeded, and a captured run replays the same scenario', () => {
+ const params = makeParams(
+ {
+ returnModel: 'historical-block-bootstrap',
+ annualWithdrawal: 40_000,
+ horizonYears: 30,
+ },
+ { startingBalance: 1_000_000, allocationPreset: 'equity-60' },
+ );
+ const first = runMonteCarloSimulation(params);
+ const second = runMonteCarloSimulation(params);
+ expect(second.endingBalances).toEqual(first.endingBalances);
+
+ // Variable draws per year must not desynchronise the capture re-run
+ const captured = runMonteCarloSimulation({
+ ...params,
+ captureRunDetail: 7,
+ });
+ expect(captured.endingBalances).toEqual(first.endingBalances);
+ const fundedRows = captured.runDetail!.filter(row => !row.afterDepletion);
+ expect(fundedRows.at(-1)!.endBalance).toBe(
+ Math.round(first.endingBalances[7]),
+ );
+ });
+ });
+
it('runs one scenario per start year in sequence mode', () => {
const result = runMonteCarloSimulation(
makeParams(
diff --git a/packages/desktop-client/src/components/reports/reports/monte-carlo/monteCarloSimulation.ts b/packages/desktop-client/src/components/reports/reports/monte-carlo/monteCarloSimulation.ts
index 277eaf8625..30a5bfb7f6 100644
--- a/packages/desktop-client/src/components/reports/reports/monte-carlo/monteCarloSimulation.ts
+++ b/packages/desktop-client/src/components/reports/reports/monte-carlo/monteCarloSimulation.ts
@@ -156,6 +156,12 @@ export const MAX_FORMATTABLE_AMOUNT = MAX_SAFE_NUMBER;
export const MIN_SIMULATION_COUNT = 1000;
export const MAX_SIMULATION_COUNT = 10000;
+// The block bootstrap model's average run of consecutive historical years.
+// 10 gives each year a 90% chance of continuing to the next, a common
+// choice for annual market data; 1 never continues (the shuffled model)
+export const MIN_HISTORICAL_BLOCK_LENGTH = 1;
+export const MAX_HISTORICAL_BLOCK_LENGTH = 30;
+export const DEFAULT_HISTORICAL_BLOCK_LENGTH = 10;
export const MIN_HORIZON_YEARS = 1;
export const MAX_HORIZON_YEARS = 100;
@@ -488,9 +494,15 @@ export type MonteCarloConfig = {
withdrawalStrategy: MonteCarloWithdrawalStrategy;
/**
* How yearly returns are generated: random normal draws, random samples
- * of historical years, or replays of actual historical sequences
+ * of historical years, random blocks of consecutive historical years,
+ * or replays of actual historical sequences
*/
returnModel: MonteCarloReturnModel;
+ /**
+ * Block bootstrap model only: the average number of consecutive
+ * historical years in a block before jumping to a new random year
+ */
+ historicalBlockLength: number;
/** Dynamic withdrawal adjustment rule applied at the start of each year */
withdrawalRule: MonteCarloWithdrawalRuleConfig;
/** Minimum yearly spending in minor units (today's money); 0 = no floor */
@@ -534,6 +546,7 @@ export const MONTE_CARLO_DEFAULTS: MonteCarloConfig = {
currentAge: 60,
targetAge: 90,
simulationCount: 5000,
+ historicalBlockLength: DEFAULT_HISTORICAL_BLOCK_LENGTH,
};
/** Simulated years, derived from the configured ages */
@@ -677,6 +690,8 @@ export function monteCarloConfigFromMeta(
targetAge: meta?.targetAge ?? MONTE_CARLO_DEFAULTS.targetAge,
simulationCount:
meta?.simulationCount ?? MONTE_CARLO_DEFAULTS.simulationCount,
+ historicalBlockLength:
+ meta?.historicalBlockLength ?? MONTE_CARLO_DEFAULTS.historicalBlockLength,
};
}
@@ -1620,6 +1635,17 @@ export function runMonteCarloSimulation(
MAX_SIMULATION_COUNT,
);
+ // Block bootstrap: with an average block of L years, each year carries
+ // on to the next historical year with probability 1 - 1/L
+ const blockContinueProbability =
+ 1 -
+ 1 /
+ clamp(
+ params.historicalBlockLength,
+ MIN_HISTORICAL_BLOCK_LENGTH,
+ MAX_HISTORICAL_BLOCK_LENGTH,
+ );
+
const random = mulberry32(params.seed ?? DEFAULT_SIMULATION_SEED);
const nextNormal = makeNormalSampler(random);
@@ -1694,6 +1720,9 @@ export function runMonteCarloSimulation(
// This replay's realized inflation path: each year draws its own rate
// when inflation volatility is set
let cumulativeInflation = 1;
+ // Block bootstrap: the historical year the previous simulated year
+ // used, so this year can carry on from it; -1 before the first year
+ let previousHistoryIndex = -1;
let ratchetStreak = 0;
let withdrawnSum = 0;
let depleted = false;
@@ -1771,6 +1800,18 @@ export function runMonteCarloSimulation(
let historyIndex = -1;
if (returnModel === 'historical-bootstrap') {
historyIndex = Math.floor(random() * historyCount);
+ } else if (returnModel === 'historical-block-bootstrap') {
+ // Stationary block bootstrap: carry on to the next historical
+ // year with the continue probability (wrapping past the end of
+ // the data, as the replay model does), otherwise start a new
+ // block at a random year. The first year always starts a block.
+ // Crashes keep their aftermath within a block, yet every
+ // scenario is a different chain of blocks
+ historyIndex =
+ previousHistoryIndex >= 0 && random() < blockContinueProbability
+ ? (previousHistoryIndex + 1) % historyCount
+ : Math.floor(random() * historyCount);
+ previousHistoryIndex = historyIndex;
} else if (returnModel === 'historical-sequence') {
historyIndex = (simulationIndex + year - 1) % historyCount;
}
diff --git a/packages/docs/docs/experimental/monte-carlo-analysis.md b/packages/docs/docs/experimental/monte-carlo-analysis.md
index ae2291e8ca..8b35584657 100644
--- a/packages/docs/docs/experimental/monte-carlo-analysis.md
+++ b/packages/docs/docs/experimental/monte-carlo-analysis.md
@@ -48,11 +48,13 @@ The tab is organized into four small groups - **Your plan**, **Simulation**, **I
- **Return model** - how the simulation invents each year's investment returns:
- **Random (normal distribution)**: each year's return is drawn randomly around the expected return and volatility you set on each pot. Think of it as a weighted coin flip, year after year. Every pot lives through the same simulated market year - a good year is good for all your pots, scaled by each pot's volatility - so two pots holding the same investments earn the same return.
- **Historical returns, shuffled**: instead of invented numbers, each simulated year is a real year from US market history (1928 onwards), picked in random order. Real crashes like 1931 and 2008 are in the deck, and each sampled year brings its own actual US inflation with it, so high-inflation years keep their high-inflation markets.
+ - **Historical returns, in blocks**: each replay is stitched together from runs of consecutive real years. A replay starts at a random year of US market history and usually carries on to the next real year, so a crash is followed by the years that actually followed it; every so often it jumps to a different point in history and carries on from there. Each year brings its own actual US inflation with it. This keeps the multi-year patterns that shuffling throws away, while still giving you thousands of different replays instead of one per starting year. A run that reaches the end of the data carries on from 1928.
- **Historical sequences (replay)**: each replay is actual history, played in order from a different starting year - "what if you retired in 1929?", "what if you retired in 1972?", and so on. Each replayed year brings its own actual US inflation with it. This is the strictest test of bad timing, because real crashes and recoveries happen in their true order.
- Both historical models only take real returns for pots with an asset mix - an allocation preset, or a complete custom mix. A **Custom** pot always draws random returns around its own expected return and volatility, which is useful for assets history can't stand in for, such as an annuity or property. If every pot is Custom, a historical model's returns behave just like the random model's - though spending still follows each historical year's real inflation.
+ All three historical models only take real returns for pots with an asset mix - an allocation preset, or a complete custom mix. A **Custom** pot always draws random returns around its own expected return and volatility, which is useful for assets history can't stand in for, such as an annuity or property. If every pot is Custom, a historical model's returns behave just like the random model's - though spending still follows each historical year's real inflation.
- **Simulations** - how many replays to run (1,000 to 10,000). More replays give steadier numbers but take slightly longer. When using historical sequences, this field is disabled because there is exactly one replay per historical starting year.
+- **Block length (years)** - only shown for **Historical returns, in blocks**. How many consecutive real years are replayed, on average, before a replay jumps to a different point in history (1 to 30). Longer blocks keep more of history's multi-year patterns, such as a crash and its slow recovery; shorter blocks mix history up more, and a length of 1 behaves like the shuffled model. The default of 10 years is a common choice.
- **Inflation - Mean (%)** - the average yearly rise in prices. When set, your planned spending grows with it so your spending power keeps up. Leave it blank to take exactly the same amount every year. With a historical return model, the two inflation inputs are replaced by an **Adjust spending with inflation** checkbox: each simulated year then uses that year's actual US inflation, keeping inflation and market returns paired the way they really were.
- **Inflation - Std dev (%)** - real-world inflation bounces around from year to year rather than staying fixed. When set, each simulated year draws its own inflation rate around the mean, separately in every replay. The default of 2% is roughly how much US inflation has varied in recent decades; set it to 0 to use the fixed mean rate every year. Only used by the random return model.
- **Manage surplus - Assume any unspent money is spent** - what happens to money the plan doesn't spend in a year, such as income beyond your spending. Unticked (the default), a **Surplus cash** pot holds everything the plan doesn't spend and is drawn on first when spending needs funding; tick it and unspent money simply leaves the plan. See [Income](#income) for the details.
diff --git a/packages/loot-core/src/types/models/dashboard.ts b/packages/loot-core/src/types/models/dashboard.ts
index 2b4875fc02..0e468e033e 100644
--- a/packages/loot-core/src/types/models/dashboard.ts
+++ b/packages/loot-core/src/types/models/dashboard.ts
@@ -142,6 +142,7 @@ export type MonteCarloWithdrawalStrategy =
export type MonteCarloReturnModel =
| 'normal'
| 'historical-bootstrap'
+ | 'historical-block-bootstrap'
| 'historical-sequence';
export type MonteCarloWithdrawalRuleType =
@@ -315,6 +316,8 @@ export type MonteCarloWidget = AbstractWidget<
/** Age the pot must last to; the horizon is targetAge - currentAge */
targetAge?: number;
simulationCount?: number;
+ /** Block bootstrap model: average run of consecutive years */
+ historicalBlockLength?: number;
} | null
>;
diff --git a/upcoming-release-notes/monte-carlo-block-bootstrap.md b/upcoming-release-notes/monte-carlo-block-bootstrap.md
new file mode 100644
index 0000000000..b8a1747d3d
--- /dev/null
+++ b/upcoming-release-notes/monte-carlo-block-bootstrap.md
@@ -0,0 +1,6 @@
+---
+category: Enhancements
+authors: [MikesGlitch]
+---
+
+Add a historical return model to the Monte Carlo report that replays blocks of consecutive real years