Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

feat(data-warehouse): add payout, disbursement and income tables to mono (#123052)

Tom Owersb909976b60bb
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md
index 6a02b417203a..fee96f25e757 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md
@@ -8189,10 +8189,10 @@ Diffed against: <https://docs.mono.co/llms.txt>
- [x] `Fetch all Payments (GET /v2/payments/transactions)` — analyze payment collection revenue, failures, fees, and refunds separately from linked bank transactions (high)
- [x] `Get all Mandates (GET /v3/payments/mandates)` — track direct debit authorizations, their states, and customer relationships for recurring collection analysis (high)
- [x] `Retrieve All Debits (GET /v3/payments/mandates/debits)` — measure successful and failed mandate collections, collected amounts, and debit fees (high)
-- [ ] `Payout (GET /v2/payments/payouts)` — reconcile payment collections with settlement amounts and payout status (high)
-- [ ] `Payout Transactions (GET /v2/payments/payout/{payout}/transactions)` — match individual collections to settlements and explain payout totals (high)
-- [ ] `Disbursements and distributions (GET /v3/payments/disburse/disbursements; GET /v3/payments/disburse/disbursements/{batch_id}/distributions)` — analyze outgoing payment batches and recipient distributions, including payment status and amounts (high)
-- [ ] `Income Records (GET /v2/accounts/{id}/income-records)` — analyze completed income assessments and compare customer earning capacity over time (medium)
+- [x] `Payout (GET /v2/payments/payouts)` — reconcile payment collections with settlement amounts and payout status (high). Added as `payouts`. Mono requires a `status` filter and documents only `settled`, so the table holds settled payouts
+- [ ] `Payout Transactions (GET /v2/payments/payout/{payout}/transactions)` — match individual collections to settlements and explain payout totals (high). Skipped: the endpoint requires an `account` query parameter that payout records do not carry, so it cannot fan out from `payouts`
+- [x] `Disbursements and distributions (GET /v3/payments/disburse/disbursements; GET /v3/payments/disburse/disbursements/{batch_id}/distributions)` — analyze outgoing payment batches and recipient distributions, including payment status and amounts (high). Added as `disbursements` and `disbursement_distributions`
+- [x] `Income Records (GET /v2/accounts/{id}/income-records)` — analyze completed income assessments and compare customer earning capacity over time (medium). Added as `income_records`
- [ ] `Assets (GET /v2/accounts/{id}/assets)` — analyze customer investment holdings that bank transaction tables do not expose (medium)
- [ ] `Earnings (GET /v2/accounts/{id}/earnings)` — measure investment income and portfolio returns alongside customer holdings (medium)
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/mono/canonical_descriptions.py b/products/warehouse_sources/backend/temporal/data_imports/sources/mono/canonical_descriptions.py
index d2690bd67c0a..e68f7dd79f5b 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/mono/canonical_descriptions.py
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/mono/canonical_descriptions.py
@@ -117,4 +117,60 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {
"date": "When the debit was made.",
},
},
+ "payouts": {
+ "description": "Settled payouts of DirectPay and Direct Debit collections to the business's settlement account.",
+ "docs_url": "https://docs.mono.co/api/money-operations/payout",
+ "columns": {
+ "id": "The payout identifier.",
+ "status": "The payout status. Mono returns only settled payouts for this table.",
+ "amount": "The payout amount before fees, in the currency's smallest unit, such as kobo.",
+ "fee": "The fee deducted from the payout, in the currency's smallest unit.",
+ "settled_amount": "The amount paid to the settlement account after fees.",
+ "settled_account": "The beneficiary bank, account name, and account number that received the payout.",
+ "currency": "The payout currency.",
+ "date": "The date of the payout.",
+ "bank": "The identifier of the settlement bank account.",
+ "processor": "The processor that sent the payout.",
+ "message": "A message that describes the payout result.",
+ },
+ },
+ "disbursements": {
+ "description": "Disbursement batches that send funds from a source account to one or more recipients.",
+ "docs_url": "https://docs.mono.co/api/disburse/fetch-all-disbursements",
+ "columns": {
+ "id": "The disbursement batch identifier.",
+ "reference": "The reference the business gave the batch.",
+ "source": "The source of the funds, such as mandate.",
+ "status": "The batch status, such as processing or completed.",
+ "total_amount": "The total amount of the batch in the currency's smallest unit, such as kobo.",
+ "total_fee": "The total fee for the batch in the currency's smallest unit.",
+ "created_at": "When the batch was created.",
+ },
+ },
+ "disbursement_distributions": {
+ "description": "Individual recipient payments inside a disbursement batch.",
+ "docs_url": "https://docs.mono.co/api/disburse/fetch-all-distributions-in-batch",
+ "columns": {
+ "batch_id": "The identifier of the disbursement batch.",
+ "id": "The distribution identifier.",
+ "reference": "The reference of the distribution.",
+ "account_number": "The recipient's bank account number.",
+ "bank_code": "The code of the recipient's bank.",
+ "amount": "The amount sent to the recipient in the currency's smallest unit, such as kobo.",
+ "narration": "The narration sent with the payment.",
+ "status": "The distribution status, such as completed.",
+ "recipient_email": "The recipient's email address.",
+ },
+ },
+ "income_records": {
+ "description": "Completed income assessments of linked accounts.",
+ "docs_url": "https://docs.mono.co/api/bank-data/accounts/income-records",
+ "columns": {
+ "account_id": "The identifier of the linked account.",
+ "income": "The income assessment, with the income summary, the income streams, and the totals.",
+ "app": "The identifier of the Mono app that requested the assessment.",
+ "created_at": "When the assessment was created.",
+ "updated_at": "When the assessment was last updated.",
+ },
+ },
}
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/mono/mono.py b/products/warehouse_sources/backend/temporal/data_imports/sources/mono/mono.py
index f4f8accc0a49..5b51aaaefa7a 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/mono/mono.py
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/mono/mono.py
@@ -32,10 +32,13 @@ from products.warehouse_sources.backend.temporal.data_imports.sources.generated_
from products.warehouse_sources.backend.temporal.data_imports.sources.mono.settings import (
AUTH_ERROR,
BASE_URL,
+ DATA_SELECTORS,
DATE_FILTERS,
ENDPOINTS,
+ FANOUT_PARENTS,
INCREMENTAL_FIELDS,
PAGE_SIZE,
+ PAYOUT_STATUS,
PERMISSION_ERROR,
PRIMARY_KEYS,
)
@@ -135,8 +138,10 @@ def incremental_start(
def endpoint_paginator(endpoint: str) -> PaginatorConfig | None:
if endpoint in ("institutions", "mandate_debits"):
return "single_page"
- if endpoint == "payments":
+ if endpoint in ("payments", "payouts"):
return PageNumberPaginator(base_page=1, total_path="meta.paging.pages")
+ if endpoint in ("disbursements", "disbursement_distributions"):
+ return PageNumberPaginator(base_page=1, total_path="data.pagination.pages")
if endpoint == "mandates":
# The documented response has no paging metadata, so stop at the first empty page.
return PageNumberPaginator(base_page=1)
@@ -176,13 +181,18 @@ def mono_source(
"paginate": "true",
"limit": PAGE_SIZE,
}
+ elif endpoint in FANOUT_PARENTS:
+ parent, placeholder = FANOUT_PARENTS[endpoint]
+ params[placeholder] = {"type": "resolve", "resource": parent, "field": "id"}
elif endpoint in ("payments", "mandates"):
params["limit"] = PAGE_SIZE
+ elif endpoint == "payouts":
+ params["status"] = PAYOUT_STATUS
endpoint_config: Endpoint = {
"path": path,
"params": params,
- "data_selector": "data.payments" if endpoint == "payments" else "data",
+ "data_selector": DATA_SELECTORS.get(endpoint, "data"),
"data_selector_required": True,
}
paginator = endpoint_paginator(endpoint)
@@ -199,14 +209,18 @@ def mono_source(
"endpoint": endpoint_config,
}
resources: list[str | EndpointResource] = []
- if endpoint == "transactions":
+ if endpoint in FANOUT_PARENTS:
+ parent, _ = FANOUT_PARENTS[endpoint]
+ parent_endpoint: Endpoint = {
+ "path": schema_for_resource(ENDPOINTS, parent),
+ "data_selector": DATA_SELECTORS.get(parent, "data"),
+ "data_selector_required": True,
+ }
+ parent_paginator = endpoint_paginator(parent)
+ if parent_paginator is not None:
+ parent_endpoint["paginator"] = parent_paginator
resource["include_from_parent"] = ["id"]
- resources.append(
- {
- "name": "accounts",
- "endpoint": {"path": "accounts", "data_selector": "data", "data_selector_required": True},
- }
- )
+ resources.append({"name": parent, "endpoint": parent_endpoint})
resources.append(resource)
def save_checkpoint(state: dict[str, Any] | None) -> None:
@@ -223,8 +237,9 @@ def mono_source(
initial_paginator_state=resume.paginator_state if resume else None,
)
result = next(item for item in built if item.name == endpoint)
- if endpoint == "transactions":
- result.add_map(rename_parent_fields("accounts", {"id": "account_id"}))
+ if endpoint in FANOUT_PARENTS:
+ parent, placeholder = FANOUT_PARENTS[endpoint]
+ result.add_map(rename_parent_fields(parent, {"id": placeholder}))
return SourceResponse(
name=endpoint,
items=lambda: result,
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/mono/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/mono/settings.py
index c69dcc8fac67..c7ad92dc394f 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/mono/settings.py
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/mono/settings.py
@@ -1,7 +1,7 @@
from products.warehouse_sources.backend.types import IncrementalField, IncrementalFieldType
BASE_URL = "https://api.withmono.com"
-# Relative paths follow the pinned API version. Mono serves bank coverage and Direct Debit only on v3.
+# Relative paths follow the pinned API version. Mono serves bank coverage, Direct Debit and Disburse only on v3.
ENDPOINTS = {
"customers": "customers",
"accounts": "accounts",
@@ -10,6 +10,10 @@ ENDPOINTS = {
"payments": "payments/transactions",
"mandates": f"{BASE_URL}/v3/payments/mandates",
"mandate_debits": f"{BASE_URL}/v3/payments/mandates/debits",
+ "payouts": "payments/payouts",
+ "disbursements": f"{BASE_URL}/v3/payments/disburse/disbursements",
+ "disbursement_distributions": f"{BASE_URL}/v3/payments/disburse/disbursements/{{batch_id}}/distributions",
+ "income_records": "accounts/{account_id}/income-records",
}
PRIMARY_KEYS = {
"customers": ["id"],
@@ -19,7 +23,26 @@ PRIMARY_KEYS = {
"payments": ["id"],
"mandates": ["id"],
"mandate_debits": ["mandate", "reference"],
+ "payouts": ["id"],
+ "disbursements": ["id"],
+ "disbursement_distributions": ["batch_id", "id"],
+ # Income records carry no id, so the account and the assessment time identify a record.
+ "income_records": ["account_id", "created_at"],
}
+# Child endpoint -> (parent endpoint, path placeholder that takes the parent id).
+FANOUT_PARENTS = {
+ "transactions": ("accounts", "account_id"),
+ "income_records": ("accounts", "account_id"),
+ "disbursement_distributions": ("disbursements", "batch_id"),
+}
+DATA_SELECTORS = {
+ "payments": "data.payments",
+ "payouts": "data.payouts",
+ "disbursements": "data.disbursements",
+ "disbursement_distributions": "data.distributions",
+}
+# Mono requires a payout status and documents only "settled".
+PAYOUT_STATUS = "settled"
INCREMENTAL_FIELDS: dict[str, list[IncrementalField]] = {
"transactions": [
{
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/mono/tests/test_mono.py b/products/warehouse_sources/backend/temporal/data_imports/sources/mono/tests/test_mono.py
index fe8b25b16fd4..65f38e3043cb 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/mono/tests/test_mono.py
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/mono/tests/test_mono.py
@@ -129,6 +129,16 @@ def payments_page(rows: list[dict[str, Any]], pages: int) -> ScriptedResponse:
)
+def payouts_page(rows: list[dict[str, Any]], pages: int) -> ScriptedResponse:
+ return ScriptedResponse(
+ json={"status": "successful", "data": {"payouts": rows}, "meta": {"paging": {"pages": pages, "next": None}}}
+ )
+
+
+def disburse_page(key: str, rows: list[dict[str, Any]], pages: int) -> ScriptedResponse:
+ return ScriptedResponse(json={"status": "successful", "data": {key: rows, "pagination": {"pages": pages}}})
+
+
@time_machine.travel("2026-06-15T12:00:00Z", tick=False)
@pytest.mark.parametrize(
"endpoint,incremental,watermark,script,expected_path,expected_queries,expected_ids",
@@ -178,6 +188,15 @@ def payments_page(rows: list[dict[str, Any]], pages: int) -> ScriptedResponse:
[{"page": ("1",), "limit": ("100",), "start": ("09-06-2026",), "end": ("15-06-2026",)}],
["payment-1"],
),
+ (
+ "payouts",
+ False,
+ None,
+ [payouts_page([{"id": "payout-1"}], 2), payouts_page([{"id": "payout-2"}], 2)],
+ "/v2/payments/payouts",
+ [{"page": (str(number),), "status": ("settled",)} for number in (1, 2)],
+ ["payout-1", "payout-2"],
+ ),
(
"mandate_debits",
False,
@@ -218,6 +237,69 @@ def test_payment_and_coverage_tables_request_the_right_version_filters_and_pages
assert all(request.headers["mono-sec-key"] == CONFIG.api_key for request in result.requests)
+ "endpoint,script,expected_requests,expected_keys",
+ [
+ (
+ "income_records",
+ [
+ page([{"id": "account-a"}], "https://api.withmono.com/v2/accounts?page=2"),
+ page(
+ [{"created_at": "2025-05-23T09:48:43.276Z", "income": {"total_income": 16300}}],
+ "https://api.withmono.com/v2/accounts/account-a/income-records?page=2",
+ ),
+ page([{"created_at": "2025-04-01T08:00:00.000Z", "income": {"total_income": 9000}}]),
+ page([{"id": "account-b"}]),
+ page([{"created_at": "2025-05-23T09:48:43.276Z", "income": {"total_income": 500}}]),
+ ],
+ [
+ ("/v2/accounts", "1"),
+ ("/v2/accounts/account-a/income-records", "1"),
+ ("/v2/accounts/account-a/income-records", "2"),
+ ("/v2/accounts", "2"),
+ ("/v2/accounts/account-b/income-records", "1"),
+ ],
+ [
+ ("account-a", "2025-05-23T09:48:43.276Z"),
+ ("account-a", "2025-04-01T08:00:00.000Z"),
+ ("account-b", "2025-05-23T09:48:43.276Z"),
+ ],
+ ),
+ (
+ "disbursement_distributions",
+ [
+ disburse_page("disbursements", [{"id": "batch-a"}], 2),
+ disburse_page("distributions", [{"id": "distribution-1"}], 2),
+ disburse_page("distributions", [{"id": "distribution-2"}], 2),
+ disburse_page("disbursements", [{"id": "batch-b"}], 2),
+ disburse_page("distributions", [{"id": "distribution-1"}], 1),
+ ],
+ [
+ ("/v3/payments/disburse/disbursements", "1"),
+ ("/v3/payments/disburse/disbursements/batch-a/distributions", "1"),
+ ("/v3/payments/disburse/disbursements/batch-a/distributions", "2"),
+ ("/v3/payments/disburse/disbursements", "2"),
+ ("/v3/payments/disburse/disbursements/batch-b/distributions", "1"),
+ ],
+ [("batch-a", "distribution-1"), ("batch-a", "distribution-2"), ("batch-b", "distribution-1")],
+ ),
+ ],
+)
+def test_fanout_tables_walk_every_parent_page_and_key_rows_by_parent(
+ endpoint: str,
+ script: list[ScriptedResponse],
+ expected_requests: list[tuple[str, str]],
+ expected_keys: list[tuple[str, str]],
+) -> None:
+ result = driver().run(endpoint, script)
+
+ assert result.raised is None
+ assert [(request.path, request.param("page")) for request in result.requests] == expected_requests
+ assert result.response is not None
+ assert [tuple(row[key] for key in result.response.primary_keys or []) for row in result.rows] == expected_keys
+ assert all(not any(key.startswith("_") for key in row) for row in result.rows)
+
+
@pytest.mark.parametrize(
"start,end,expected_query",
[