Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

feat(warehouse_sources): add gusto departments, benefits, ach and reimbursement tables (#121108)

Tom Owers249c1e2e4cf9
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md
index 955475096af6..e68d9325d681 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md
@@ -5675,19 +5675,19 @@ Note: developer.getguru.com is a ReadMe site with no downloadable OpenAPI (opena
## Gusto — gaps
-Today (10): `companies`, `contractor_payments`, `contractors`, `employees`, `jobs`, `locations`, `pay_periods`, `pay_schedules`, `payrolls`, `tax_payments`
+Today (14): `ach_transactions`, `companies`, `company_benefits`, `contractor_payments`, `contractors`, `departments`, `employees`, `jobs`, `locations`, `pay_periods`, `pay_schedules`, `payrolls`, `recurring_reimbursements`, `tax_payments`
Diffed against: <https://docs.gusto.com/embedded-payroll/llms.txt>
-- [ ] `Departments (GET /v1/companies/{company_uuid}/departments)` — resolve department_uuid values on employees and analyze workforce composition by department (high)
-- [ ] `ACH transactions (GET /v1/companies/{company_uuid}/ach_transactions)` — reconcile payroll funding and payment movements with bank transaction status and failures (high)
-- [ ] `Company benefits (GET /v1/companies/{company_id}/company_benefits)` — explain employee deductions and employer contributions using named benefit plans (high)
+- [x] `Departments (GET /v1/companies/{company_uuid}/departments)` — resolve department_uuid values on employees and analyze workforce composition by department (high)
+- [x] `ACH transactions (GET /v1/companies/{company_uuid}/ach_transactions)` — reconcile payroll funding and payment movements with bank transaction status and failures (high)
+- [x] `Company benefits (GET /v1/companies/{company_id}/company_benefits)` — explain employee deductions and employer contributions using named benefit plans (high)
- [ ] `Earning types (GET /v1/companies/{company_id}/earning_types)` — separate standard earnings from custom pay categories in payroll analysis (medium)
- [ ] `Employee benefits (GET /v1/company_benefits/{company_benefit_id}/employee_benefits)` — measure benefit enrollment, employee deductions, and employer contribution arrangements (medium)
- [ ] `Employment history (GET /v1/employees/{employee_id}/employment_history)` — analyze terminations, rehires, tenure, and workforce changes beyond current employment status (medium)
- [ ] `Time off requests (GET /v1/companies/{company_uuid}/time_off/requests)` — track requested and approved leave, absence patterns, and approval delays (medium)
- [ ] `Employee time off activities (GET /v1/employees/{employee_uuid}/time_off_activities)` — explain leave balance changes through accrual, usage, and adjustment activity (medium)
-- [ ] `Recurring reimbursements (GET /v1/employees/{employee_id}/recurring_reimbursements)` — forecast recurring employee reimbursements separately from amounts already paid through payroll (medium)
+- [x] `Recurring reimbursements (GET /v1/employees/{employee_id}/recurring_reimbursements)` — forecast recurring employee reimbursements separately from amounts already paid through payroll (medium)
- [ ] `Events (GET /v1/events)` — track payroll and employee changes as events instead of inferring them from repeated snapshots (medium)
- [x] `Tax payments (GET /v1/companies/{company_uuid}/tax_payments)` — reconcile payroll tax liabilities with what Gusto paid each agency and when (medium)
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/gusto/canonical_descriptions.py b/products/warehouse_sources/backend/temporal/data_imports/sources/gusto/canonical_descriptions.py
index 733c38d311e6..3ff0eaf3e1bf 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/gusto/canonical_descriptions.py
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/gusto/canonical_descriptions.py
@@ -197,4 +197,73 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {
"_company_uuid": "The company this tax payment belongs to.",
},
},
+ "departments": {
+ "description": "A department within a company, with the employees and contractors assigned to it.",
+ "docs_url": "https://docs.gusto.com/embedded-payroll/reference/get-companies-departments",
+ "columns": {
+ "uuid": "The unique identifier of the department.",
+ "company_uuid": "The company the department belongs to.",
+ "title": "The name of the department.",
+ "employees": "The employees assigned to the department, as a list of objects that each carry the employee's `uuid`.",
+ "contractors": "The contractors assigned to the department, as a list of objects that each carry the contractor's `uuid`.",
+ "version": "The current version of the department object, used for optimistic concurrency.",
+ "_company_uuid": "The company this department belongs to.",
+ },
+ },
+ "company_benefits": {
+ "description": "A benefit a company offers its employees, tying a Gusto-supported benefit type to the company's own offering of it.",
+ "docs_url": "https://docs.gusto.com/embedded-payroll/reference/get-v1-companies-company_id-company_benefits",
+ "columns": {
+ "uuid": "The unique identifier of the company benefit.",
+ "company_uuid": "The company offering the benefit.",
+ "benefit_type": "The numeric id of the Gusto-supported benefit type (e.g. 1 for medical insurance).",
+ "description": "The company's name for the benefit, distinguishing multiple offerings of the same benefit type.",
+ "active": "Whether the benefit is open for employee participation.",
+ "source": "Where the benefit came from: internal, external, or partnered.",
+ "partner_name": "The name of the partner that created the benefit, if any.",
+ "deletable": "Whether the benefit can still be deleted, i.e. it has never been used in payroll or had employees enrolled.",
+ "supports_percentage_amounts": "Whether employee deductions and company contributions can be set as a percentage of pay.",
+ "responsible_for_employer_taxes": "For third-party sick pay, whether the employer pays employer taxes while an employee is on leave.",
+ "responsible_for_employee_w2": "For third-party sick pay, whether the employer files W-2 forms for an employee on leave.",
+ "catch_up_type": "The SECURE 2.0 catch-up contribution type, for pre-tax 401(k) and 403(b) benefits.",
+ "version": "The current version of the company benefit object, used for optimistic concurrency.",
+ "_company_uuid": "The company this benefit belongs to.",
+ },
+ },
+ "ach_transactions": {
+ "description": "An ACH bank transfer Gusto made or collected for a payroll or contractor payment, with its settlement status.",
+ "docs_url": "https://docs.gusto.com/embedded-payroll/reference/get-ach-transactions",
+ "columns": {
+ "uuid": "The unique identifier of the ACH transaction.",
+ "company_uuid": "The company the transaction belongs to.",
+ "payment_event_type": "The kind of payment that triggered the transaction: Payroll or ContractorPayment.",
+ "payment_event_uuid": "The payroll or contractor payment that triggered the transaction.",
+ "payment_event_check_date": "The check date of the payroll or contractor payment.",
+ "recipient_type": "Whether the counterparty is an Employee or a Contractor. Null for company-level transactions.",
+ "recipient_uuid": "The employee or contractor the money moves to or from.",
+ "transaction_type": "What the transaction moves (e.g. Credit employee pay).",
+ "payment_direction": "Whether money is sent to the recipient (credit) or collected from them (debit).",
+ "payment_status": "The transaction's status: unsubmitted, submitted, successful, or failed.",
+ "payment_date": "The date the money moves.",
+ "amount": "The amount moved by the transaction. Always non-negative; direction is in payment_direction.",
+ "error_code": "The ACH return code when the transaction failed. Null when there is no error.",
+ "description": "The description shown on the recipient's bank statement.",
+ "_company_uuid": "The company this transaction belongs to.",
+ },
+ },
+ "recurring_reimbursements": {
+ "description": "An active reimbursement paid to an employee on every payroll, separate from one-off reimbursements already paid.",
+ "docs_url": "https://docs.gusto.com/embedded-payroll/reference/get-v1-employees-employee_id-recurring_reimbursements",
+ "columns": {
+ "uuid": "The unique identifier of the recurring reimbursement.",
+ "employee_uuid": "The employee who receives the reimbursement.",
+ "description": "What the reimbursement is for.",
+ "amount": "The dollar amount reimbursed each payroll.",
+ "created_at": "When the recurring reimbursement was created.",
+ "updated_at": "When the recurring reimbursement was last updated.",
+ "version": "The current version of the reimbursement object, used for optimistic concurrency.",
+ "_company_uuid": "The company that employs the employee.",
+ "_employee_uuid": "The employee this reimbursement belongs to.",
+ },
+ },
}
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/gusto/gusto.py b/products/warehouse_sources/backend/temporal/data_imports/sources/gusto/gusto.py
index 8d71fc411ec2..916c2f760d94 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/gusto/gusto.py
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/gusto/gusto.py
@@ -377,6 +377,12 @@ def get_rows(
resumable_source_manager.save_state(
GustoResumeConfig(company_index=company_index, employee_index=employee_index + 1)
)
+ elif not config.paginated:
+ # These lists document no `page`/`per` and return everything at once, so one request covers them.
+ body, _ = client.get_json(config.path.format(company_uuid=company_uuid))
+ rows = _stamp_parents(extract_rows(config, body), parents)
+ if rows:
+ yield rows
else:
first_page = resume.next_page if (resuming_here and resume is not None) else 1
yield from _paginate(
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/gusto/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/gusto/settings.py
index 0114364db64f..bb6dc207406e 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/gusto/settings.py
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/gusto/settings.py
@@ -3,14 +3,14 @@ from dataclasses import dataclass
from products.warehouse_sources.backend.types import IncrementalField, IncrementalFieldType
-@dataclass
+@dataclass(frozen=True)
class GustoEndpointConfig:
name: str
# Path under the API host. `{company_uuid}` / `{employee_uuid}` are filled in per parent row.
path: str
primary_keys: list[str]
- # Company-scoped collections paginate with `page`/`per`; the derived `companies` table and the
- # per-employee child lists come back whole in a single response.
+ # Most company-scoped collections paginate with `page`/`per`; the derived `companies` table, a
+ # few small company lists, and the per-employee child lists come back whole in a single response.
paginated: bool = True
# Money-movement endpoints take a `start_date`/`end_date` check-date window instead of being
# walked in full. The value is the row field the window filters on, which is also the only
@@ -94,6 +94,32 @@ GUSTO_ENDPOINTS: dict[str, GustoEndpointConfig] = {
path="/v1/companies/{company_uuid}/tax_payments",
primary_keys=["uuid"],
),
+ "departments": GustoEndpointConfig(
+ name="departments",
+ path="/v1/companies/{company_uuid}/departments",
+ primary_keys=["uuid"],
+ paginated=False,
+ ),
+ "company_benefits": GustoEndpointConfig(
+ name="company_benefits",
+ path="/v1/companies/{company_uuid}/company_benefits",
+ primary_keys=["uuid"],
+ paginated=False,
+ ),
+ # ACH transactions take no date filter, and `payment_status` / `error_code` change after a
+ # transaction is created, so full refresh keeps failures and settlements current.
+ "ach_transactions": GustoEndpointConfig(
+ name="ach_transactions",
+ path="/v1/companies/{company_uuid}/ach_transactions",
+ primary_keys=["uuid"],
+ ),
+ "recurring_reimbursements": GustoEndpointConfig(
+ name="recurring_reimbursements",
+ path="/v1/employees/{employee_uuid}/recurring_reimbursements",
+ primary_keys=["_employee_uuid", "uuid"],
+ paginated=False,
+ fans_out_over_employees=True,
+ ),
}
ENDPOINTS = tuple(GUSTO_ENDPOINTS.keys())
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/gusto/tests/test_gusto.py b/products/warehouse_sources/backend/temporal/data_imports/sources/gusto/tests/test_gusto.py
index 1bc9263144cb..395cf1699b4e 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/gusto/tests/test_gusto.py
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/gusto/tests/test_gusto.py
@@ -250,6 +250,21 @@ class TestGetRowsCompanies:
assert [row["name"] for row in result.rows] == ["Acme", "Globex"]
+class TestGetRowsUnpaginated:
+ def test_full_list_is_read_in_one_request(self) -> None:
+ # A page-size response must not trigger a page 2 that would re-read the same list.
+ departments = [{"uuid": f"d-{index}"} for index in range(100)]
+ result = _driver().run(
+ "departments",
+ [_token(), ScriptedResponse(json=_me_body(["c-1"])), ScriptedResponse(json=departments)],
+ )
+ assert result.raised is None
+ assert len(result.rows) == 100
+ assert {row["_company_uuid"] for row in result.rows} == {"c-1"}
+ assert result.paths == ["/oauth/token", "/v1/me", "/v1/companies/c-1/departments"]
+ assert result.params("page")[-1] is None
+
+
class TestGetRowsFanOut:
def test_resumes_partway_through_a_company_employee_list(self) -> None:
result = _driver().run(
@@ -323,7 +338,7 @@ class TestGustoSourceResponse:
assert result.response.primary_keys == GUSTO_ENDPOINTS[endpoint].primary_keys
assert result.response.sort_mode == "asc"
- @parameterized.expand([("jobs",), ("contractor_payments",), ("pay_periods",)])
+ @parameterized.expand([("jobs",), ("recurring_reimbursements",), ("contractor_payments",), ("pay_periods",)])
def test_fan_out_and_keyless_tables_include_their_parent_in_the_key(self, endpoint: str) -> None:
# Child rows from every parent land in one table, so a per-parent identifier alone would
# collide and make each merge multi-match.