Skip to content

armgit / macro

Public Imported from GitHub
0

Macro is a unified workspace for teams: email, chat, docs, tasks, agents, calls, and CRM — @-linked together with shared AI memory.

feat(infra): grant Grafana admin access to Macro Workspace (#7854)

Wolf Mermelstein488f93102959
diff --git a/infra/stacks/observability/README.md b/infra/stacks/observability/README.md
index 5a5cff6c2e..320248233e 100644
--- a/infra/stacks/observability/README.md
+++ b/infra/stacks/observability/README.md
@@ -16,7 +16,7 @@ packages, configuration, service users, resource limits and startup dependencies
NixOS modules generate application configuration from Nix attributes. Alloy's
pipelines live in native `.alloy` files read by Nix. Pulumi sends only
-validated runtime values such as hostnames, bucket names and approved identities.
+validated runtime values such as hostnames and bucket names.
Datadog instrumentation, collection and alerts remain unchanged. Nothing sends
application telemetry here until a subsequent dual-export change is deployed.
@@ -98,23 +98,23 @@ before enabling broad production ingestion.
Production URLs are `https://grafana.macro-internal.com` and `https://otlp.macro-internal.com`;
dev uses `grafana-dev.macro-internal.com` and `otlp-dev.macro-internal.com`.
-Grafana uses a dedicated **Google Workspace OAuth client**. Only explicitly
-approved lowercase `@macro.com` emails in `allowedEmails` may sign in; membership
-in the domain alone is insufficient. Named `adminEmails` get Grafana server
-administrator access; other approved users are Viewers. Admins must also appear
-in the allowlist. Unknown identities map to an invalid role and are denied with
-`role_attribute_strict`. Google hosted-domain validation, ID-token signature
-validation, PKCE and refresh-token checking are enabled. Workspace administrators
-must enforce MFA for the approved accounts. No password login, default admin,
-anonymous access or public signup is enabled.
-
-Sessions have a one-hour inactivity limit and eight-hour maximum. Removing an
-email from configuration blocks the next login; it does **not** immediately
-invalidate an existing session. For offboarding, first disable the user and
-revoke their sessions in Grafana using another admin, then update the allowlist.
-Keep at least two admins. This uses OSS role mapping, not Enterprise team sync.
-All approved users can query the pilot's telemetry; there is no per-service data
-isolation. Apply redaction in the producer pipeline before copying sensitive data.
+Grafana uses a **Google Workspace OAuth client**. Every member of the `macro.com`
+Workspace may sign in and receives Grafana server administrator and organization
+administrator access. Google hosted-domain validation and the `macro.com` domain
+restriction reject accounts outside that Workspace, including personal Google
+accounts. ID-token signature validation, PKCE and refresh-token checking are enabled.
+Workspace administrators must enforce MFA. No password login, default admin or
+anonymous access is enabled; account creation happens only after Google login.
+The role policy is declared directly in `nixos/grafana.nix`; there are no per-user
+Pulumi access lists.
+
+Sessions have a one-hour inactivity limit and eight-hour maximum. Suspending a
+Workspace account does **not** immediately invalidate an existing Grafana session.
+For offboarding, disable the Workspace account and use another Grafana admin to
+disable the Grafana user and revoke their sessions. This uses OSS role mapping,
+not Enterprise team sync. All team members can administer Grafana and query the
+pilot's telemetry; there is no per-service data isolation. Apply redaction in the
+producer pipeline before copying sensitive data.
Only the ALB is internet-facing, on TLS 1.2/1.3 port 443. The host has no public IP,
SSH key or SSH ingress. Its only ingress is port 8080 from the ALB security group.
@@ -124,7 +124,7 @@ services bind to loopback. The default ALB action is 404. No HTTP listener is op
Grafana data sources use a separate internal nginx listener on port 8081 that
allows only named query endpoints and their required HTTP methods. This matters
-because Viewers can call Grafana's data-source proxy: direct backend URLs would
+because users can call Grafana's data-source proxy: direct backend URLs would
also expose maintenance endpoints such as Tempo `/shutdown` and Loki `/flush`
(including mutating GET requests). Port 8081 binds only to loopback, and
Alloy's ingestion path is separate. New data-source features may require reviewed
@@ -142,9 +142,9 @@ Secrets Manager holds Google client credentials, the Grafana encryption key and
the ingestion token. EC2 retrieves them at startup into `/run` with restricted
file permissions. Systemd `LoadCredential` gives Grafana only its three credentials
and ingress Alloy only the OTLP token. Secret values never enter Pulumi state,
-user-data, environment variables or configuration committed here. Grafana's nonsecret host
-and approved-user role expression are supplied through environment variables;
-the generated INI refers to them through Grafana's environment provider. Secrets
+user-data, environment variables or configuration committed here. Grafana's nonsecret hostname
+and public URL are supplied through environment variables; the generated INI
+refers to them through Grafana's environment provider. Secrets
remain file references in the Nix store, never secret values. The secret must use the
AWS-managed Secrets Manager encryption key; a customer-managed key needs an
explicit scoped KMS policy addition. IMDSv2 is required with hop limit 1. Only Loki, Tempo, the bootstrap/secret helper,
@@ -160,26 +160,28 @@ and AWS account `569036502058`, region `us-east-2`. Deployment requires the
existing public `macro-internal.com` Route53 zone; this stack creates its own VPC/NAT and
DNS-validated regional ACM certificate. Region validation rejects `us-east-1`.
-1. Create a dedicated Google OAuth Web application under the company's Google
- organization with an **Internal** consent audience. Register exactly
+1. Use a Google OAuth Web client managed by the company. For a new dedicated
+ client, choose an **Internal** consent audience. When reusing the application
+ client, preserve its existing audience and redirect URIs; Grafana independently
+ enforces the `macro.com` Workspace restriction. Add the exact redirect URI
`https://grafana-dev.macro-internal.com/login/google` for dev or
- `https://grafana.macro-internal.com/login/google` for prod. Prefer separate clients and
- secrets per environment. Confirm Workspace MFA enforcement.
+ `https://grafana.macro-internal.com/login/google` for prod. Prefer separate clients
+ and secrets per environment. Confirm Workspace MFA enforcement.
2. Through the approved secret-management process, create a Secrets Manager JSON
secret in **us-east-2** containing `google_client_id`, `google_client_secret`,
`grafana_secret_key`, and `otlp_token`. Generate independent cryptographically
random values of at least 32 characters for the last two. Preserve
`grafana_secret_key` through recovery; changing it can make stored credentials
unreadable. Never paste secret values into shell commands, this file or PRs.
-3. Select approved users and admins. Select an existing **us-east-2** SNS topic
+3. Select an existing **us-east-2** SNS topic
with a confirmed, monitored subscription for infrastructure alarms. Its
delivery destination must remain accessible during a production outage.
Configuration rejects secrets and topics in another region; the host reads
the local secret directly, without fetching credentials from production.
4. Build, publish and smoke-boot the NixOS image as described below. Set its
reviewed Ohio AMI ID; there is deliberately no mutable "latest image" lookup.
-5. Set the nonsecret configuration below, substituting actual identifiers. No
- example account or placeholder is authorized automatically.
+5. Set the nonsecret configuration below, substituting actual identifiers.
+ Placeholder credentials are not usable for deployment.
```bash
\cd infra
@@ -190,9 +192,6 @@ pulumi config set aws:region us-east-2
pulumi config set amiId '<reviewed NixOS AMI ID in Ohio>'
pulumi config set secretArn '<existing Secrets Manager ARN>'
pulumi config set alarmTopicArn '<existing monitored SNS topic ARN>'
-pulumi config set --path 'allowedEmails[0]' '<[email protected]>'
-pulumi config set --path 'allowedEmails[1]' '<[email protected]>'
-pulumi config set --path 'adminEmails[0]' '<[email protected]>'
pulumi preview --diff
```
@@ -204,8 +203,8 @@ an explicit migration and recovery plan for its protected storage.
Before enabling application traffic, validate all of these against AWS:
-- Approved Viewer and Admin can log in; an unapproved Workspace account and a
- personal Google account cannot. Viewer cannot administer Grafana.
+- A `macro.com` Workspace member can log in with server and organization admin
+ access. A different Workspace domain and personal Google accounts cannot log in.
- All three data sources pass their health checks. A small OTLP fixture appears
in logs, traces and metrics with the expected `service.name`.
- Missing/wrong tokens fail; valid token cannot read backend APIs. No backend
@@ -274,11 +273,11 @@ sudo journalctl -u grafana -u loki -u tempo -u prometheus -u alloy -u alloy-inge
Nonsecret runtime values arrive as versioned, compressed JSON in EC2 user-data.
`read-user-data.py` retrieves them through IMDSv2 and validates the exact schema,
-region, hostnames, bucket/volume/secret identifiers and approved identity lists.
+region, hostnames and bucket/volume/secret identifiers.
It writes a systemd environment file, nginx hostname maps, the secret's identifier
and the expected volume ID under `/opt/observability`. It does not execute
user-data or read secret values. The standard NixOS user-data evaluator is disabled.
-Schema version 3 rejects older payloads, unknown settings and caller-supplied files.
+Schema version 4 rejects older payloads, unknown settings and caller-supplied files.
`observability-bootstrap` mounts that exact EBS volume and creates each service's
data directory using stable, named NixOS users. `prepare-volume.sh` formats only
diff --git a/infra/stacks/observability/index.ts b/infra/stacks/observability/index.ts
index 4d99c5719a..3897c547e1 100644
--- a/infra/stacks/observability/index.ts
+++ b/infra/stacks/observability/index.ts
@@ -22,8 +22,6 @@ const otlpHost = `otlp${suffix}.${baseDomain}`;
const tags = { project: 'observability', environment: stack };
const durable = { protect: true, retainOnDelete: true };
const secretArn = config.require('secretArn');
-const allowedEmails = config.requireObject<string[]>('allowedEmails');
-const adminEmails = config.requireObject<string[]>('adminEmails');
const alarmTopicArn = config.require('alarmTopicArn');
// Built from this stack's pinned NixOS flake and published in Ohio.
const amiId = config.require('amiId');
@@ -35,8 +33,6 @@ validateSettings({
region,
grafanaHost,
otlpHost,
- allowedEmails,
- adminEmails,
secretArn,
volumeId: 'vol-0',
logsBucket: 'validate-logs',
@@ -259,8 +255,6 @@ const instance = new aws.ec2.Instance(
region,
grafanaHost,
otlpHost,
- allowedEmails,
- adminEmails,
secretArn,
volumeId,
logsBucket,
diff --git a/infra/stacks/observability/nixos/grafana.nix b/infra/stacks/observability/nixos/grafana.nix
index ff61f97816..88715066b0 100644
--- a/infra/stacks/observability/nixos/grafana.nix
+++ b/infra/stacks/observability/nixos/grafana.nix
@@ -45,7 +45,7 @@
use_refresh_token = true;
validate_id_token = true;
jwk_set_url = "https://www.googleapis.com/oauth2/v3/certs";
- role_attribute_path = "$__env{GRAFANA_ROLE_EXPRESSION}";
+ role_attribute_path = "\"'GrafanaAdmin'\"";
role_attribute_strict = true;
allow_assign_grafana_admin = true;
skip_org_role_sync = false;
diff --git a/infra/stacks/observability/nixos/read-user-data.py b/infra/stacks/observability/nixos/read-user-data.py
index 2c20f6001c..9ae2205c82 100644
--- a/infra/stacks/observability/nixos/read-user-data.py
+++ b/infra/stacks/observability/nixos/read-user-data.py
@@ -7,13 +7,13 @@ import urllib.request
CONFIG_ROOT = Path('/opt/observability')
SETTING_KEYS = {
- 'region', 'grafanaHost', 'otlpHost', 'allowedEmails', 'adminEmails',
+ 'region', 'grafanaHost', 'otlpHost',
'secretArn', 'volumeId', 'logsBucket', 'tracesBucket',
}
def validate_payload(payload):
- if not isinstance(payload, dict) or set(payload) != {'version', 'settings'} or payload['version'] != 3:
+ if not isinstance(payload, dict) or set(payload) != {'version', 'settings'} or payload['version'] != 4:
raise ValueError('Unsupported observability user-data schema')
settings = payload['settings']
if not isinstance(settings, dict) or set(settings) != SETTING_KEYS:
@@ -32,32 +32,20 @@ def validate_payload(payload):
raise ValueError(f'Invalid runtime setting: {key}')
if settings['grafanaHost'] == settings['otlpHost']:
raise ValueError('UI and ingestion require separate hostnames')
- for key in ['allowedEmails', 'adminEmails']:
- emails = settings[key]
- if not isinstance(emails, list) or not emails or any(
- not isinstance(email, str) or not re.fullmatch(r'[a-z0-9._+-]+@macro\.com', email)
- for email in emails
- ):
- raise ValueError(f'Invalid approved identity list: {key}')
- if not set(settings['adminEmails']).issubset(settings['allowedEmails']):
- raise ValueError('Every admin must also be approved')
return settings
def write_runtime(payload, output_root):
settings = validate_payload(payload)
- admin_list = json.dumps(settings['adminEmails'], separators=(',', ':'))
- allowed_list = json.dumps(settings['allowedEmails'], separators=(',', ':'))
environment = {
'AWS_REGION': settings['region'],
'GRAFANA_HOST': settings['grafanaHost'],
'GRAFANA_ROOT_URL': 'https://' + settings['grafanaHost'] + '/',
'LOGS_BUCKET': settings['logsBucket'],
'TRACES_BUCKET': settings['tracesBucket'],
- 'GRAFANA_ROLE_EXPRESSION': f"contains(`{admin_list}`, email) && 'GrafanaAdmin' || contains(`{allowed_list}`, email) && 'Viewer' || 'Denied'",
}
# systemd EnvironmentFile accepts double-quoted, escaped values. Inputs are
- # validated ASCII; no shell ever evaluates these values or the role expression.
+ # validated ASCII; no shell ever evaluates these values.
files = {
'runtime.env': ''.join(f'{key}={json.dumps(value)}\n' for key, value in environment.items()),
'hosts.conf': (
diff --git a/infra/stacks/observability/render.ts b/infra/stacks/observability/render.ts
index 1ec03801e8..95058ae212 100644
--- a/infra/stacks/observability/render.ts
+++ b/infra/stacks/observability/render.ts
@@ -4,7 +4,7 @@ import { type Settings, validateSettings } from './settings';
// Nix owns service configuration. User data carries only validated runtime values.
export function renderUserData(settings: Settings): string {
validateSettings(settings);
- const compressed = gzipSync(JSON.stringify({ version: 3, settings }));
+ const compressed = gzipSync(JSON.stringify({ version: 4, settings }));
if (compressed.length > 16 * 1024) {
throw new Error('Configuration exceeds the EC2 user-data limit');
}
diff --git a/infra/stacks/observability/settings.ts b/infra/stacks/observability/settings.ts
index bb106e9ce2..b5b879d3c6 100644
--- a/infra/stacks/observability/settings.ts
+++ b/infra/stacks/observability/settings.ts
@@ -2,8 +2,6 @@ export interface Settings {
region: string;
grafanaHost: string;
otlpHost: string;
- allowedEmails: string[];
- adminEmails: string[];
secretArn: string;
volumeId: string;
logsBucket: string;
@@ -49,23 +47,6 @@ export function validateSettings(settings: Settings): void {
if (settings.grafanaHost === settings.otlpHost) {
throw new Error('UI and ingestion require separate hostnames');
}
- if (!settings.allowedEmails.length || !settings.adminEmails.length) {
- throw new Error('Configure approved users and at least one admin');
- }
- for (const email of [...settings.allowedEmails, ...settings.adminEmails]) {
- if (!/^[a-z0-9._+-]+@macro\.com$/.test(email)) {
- throw new Error(
- 'Access lists require lowercase macro.com email addresses'
- );
- }
- }
- if (
- settings.adminEmails.some(
- (email) => !settings.allowedEmails.includes(email)
- )
- ) {
- throw new Error('Every admin must also be in allowedEmails');
- }
if (
!/^arn:aws:secretsmanager:[a-z0-9-]+:\d{12}:secret:[a-zA-Z0-9/_+=.@-]+$/.test(
settings.secretArn