Macro is a unified workspace for teams: email, chat, docs, tasks, agents, calls, and CRM — @-linked together with shared AI memory.
TypeScript agentaiai-agentsall-in-onecrmcrm-systememaillinearmcpmessagingnotesnotionofficerustslackslack-alternativestartupstartup-toolstypescriptworkspace
fix(search): prevent accidental agent index creation (#6495)
Eric Hayese768fde12edf
diff --git a/crates/opensearch_client/src/upsert/agent_session.rs b/crates/opensearch_client/src/upsert/agent_session.rsindex a1f53bd669..06bd805879 100644--- a/crates/opensearch_client/src/upsert/agent_session.rs+++ b/crates/opensearch_client/src/upsert/agent_session.rs@@ -174,6 +174,9 @@ pub(crate) async fn reconcile_agent_session(&args.agent_session_id,)).routing(routing)+ // Normal writes must never auto-create an index under the alias name.+ // Backfills may explicitly target a pre-provisioned physical index.+ .require_alias(index_override.is_none()).body(parent_document(args)).refresh(opensearch::params::Refresh::WaitFor).send()@@ -201,6 +204,7 @@ pub(crate) async fn reconcile_agent_session(let response = client.bulk(BulkParts::Index(index))+ .require_alias(index_override.is_none()).body(body).refresh(opensearch::params::Refresh::WaitFor).send()diff --git a/crates/opensearch_client/src/upsert/agent_session/test.rs b/crates/opensearch_client/src/upsert/agent_session/test.rsindex 4ebf4579f5..6ee43fdc44 100644--- a/crates/opensearch_client/src/upsert/agent_session/test.rs+++ b/crates/opensearch_client/src/upsert/agent_session/test.rs@@ -71,6 +71,80 @@ fn index_override_selects_a_physical_backfill_index() {);}+/// Capture actual HTTP requests so both index and bulk serialization are covered.+async fn capture_reconcile_requests(index_override: Option<&str>) -> Vec<String> {+ use std::io::{BufRead, Read, Write};+ use std::net::TcpListener;+ use std::time::Duration;++ let listener = TcpListener::bind("127.0.0.1:0").unwrap();+ let address = listener.local_addr().unwrap();+ let server = std::thread::spawn(move || {+ let mut requests = Vec::new();+ for body in [+ r#"{}"#,+ r#"{"errors":false}"#,+ r#"{"failures":[],"timed_out":false}"#,+ ] {+ let (mut stream, _) = listener.accept().unwrap();+ stream+ .set_read_timeout(Some(Duration::from_secs(5)))+ .unwrap();+ let mut reader = std::io::BufReader::new(&mut stream);+ let mut request = String::new();+ reader.read_line(&mut request).unwrap();+ requests.push(request);+ let mut content_length = 0;+ loop {+ let mut line = String::new();+ reader.read_line(&mut line).unwrap();+ if line == "\r\n" {+ break;+ }+ if let Some(value) = line.to_ascii_lowercase().strip_prefix("content-length:") {+ content_length = value.trim().parse::<usize>().unwrap();+ }+ }+ reader.read_exact(&mut vec![0; content_length]).unwrap();+ write!(stream, "HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}", body.len()).unwrap();+ }+ requests+ });+ let client =+ crate::OpensearchClient::new(format!("http://{address}"), "".into(), "".into()).unwrap();+ let result = client+ .reconcile_agent_session(&args(), index_override)+ .await;+ result.unwrap();+ server.join().unwrap()+}++#[tokio::test]+async fn normal_parent_and_bulk_writes_require_an_alias() {+ let requests = capture_reconcile_requests(None).await;+ assert!(+ requests[0].starts_with("POST /agent_sessions/_doc/session-1?"),+ "{requests:?}"+ );+ assert!(requests[1].starts_with("POST /agent_sessions/_bulk?"));+ for request in &requests[..2] {+ assert!(request.contains("require_alias=true"), "{request}");+ }+}++#[tokio::test]+async fn explicit_backfill_overrides_allow_a_physical_index() {+ let requests = capture_reconcile_requests(Some("scratch-index")).await;+ assert!(+ requests[0].starts_with("POST /scratch-index/_doc/session-1?"),+ "{requests:?}"+ );+ assert!(requests[1].starts_with("POST /scratch-index/_bulk?"));+ for request in &requests[..2] {+ assert!(request.contains("require_alias=false"), "{request}");+ }+}+/// Uses only a uniquely named scratch index; never changes application indices.#[tokio::test]#[ignore = "requires local OpenSearch at http://localhost:9200"]diff --git a/infra/stacks/opensearch/helpers/README.md b/infra/stacks/opensearch/helpers/README.mdindex c6ab7e3782..ee5a7daa1a 100644--- a/infra/stacks/opensearch/helpers/README.md+++ b/infra/stacks/opensearch/helpers/README.md@@ -42,11 +42,79 @@ that can be swapped without a code deploy.All migration scripts default to `DRY_RUN=true`; pass `DRY_RUN=false` to apply.+## Manual provisioning++Provision indices and aliases manually using the canonical declarations in+`create_indices.ts`, and verify them before deploying their consumers. Deployment+does not create or reconcile indices.++Normal agent-session parent and bulk writes set `require_alias=true`. A missing+alias fails the write instead of automatically creating a bare index with an+inferred mapping. Explicit backfill `index_override` requests may still target+a physical index. This guard does not provision a schema or recover events+already dropped by the consumer.++## One-time repair: incorrectly auto-created agent-session index++This is an operator procedure, separate from deployment. Confirm that+`agent_sessions` is a bare physical index with the incorrect mapping, not an+existing alias. If it is already an alias, inspect its target before proceeding.+Configure the intended cluster's usual `OPENSEARCH_*` credentials and use the+production VPC connection/tunnel when appropriate.++Create the versioned destination from the canonical declaration and verify it:++```sh+ENVIRONMENT=prod INDEX=agent_sessions bun scripts/create_indices.ts+ENVIRONMENT=prod INDEX=agent_sessions DRY_RUN=false bun scripts/create_indices.ts+ENVIRONMENT=prod INDEX=agent_sessions bun scripts/verify_mappings.ts+```++Inspect `GET /agent_sessions_v1/_mapping` against `AGENT_SESSIONS_V1_BODY` in+`create_indices.ts`: the existing verifier checks field presence and types,+not all mapping parameters. Confirm `dynamic:false`, the+`agent_session_relation` join from `agent_session` to `message`, field-alias+paths, and date formats before proceeding.++A bare index blocks alias creation, so creation deliberately leaves the alias+for this manual cutover. If the destination already contains data, inspect it+before deciding to use it. Wait for `agent_sessions_v1` health to be at least+yellow (all primary shards available), and recheck the source/alias state.++For a future-writes-only repair, submit this **single** request through the+OpenSearch API after explicitly accepting deletion of the old search projection:++```http+POST /_aliases+{+ "actions": [+ { "remove_index": { "index": "agent_sessions" } },+ { "add": { "index": "agent_sessions_v1", "alias": "agent_sessions", "is_write_index": true } }+ ]+}+```++Do not issue a separate DELETE followed by alias creation: a writer could+recreate the bare index between those operations. The atomic request deletes+the old index and its search data; Postgres session data is untouched. It does+not reindex or backfill. The generic `reindex_with_alias_swap.ts` helper copies+old documents, so it is not a substitute for this no-copy operation.++Require an acknowledged response, then run `verify_mappings.ts` with+`INDEX=agent_sessions`. Inspect `GET /_alias/agent_sessions`: its sole target+must be `agent_sessions_v1` and `is_write_index` must be true or omitted, never+false. Create a new session, complete a turn, and verify its title and unique+phrases from both authors are searchable.+Check indexing errors. In-flight reconciles can straddle the cutover; backfill+those sessions and any desired historical sessions using the existing+`POST /internal/backfill/agent-sessions` endpoint. Record the repair outcome in+the incident runbook.+## Adding a field to an existing indexThe index bodies in `create_indices.ts` are the source of truth for mappings.Every body is `dynamic: 'false'`, so a field the service writes but the live-mapping lacks is dropped silently — and a search over it matches nothing+mapping lacks remains in `_source` but is not indexed — and a search over it matches nothingrather than erroring. That is how `call_records_v2` shipped without`properties`/`name` and left tag filters on calls returning empty (macro-2731).diff --git a/services/search_processing_service/README.md b/services/search_processing_service/README.mdindex 8293b1045d..487ae1a2f8 100644--- a/services/search_processing_service/README.md+++ b/services/search_processing_service/README.md@@ -42,6 +42,12 @@ the parent and both authors' folded messages, refreshes them, then removesdocuments from older projection generations. Missing sessions delete theirentire projection, even when triggered by an older event.+Provision and verify the `agent_sessions` index and alias manually before+deploying this consumer; deployment does not create indices. Normal parent and+bulk writes require an alias, so a missing alias fails instead of automatically+creating an index. An explicit backfill `index_override` permits physical+indices. Use the OpenSearch helper runbook for an explicit repair.+For rollout:1. Verify the `agent_sessions` alias exists with the join mapping defined in