Macro is a unified workspace for teams: email, chat, docs, tasks, agents, calls, and CRM — @-linked together with shared AI memory.
TypeScript agentaiai-agentsall-in-onecrmcrm-systememaillinearmcpmessagingnotesnotionofficerustslackslack-alternativestartupstartup-toolstypescriptworkspace
Wire property events into realtime Soup (#5269)
Sean Ayeb472c7c5856f
diff --git a/Cargo.lock b/Cargo.lockindex 3bea352058..6c334a5526 100644--- a/Cargo.lock+++ b/Cargo.lock@@ -13607,6 +13607,7 @@ dependencies = ["broadcast","channels","chat",+ "chrono","documents","email","entity_access",@@ -13616,7 +13617,9 @@ dependencies = ["macro_event_topics","macro_user_id","model-entity",+ "models_properties","projects",+ "properties","rdkafka","rootcause","serde",diff --git a/crates/properties/README.md b/crates/properties/README.mdindex 71f25132d1..d74ef930a0 100644--- a/crates/properties/README.md+++ b/crates/properties/README.md@@ -127,8 +127,11 @@ property definition. A future definition-edit operation can add that event as aforward-compatible enum variant. TAG definition creation uses`property.created`; there is no separate `tag_set.created` event.-The launch is producer-only. No `macro.properties` consumer ships in this-repository change.+The `soup_realtime` consumer subscribes to `macro.properties`. It maps entity+property updates, deletions, and clears to updated Soup entities and fans them+out to every current accessor. Task property entities map to their document+representation. Definition and option lifecycle events remain ignored because+they do not identify every affected entity.## Producer compositiondiff --git a/crates/soup_realtime/Cargo.toml b/crates/soup_realtime/Cargo.tomlindex e4e9af7225..309cb2aa08 100644--- a/crates/soup_realtime/Cargo.toml+++ b/crates/soup_realtime/Cargo.toml@@ -11,7 +11,9 @@ consumer = ["dep:documents","dep:email","dep:kafka_util",+ "dep:models_properties","dep:projects",+ "dep:properties","dep:rdkafka","dep:tokio","macro_event_broker/outbound",@@ -44,7 +46,9 @@ macro_event_broker = { path = "../macro_event_broker", default-features = falsemacro_event_topics = { path = "../macro_event_topics" }macro_user_id = { path = "../macro_user_id" }model-entity = { path = "../model-entity" }+models_properties = { path = "../models_properties", optional = true }projects = { path = "../projects", default-features = false, features = ["service"], optional = true }+properties = { path = "../properties", default-features = false, features = ["ports"], optional = true }rdkafka = { workspace = true, optional = true }rootcause = { workspace = true, optional = true }serde = { workspace = true }@@ -54,6 +58,7 @@ tracing = { workspace = true, optional = true }workspace-hack = { version = "0.1", path = "../workspace-hack" }[dev-dependencies]+chrono = { workspace = true }serde_json = { workspace = true }tokio = { workspace = true, features = ["macros", "rt-multi-thread", "test-util", "time"] }uuid = { workspace = true }diff --git a/crates/soup_realtime/README.md b/crates/soup_realtime/README.mdindex 9cffc0daf4..d8ac87477e 100644--- a/crates/soup_realtime/README.md+++ b/crates/soup_realtime/README.md@@ -1,8 +1,8 @@# soup_realtime-Hexagonal components for turning existing document, project, chat, email, and channel Kafka events into recipient-targeted Soup patches.+Hexagonal components for turning existing document, project, chat, email, channel, and property Kafka events into recipient-targeted Soup patches.-The inbound consumer maps each source event to `Patch::Updated(Entity)` or `Patch::Deleted(Entity)`. The domain service expands current access and publishes one patch per recipient to `macro.soup`; it does not hydrate or serialize `SoupItem` values. Channel threads use their containing channel as the access source.+The inbound consumer maps each source event to `Patch::Updated(Entity)` or `Patch::Deleted(Entity)`. Entity-property updates, deletions, and clears map to updated Soup entities; property task entities map to their document representation. Property definition and option lifecycle events are ignored because they do not identify every affected entity. The domain service expands current access and publishes one patch per recipient to `macro.soup`; it does not hydrate or serialize `SoupItem` values. Channel threads use their containing channel as the access source.The Soup topic carries `Patch<(MacroUserIdStr<'static>, Entity<'static>)>` and is also keyed by recipient user ID. Delivery is at least once: offsets are committed only after successful fan-out, and exhausted retries stop the consumer so its supervisor can restart it for redelivery. Downstream consumers must tolerate duplicate patches.diff --git a/crates/soup_realtime/src/inbound/kafka_consumer.rs b/crates/soup_realtime/src/inbound/kafka_consumer.rsindex 5946a26a3c..7d9b0c706e 100644--- a/crates/soup_realtime/src/inbound/kafka_consumer.rs+++ b/crates/soup_realtime/src/inbound/kafka_consumer.rs@@ -18,7 +18,9 @@ use macro_event_broker::{KafkaConsumerAdapter, MacroEvent as _, MacroEventCollection as _, MacroEventConsumerService,};use model_entity::{Entity, EntityType};+use models_properties::EntityType as PropertyEntityType;use projects::domain::events::{ProjectMacroEvent, ProjectTopicEvent};+use properties::domain::events::{PropertyMacroEvent, PropertyTopicEvent};use rdkafka::consumer::CommitMode;use rdkafka::message::{BorrowedMessage, Message as _};use rootcause::prelude::{Report, ResultExt as _};@@ -48,6 +50,7 @@ macro_event_broker::declare_topics!(ChatMacroEvent,EmailMacroEvent,ChannelMacroEvent,+ PropertyMacroEvent,);/// Total service attempts before returning for supervisor-driven redelivery.@@ -326,6 +329,45 @@ fn patches_from_channel_event(event: &ChannelTopicEvent) -> Vec<SoupRealtimePatc}}+fn soup_entity_type_from_property(entity_type: PropertyEntityType) -> Option<EntityType> {+ match entity_type {+ PropertyEntityType::CallRecord => Some(EntityType::Call),+ PropertyEntityType::Chat => Some(EntityType::Chat),+ PropertyEntityType::Company => Some(EntityType::CrmCompany),+ PropertyEntityType::Document | PropertyEntityType::Task => Some(EntityType::Document),+ PropertyEntityType::Project => Some(EntityType::Project),+ PropertyEntityType::Thread => Some(EntityType::EmailThread),+ // Soup channels do not expose properties, and users are not Soup items.+ PropertyEntityType::Channel | PropertyEntityType::User => None,+ }+}++fn property_update(entity_type: PropertyEntityType, entity_id: &str) -> Vec<SoupRealtimePatch> {+ soup_entity_type_from_property(entity_type)+ .map(|entity_type| update(entity_type, entity_id))+ .into_iter()+ .collect()+}++fn patches_from_property_event(event: &PropertyTopicEvent) -> Vec<SoupRealtimePatch> {+ match event {+ PropertyTopicEvent::EntityPropertyUpdated(metadata) => {+ property_update(metadata.entity_type, &metadata.entity_id)+ }+ PropertyTopicEvent::EntityPropertyDeleted(metadata) => {+ property_update(metadata.entity_type, &metadata.entity_id)+ }+ PropertyTopicEvent::EntityPropertiesCleared(metadata) => {+ property_update(metadata.entity_type, &metadata.entity_id)+ }+ PropertyTopicEvent::Created(_)+ | PropertyTopicEvent::Deleted(_)+ | PropertyTopicEvent::OptionCreated(_)+ | PropertyTopicEvent::OptionUpdated(_)+ | PropertyTopicEvent::OptionDeleted(_) => Vec::new(),+ }+}+fn patches_from_event(event: &DeclaredMacroEvent) -> Vec<SoupRealtimePatch> {match event {DeclaredMacroEvent::DocumentMacroEvent(event) => {@@ -341,6 +383,9 @@ fn patches_from_event(event: &DeclaredMacroEvent) -> Vec<SoupRealtimePatch> {DeclaredMacroEvent::ChannelMacroEvent(event) => {patches_from_channel_event(&event.event().event)}+ DeclaredMacroEvent::PropertyMacroEvent(event) => {+ patches_from_property_event(&event.event().event)+ }}}diff --git a/crates/soup_realtime/src/inbound/kafka_consumer/test.rs b/crates/soup_realtime/src/inbound/kafka_consumer/test.rsindex 00c28d745c..c823013078 100644--- a/crates/soup_realtime/src/inbound/kafka_consumer/test.rs+++ b/crates/soup_realtime/src/inbound/kafka_consumer/test.rs@@ -8,6 +8,7 @@ use channels::domain::{models::ChannelSender,};use chat::domain::events::{ChatMessageDeletedMetadata, ChatTopicEvent, ChatUpdatedMetadata};+use chrono::Utc;use documents::domain::events::{DocumentCreatedMetadata, DocumentDeletedMetadata, DocumentInteractionMetadata,DocumentUpdatedMetadata, InteractionReason,@@ -18,6 +19,9 @@ use email::domain::events::{use macro_event_broker::{Event, EventBrokerError, MacroEventCollection as _, MessageParts};use macro_user_id::user_id::MacroUserIdStr;use projects::domain::events::{ProjectDeletedMetadata, ProjectTopicEvent};+use properties::domain::events::{+ EntityPropertiesClearedMetadata, EntityPropertyDeletedMetadata, EntityPropertyUpdatedMetadata,+};use uuid::Uuid;use super::*;@@ -104,6 +108,7 @@ fn subscribes_to_all_existing_soup_source_topics() {"macro.chats","macro.email","macro.channels",+ "macro.properties",]);}@@ -253,6 +258,71 @@ fn deleted_chat_messages_do_not_change_soup() {assert!(patches_from_chat_event(&event).is_empty());}+#[test]+fn task_property_updates_map_to_document_updates() {+ let event = PropertyTopicEvent::EntityPropertyUpdated(EntityPropertyUpdatedMetadata {+ entity_property_id: Uuid::now_v7(),+ entity_id: DOCUMENT_ID.to_string(),+ entity_type: PropertyEntityType::Task,+ property_definition_id: Uuid::now_v7(),+ actor_user_id: Some(user()),+ value: None,+ updated_at: Utc::now(),+ });++ let patches = patches_from_property_event(&event);+ assert_eq!(patches.len(), 1);+ assert!(matches!(patches[0].patch, Patch::Updated(_)));+ assert_eq!(patch_entity(&patches[0]).entity_type, EntityType::Document);+ assert_eq!(patch_entity(&patches[0]).entity_id, DOCUMENT_ID);+}++#[test]+fn deleting_or_clearing_properties_updates_the_soup_entity() {+ let thread_id = Uuid::now_v7().to_string();+ let deleted = PropertyTopicEvent::EntityPropertyDeleted(EntityPropertyDeletedMetadata {+ entity_property_id: Uuid::now_v7(),+ entity_id: thread_id.clone(),+ entity_type: PropertyEntityType::Thread,+ property_definition_id: Uuid::now_v7(),+ actor_user_id: Some(user()),+ });+ let company_id = Uuid::now_v7().to_string();+ let cleared = PropertyTopicEvent::EntityPropertiesCleared(EntityPropertiesClearedMetadata {+ entity_id: company_id.clone(),+ entity_type: PropertyEntityType::Company,+ actor_user_id: Some(user()),+ });++ let deleted = patches_from_property_event(&deleted);+ assert!(matches!(deleted[0].patch, Patch::Updated(_)));+ assert_eq!(+ patch_entity(&deleted[0]).entity_type,+ EntityType::EmailThread+ );+ assert_eq!(patch_entity(&deleted[0]).entity_id, thread_id);++ let cleared = patches_from_property_event(&cleared);+ assert!(matches!(cleared[0].patch, Patch::Updated(_)));+ assert_eq!(+ patch_entity(&cleared[0]).entity_type,+ EntityType::CrmCompany+ );+ assert_eq!(patch_entity(&cleared[0]).entity_id, company_id);+}++#[test]+fn property_events_for_non_property_soup_items_are_ignored() {+ for entity_type in [PropertyEntityType::Channel, PropertyEntityType::User] {+ let event = PropertyTopicEvent::EntityPropertiesCleared(EntityPropertiesClearedMetadata {+ entity_id: DOCUMENT_ID.to_string(),+ entity_type,+ actor_user_id: Some(user()),+ });+ assert!(patches_from_property_event(&event).is_empty());+ }+}+#[test]fn email_state_events_map_to_updated_or_deleted_patches() {let thread_id = Uuid::now_v7();