Open-source AI teammates in Slack, Teams, Discord, Telegram and Mattermost. Self-host it, with any agent harness and any model.
Python agent-orchestrationaiai-agentschatopsdiscordlangchainmcpmicrosoft-teamsmulti-agentopen-sourceopenaiself-hostedslack
Merge remote-tracking branch 'origin/asyncpg-message-write' into scale-core
Abel Dantas6f2e5cc4edc6
diff --cc CHANGELOG.mdindex bc67fcf48,430b32ccd..b7474239c--- a/CHANGELOG.md+++ b/CHANGELOG.md@@@ -159,18 -142,43 +159,55 @@@ version of their own to them without alcompares it only when someone signs in.#### Changed+- **Workspaces belong to organizations. Upgrade by replacing every pod, and+ downgrade the database before rolling back.** Every existing workspace gets+ an organization of its own, and its owners become the organization's admins.+ An organization admin administers every workspace in it without being added+ to each. A release from before organizations doesn't know about organization+ admins: if old and new pods serve together during a rolling update, or after+ a rollback that keeps the new database, an owner removed from a workspace by+ an old pod is still the organization's admin and can come back into it. So+ for this release, replace every switch-core pod at once rather than rolling+ them, and to roll back, first downgrade the database to the revision before+ `238c68b7306e` (`alembic downgrade 238c68b7306e-1`), then start the older+ release.+ - **The statements every message write costs run on asyncpg directly.**+ Taking the room's lock, reading its head, storing the agents' own events+ and reading the room's agent members again under the lock go straight to+ asyncpg on the transaction's connection (`db/raw.py`), as the delivery+ reads do, rather than through SQLAlchemy's execution.+ - **The two reads behind every delivery run on asyncpg directly.** A+ stream reads an agent's events, and counts its unread messages, for every+ event it delivers, and under load SQLAlchemy's work around each statement+ cost more than Postgres took to run it. `db/raw.py` runs a statement+ SQLAlchemy compiled once on the session's own asyncpg connection, in its+ transaction and under its tenant, with SQLAlchemy's type conversions and+ timed into `db.query.duration` as every other statement is.+ - **Pods start in seconds again with many rooms.** Every process checks at+ startup that each room has its clients, under a lock that lets one+ process at a time through, and read each room on its own to do it. With+ tens of thousands of rooms that took about 100 seconds, so processes+ starting together were restarted while they waited. Rooms are now read a+ thousand at a time.+ - **Idle agents behind a controller cost the database far less.** A+ controller stream's pass, including the one after every idle interval,+ read each of its agents' events in a session of its own; it now reads all+ their heads in one query and reads only the agents with something new. And+ each process's 30-second re-check of controllers held by another process+ read every agent's rooms with a query of its own, which grew with+ processes times agents; it is now one query per controller.+ - **A delivery to an agent behind a controller takes two database+ transactions instead of four.** The controller's stream read the heads of+ all its agents to find the work, again after delivering, and again before+ waiting. It now reads them once a pass, and an event that arrives after+ that read wakes the stream for the next pass.+ - **A controller keeps its socket when a delivery pass cannot get a+ database connection.** When a pod was too busy to hand its streams a+ connection within the pool timeout, the controller's stream ended and its+ socket closed, and every agent on it reconnected, which needs more of+ what was short. The pass is now tried again after a short wait, doubling+ to 5 seconds, and the socket closes only if it is still failing after a+ minute.- **An agent's stream reads in one database session per pass, and pooledconnections are no longer pinged before use.** A pass that delivered amessage opened a session for each of its reads, six pool checkouts and six@@@ -236,35 -235,27 +273,54 @@@group whose subgroup would move up next to a namesake is refused too, andthe dashboard now shows why. Names can no longer contain `/` or be blank.Existing duplicates are left as they are and can be reached by id.+ - **Delivering an event to an agent behind a controller no longer reads the+ agent's head first.** The notification of a new agent event already carries+ its number, and the controller's stream now keeps it as the agent's head, so+ a delivery costs the agent's own read alone, where it read the heads of all+ the controller's agents first. The heads are still read after each idle+ interval, when an agent is attached, and after the listener reconnects.+ - **Delivering an event to an agent costs three statements, where it cost+ seven.** The rooms dropped after the agent's cursor are read with its+ events, and the unread count reads the agent's baseline, the room's window+ and the agent's user id in the statement that counts.+ - **Writing a message no longer costs four statements for each agent it is+ for.** The agents' own events are stored together, in two statements and+ one savepoint, where each agent's event took a savepoint, a numbering, an+ insert and a release of its own.+ - **A controller's stream waits on three events, where it waited on one per+ agent and two more.** Every attached agent's doorbell now rings the stream's+ one bell.+ - **Reading an agent's own events no longer builds its query each time.**+ Each shape of the read is built once and run with bind parameters.#### Fixed+- **On Slack, the 👀 and the activity card no longer arrive after the agent's+ reply.** Every running turn's card was redrawn every 5 seconds to tick its+ clock. Slack closes a streamed card after 10 minutes, so on a longer turn+ each tick was a full message edit, and Slack allows about 50 of those a+ minute for the whole workspace. A few long turns used that up, Slack made the+ bridge wait, and new turns' cards and 👀 waited behind the clock redraws. The+ clock now moves once a minute, and a running card shows whole minutes+ ("Working… 3m"). Clock redraws stop while Slack is making the bridge wait,+ and new turns go ahead of them. The 👀 goes on as soon as a turn starts+ rather than once its card has posted. A card is redrawn as soon as Slack+ closes its stream, instead of showing still-running steps as failed until+ the next change. A turn whose first draw into its stream fails keeps that+ card rather than posting a second one under it.+- **An agent is told why a machine's CLI cannot be used.** When+ `create_agent` or `update_agent_detail` was refused because a provider was+ not logged in, it said the owner needed to log in, even when the machine had+ reported a login its agents could not use, which logging in does not fix+ (a Codex login in the OS keyring, say). The refusal now gives what the+ machine said instead, and `list_machines` shows it as each provider's+ `problem`.+- **A Teams app on a federated credential can deploy from the Helm chart.**+ switch-core's migrate init container and the pre-upgrade migration Job were+ given the Teams app's settings but not its projected token, so+ `TEAMS_APP_FEDERATED_TOKEN_FILE` named a missing file and both refused to+ load their configuration: with `switchCore.teamsApp.credential: federated`,+ no release could migrate or start. The two that migrate are no longer given+ the Teams app at all.- **The agent HTTP API answers a refused or malformed request with a 4xx, nota 500.** Inviting an agent to a room the caller may not change answered 500instead of 403, and so did looking up a room the agent is not in. A roomdiff --cc core/switch_core/db/tenant_session.pyindex 683d7c756,cdc7c65e9..401647dac--- a/core/switch_core/db/tenant_session.py+++ b/core/switch_core/db/tenant_session.py@@@ -207,21 -197,12 +207,21 @@@ def _refuse_a_flush_that_driftedsession: Session, flush_context: object, instances: object) -> None:"""The same check, on the path `do_orm_execute` does not see."""- _refuse_drift(session)+ refuse_drift(session)- def _refuse_drift(session: Session) -> None:+ def refuse_drift(session: Session) -> None:if _STAMPED not in session.info:return+ stamped_organization = session.info.get(_STAMPED_ORGANIZATION)+ bound_organization = current_organization_id()+ if bound_organization != stamped_organization:+ raise TenantBindingDriftError(+ f"this session's transaction was opened with organization "+ f"{stamped_organization!r} and the organization bound now is "+ f"{bound_organization!r}. Open the session inside the "+ "organization_scope, or commit this transaction before rebinding."+ )stamped = session.info[_STAMPED]bound = current_tenant_id()if bound == stamped: