2.3 KiB
2.3 KiB
ADR-0003 — RabbitMQ as Event and Job Backbone
- Status: Accepted
- Date: 2026-09-07
- Deciders: ASF Architecture Team
Context
ASF has two asynchronous messaging needs:
- Domain events (facts that happened —
artifact.created,deployment.completed, …) used for decoupled side effects. - Asynchronous jobs (AI inference, indexing, code analysis, testing) that need separate queues with priorities and horizontal workers.
A single backbone that handles both patterns is preferred to reduce infrastructure surface.
Decision
Adopt RabbitMQ 3.13+ as the sole event/job backbone.
Logical queues
| Queue | Purpose | Priority |
|---|---|---|
asf.ai.interactive |
Latency-sensitive user-facing AI work | High |
asf.ai.background |
Batch / low-priority AI work | Normal |
asf.indexing |
Repository / document indexing | Low |
asf.code_analysis |
Static analysis tasks | Normal |
asf.testing |
Test execution requests | Normal |
asf.devops |
Deployment / CI orchestration | Normal |
asf.notifications |
Emails / webhooks / in-app notif. | Normal |
Source of truth
RabbitMQ delivers work. Persistent job state (status, timestamps,
usage, error, correlation) lives in MongoDB jobs collection. Workers
update Mongo as they progress through queued → running → completed|failed.
RabbitMQ is never the permanent system of record for job state.
Event format
Every event message contains:
event_id, event_type, timestamp, tenant_id, project_id?,
correlation_id, causation_id?, payload
Large payloads / binaries go to MinIO; the message carries only a reference.
Consequences
Positive
- One broker, well-understood routing / priority / TTL / DLX semantics.
- Workers are stateless, horizontally scalable processes.
- At-least-once delivery matches domain; idempotency keys on workers prevent duplicate side-effects.
Negative
- Exactly-once is not provided — consumers must be idempotent.
- Operational complexity of a broker (mitigated by managed services in prod).