62 lines
2.3 KiB
Markdown
62 lines
2.3 KiB
Markdown
# ADR-0003 — RabbitMQ as Event and Job Backbone
|
|
|
|
- **Status:** Accepted
|
|
- **Date:** 2026-09-07
|
|
- **Deciders:** ASF Architecture Team
|
|
|
|
## Context
|
|
|
|
ASF has two asynchronous messaging needs:
|
|
1. **Domain events** (facts that happened — `artifact.created`,
|
|
`deployment.completed`, …) used for decoupled side effects.
|
|
2. **Asynchronous jobs** (AI inference, indexing, code analysis, testing)
|
|
that need separate queues with priorities and horizontal workers.
|
|
|
|
A single backbone that handles both patterns is preferred to reduce
|
|
infrastructure surface.
|
|
|
|
## Decision
|
|
|
|
Adopt **RabbitMQ 3.13+** as the sole event/job backbone.
|
|
|
|
### Logical queues
|
|
|
|
| Queue | Purpose | Priority |
|
|
|------------------------|----------------------------------------|----------|
|
|
| `asf.ai.interactive` | Latency-sensitive user-facing AI work | High |
|
|
| `asf.ai.background` | Batch / low-priority AI work | Normal |
|
|
| `asf.indexing` | Repository / document indexing | Low |
|
|
| `asf.code_analysis` | Static analysis tasks | Normal |
|
|
| `asf.testing` | Test execution requests | Normal |
|
|
| `asf.devops` | Deployment / CI orchestration | Normal |
|
|
| `asf.notifications` | Emails / webhooks / in-app notif. | Normal |
|
|
|
|
### Source of truth
|
|
RabbitMQ delivers work. **Persistent job state** (status, timestamps,
|
|
usage, error, correlation) lives in MongoDB `jobs` collection. Workers
|
|
update Mongo as they progress through `queued → running → completed|failed`.
|
|
|
|
RabbitMQ is **never** the permanent system of record for job state.
|
|
|
|
### Event format
|
|
Every event message contains:
|
|
```
|
|
event_id, event_type, timestamp, tenant_id, project_id?,
|
|
correlation_id, causation_id?, payload
|
|
```
|
|
|
|
Large payloads / binaries go to MinIO; the message carries only a
|
|
reference.
|
|
|
|
## Consequences
|
|
|
|
### Positive
|
|
- One broker, well-understood routing / priority / TTL / DLX semantics.
|
|
- Workers are stateless, horizontally scalable processes.
|
|
- At-least-once delivery matches domain; idempotency keys on workers
|
|
prevent duplicate side-effects.
|
|
|
|
### Negative
|
|
- Exactly-once is not provided — consumers must be idempotent.
|
|
- Operational complexity of a broker (mitigated by managed services in prod).
|