asf/docs/adr/ADR-0002-mongodb-primary-store.md

ADR-0002 — MongoDB as Primary Persistence

• Status: Accepted
• Date: 2026-09-07
• Deciders: ASF Architecture Team

Context

ASF requires a primary store for Organizations, Projects, Artifacts, ArtifactRelationships, Jobs, AuditEvents and similar tenant-scoped records.

Key properties required:

• Flexible schema: the generic Artifact model needs extensible metadata.
• Document-level locality: an Artifact document bundles relationships list, metadata and version.
• Horizontal scaling and multi-tenancy (every query is scoped by tenant_id).
• Async/await Python driver support.

Decision

Adopt MongoDB 7.x as the only primary operational database.

Use compound indexes prefixed with tenant_id for all tenant-owned collections. Collections are intentionally few:

• organizations
• users
• teams
• projects
• artifacts
• artifact_relationships
• integrations
• jobs
• ai_executions
• audit_events

We do NOT create a collection per artifact type. Every artifact type (requirement, task, commit, test, deployment…) lives in the single artifacts collection discriminated by type.

MinIO holds large binaries / reports; MongoDB stores the object reference and metadata.

Consequences

Positive

• Flexible metadata on Artifacts matches the "Everything is an Artifact" principle without schema migrations for every new artifact subtype.
• Compound (tenant_id, project_id, …) indexes give fast tenant-scoped reads and guarantee isolation at query time.
• Native async driver (motor) integrates naturally with FastAPI.

Negative

• No foreign-key constraints across collections — referential integrity is an application responsibility (mitigated via service-layer validations and tests).
• Relationship graph traversal requires multiple queries or the $graphLookup aggregation stage. The schema is intentionally designed so that a future graph-database projection is possible without rewriting domain logic.

Follow-up

• ADR for graph-database introduction once traversal patterns stabilize.
• ADR for vector / embedding store (Semantic Context Engine phase).