66 lines
2.0 KiB
Markdown
66 lines
2.0 KiB
Markdown
# ADR-0004 — AI Gateway Abstraction Layer
|
|
|
|
- **Status:** Accepted
|
|
- **Date:** 2026-09-07
|
|
- **Deciders:** ASF Architecture Team
|
|
|
|
## Context
|
|
|
|
ASF requires LLM capabilities for:
|
|
- Code analysis
|
|
- Requirement decomposition
|
|
- Test generation
|
|
- Context-aware chat ("Ask ASF")
|
|
- Incident analysis
|
|
|
|
Target inference backends include on-prem DGX Spark, OpenAI-compatible
|
|
local APIs, commercial providers, and customer-hosted endpoints.
|
|
|
|
Tenants may have different AI policies (allowed models, budgets,
|
|
geographic restrictions).
|
|
|
|
## Decision
|
|
|
|
Introduce an **ASF AI Gateway** service (`services/asf-ai/`) with a
|
|
stable Python interface.
|
|
|
|
### Interface
|
|
|
|
Business code requests **capabilities**, never models:
|
|
|
|
```python
|
|
class AIGateway(Protocol):
|
|
async def analyze_code(self, request: CodeAnalysisRequest) -> AnalysisResult: ...
|
|
async def generate_tests(self, request: TestGenerationRequest) -> GeneratedTests: ...
|
|
async def chat(self, request: ChatRequest, ctx: AIContext) -> ChatResponse: ...
|
|
async def decompose_requirement(self, req: RequirementDecomposeRequest) -> Decomposition: ...
|
|
```
|
|
|
|
Provider adapters implement the gateway. Model selection, routing,
|
|
priority queuing, usage metering, cost tracking and tenant AI policy
|
|
live inside the gateway — not in callers.
|
|
|
|
### Bootstrap
|
|
|
|
The first implementation ships with a `MockProvider` that returns
|
|
deterministic canned responses with realistic latencies. Real providers
|
|
(OpenAI-compatible, NVIDIA, etc.) are added as adapters later without
|
|
touching domain/business code.
|
|
|
|
## Consequences
|
|
|
|
### Positive
|
|
- No LLM provider lock-in — swap / add adapters freely.
|
|
- Usage and cost tracking centralized.
|
|
- Tenant-level policy enforceable at one boundary.
|
|
- Tests for business logic run fast against the mock — no external API
|
|
calls, no flakiness.
|
|
|
|
### Negative
|
|
- One more abstraction — the initial API surface must be designed
|
|
carefully so future providers truly fit.
|
|
|
|
### Mitigation
|
|
- Provider adapters are written early for at least two real providers,
|
|
to validate the abstraction before domain code depends heavily on it.
|