Routing Engine
The routing engine is the core of Layerr. It is not rule-based configuration. It is a dynamic scoring system that analyses the incoming request, the workspace profile, the workload context, and real-time provider conditions to pick the best model for the job.
Key Files
Section titled “Key Files”| File | Purpose |
|---|---|
routing/enrichment.ts | Enriches routing decisions with workspace, provider, and workload context |
routing/prompt-builder.ts | Builds the orchestration prompt for the strategy engine |
routing/types.ts | Core routing types: RoutingInput, RoutingDecision, EnrichedDecision |
orchestration/orchestrator.ts | Main orchestrator that coordinates the full routing pipeline |
orchestration/fallback.ts | Builds fallback chains when the primary provider fails |
providers/resolution.ts | Resolves provider URLs, credentials, and capability matching |
The Routing Pipeline
Section titled “The Routing Pipeline”
1. Enrichment
Section titled “1. Enrichment”enrichRoutingDecision takes a raw routing input and enriches it with:
- Workspace configuration (active profile, budgets, preferences)
- Provider health status (latency, error rate, rate limit headroom)
- Workload context (language, stack, complexity estimate)
- Historical performance (previous similar requests, calibration data)
2. Scoring
Section titled “2. Scoring”The QualityScoringEngine.scoreTrace method scores each candidate provider on:
- Capability match: Can this model handle the detected language/stack?
- Cost efficiency: Cost per token for this workload type
- Latency: Recent p50/p99 response times
- Reliability: Error rate and circuit breaker status
- Confidence grade: Historical accuracy on similar requests
3. Fallback Chain
Section titled “3. Fallback Chain”buildFallbackChain constructs an ordered list of providers to try:
- Primary: Highest scored provider
- Secondary: Next best, different provider family (diversification)
- Tertiary: Cheapest viable option
- Emergency: Self-hosted / open-source fallback
4. Provider Resolution
Section titled “4. Provider Resolution”resolveProviderChain resolves URLs, injects credentials, and verifies rate limit headroom before attempting the request.
Routing Decisions
Section titled “Routing Decisions”Every routing decision produces a RoutingDecision record with:
- Chosen provider and model
- Full candidate list with scores
- Fallback chain
- Rationale (human-readable explanation)
- Estimated cost
- Confidence score
These decisions are stored and replayable. See Replay Platform for details.