Routing Engine

The Routing Engine is the heart of Layerr. It takes an incoming request, enriches it with context, scores providers, builds a fallback chain, and produces an execution plan, all within milliseconds.

The Routing Pipeline

[1] Classify Intent          -- workflow/classifier.ts
         |
         v
[2] Analyze Workload         -- workload/analyzer.ts
         |
         v
[3] Resolve Strategy         -- strategy/engine.ts
         |
         v
[4] Enrich Routing Decision  -- routing/enrichment.ts (THE CORE)
         |
         v
[5] Score Candidates         -- scoring/scorer.ts
         |
         v
[6] Build Fallback Chain     -- orchestration/fallback.ts
         |
         v
[7] Resolve Provider Chain   -- providers/resolution.ts
         |
         v
[8] Execute                  -- gateway/handler.ts

Core Enrichment Function

The enrichment function (routing/enrichment.ts) is where the magic happens:

function enrichRoutingDecision(
  intent: IntentClassification,
  workload: WorkloadProfile,
  strategy: Strategy,
  providers: ProviderMetadata[],
  telemetry: ProviderTelemetry
): RoutingDecision {
  // 1. Filter ineligible providers (capability mismatch)
  const eligible = resolveCapabilities(workload, providers);

  // 2. Score each eligible provider
  const scored = eligible.map(p => scoreTrace(p, workload, strategy, telemetry));

  // 3. Apply strategy weights
  const weighted = applyStrategyWeights(scored, strategy.weights);

  // 4. Build fallback chain
  const fallbackChain = buildFallbackChain(weighted, strategy.constraints);

  return {
    primary: weighted[0],
    fallbackChain,
    explanation: buildRoutingExplanation(weighted, strategy)
  };
}

Prompt Builder

The routing prompt builder (routing/prompt-builder.ts) assembles prompts that help the routing system understand the request better. It includes:

1. Request summary: What the user is asking for
2. File context: Active files, language, stack
3. Historical patterns: What worked for similar requests
4. Workspace preferences: Strategy, budget, restrictions

Fallback Chain Construction

The fallback builder (orchestration/fallback.ts) ensures resilience:

Diversity Rules

Rule	Description
Provider diversity	No two consecutive providers from the same company
Model family diversity	Mix of model architectures for resilience
Geographic diversity	Different regions for availability
Tier diversity	Mix of premium and standard models

Fallback Ordering

function buildFallbackChain(
  scoredProviders: ScoredCandidate[],
  constraints: StrategyConstraints
): ProviderCandidate[] {
  // 1. Sort by composite score
  const ordered = scoredOrder(scoredProviders);

  // 2. Apply diversity rules
  const diverse = applyDiversityRules(ordered);

  // 3. Respect constraints (max cost, min quality)
  return filterByConstraints(diverse, constraints);
}

Routing Types

Key type definitions (routing/types.ts):

interface RoutingDecision {
  primary: ScoredCandidate;
  fallbackChain: ProviderCandidate[];
  scores: ScoreBreakdown;
  explanation: Explanation;
  estimatedCost: number;
  estimatedLatency: number;
}

interface ScoreBreakdown {
  quality: number;
  speed: number;
  cost: number;
  health: number;
  workspaceFit: number;
  weightedTotal: number;
}

Real-Time Considerations

The routing engine is designed for sub-100ms decision latency:

Optimization	Implementation
Cached intelligence	Provider intelligence cached in memory
Pre-computed scores	Quality scores updated asynchronously
Lazy scoring	Only score top-N candidates, not all
Fast path	Direct routing for high-confidence simple requests

File Reference

File	Purpose
routing/enrichment.ts	Core routing enrichment logic
routing/prompt-builder.ts	Routing prompt assembly
routing/types.ts	Routing type definitions
orchestration/fallback.ts	Fallback chain construction
src/workspace/control/RoutingPreview.tsx	Frontend routing preview
src/features/replay/RoutingExplanationTimeline.tsx	Routing explanation timeline
src/features/replay/DecisionBreakdownPanel.tsx	Score breakdown panel

Failure Modes

Failure	Cause	Mitigation
All providers exhausted	System-wide outage	Return 502 with full trace
No eligible providers	Capability mismatch	Return 400 with suggested alternatives
Budget exceeded	Workspace over limit	Return 402 with budget status
Timeout during routing	Slow health checks	Use cached health scores
Ambiguous intent	Low classification confidence	Route to most capable model