Replay & Trace Comparison

The Replay Layer captures every orchestration decision as an immutable Trace and provides tools to compare, replay, and analyse those traces. This is the foundation of Layerr’s self-improving nature.

What Is a Trace?

A trace is a complete record of one request’s journey through the Layerr system:

interface Trace {
  traceId: string;
  workspaceId: string;
  createdAt: Date;
  intent: IntentClassification;
  workload: WorkloadProfile;
  strategy: string;
  routingDecision: {
    primaryProvider: string;
    primaryModel: string;
    fallbackChain: string[];
    scores: ScoreBreakdown;
  };
  execution: {
    attempts: Attempt[];
    finalProvider: string;
    finalModel: string;
    latencyMs: number;
    tokensIn: number;
    tokensOut: number;
    costUsd: number;
  };
  explanation: Explanation;
}

Trace Storage

Traces are persisted in SQLite (storage/traces.ts and storage/replay.ts):

Function	Purpose
`listRecentTraces()`	Retrieves recent traces for a workspace
`compareTraceIds()`	Compares two traces side-by-side
`analyzeRecentTraces()`	Aggregates trends across recent traces
`buildReplayTrace()`	Reconstructs a full trace from raw data

Replay Studio

The Replay Studio (src/features/replay/OrchestrationReplayStudio.tsx) allows users to:

Replay a request: Send the same request again and compare outcomes
Test alternative strategies: “What if I had used Speed instead of Quality?”
Compare providers: See how Claude vs GPT-4o handled the same prompt
View economic impact: Compare costs across replays (src/replay/ReplayEconomicOverlay.tsx)

Trace Comparator

The comparator (replay/comparator.ts) produces diff-style comparisons:

Dimension	Comparison
Latency	Delta in milliseconds
Cost	Delta in USD
Quality	Side-by-side quality scores
Provider	Which provider/model was used
Explanation	Why different decisions were made

Trend Analysis

The trend engine (replay/analysis.ts) detects patterns across traces:

Analysis	Description
`analyzeTrend()`	Detects latency/cost/quality trends over time
Provider degradation	Identifies providers whose quality is declining
Strategy drift	Detects when a strategy is no longer optimal
Workload evolution	Tracks how workspace workloads change over time

Economic Replay

Every replay includes economic analysis (src/replay/ReplayEconomicOverlay.tsx):

Metric	Description
Base Cost	Cost of the original execution
Comparison Cost	Cost of the replayed/alternative execution
Savings	Amount saved (or overspent) by the chosen route
Cost Efficiency	Quality per dollar

Frontend Components

Component	File	Purpose
`ReplayComparisonView`	`src/replay/ReplayComparisonView.tsx`	Side-by-side trace comparison
`ReplayEconomicOverlay`	`src/replay/ReplayEconomicOverlay.tsx`	Cost analysis overlay
`TrendAnalysisView`	`src/replay/TrendAnalysisView.tsx`	Trend detection UI
`OrchestrationReplayStudio`	`src/features/replay/OrchestrationReplayStudio.tsx`	Full replay studio

File Reference

File	What It Does
`replay/comparator.ts`	Trace comparison logic. Produces diff-style outputs
`replay/analysis.ts`	Trend analysis and pattern detection across traces
`storage/replay.ts`	Replay storage and retrieval layer
`storage/traces.ts`	Core trace persistence
`src/replay/ReplayComparisonView.tsx`	Frontend comparison UI
`src/replay/ReplayEconomicOverlay.tsx`	Cost analysis UI
`src/replay/TrendAnalysisView.tsx`	Trend detection UI
`src/features/replay/`	Complete replay feature module (studio, models, hooks)

Integration

Execution Engine → writes traces after every request
Explainability → stores explanations as part of each trace
Economics → reads traces for cost attribution and savings analysis
Adaptive Learning → reads trace histories to learn from outcomes
Quality Evaluation → uses replay for A/B testing and calibration
Benchmarks → replays traces against new providers for benchmarking