Provider System

The Provider System is the backbone of Layerr’s multi-provider orchestration. It abstracts every LLM provider behind a unified interface, handles provider discovery, health monitoring, and adapts requests/responses across different APIs.

Architecture

Provider Registry
    |
    v
[Capability Matching] -- filters by workload requirements
    |
    v
[Health Filtering]  -- removes unhealthy providers
    |
    v
[Resolution]        -- resolves URLs, handles auth
    |
    v
[Scoring]           -- ranks remaining candidates
    |
    v
[Adapter]           -- transforms to provider-specific format
    |
    v
[Execution]         -- sends request via gateway

Provider Registry

The registry (providers/registry.ts) maintains metadata for every connected provider:

Field	Description
id	Unique provider identifier (e.g., “openai-prod”)
type	Provider type: openai, anthropic, ollama, custom
name	Human-readable name
baseUrl	API base URL
models	Supported models with metadata
capabilities	Feature flags (code, reasoning, vision, etc.)
pricing	Token pricing per model
region	Geographic region
status	active, inactive, maintenance

Provider Types

Type	Adapter File	Description
OpenAI-compatible	providers/openai-compat/adapter.ts	OpenAI, Groq, Together, Anyscale, custom OpenAI-compatible
Anthropic	Part of OpenAI-compat adapter with Claude-specific transforms	Claude, Claude Instant
Ollama	providers/ollama/adapter.ts	Local self-hosted models
Custom	User-defined adapter	Any provider with custom API

Health Monitoring

The health system (providers/health/aggregator.ts) tracks:

Metric	Source	Update Frequency
Latency p50/p99	Execution traces	Every request
Error rate	Execution outcomes	Every request
Uptime	Health probes	Every 60 seconds
Success rate	Trace analysis	Rolling 1-hour window
Throughput	Request counts	Real-time

Health Adjustments

The health adjuster (providers/health/adjustments.ts) dynamically modifies provider scores:

• Error spike penalty: Temporary score reduction after consecutive failures
• Latency degradation: Score reduction if p99 latency exceeds threshold
• Recovery boost: Score restoration after sustained healthy period
• Region outage: Zero score for providers in affected regions

Provider Intelligence

Provider intelligence (providers/intelligence.ts) maintains a knowledge base about each provider:

Intelligence	Description
Model capabilities	Which tasks each model excels at (code, creative, reasoning)
Cost efficiency	Tokens per dollar for different workload types
Quality patterns	Historical quality scores per workload category
Failure modes	Common failure patterns and their predictors
Seasonal trends	Time-of-day and day-of-week performance variations

Capability Matching

The capability system ensures providers can actually handle the workload:

Capability	Description
code	Code generation and understanding
reasoning	Multi-step logical reasoning
vision	Image understanding
long-context	>32k context window
function-calling	Tool/function calling
json-mode	Structured JSON output
streaming	Real-time streaming responses
low-latency	<500ms time-to-first-token

Resolution

The resolver (providers/resolution.ts) handles:

1. URL resolution: Converts provider IDs to actual API URLs
2. Authentication: Injects API keys from secure storage
3. Proxy configuration: Routes through enterprise proxies if configured
4. SSL/TLS: Certificate validation and custom CA support

File Reference

File	Purpose
providers/registry.ts	Provider registry and CRUD
providers/registry-db.ts	Database layer for provider persistence
providers/registry-types.ts	TypeScript type definitions
providers/resolution.ts	URL and auth resolution
providers/metadata.ts	Provider metadata management
providers/intelligence.ts	Provider intelligence and historical patterns
providers/capabilities.ts	Capability definitions and matching
providers/openai-compat/adapter.ts	OpenAI-compatible API adapter
providers/ollama/adapter.ts	Ollama local model adapter
providers/health/aggregator.ts	Health metrics aggregation
providers/health/adjustments.ts	Dynamic health score adjustments

Frontend Components

Component	File	Purpose
ProviderTopologyMap	src/providers/ProviderTopologyMap.tsx	Visual provider network map
ProviderCapabilityMatrix	src/providers/ProviderCapabilityMatrix.tsx	Capability comparison grid
ProviderDetailPanel	src/providers/ProviderDetailPanel.tsx	Provider detail view
ProviderIntelligenceInspector	src/providers/ProviderIntelligenceInspector.tsx	Provider intelligence dashboard
ProviderRecommendations	src/providers/ProviderRecommendations.tsx	Provider setup recommendations
LocalVsCloudSplit	src/providers/LocalVsCloudSplit.tsx	Local vs cloud provider comparison
FallbackChainViz	src/providers/FallbackChainViz.tsx	Visual fallback chain
OrchestrationProviderCard	src/providers/OrchestrationProviderCard.tsx	Provider card component