System Overview & Routing Flow
Axon is an SDK-based, multi-tier inference LLM router that sits directly in your application process. It categorizes prompts dynamically and routes them to the optimal model tier, achieving superior cost and latency efficiency without sacrificing quality on critical requests.
The 3-Tier Model Architecture
Rather than forcing developers to make manual, static choices between expensive flagship models and lightweight budget models, Axon organizes available models into three tiers:
Top-tier reasoning models (e.g. gpt-4o, claude-3-5-sonnet, o1). Reserved for tasks with high reasoning depth, irreversible side-effects, or large architectural blast radius.
Mid-tier models offering great general performance at reasonable latency and price. Handles the vast majority of day-to-day coding, transformations, and standard generations.
Ultra-fast, economical models (e.g. claude-3-5-haiku, gemini-1.5-flash). Ideal for trivial rewording, typo corrections, regex generation, and simple factual lookups.
Axon operates strictly on tiers. The developer assigns specific models and API keys to each tier. If a developer assigns claude-3-5-haiku to the Frontier tier, Axon routes frontier-classified prompts to that model without judgment.
The Two-Stage Inference Engine
Routing is executed in two consecutive stages to minimize latency and LLM token overhead:
- Stage 1 — The Gate (Deterministic Heuristics):
The prompt is evaluated in-process against sub-millisecond heuristic rules. If it satisfies the narrow trivial-allowlist and doesn't trigger any scope denylist vetoes, it is assigned directly to the Fast tier with zero LLM overhead and zero classification cost.
- Stage 2 — The Judge (Rubric-based LLM Classification):
If any heuristic rule fails or a scope veto fires, the prompt escalates to the Judge model. The Judge evaluates the prompt across four independent categorical axes (Blast Radius, Irreversibility, Reasoning Depth, Ambiguity) and determines the final tier via a deterministic table lookup.
Severity as a Hard Floor
In Axon, Severity is the primary hard floor for tier classification. When a prompt requires frontier-level capabilities (such as irreversible actions or deep multi-step reasoning), lesser tiers are excluded.
Unlike early experimental routers that attempted to blend numeric scores (1-100) or perform subjective tie-breaking between eligible tiers, Axon uses a deterministic categorical table lookup. The table produces exactly one tier, eliminating arbitrary tie-breaks or random routing fluctuations.
In-Process & Stateless Design
Axon does not run an external SaaS proxy or intermediary server:
- Zero Network Proxy Hop: Routing and model execution happen entirely inside your Node.js or TypeScript application runtime.
- Developer Data Privacy: Prompts and API keys never leave your process to an Axon-managed cloud. They go straight from your app to the respective model providers.
- Zero Visualization Mandate: Axon returns structured metrics (
costSaved,latencySaved,tier) directly to your code, leaving presentation or logging entirely up to your application architecture.