Reliability & Internals

Cost & Latency Baselines

Every successful call to infer() returns metrics tracking the financial and latency delta achieved by dynamic routing: costSaved and latencySaved.

Baseline Formula

Savings are calculated relative to the developer's configured Fallback Tier—not arbitrarily against the most expensive Frontier model on every prompt:

costSaved = (Cost at Fallback Tier) − (Cost at Allocated Tier)

Because the developer decides which models they trust for each tier, the fallback tier represents the baseline choice the developer would have statically used if Axon was not present.

Pricing Resolution & Merged Tables

Token costs are resolved from a merged pricing table:

  1. DEFAULT_PRICING: Built-in pricing map shipping with Axon covering standard flagship and flash models from OpenAI, Anthropic, and Google.
  2. config.pricing: Developer-provided pricing overrides or custom additions (e.g. self-hosted models or private provider endpoints).
  3. Precedence: Developer-supplied pricing entries always win in case of conflict.
const axon = new Axon({
  tiers: { ... },
  fallbackTier: "balanced",
  pricing: {
    // Custom model pricing (dollars per 1M tokens)
    "meta-llama/llama-3.1-70b": {
      promptPricePerMillion: 0.59,
      completionPricePerMillion: 0.79,
    },
  },
});

The Null-vs-Zero Rule

Axon enforces strict telemetry semantics regarding missing data:

ScenarioReturn ValueMeaning
Calculated 0 delta0Allocated tier is the fallback tier; 0 delta saved.
Missing pricing / baseline datanullSavings could not be computed (distinct from zero).
Never returnedNaN / undefinedGuaranteed never returned; strict numbers or null only.
Non-Blocking Observability

Missing token pricing never blocks or throws in infer(). Missing pricing emits a one-time console warning during new Axon(...) instantiation and cleanly returns null for costSaved at runtime. Cost tracking is pure observability, never a routing bottleneck.