Reliability & Internals

Failure Handling & BYOK Posture

In modern AI applications, API key invalidity and provider outages are not software bugs—they are routine runtime conditions. Axon is designed with a resilient, Bring-Your-Own-Key (BYOK) posture: failures produce structured status objects, fallback tiers maintain availability, and critical irreversible tasks are protected with strict confirmation stops.

Consistent Status Vocabulary

Axon uses the exact same structured status format for proactive health checks (before requests) and reactive failure reporting (during live inference):

type HealthStatus = Record<
  'frontier' | 'balanced' | 'fast' | 'fallback',
  'ok' | `failed: ${string}`
>;

// Example Health Check Output:
{
  frontier: 'ok',
  balanced: 'ok',
  fast: 'failed: invalid key',
  fallback: 'ok'
}

The Fallback Routing Mechanism

If an allocated model fails during a live request (due to a timeout, rate limit, or invalid key), Axon automatically attempts the prompt using the developer's configured fallbackTier:

  • Availability Over Optimization: In failure scenarios, delivering a valid response takes precedence over strict severity tiering.
  • Observability: The result is marked as InferDegraded with usedFallback: true, identifying the exact stage and provider reason for the primary tier's failure.

The Irreversibility Safety Stop

There is one deliberate, critical exception to the automatic fallback rule:

No Silent Downgrade on Irreversible Tasks

If the Judge classified a prompt as Irreversible (e.g. executing financial transactions, permanent data deletions, or publishing content) and the Frontier tier fails, Axon will NOT silently redirect the request to a degraded fallback model.

Instead, execution stops immediately and returns an InferStopped result:

const result = await axon.infer("Drop obsolete production partitions and re-index", {
  context: { codeContext: "DB migration script" },
});

if (!result.ok && result.needsConfirmation) {
  // result is InferStopped:
  console.warn("Frontier model failed for an irreversible prompt!");
  console.log("Reason:", result.reason);
  console.log("Proposed Fallback:", result.proposedFallback);

  // Present a confirmation prompt to your end-user:
  // "Frontier tier unavailable. Would you like to attempt this database operation with Balanced tier?"
}

Notice that in TypeScript, result.response does not exist on InferStopped. This compile-time guard ensures developers cannot accidentally execute an unverified or empty completion.

Unrecoverable Failures

If the primary allocated tier fails and the fallback tier also fails (or if the allocated tier was already the fallback tier and failed), Axon throws a runtime error. The error message combines diagnostic details from both provider attempts:

Error: [Axon] Inference failed.
  Allocated (frontier): failed: 429 Too Many Requests (OpenAI)
  Fallback (balanced): failed: invalid api key (Anthropic)

Internal Provider Result Architecture

Internally, provider adapters never throw raw exceptions for standard network or authentication issues. Every adapter call returns a strongly typed discriminated union:

type ProviderResult =
  | {
      ok: true;
      text: string;
      latencyMs: number;
      usage: ProviderUsage;
      provider: ProviderId;
      model: string;
    }
  | {
      ok: false;
      status: string;
      reason: string;
    };

This internal contract ensures that metrics capture (tokens, latency) and failure recovery occur cleanly without fragile try/catch side-effects.