Reliability architecture

We treat "stability" as the product's number-one feature. This page explains how we achieve that, and what happens when something goes wrong.

Why is reliability so hard?

Developers who connect directly to the official API run into things like:

• Regional rate limiting / unavailability
• A single payment channel or account getting banned
• Compatibility issues during model version switches
• Queuing at peak hours and spikes in first-token latency

If a relay service is just a simple proxy, these problems are passed through as-is to users, or even made worse. Our approach is to design the system on the premise that "upstreams are unstable," rather than treating that as an exception.

Our multi-source routing architecture

For every API call, the request is handled along the following path:

Core mechanisms:

• Multi-upstream pool: multiple independent channels sit behind the same model, avoiding single points of failure
• Health probing: we continuously monitor each upstream's success rate, first-token latency, and rate-limit rate, and unhealthy channels are automatically down-weighted or removed
• Automatic failover: when the primary channel returns 5xx / rate limiting / timeouts, the request switches to a backup channel within milliseconds, usually with no impact on your side
• Stream interruption reconnect: when a streaming response drops midway, we attempt to resume it on protocols that support it

Service level (SLA)

Our external commitments are grounded in "what we can actually deliver" — we don't make empty promises.

The Economy tier is for individual developers who want the best value and can tolerate occasional fluctuations; if your business has hard reliability requirements, choose the Enterprise tier or contact our sales team for a custom plan.

How availability is measured:

• We count only the request success rate "on the LMU AI side," excluding user-side network problems, parameter errors, and content refusals from the upstream model itself
• A single failure lasting < 60 seconds is not counted
• The monthly report includes success rates and P95 latency broken down by model and by time period

What happens during an incident

Brief jitter (< 5 minutes)

• The routing layer switches upstreams automatically, with no announcement
• The status page records the event

A single upstream outage (< 1 hour)

• The routing layer completes the switch; if it's visible to some users/models, we post it on the status page
• Standard and Enterprise users are notified by email

Regional or simultaneous multi-upstream impact

• The event is pinned on the status page with periodic updates
• Enterprise users get progress updates by email in real time
• A postmortem is published within 48 hours after the event ends

What we don't do

To avoid misleading users, we currently do not promise the following:

• Zero-latency, seamless switching: failover introduces extra latency from milliseconds to seconds, so it will never be completely seamless
• Cross-model compatibility compensation: if an upstream model itself is retired (for example, a model version is discontinued by the provider), we announce it rather than silently switching to a different model
• Unlimited retries: failed requests are retried a limited number of times per policy, so you don't pay for unexpected repeat calls

FAQ

Last updated: 2026-08-26