Architecture
Olyx sits between your application and the model path you choose for a project. Think of this page as the map: it shows where requests go and where the important boundaries sit.
During closed beta, read the architecture docs as the product shape Olyx is stabilizing: SDK-first execution, project-scoped routing, traceable model calls, and optional outbound agent routes for selected private deployments. For how-to detail, follow the linked guides instead of treating this page as the full reference.
SDK Request Path
The SDKs all expose the same flow in different languages: create a trace, call execute, complete the trace, then read
the result when you need details. For examples, start with the language pages under SDKs and
the lifecycle guide in Traces.
The Request Flow
The request flow stays the same across languages: accept the SDK call, resolve the project and route, execute the selected model path, and record the trace step.
The trace does not replace application logs. It gives you an AI-specific record of the selected model, the steps that ran, how long they took, what they cost, and where the workflow stopped if it failed.
System Boundaries
Olyx is not meant to own every system around your AI application. Your application still owns business logic, tool execution, and user-facing workflows.
| Boundary | What crosses it |
|---|---|
| Application to Olyx | Inputs, messages, metadata, tool results, and trace identifiers |
| Olyx to provider | Provider-ready request data and model parameters |
| Application to tools | Tool calls your application chooses to execute locally or through MCP |
| Agent to private systems | Work dispatched to selected private routes in beta deployments |
| Olyx to traces | Step records, timing, routing, cost, and summary fields |
Keep provider credentials out of application call sites when using the governed path. Application services should hold Olyx API keys; provider keys and model configuration belong in project settings.
Request Lifecycle
Use this lifecycle when you are debugging a request. Each stage should leave enough information to explain what happened without guessing from provider logs alone.
| Stage | What happens | What to inspect |
|---|---|---|
| Create trace | Your app opens an operational record for the workflow. | Trace metadata and revenue, if provided |
| Execute | Olyx resolves the project, chooses a route, and calls the selected model path. | Step output, selected model, status |
| Continue tools | If the model asks for a tool, your app executes it and sends results back. | tool_call and tool_result steps |
| Complete trace | Olyx calculates summary fields and optimization grade. | Cost, latency, grades, chain depth |
| Read trace | Your app or dashboard retrieves the diagnostic record. | Summary, graph, steps, routing decision |
For tool-heavy workflows, see MCP and Performance. For cost and grade interpretation, see Cost Intelligence.
Private Agent Routes
Private routes use an outbound-only Olyx Agent for selected beta deployments. The agent runs inside your infrastructure and initiates the connection to Olyx. This keeps private model and tool endpoints off the public internet while letting the application keep the same SDK workflow.
Private routes are useful when a workload needs local model access, internal tool access, or stricter network placement. Start with the standard gateway path unless your data, provider, or network requirements need the agent.
At the call site, the app still uses execute; the routing metadata tells Olyx when to pick the private path. See the
private-agent details in Performance and Traces.
Routing behavior depends on project configuration. If a private route is unavailable for a workload that requires one, the request should stop rather than silently switching to an unintended model path.
Failure Model
The supported path is designed to stop cleanly when the gateway cannot authenticate, route, execute, or record the work. This is more useful than a hidden fallback because developers can see the failure and fix the configuration.
| Failure condition | Expected behavior |
|---|---|
| Missing or revoked API key | Request is rejected before execution |
| Trace not found or wrong project | Request is rejected for that key/project scope |
| No model route configured | Request stops with an error instead of guessing a model |
| Provider error | Error is returned and the trace can record how far the request got |
| Spend or loop protection trips | Key or request is blocked according to project settings |
The SDKs also expose an opt-in fail-open mode for selected non-sensitive workloads. Use it carefully: bypassed calls are outside the normal Olyx trace path and should not be used for data that requires governance.
Data Isolation
Traces are scoped to the project and customer context associated with the API key or session. A project-scoped key should only see that project’s traces and configuration. This is why production services should use project-scoped keys rather than a broad administrative credential.
Data retention is configured in the project and model settings. During closed beta, verify retention behavior in the environment you deploy rather than treating docs as a compliance commitment.
Replay & Optimization
Once a trace exists, replays let you compare how the same recorded workflow might behave with a different model. The original trace remains the source record; replay work produces separate replay results or replay traces for comparison.
const job = await client.replays.create({
traceId: trace.data.id,
forceModel: "gpt-4o-mini",
maxCost: 0.005,
});job = client.replays.create(
trace_id=trace.id,
force_model="gpt-4o-mini",
max_cost=0.005,
)job = client.replays.create(
trace_id: trace.id,
force_model: "gpt-4o-mini",
max_cost: 0.005
)Use replays to evaluate a route before changing production behavior. See Replays for the full guide.
Summary
The Olyx architecture is intentionally narrow at the application boundary: one SDK path for execution, one trace record for inspection, and optional agent routes when a private deployment needs them. That keeps the closed-beta product easy to evaluate without presenting the system as a finished enterprise platform.