INFRASTRUCTURE
Your fleet, at a glance
Clusters, nodes and node groups, with reservations to hold capacity and power profiles that idle a cluster when it is quiet. Every GPU cell shows whether it is busy, held or free.
Your GPUs · hosted providers · one API
inferos:~$ inferos serve your-model
Run AI on your own GPUs.
Serve it like a cloud.
Inferos connects your GPU clusters, deploys open models onto them, and serves them next to hosted providers through one OpenAI-compatible gateway. Then it measures how fast they answer, and how well.
Public, no sign-in: the docs and the inference leaderboard.
FROM BARE METAL TO AN API
Four steps. One workspace.
Everything from the first node to the first scored answer happens in one place, with one account and one set of keys.
- 01
Connect a cluster
Install the controller and run one command on each machine. You review a node's hardware before it joins. The controller dials out, so no inbound port is opened.
- 02
Deploy a model
Pick a model and see whether it fits your GPUs now. The engine, precision and placement are suggested for you, and every settings change is a revision you can roll back to.
- 03
Serve it
About a minute after its first replica is ready, the model answers on your gateway, next to hosted providers, behind API keys with their own model, rate and spend limits.
- 04
Measure it
Benchmark speed under real load and judge answer quality on your own data and your real traffic. Compare a change with a baseline before it ships.
THE PLATFORM
Everything a model needs in production.
Each part has its own console. They share one sign-in and one set of permissions.
INFERENCE
Deployments that roll forward and back
vLLM, SGLang, TEI, Ollama or plowrt. Follow each replica from submitted to warmed up; scale in place without a restart.
AI GATEWAY
One API for every model
Chat, images, video, speech and transcription where the model supports them. Failover by priority, split by weight, and limits per account, model or key.
EVALS
Quality you can track
Judge real conversations as they happened, or run datasets through prompts and models. Write your own criteria in plain words.
BENCHMARKS
Speed under real load
Time to first token, time between tokens and throughput at every load level: sweeps, fixed rates, traffic replays and soaks.
AGENTS · PREVIEW
Coding agents on your cluster
Claude Code, Codex or OpenCode work on a goal in a sandboxed job on your own machines, and report every step.
COMMAND LINE AND API
Script anything you can click
Use the inferos CLI, or the REST API with its OpenAPI reference. Gate a release in CI: the run fails when a model scores below your mark.
AI GATEWAY
Change the base URL. Keep your code.
Point any OpenAI SDK at your gateway. The same request reaches a model on your GPUs or a hosted provider, whichever is healthy and allowed.
- Streaming chat completions
- Failover to the next target by priority
- Conversations stay put: KV-aware routing reuses a replica's cache
- Per-key policy: allowed models, rate and spend limits
from openai import OpenAI
import os
client = OpenAI(
base_url="https://your-host/api/v1",
api_key=os.environ["INFEROS_API_KEY"],
)
stream = client.chat.completions.create(
model="your-model",
messages=[{"role": "user", "content": "Hello"}],
stream=True,
)import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://your-host/api/v1",
apiKey: process.env.INFEROS_API_KEY,
});
const reply = await client.chat.completions.create({
model: "your-model",
messages: [{ role: "user", content: "Hello" }],
});curl https://your-host/api/v1/chat/completions \
-H "Authorization: Bearer $INFEROS_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "your-model",
"messages": [{"role": "user", "content": "Hello"}]}'MEASURE
Know it's fast. Know it's right.
Speed and quality are measured by the platform itself, on your deployments, and kept private to your account.
Benchmarks
throughput against speed per user · illustrationEvals
a conversation's scorecard · example- Goal completion92%
- Relevance88%
- Instruction following84%
- Memory79%
- Consistency95%
- Hallucination90%
A judge model reads each conversation and cites the evidence for every score. A score the judge could not give is shown as not scored, never as a pass.
A public leaderboard you can check.
Results come from fixed, versioned test profiles, are measured by the platform and reviewed before they are published. Every number links to its methodology.
CONTROL
Your hardware. Your data. Your rules.
- 01
Outbound only
Each cluster's controller opens the connection to Inferos. Your network needs no inbound rule.
- 02
Private node network
Controller and nodes talk over WireGuard. Every node is reviewed before it runs work.
- 03
Secrets stay sealed
Provider keys and tokens are stored encrypted and never shown again. Engines and agents use them by name.
- 04
Accounts kept apart
Runs, datasets, traffic and keys belong to one account. Results reach the public leaderboard only when the owner submits them.
- 05
Access per product
Grant read or write per product to a person, a group or a whole team. An API key carries its own limits.
- 06
Audited
Administrative actions, from granting access to publishing a result, are recorded in an audit log.
Put your GPUs to work.
Send a first request today.
Connect a cluster, deploy a model and call it through your gateway.