OpenRouter Batch API: half-price async for eval queues (not live phone turns)

OpenRouter’s Batch API, announced Sep 22, 2026, is the missing cost lever next to flash defaults and the Descript-style model-eval queue. You submit work that does not need a realtime answer. The provider picks when to run it inside a 24-hour window. In exchange you generally pay about half the normal per-token price — sometimes less. It ships on 70+ models today.

Across 230k+ batches that completed in their two-week beta, the median finished in 7 minutes and the p90 in about 1 hour (p99 ~10.3h). You rarely wait anywhere near a day. That is the shape I want for overnight eval corpora, ticket backlog summarization, embedding backfills, and “run this harness on the new model” jobs — not for the live phone turn sitting on hold.

What you actually call

POST to https://openrouter.ai/api/v1/batches with an endpoint shape and a list of requests. Supported shapes: chat completions, responses, messages, and embeddings — the same text bodies you already send. Each request carries a custom_id so results come back independently; a few bad rows do not fail the rest of the job.

Then poll GET /api/v1/batches/:id until status is completed, failed, expired, or cancelled. Completed batches return results inline. Inputs and results stick around for 30 days (or until you DELETE). Every batch shows up in the Batches tab with model, provider, status, and cost.

Routing is single-provider per batch. By default OpenRouter picks the cheapest batch endpoint for the model after your allowlist, data policy, and BYOK settings. BYOK works: with a provider key configured, eligible providers route through your key and you pay the BYOK fee. Images and files must be public URLs. Audio, video, and OpenRouter’s own web-search plugin are not available in batch. Web search, if you somehow mix it in elsewhere, still bills at standard rates — the discount is on per-token inference.

Latency is load-shaped, not “batch means slow”

The interesting ops detail in the announcement is time-of-day, not batch size. Batches submitted between 5am and noon Pacific are significantly slower; the slowest tenth take 2–4.5 hours. Outside that window the p90 drops under ~1.1h, and after 6pm Pacific under ~50 minutes. A single-request batch finishes in 5–11 minutes depending on the hour; a batch of 1,000+ requests finishes in 12–21 minutes. Large batches take longer, but size is not the main knob — when you submit is.

For a desk that already runs overnight evals, that maps cleanly: fire the heavy corpus after US evening, poll in the morning, open the PR. Same rhythm as Descript’s Slack → Claude Tag → harness → human approve loop, just with tokens that cost half as much when latency is allowed to wander.

Where this sits next to flash and shell

I treat Batch as the third sibling in the same OpenRouter product family I’ve been drafting this month:

Flash defaults for interactive loops — rankings through Sep 21 still scream volume on DeepSeek V4.1 Flash (~16.9T weekly, +172%) and GLM 5.3 Flash (~16.9T, +45%), with Hy4 preview #3 (~12.7T), DeepSeek V4 Flash 0731 #4 (~8.91T), and GPT-5.6 Luna #5 (~8.48T). Apps stay Hermes / Claude Code / Kilo / Cline heavy. Cheap realtime stays the phone-and-ops default when a human is waiting.
Shell + Files for blast-radius control — host the sandbox, keep network default-deny, promote artifacts on purpose. That is how the agent does work without owning the machine.
Batch for anything that can wait — eval sets, labeling, embedding backfills, summarizing a ticket backlog, scoring the same prompt across a few thousand overnight rows. Model stays a parameter; cost drops ~50%; you accept variable finish times.

Yesterday’s Descript note was about eval throughput (Slack message → harness in 1–2h). Batch is about eval unit economics once that throughput exists. If “try the new model” still needs eng calendar time, Batch will not save you. If the harness already fires without a meeting, Batch is how you stop paying sync prices for overnight work.

What I would put on Batch this week

Full regression corpus after a model picker change — same cases as the Descript-style queue, half the token bill.
Embedding backfills when a retrieval index needs a new model or chunking scheme.
Ticket / transcript backlog summarization that does not block a live agent.
Prompt A/B across thousands of frozen rows overnight; read the report with coffee.

What I would not put on Batch: live inbound phone turns, warm transfer decisions, anything a caller is waiting on, or tool loops that must react in seconds. Those stay on flash (or escalated) sync paths. Batch is async by contract — variable latency is the feature you bought the discount with.

Operational checklist

One OpenRouter integration; endpoint shape already used in sync calls.
Poll until terminal status; treat expired/cancelled as first-class outcomes.
Public URLs only for images/files; no audio/video/web-search plugin in the batch payload.
Expect single-provider execution; configure BYOK if you care which key the provider sees.
DELETE when done if retention beyond 30 days is a problem; otherwise plan for the window.
Prefer evening Pacific submit windows when the desk can wait until morning.

Closing

Batch API is not a new model and not a phone stack. It is half-price inference for work that can finish in minutes-to-an-hour instead of now — 70+ models, median 7 minutes in beta, same chat/embeddings shapes you already use. Paired with flash defaults for live turns and shell/Files for blast radius, it closes the cost gap on the eval queues we already decided to run. Comparing notes beats comparing logos. Say hello.

Sources: OpenRouter — Batch API: half-price inference by bundling requests (Sep 22, 2026); OpenRouter rankings (usage through Sep 21, 2026).