Vercel AI Gateway Review: Measured Speed & Pricing (2026)

Most of the cloud inference products I tried that weekend failed before I could learn much from them. The fast flat-rate tiers at Makora and Cerebras were sold out. Groq had closed paid upgrades. A direct Baseten route hit its shared rate cap halfway through a long Claude Code turn. Vercel's AI Gateway kept working.

It has become the hosted half of my model lab. Private work stays on the M5 Max. Shareable work goes through the Gateway, and claude-gateway <model-name> lets me replace the model behind Claude Code in seconds. Vercel's public model table makes that loop unusually useful because it shows throughput and latency alongside input, output, and cache pricing. I can browse, pick, run real work, measure, and either keep the route or discard it without opening another account.

I got there by measuring. The weekend produced the full numbers in The Price Floor, the routing summary in Practical Advice on Local Inference and Cloud LLMs, and a trial of Thinking Machines' Inkling in Sampling Inkling and the Alias Pattern. This review is the part that persisted after those experiments: why the Gateway was one of the only cloud inference products I could actually use, and why it is now the fastest way I have found to learn the current model field and right-size each workload.

What the Vercel AI Gateway is

The AI Gateway is a router across hundreds of models and their provider deployments. It gives me one endpoint, one credential, and one credit balance. When I benchmarked DeepSeek V4 Flash 0731 on the evening of August 1, eight providers were serving that identical checkpoint through it. The Gateway can retry another deployment when one provider degrades, and I can pin or order providers when I need more control.

The part that matters for my daily driver: the Gateway implements Anthropic's /v1/messages protocol. There is no proxy to run and no translation layer for me to maintain. Claude Code talks to it directly, including tool calls and prompt-cache accounting.

For work that can leave my machine but shouldn't be retained, zero-data-retention options exist per provider.

Claude Code setup: two values and one explicit reset

Vercel's Claude Code setup is three exports:

export ANTHROPIC_BASE_URL="https://ai-gateway.vercel.sh"
export ANTHROPIC_AUTH_TOKEN="$AI_GATEWAY_API_KEY"
export ANTHROPIC_API_KEY=""
claude

The empty ANTHROPIC_API_KEY is deliberate. Claude Code checks it before ANTHROPIC_AUTH_TOKEN; leaving a provider key there can bypass the Gateway. That is the complete integration I measured against direct providers with the same prompts.

Vercel AI Gateway pricing: the cache-hit line is the price

A coding agent re-sends its entire context on every turn: system prompt, tool definitions, the conversation so far. Providers store the processed prefix and bill repeat reads at a discounted cache-hit rate. This means for any long-lived-context workload, the cache-hit price effectively is the price, and most provider comparisons never print it.

My own 12-day laptop ledger — honestly labeled, because it is a small personal sample, not my real workload: 4,127 assistant turns, 1.065 billion input tokens served as cache reads against 33 million cache writes and 183 thousand fresh input tokens, with 5 million tokens generated. Cache reads were 96.6% of all input, and input outnumbered output 218 to 1. At that composition, the effective price converges on the cache-hit rate:

p_effective = h · p_hit + (1 − h) · p_miss, where h = 0.966

Run the same ledger through five rate cards and the bills land two orders of magnitude apart:

Provider · model	Cache-hit price /M	12-day sample, monthly rate
Vercel AI Gateway · V4 Flash	$0.0028	$17
DeepSeek first-party · V4 Flash	$0.0028	$22
Fireworks · V4 Flash	$0.028	$88
Groq · gpt-oss-120b	50% of input	$214
Anthropic · Opus 5	$0.50	$2,117

The dollar figures scale with my toy sample and yours will differ. The ordering is what travels, because it's driven by cache composition, and any context-heavy agent workload shares that composition. The Groq row is a smaller model entirely; it's there as the cheapest fast option, and its 50% cache floor still makes it ten times the Gateway here.

On uncached prices the Gateway also undercut the model's own maker: $0.09 per million input tokens and $0.18 out, against DeepSeek's $0.14 and $0.28, with the identical $0.0028 cached-input rate. A router undercutting the first party is a strange fact about open weights, and it's true today. Baseten's direct API comes close at $0.13 in and $0.26 out — but its cached-input rate is $0.028 per million, ten times the Gateway's, which keeps the Gateway ahead for agent workloads shaped like mine.

Vercel AI Gateway throughput and latency: what I measured

Same checkpoint, same prompt, five streamed trials each, one evening. Fireworks came back at a median of 72 tokens per second end-to-end; the Gateway at 73. Both sat 30 to 45% below the published figures (107 listed for first-party, 267 for the fastest reseller), and individual runs ranged from 20 to 99 tokens per second — a wider spread than any difference between the providers.

The medians being a wash, the Gateway won on the tail. Time to first token held between 0.35 and 0.63 seconds on every single Gateway trial, while the direct provider threw 8 and 9 second outliers on otherwise sub-300ms starts. For interactive work, that tail latency shapes the experience more than the median throughput does. Consistency is what you feel.

The model table is the real interface

The public model table is the page I use most. Most catalogs tell me context size, capability, and price. Vercel puts latency and throughput in the same scan, then exposes the provider deployments underneath each model with their own cache rates and uptime. That answers the questions I actually have before a trial: will the context fit, how quickly will output arrive, what will a long agent session cost, and is there more than one healthy route?

Read that table the way I do. DeepInfra's deployment lists $0.09 in / $0.18 out with 0.35s latency and 84.5 t/s; DeepSeek first-party lists $0.14 / $0.28 at 1.03s and 62.1 t/s. Price alone would tell me to try DeepInfra. Throughput and latency tell me I am unlikely to pay for that saving by staring at a slow stream. The cached-read price — $0.0028 across the board here, the line that dominates an agent bill — is printed in its own column. Baseten's 99.83% uptime sits next to everyone else's 100%, another useful input for a service that may route among them.

Published performance numbers are a shortlist, not a promise. My own trials landed 30 to 45% below the listed throughput, which is why the next step is always a real task. The Gateway removes almost all of the friction from taking that step: one key already fronts the catalog, so a different model does not require another account, SDK, or billing relationship.

Availability: the week everything else was sold out

Availability was the first benchmark. Vercel was one of the only cloud inference products I could actually use that week. I tried to buy fast inference from three vendors in one evening and every one turned me away: Makora's $20 and $200 plans both showed sold out, Cerebras Code Pro at $50 and Code Max at $200 were sold out, and Groq's paid Developer tier had been closed to upgrades for months. Groq's free tier was capped at 6,000 tokens per minute, less than a single agent-session context.

The metered tiers remained open because a metered customer is profitable at any volume. Fast inference at retail subscription prices was the scarce product in August 2026. The Gateway gave me a route around that market: I could prefund one balance, use the model page to see what was actually available, and change models without waiting for another vendor to sell me a seat.

The failover showed up in practice the same weekend. While trialing Inkling through Baseten's direct endpoint, a long streaming read hit the shared 100k tokens/min cap mid-turn, the request 429'd, and Claude Code started burning the pool on retries. The fix cost about one command: switch the wrapper back to the Gateway, which absorbed the burst and failed over between providers while I finished the task. The post itself was finished on DeepSeek Flash via the Gateway after Inkling lost the thread on long-form writing — same CLI, different endpoint, one-line change.

claude-gateway <model-name> closes the loop

Every route I use with Claude Code lives behind a script in ~/.local/bin/. The named wrappers hold onto models that have already earned a job: claude-deepseek, claude-laguna, and claude-inkling. claude-gateway <model-name> is the audition room. Any provider/model ID in Vercel's catalog becomes a Claude Code backend in one command.

Here is the complete Gateway wrapper:

#!/usr/bin/env bash
# claude-gateway <provider/model> [claude args...]
set -euo pipefail

MODEL="${1:?usage: claude-gateway <provider/model> [claude args...]}"
shift

if [[ "$MODEL" != */* ]]; then
  echo "claude-gateway: use a provider/model ID from the Vercel catalog" >&2
  exit 1
fi

KEY="${AI_GATEWAY_API_KEY:-}"
if [[ -z "$KEY" && -r "$HOME/.config/ai-gateway/key" ]]; then
  KEY="$(<"$HOME/.config/ai-gateway/key")"
fi
if [[ -z "$KEY" ]]; then
  echo "claude-gateway: AI Gateway key not found" >&2
  exit 1
fi

CATALOG="$(curl -fsS --max-time 8 https://ai-gateway.vercel.sh/v1/models)"
if ! printf '%s' "$CATALOG" | grep -Fq -- "\"id\":\"$MODEL\""; then
  echo "claude-gateway: '$MODEL' is not in the current catalog" >&2
  exit 1
fi

export ANTHROPIC_BASE_URL="https://ai-gateway.vercel.sh"
export ANTHROPIC_AUTH_TOKEN="$KEY"
export ANTHROPIC_API_KEY=""
export ANTHROPIC_MODEL="$MODEL"
export ANTHROPIC_DEFAULT_HAIKU_MODEL="$MODEL"
export ANTHROPIC_DEFAULT_SONNET_MODEL="$MODEL"
export ANTHROPIC_DEFAULT_OPUS_MODEL="$MODEL"
export CLAUDE_CODE_SUBAGENT_MODEL="$MODEL"
export CLAUDE_CODE_MAX_OUTPUT_TOKENS=32000
export DISABLE_TELEMETRY=1
export DISABLE_NON_ESSENTIAL_MODEL_CALLS=1
export DISABLE_ERROR_REPORTING=1

exec claude "$@"

Three design decisions worth calling out:

Use the catalog's real ID. Vercel model IDs include the creator prefix: deepseek/deepseek-v4-flash-0731, not deepseek-v4-flash-0731. The unauthenticated GET /v1/models check catches a typo or removed model before Claude Code can start a retry loop.

Prompt caching stays on. The local wrappers disable Claude's prompt caching because their local engines manage the KV cache themselves. On the Gateway, a V4 Flash cache read cost $0.0028/M against $0.09/M for fresh input in my August snapshot. Disabling it would multiply the dominant line in my agent workload.

One credential covers the catalog. claude-gateway deepseek/deepseek-v4-flash-0731 and claude-gateway thinkingmachines/inkling use the same key, endpoint, and balance. The model ID is the moving part.

The full roster

• claude-deepseek — local DeepSeek V4 Flash via ds4, which serves the Anthropic-style API directly. 30 to 40 tokens per second on my M5 Max, works on an airplane, zero data egress. Prompt caching is off because ds4 manages its own disk KV cache — warm starts drop from 66 seconds to 10.
• claude-laguna — the local lab: Llama-3.3-70B and Qwen2.5-Coder-32B hot on Metal via llama-server (llama70 on :8080, coder32 on :8081). Prompt caching is off because llama-server handles cache internally. These two models together fill the M5 Max's 128 GB, so claude-deepseek and claude-laguna are mutually exclusive.
• claude-inkling — Thinking Machines' Inkling (975B/41B MoE). Default route is the Gateway; INKLING_VIA=baseten selects the direct endpoint once workspace verification lifts the shared rate cap. Two routes, same wrapper, one env-var switch.
• claude-gateway <model-name> — the general case. Any model the Gateway lists becomes a Claude Code backend with one command. Prompt caching is on, the catalog is validated at launch, and the model ID is the only thing that changes between runs.

Browse, pick, run, measure

The model table and the alias are two halves of one workflow. The table is how I form a shortlist: context, latency, throughput, cache-hit price, and available deployments. The alias puts one candidate under real Claude Code work immediately. If it fails the task, I replace one string and keep moving.

A concrete example from last week: Inkling kept losing the thread while I finished a long post. The Gateway page showed DeepSeek V4 Flash at $0.09/$0.18 with a 0.35s latency listing on DeepInfra's deployment. It was cheap, fast enough, and capable enough for prose editing. The switch was one command: claude-gateway deepseek/deepseek-v4-flash-0731. No new account, key, or bill. The post was done 20 minutes later.

That loop is how I learn the model field and right-size my routes. A model earns a permanent named alias only after it survives representative work. Vercel gives me the measurements to choose the next candidate and the common interface to try it while the comparison is still fresh.

Where the Gateway fits in my routing

Work	Where it runs	Why
Client and proprietary code	MacBook, local ds4	Nothing leaves the machine
Personal projects, open source	V4 Flash via AI Gateway	Cheapest capable option, caching included
Fast iteration bursts	Fast metered silicon, when a seat exists	Wrong answers are cheap, speed compounds
Hardest problems	Frontier API	Quality per attempt beats price per token

The Gateway holds the default slice: everything shareable that benefits from a frontier-class open model at pennies, with caching included and no availability risk riding on one vendor.

What I'd re-check before trusting this review

The "for now" is load-bearing. Providers reprice monthly, the Gateway's routing changes under the hood, and the 30 to 45 percent gap I measured between listed and actual throughput says published numbers age fast. My cost sample is 12 days from a new personal machine — the composition (96.6% cache reads) transfers to any agent workload, but the dollar figures don't claim to represent anyone else's bill, including mine at the office. I plan to re-run the same benchmarks quarterly, and the scripts made that cheap.

Verdict: should you use the Vercel AI Gateway?

If you run agent workloads against open-weight models, the Vercel AI Gateway is currently the cheapest capable metered option I've measured, with the most consistent time-to-first-token, a cache-read price that respects how agents actually spend tokens, and a model browsing page that makes comparison shopping a two-minute task instead of an afternoon. It's the default lane in my routing and the claude-gateway alias is how new models earn a place in it.

It is not the whole answer. Work that can't leave the machine stays local. Flat-rate fast silicon would be great if anyone would sell me a seat. And the hardest problems still go to the frontier API, where quality per attempt beats price per token. But for the large middle slice — personal projects, open source, daily agent sessions on shareable code — the Gateway won on measurements, and the win is re-checkable on a quarterly clock. That's the strongest endorsement I can give anything in this market: I measured it, and I can measure it again.