kaggle-tpu-lab

What it does

• OpenAI-compatible endpoint usable as Claude Code / Codex CLI / opencode backend — the repo's own measurements: ~130 tok/s single-stream decode with MTP speculative decoding (78 without)
• Native 262,144-token context at bf16, TP=8 — a 225k-token prompt prefilled in ~28 s (repo-measured)
• Live endpoint in ~22 min with the datasets attached (~12 with --text-only, ~6 with --fast-start)
• Either Copy & Edit the published Kaggle notebook, or push and serve from the CLI with python launch.py serve

When to use it

For developers who want a free, long-context OpenAI-compatible backend for coding agents without a cloud account or GPU budget — experiments, load tests of agent workflows, or trying a 262k context window that most hosted tiers do not offer.

Getting started

# Kaggle account with TPU access (phone verify; free tier ≈ 20 TPU hours/week)
pip install kaggle
# kaggle.com → Settings → API → Create New Token → ~/.kaggle/kaggle.json

git clone https://github.com/ARahim3/kaggle-tpu-lab
cd kaggle-tpu-lab
python launch.py serve
# endpoint URL and API key stream to your terminal when ready

Things to know

Throughput and latency numbers are the repo author's own measurements, not a benchmark you can rely on contractually. Kaggle's free tier queues for TPU slots, needs phone verification, and allows roughly 20 TPU hours per week — sessions are ephemeral, so the endpoint is for experiments, not production.