monkway is an LLM gateway and router, built from scratch. It takes one request, routes it to the right provider, retries and falls back on failure, caches answers, and tracks cost and limits. Built in the open, and taught one piece at a time.
The idea
Picture a gatekeeper monk at the monastery gate. Every traveler arrives at one door. That one monk is the whole gateway, and every feature monkway has is just one more thing he does:
Greets every request at one door
A single API in front of many providers (the proxy).
Picks which path to send it down
Routing by cost, latency, or quality.
Reroutes when a door is closed
Retry with backoff, then fall over to another provider.
Recognizes travelers seen before
Exact and semantic caching, so a question is not paid for twice.
Keeps the ledger
Token, cost, and rate-limit tracking, with per-key budgets.
How it's built
A single-provider proxy with retry, backoff, and structured logging of tokens, latency, and cost. Small in lines, the floor under it all.
Multi-provider routing, a semantic cache, cross-provider fallback, and per-key budgets and rate limits. Where it becomes a system.
The course
19 lessonsAn interactive course that teaches the concepts behind monkway, then leaves the implementation to you, which is where the understanding sticks. Six units, nineteen lessons, each with a demo you can drive. Start at the gate.
The gate
What a gateway is, and why one door in front of many models.
The proxy
Forwarding a request, and what "the same API" really means.
When it fails
Retries, backoff and jitter, timeouts, and idempotency.
Choosing a path
Routing policy: cost vs latency vs quality, and fallback.
Remembering answers
Exact vs semantic caching, embeddings, nearest-neighbor lookup.
The ledger
Token accounting, cost, rate limits, and per-key budgets.