Knappstack is an open, generalized version of my daily agentic-engineering practice, and it's available on GitHub. It started as a private working set and got sanitized for public use: principles, a development lifecycle, review and release discipline, durable planning, and the newest layer, local-first orchestration. This page is the operating manual. Read it top to bottom to understand the model, or jump to the runbook if you just want to run it.
The thesis: generation is cheap, judgment is the bottleneck
AI can now generate more code than any team can carefully validate, and that inverts where the risk lives. The scarce resource is no longer typing speed. It's judgment: deciding what to build, catching the plausible-but-wrong output, and knowing what's safe to ship. Every part of Knappstack exists to protect judgment while letting generation run fast.
Two ideas do most of the work. The first is gates over vibes: a design gate before code, a review gate before merge, a staging gate before production. Each gate is a place where a human or a frontier model deliberately decides, rather than letting momentum carry a change forward. The second is that quality is set by who decides while cost is set by who drafts, so you push the drafting down to the cheapest capable model and keep the deciding at the top. The rest of this manual is those two ideas made concrete.
The pipeline
The commands compose into one delivery flow. Each has a single canonical home, and you invoke them by name to run a long-form runbook.
/brainstorm → /plan → build → /peer-review → /release-prep → /release-prod
(design gate) (durable (agent (cross-model (staging gate, (merge +
plan) builds) hardening → human-approved) distribute)
bot-clean PR)Two commands are cross-cutting, available at any phase. /checkpoint saves state and writes a cross-conversation handoff whenever a session is about to end or context is filling up. /debug drops in whenever something breaks mid-build, and it's also where review findings get remediated. Two more are standing instructions consulted throughout rather than at a single step: an engineering-principles decision framework, and a /principal-consultant communication protocol.
The gates, one line each
| Gate | What it enforces |
|---|---|
| /brainstorm | No code until a design is approved. Drives requirements like a senior PM, one question at a time, a few approaches with a recommendation. |
| /plan | Durable planning. A plan file with a state header and verbatim goal capture, so decisions survive across sessions and agents without drifting into paraphrase. |
| /peer-review | Autonomous hardening. Adversarial cross-model review (one model critiques another), drives to convergence, opens a bot-clean PR. |
| /release-prep → /release-prod | A staging gate that hardens, verifies the build, and waits for explicit human approval, then a production step that merges and distributes. The merge only happens after a human says go. |
The orchestration layer: cheap volume, frontier verification
The pipeline above is the delivery spine. The newest layer sits underneath it and answers a different question: as you delegate more and more work to AI, how do you keep cost from exploding without letting quality rot? I call the answer local-first orchestration. It's "local" because on a machine with capable local models the cheap tier is effectively free, but it works identically with a cheap cloud tier. Read "local" as "the cheapest model that clears the bar."
The two dials people conflate
"Spend less on AI" usually reads as one dial, use a weaker model, which collapses two independent choices:
- Orchestrator tier. How capable is the model that plans, decomposes, reviews, and decides. Turn this down and everything degrades.
- Delegation depth. How much of the labor (drafts, summaries, classification, extraction, first-pass review, bulk transforms, bounded code-gen) leaves the orchestrator for a cheaper worker. Turn this up and cost drops with no quality loss, as long as the orchestrator still authors the spec and verifies the result.
Local-first orchestration moves the second dial hard while holding the first at the top. The frontier model's token volume drops; its quality never does. Four pieces make that trade safe.
1. The broker (one door for all cheap inference)
If you run local or self-hosted models, route all of them through a single gateway that owns model lifecycle and queueing. Every session, job, and teammate's app calls the same endpoint, and nothing else loads a model directly. That one admission queue is the collision fix by construction. It accounts for memory, fails closed with a clean "busy" rather than an out-of-memory crash when a model won't fit, and enforces priority classes so a heavy batch job never starves a person at a keyboard. Callers ask for a role (reasoning, code, classification), not a specific model, so upgrades propagate everywhere without touching a client. For a pure-cloud setup this layer is just "use one gateway with per-key budgets and retries." The invariant that matters is that one place owns model selection and cost limits, not scattered per-call config.
2. The router (what may leave the frontier model)
A deterministic table, not a model's judgment call, decides where each subtask goes. Key it on task-type and risk, and return one of three verdicts:
| Verdict | Meaning |
|---|---|
| cheap-first | Route to a cheap model; the orchestrator verifies. Drafts, summaries, classification, extraction, research digestion, first-pass code review, and bounded code-gen (a single function or module against an explicit spec plus tests, scaffolding, mechanical refactors, doc comments). |
| frontier-only | The orchestrator does it itself. Multi-file design, API shape, concurrency, security-sensitive code, final verification, client-facing output. |
| never-cheap | Hard refuse. No cost mode overrides it. Anything with legal, financial, or safety consequence, or an irreversible outward action. |
Start static. A hand-written table is auditable, and you can unit-test that never-cheap types are refused in every mode. Add learned routing only if the static table plateaus; it's an optimization, not a foundation.
3. The supervision protocol (the quality guarantee)
This is the layer that permits aggression everywhere else. A delegated subtask is a contract. The orchestrator supplies the task, the payload, and explicit acceptance criteria; the router picks the model; the worker returns its output plus a self-reported confidence; and the orchestrator verifies the output against the criteria. That verification stays at frontier quality, and it is the quality floor. On failure it re-routes or escalates, bounded to two retries, then hands the task up. Two corollaries are worth stating outright:
- Bounded code-gen is safe because of this loop. A cheap model writing a function is fine when the frontier model wrote the spec and tests and reviews the diff, and the release gate still stands between it and production. Without the loop, don't.
- Verification quality equals orchestrator quality. Swapping a weaker model into the orchestrator seat lowers the floor for everything. Never ship code whose only reviewer was a cheap model. Keep a frontier verifier on anything that merges.
4. The economy ladder (how aggressive, as one setting)
Expose the aggressiveness as a small ladder the whole system reads, a routing bias rather than prose each session re-interprets:
| Mode | Posture |
|---|---|
| 0, frontier-first | Cheap tier only for always-cheap background jobs. Simplest and priciest. |
| 1, balanced | Cheap-tier discipline plus opportunistic cheap drafts and review. |
| 2, bulk (sensible default) | Every delegable subtask routes cheap-first; the frontier model stays full-quality as orchestrator and verifier. Sub-dials tune how far it goes: lite (clearly-safe types only), std (the full list), max (everything not never-cheap, tolerating a retry before escalating). |
| 3, austerity | Mode 2 plus a cheaper orchestrator tier and deferred nice-to-haves. |
The mode parameterizes the router. It never changes the never-cheap floor, and it never lowers orchestrator quality except at the explicit top of the ladder. The whole stack is orchestrator-agnostic by construction, so it works under any capable frontier model, because the mode governs routing regardless of who sits in the seat.
The manual: how to actually run it
Everything above is the why. Here's the how, the external and hardware-agnostic version of the operating manual I run day to day.
The command surface
The whole surface is small on purpose, a handful of commands with nothing else to memorize:
| Command | What it does |
|---|---|
/brainstorm | Design gate. Drive requirements to an approved design before any code. |
/plan | Write a durable plan that survives across sessions. |
/peer-review | Harden the change with adversarial cross-model review; produce a bot-clean PR. |
/release-prep → /release-prod | Staging gate (human-approved), then production merge and distribute. |
/checkpoint | Save session state and a handoff for the next session. |
/debug | Root-cause-first debugging; the remediate path for review findings. |
One canonical instruction file
Every AI harness you use (CLI, desktop app, IDE) should load one canonical rules file, shared by symlink and verified so it can never silently fork. Drift between "the rules the CLI follows" and "the rules the IDE follows" is a real and corrosive failure mode, and making one file the single source, checked automatically, removes it. In Knappstack that file carries the laws of agency (don't act beyond your authority without explicit human approval), the release discipline, and the routing bias.
Switching the orchestrator
Because the stack is orchestrator-agnostic, you can put a different frontier model in the driver's seat without touching anything else. The rules file, the router, and the cheap-model door all work unchanged. In practice that means an environment swap (base URL plus key) before launching your agent CLI, wrapped in a one-command launcher so it's frictionless. One rule survives every swap: anything that ships code keeps a frontier-quality verification pass, because verification quality equals orchestrator quality.
Asking cheap models directly
Two everyday moves save the most tokens, because bulk comprehension and drafting are the biggest sinks. The first is a tiny "ask the cheap fleet" command for questions that don't need the frontier model. The second, and the bigger win, is a "read these files and answer" command that has a cheap model read a corpus and return a cited digest the session verifies, instead of pulling thousands of tokens of source into the expensive model's context just to understand it. Use it for comprehension and survey, not for files you're about to edit (edits need the exact bytes) or where a paraphrase could hide the answer.
Measurement: the two numbers to watch
Instrument every delegated call with its task-type, route, and verification verdict. Two numbers tell you whether the system is working:
- Percent of subtasks served cheap. Should rise as you tune the router.
- Escalation rate. Should stay flat and low. Rising escalations mean the router is sending work the cheap models can't do, so retune rather than absorb the retries.
Success is the pair moving together: cheap share up, escalation flat, frontier spend down. If cheap share rises but escalations spike, you've just moved cost from tokens to wasted round-trips, which is worse, not better.
Keeping the cheap models good
Roles only stay cheap and good if the model behind each role keeps up, so automate it. On a cadence, discover newer candidates, size-gate them, evaluate on a fixed set of golden tasks, and promote only a clear win, with an absolute floor on critical tasks and the prior model retained for one-command rollback. Because callers address roles, a promotion propagates everywhere with no client change.
Adopt it incrementally
You don't need the whole stack on day one. A sane order:
- Start with the gates. A design gate before code and a cross-model review gate before merge deliver most of the quality benefit with none of the infrastructure.
- Add durable planning once work spans multiple sessions and you feel decisions drifting.
- Add the supervision protocol by hand. Consciously route routine subtasks to a cheap tier and verify against a stated bar before accepting. That's the whole discipline with zero infrastructure.
- Formalize the router and the two telemetry numbers once the manual routing pays off.
- Stand up the single-broker gateway only if and when you self-host models. Until then, a per-key cloud gateway covers it.