I bought a fully specced MacBook specifically for side experiments: local inference, fine-tuning, agentic builds. I've run three lately. I built pt, an agentic terminal that merged almost 400 PRs in two weeks. I fine-tuned a small local model to draw Mermaid flowcharts. And I kept extending ctx, my personal context engine. The terminal and the fine-tune both failed, for the same reason: I measured them with instruments I built myself, and the instruments lied. Verification is paramount. This post is the evidence, and what I'm building because of it.
The agent factory
pt is an agentic terminal I wrote in Go: ten days, 89 internal packages, workers on rented remote machines, byte-verified delivery before anything became a pull request. Close to 400 PRs merged in two weeks, with 73 on the biggest day. The full build is documented here; this is the tour it recorded of itself:
It had one rule, written into the repo and enforced in review: the home surface answers what should I drive to closed today. After two weeks I had driven zero real workdays through it. I had optimized a factory I never used for its intended job, and the merged PRs made that harder to admit, because every one was a green number I had generated myself.
The fine-tune
I wanted diagramming to be free and local: hand a small model a plain-English description of a process, get back a correct Mermaid flowchart. I trained LoRA adapters for Qwen3 (0.6B through 4B) in MLX, with a teacher model supplying ground-truth graphs.
On my own gate, the model looked finished: 98% on a frozen 50-case suite, up from 0% zero-shot. On a gate I had never trained toward (120 cases written by a different model, in six domains my corpus never touched) it collapsed.
| Round (1.7B) | Frozen gate | Held-out gate (fuzzy) | Held-out (strict) |
|---|---|---|---|
| r1 — full corpus | 94% | 0.8% | 0% |
| r2 — mixed registers | 94% | 0.8% | 0% |
| r3 — open-vocabulary labels | 88% | 12.5% | 3.3% |
| r3.5 — messiness tiers | 98% | 15.0% | 2.5% |
| r4 — rejection sampling | 98% | 17.5% | 4.2% |
| r5 — filtered pool | 92% | 14.2% | 4.2% |
Six rounds of corpus work moved the held-out number from under 1% to about 17%, then it stopped. The model had learned my generator, not the task. It could emit a plausible flowchart with the right shape and the wrong content. Reading arbitrary prose was the real job, and my synthetic corpus had none in it.
Verification is paramount
Both experiments aced instruments I built, and both numbers lied the same way. The fine-tune's scorer shared the corpus's blind spot. Merged PRs proved code moved through the factory and said nothing about whether the product worked. The thing being tested cannot grade itself.
The difference is palpable at the keyboard. When your agentic loop can see its own work, you essentially get one-shotted features: it catches the final three issues itself, patches them, verifies the fixes, and hands the work back finished. When the loop can't verify, you burn tokens and hours in frustration. And that frustration is more alienating than I expected, because you can essentially build anything you can imagine now, and you know exactly how much faster it would go if the loop could check itself.
Holding the whole graph of work
That lesson is what ctx is being built around. Its job is to hold the entire graph of my work: every task, its inputs, the actions taken, the artifacts produced, the evidence collected, and the decisions made, across every agent and machine I use. In that graph, completion is a state a verifier grants, not a claim an agent makes.
Here it is live. A release task's all PRs merged gate sits in FAILED because ctx re-read GitHub and found a sixth PR had entered scope. The agent's summary said done. The gate didn't care, and the task stayed open with the follow-up attached.
What ctx already does well
Two ctx capabilities have run untouched since the day I shipped them: shared memory and secrets.
One memory across every agent
Every agent I run (Claude Code, Codex, Crush, pi, OpenCode, the one on my phone) starts with the same memories, outcomes, file paths and decisions. No re-explaining who my clients are, what's due this week, or what I decided last month.
Credentials agents can use, but never see
Agents act with my credentials without ever holding them. Values live in Cloudflare Secrets Store, agents carry revocable scoped tokens, and the Worker makes the call on their behalf.
Earlier today one of my own sessions hit the vault with a token that lacked the grant:
error 403: token lacks 'secrets' grant
My own request, failing closed. I could not talk the vault into doing otherwise.
Neither capability is coming from a frontier lab. Their memory lives inside their own product; mine has to travel across all of theirs. Their credential handling secures their agent; mine has to hold for whichever agent shows up next month.
Three bets
This round of refocusing on ctx encodes three bets.
First: that centralizing private, context-rich tools behind MCP works across every agent and every harness. One server, reachable from whatever shows up next month, instead of per-tool memory files and per-harness plumbing.
Second: that a centralized MCP can hold the authoritative graph of work state and use it to steer implementation sessions across agents and harnesses. The graph decides what's open, what's blocked, and what's verified; the agents come and go.
Third: that preserving all of my own tasks in a readable, trainable form, and auto-extracting every artifact along the way (PRs, decisions, generated images), pays off down the road. Research packets, decisions and past projects get injected into new tasks as needed, and the archive itself becomes training data for small models, custom fine-tunes and LoRAs. That's the real corpus the Mermaid experiment was missing, accumulating as a byproduct of work I was doing anyway.
Keep the model swappable
I'm trying to stop betting against model progress. Capabilities I might spend a month building keep landing in the models themselves; pt is the fresh scar. So everything that survives treats the model as a part you swap. In ctx, switching models is literally a dropdown: it switches live, bills through Cloudflare, and involves no provider keys.
The harness follows the same logic. Building pt taught me what I actually want from a daily driver, and pi turned out to be the closest existing thing to it, without most of pt's footguns. I don't maintain the glue code or the plumbing. I benefit from open-source extensions and the private ones colleagues and friends share. And I can open source my own extensions and tools instead of welding them into a terminal only I run.
Discussion
Giscus