Claude finished in an hour. We shipped the other one.
Two sessions, same spec, same repo, same engineer. Claude went first and had a working version in about an hour. Kimi K3 went the next day and took about four hours. Kimi’s branch is the one that merged.
| Claude | Kimi | |
|---|---|---|
| Time to first working version | ~1 hour | ~4 hours |
| Refinement prompts after v1 | ~4 | ~2 |
| Cost | ~$95 | ~$75 |
| Files changed | 45 | 43 |
| Lines added | 5,535 | 8,467 |
| Lines of test code | 373 | 2,465 |
| Version we shipped | — |
That reads like a contradiction. It took a diff to see why it isn’t.
The feature both of them got
The task was session hand-off in Spec Mode: an author finishes a spec draft, hands the session to a teammate, and the teammate picks it up and builds it.
It spans a hand-off modal and its reassign path, a pending/accepted/revoked state machine, an author-side status banner that gates the Go! button, a server-side rejection so that gate is enforced rather than merely hidden, and analytics on the entry points. Big enough that a model has to hold a design in its head for the whole run, small enough to finish in one sitting.
Why this is a fair fight
Two things make the comparison worth reading. The first is the harness.
p0 can run custom providers, Kimi among them, and it runs them through the Claude Agent SDK. In p0’s own provider table the custom provider maps onto the Claude harness family: same binary, same tool loop, same skills, same slash commands. Pointing a session at Kimi changes two environment variables, ANTHROPIC_BASE_URL and ANTHROPIC_AUTH_TOKEN. Nothing else about the session changes.
The second is isolation. Each run started in a brand-new worktree cut from the same development commit. Neither model ever saw the other’s work: no shared branch, no shared session, no summary of what the other had done. Same product spec both times, same standards files, same engineer. Claude ran on August 3 and Kimi on August 4, and the engineer did not read Claude’s branch before starting the Kimi session, so there was no anchoring through the human either.
The four hours weren’t slowness
Claude changed 45 files. Kimi changed 43. On that measure the two runs are the same size. But Claude added 5,535 lines and Kimi added 8,467.
Look at the split. Of Claude’s 5,535 lines, 373 are test code, about 7% of the diff, in two new test files. Of Kimi’s 8,467, 2,465 are test code, about 29%, across five test files plus a 369-line hand-off harness that other suites can now reuse. That is 6.6× more test code.
So the wall-clock gap is mostly a scope gap. Kimi did not spend four hours being slow at Claude’s job. It spent them doing a bigger job, and most of the extra was tests.
The invariant nobody would have caught in review
Some background first.
p0’s API layer is one router tree that runs on two different hosts. Normally it runs inside the desktop app on your laptop. It can also run on a headless server you drive remotely. Some of those routers depend on your cloud identity, your login and your tokens. That identity lives on the laptop, and the server only borrows a mirrored copy. Routers in that category have to be registered by name in a list that pins them to the local channel, whoever is serving the rest.
Forget the registration and nothing breaks in any way you would notice. The call routes to the remote host, finds no laptop identity there, and returns. No error. No log. The user clicks the button and the feature is inert.
The rule is written down. The repo’s own instructions file has a section on Remote Mode invariants: it names the recurring bug family here as server-side code silently depending on laptop-only state, and tells you to decide a new router’s side explicitly and add it to the list if it is laptop-scoped. That section exists because this has already happened. It names three features that had already shipped broken this way. The file is called CLAUDE.md. We are aware of how that sounds.
Kimi registered the hand-off router in the list, and left a changelog note beside the entry naming the failure it was avoiding: the router is cloud-coupled, and without the entry it does nothing when served by a remote host.
Claude never touched the list. Its hand-off router is not in it, which means its version of the feature would have done nothing at all, silently, for anyone working against a remote host. This was not a case of never going near that part of the tree. Claude modified three other files in the same directory. It was working next door.
It is not a reasoning gap. Claude’s code is good code for a single-host application, which is what p0 looks like if you infer its architecture from the source instead of reading the instructions. One model went and found the house rules. The other wrote a well-argued feature for the house it assumed it was in. And because the failure has no symptom, a human reviewer would probably have missed it too.
The same instinct shows up in smaller places. Kimi’s recipient-side hydration checks that the teammate actually has the workspace on their machine before the ownership transfer goes through, so the most likely failure in the flow leaves nothing to unwind, and a retry after a crash resumes rather than errors.
Where Claude was ahead
The differences ran in both directions.
Once a spec is handed off, the sender’s session should freeze. Both models understood that hiding a control in the interface is not the same as making it unavailable, and both enforced the freeze underneath the UI. Claude drew the line wider: four write paths, including the ones that start an agent turn. Kimi’s blocks the phase advance and hides the controls, which covers the flow a user actually walks through, but not an agent run.
Claude also read a standards document nobody pointed it at, and cited a rule from it to justify reusing an existing background timer rather than starting a second one. We checked; the rule exists. The model that missed one written rule was careful about another.
Neither branch was spotless. Claude’s hand-off router names the lock module as the authority on which states freeze a session, then hardcodes that same list again a few lines later to dodge an import cycle.
The cost surprise
Kimi’s session cost about $75. Claude’s cost about $95. Kimi’s per-token price is a fraction of Claude’s, and the two totals still landed around 20% apart.
Cheap per token is not cheap per feature. A model that works for four hours and writes 8,467 lines spends the discount back on volume. Much of what the extra time bought was tests and a reusable harness, which is a fine thing to buy. But nobody should plan a budget from the sticker price per million tokens.
What we took from it
Kimi’s branch merged as PR #627. It shipped because it was the more complete piece of work: more of the product surface, 2,465 lines of tests, a harness other suites can reuse, and the entry in the list that keeps the feature alive for people working against a remote host.
One feature, one engineer, one session each. Treat it as a data point rather than a verdict on either model. But it changed how we pick. We stopped asking which model is fastest and started asking which one reads the house rules, because that is the difference that survives review and the one that would have reached users.
Speed was the number we expected to care about. It turned out to be the one that mattered least.
— Mario