Jason Odoom

"Thing Doer"


Session

08-08
14:00
30min
Nix Knows When the Agent Is Wrong
Jason Odoom

I built a tool to help maintainers patch nixpkgs CVEs faster. But I optimized the wrong thing.

The tool - Vulnpatch, is a dashboard. It pulls together CVE intelligence, triages by what is actually being exploited, gives the maintainer the context they need and gets out of the way. The assumption underneath it was that the maintainer is the bottleneck: speed up the workflow and packages get patched faster.

I no longer believe that assumption. I was already building Vulnpatch to automate the remediation work when Mythos was announced, and it undercut the premise. Building a product to do what a frontier model could now do on its own was the wrong problem to be solving.

The difficult part was never automating the work. It is making an agent's output something a volunteer on the security team can actually trust. The work is the evidence: which package is actually affected, which upstream commit fixes it, what the new hash is, whether it still builds, whether the tests still pass and whether a maintainer has any reason to trust it. And that is before the non-trivial cases: the patches that are not simple version bumps or hash updates.

This is a talk about that shift. I will discuss Vulnpatch and Trace, an agent-owned nixpkgs fork that produces signed, reproducible CVE evidence bundles instead of drive-by pull requests. And I want to make one argument I think is worth taking seriously: Nix is unusually well-suited to AI agents because it provides strong automated feedback. Reproducible builds and passthru.tests give agents an objective verifier for many changes, whereas most repositories offer much weaker validation.

Patches are cheap. Proof is not. I will show what it takes to make an agent's work auditable enough to be worth a maintainer's time and I hope, to persuade maintainers that agents belong in their workflow. The workflows we built for human-only contribution will not survive contact with agents unchanged.

Talks
Nix Vegas Main Stage