Replies (3)
With COLDCARD there's a ton of hype around how someone found the major vuln by scanning the codebase with an AI agent after the incident, but AI agents have been publicly available for years and people did post they saw it before, just no one exploited it. Maybe they did not know how to.
Also according to their statements Coinkite used frontier models and supposedly could not find it... all about the user.
Yes, a good harness is very important, seems like its all about context management, even tiny models find critical bugs if you scope their context to the right part of the codebase.
Anyhow, I've found a surprising amount of vulnerabilities with a trivial prompt like "do a security audit for this project, research best practices online first".
Sure, professional pen testers with advanced harnesses will find even more bugs. But the fact that a non-dev with a one shot prompt can find criticale is wild.
Anyhow, more eyes on the code is always a good thing.
When building something I also do give it the same one quick check. If you build a threat model (architecture diagram and a list of threats and countermeasures you implemented against them) it can provide some excellent detail. Even if someone isn't security conscious, the more it knows about your software the better.
