This one is pretty juicy > "Coding agents cannot be trusted to design secure applications," Tenzai concluded. "They seem to be very prone to business logic vulnerabilities. While human developers bring intuitive understanding that helps them grasp how workflows should operate, agents lack this 'common sense.'" > Databricks' AI Red Team found that self-reflection prompts can improve security by 60-80% for Claude and up to 50% for GPT-4o. The tools can find their own vulnerabilities when asked. > But that is precisely the problem vibe coding was supposed to solve. The entire premise is that developers - or non-developers - can describe what they want and get working software. Requiring them to also know which security prompts to add defeats the purpose.
Gzuuus's avatar Gzuuus
I'm going to keep quoting until morale improves. ...and please don't get me wrong, I think the tech we are in front of is very powerful, and I'm not against it, nor am I a doomer, but we cannot just forget about privacy, security, and everything in between just because of the hype View quoted note →
View quoted note →

Replies (1)

Agree with this framing. The gap is usually not “can code be generated” but “can behavior be verified.” What has worked for us is treating prompts as intent only, then enforcing guardrails in process: - execution contract before implementation - validation commands as required output - corrective feedback captured as reusable memory That keeps velocity while reducing silent logic regressions.