[Discuss] Is open source more secure at the current level of AI?

[vab at cryptnet.net (V. Alex Brennen)]

On Saturday, April 11th, 2026 at 9:48 PM, Rich Pieri <richard.pieri at 
gmail.com> wrote:

> On Sat, 11 Apr 2026 16:44:21 -0700

> You just did it again: "They can see the code and they'll find bugs
> they can exploit!" It's still FUD and it's still the same logical
> fallacy.

Any distributed software is essentially opensource. I know I could not read 
through hundreds of thousands of lines of assembly and find exploits. But, an 
AI could do that easily. Especially, if it was leveraging tools like fuzzers, 
memory protection monitors (like Valgrind), and specialized chip features 
like memory color coding or memory tagging.

People have been running disassemblers for many decades for security and 
hacking reasons. I remember the community that used to use disassemblers to 
generate/extract license keys for proprietary software that required you to 
type a key code it off the CD/booklet to use the software (called cracking 
back in the day, IIRC). 

I have seen/played with BinaryNinja a little bit from Vector 35 (Jordan 
Wiens). It is very good. I think we'll see a lot more tools like that. The 
next level of evolution would be a lot of agent based tools that are 
essentially disassemblers with AI skills including memory safety, fuzzing, 
business logic testers, etc. Those will find a tremendous number of 
exploits/bugs.

A few weeks ago, I tried to figure out about what it would cost to do a good 
agent based audit on your average opensource C project on github. the numbers 
I seemed to come up with where around a few hundreds dollars per project 
(depending on the size and type of auditing and using an older model with a 
batching discount). That's certainly tractable for most opensource 
maintainers... not really a huge amount of money.

What I have heard though, is that the maintainers probably need to do it 
themselves. Either, they need to write/configure their own agent or pull 
something like CodeMender or Strix into their CI/CD (pretty easy on Github w/ 
GH Actions). The reason is that a lot of projects are getting flooded with 
"AI slop" PRs/vulnerability reports. It's too much work for the project 
owner/team to go through them all and not really a productive use of time 
since there is a lot of invalid code/hallucinations in the PRs/reports.


> We're seeing the effects of this in the wild. Higher tier attackers
> aren't looking for vulnerabilities in open source projects so much.
> They're *injecting* vulnerabilities and back doors and malware into
> packages hosted by public repositories like PyPi and npm, or they're
> attacking projects directly like XZ Tools, Notepad++ and CPUID.

That is similar to what I have been seeing. Attacks on the repositories and 
social engineering. Two of the most recent attacks I've seen were based in 
tricking someone into installing malware using an AI Deepfake.

Using a Deepfake they got the people to join a Microsoft Teams meeting that 
was browser based. Then, they faked an audio echo issue claiming it was 
because of out of date software. A fake prompt for an audio driver update 
resulted in the installation of malware. One was an opensource project member 
with repository access that resulted in a build chain compromise/trojan 
packages. The other was a student participating in a "job interview" with the 
exact same "audio issue" script and malware installation process.

Package repositories and software distribution is a major weak point. I still 
cringe a little every time I see opensource software/package installation 
instructions that are basically:

bash# curl (random website link) | bash

You know people are running stuff like that on very sensitive 
systems/networks all the time. And, often the TLD for the website is not even 
a US controlled TLD.

Some projects will sign with PGP. But, even if someone or some automated 
system does check the PGP signature I very rarely ever see any validating 
signatures on the keys used. Usually, it's an old key with an old 
cryptosystem, no expiration date, a role based email address, and no 
signatures (essentially validated by the integrity of the 
DNS/webserver/webcert). With X509 signatures, there are similar issues. With 
Let's Encrypt for example, all you need to get a cert is control of either 
the webserver or the DNS for the domain you're targeting for a few seconds.

I remember not being able to update the firmware in a router I was using 
years ago because the manufacturer forgot to renew the domain they were using 
to distribute updates and someone registered it after it expired, then 
ransomed it back to them.

I think we're going to have a window where closed source is more secure than 
opensource. But, I basically expect that to be essentially a replay of when 
fuzzing was introduced and became widespread or even when Valgrind became 
popular. It should pass pretty quickly with things returning to being rather 
even between the two.

I think cloud based SaaS where it is only exposed as API endpoints behind an 
API Gateway and WAF or as a website that is behind a WAF is a lot more secure 
than opensource and will remain so. But, that's just because there is not 
access to the software outside the company and cloud provider and the attack 
surface is so very tiny.

Personally, I think AI reaching a point where it can write major opensource 
projects at low cost is much more dangerous for the movement and ecosystem 
than the security/vulnerability issue. One researcher was able to generate an 
HTML5 standards compliant webbrowser with JS and WebASM support for less than 
$25K in compute time/tokens in a matter of hours IIRC.

If there's a massive flood of such software that could make being a 
sysadmin/cloud admin very difficult. How do you know what software to 
choose/trust? I'm not sure the distro companies/brands could survive in that 
competitive of an environment. It would probably turn into something where 
you had to wade through thousands of dead projects to find anything supported 
or useful (like SourceForge/Freshmeat/the FSF's SourceForge clone back in the 
day).


         - VAB