The current state of AI in Open Source is like going into a bike co-op and you see someone feverishly helping someone by building a bike that looks like this:
The bike owner says, that’s not actually helpful, bikes generally have pedals and a chain so then, within minutes, the volunteer builds this bike:
And then you see someone comment on the bike - “I can see pedals and a chain there, that means it’s a working bike. Well done.”
It is an eerie feeling, where a lot of people have a disturbingly similar overly positive writing style, every issue has 3 PRs linked to it, and a sense of doubt that anyone is a human.
Before AI, open source PRs generally were well meaning, quite difficult to implement code that solved the problem the user had. Sometimes they would cause more issues than they were worth, or were not the way the maintainer thought it should be implemented, but they were high effort - simply getting a PR to pass tests that makes any changes used to be quite high effort in open source. In my experience, more than 50% of engineers never merged a significant PR to an open source project because it is quite challenging, and time consuming.
Similarly, before AI, when you read comments in issues that have a ton of text, that meant that someone generally put in a lot of effort into their post. There were many people mentioning that they also had the issue, but you would never read a comment that was just completely misleading slop.
Now we live in an absurd reality where (please do not react brigade) people post 1000 word comments on issues that are completely meaningless in the context of the conversation, although it kind of looks meaningful. In the linked example above, this issue is for the turbopack build, and the code references are for webpack build, which is an entirely separate build path. Now, this engineer may have made a mistake and confused the two, which is possible, but an ai detector gave me 99% AI generated. Additionally their github profile includes lots of references to MCP / Claude.
I think that most AI generated comments should be outright hidden immediately. Now, if English is not your first language, I understand translating using AI, or using AI as some kind of proof checker is completely fair game. Additionally, letting AI research for you and then using that is also fine. However, posting purely AI generated comments that are hundreds of words long is a toxic contribution that wears down everyone that has to read it. On the other hand, if you use AI to create a really helpful comment that is actually relevant and helpful to the conversation, then it’s completely fine.
I do think that AI generated PRs can be good, and should be allowed. However, they should be something a maintainer is actively asking for. Essentially, unless an issue is marked as “welcoming contributions” then there should not be a PR opened on it by a non-contributor. Additionally, contributors may want to disable AI generated contributions, and they should have the means to do so. Finally, any AI generated code should be explicit in the PR.
Realistically, the core problem here isn’t that contributors are submitting AI generated comments and PRs, it’s that the people submitting them are ignorant and over confident with their AI contributions. Even if you have a code of conduct that states that AI generated contributions must be verified and helpful before they are posted, junior engineers following those instructions would not necessarily have the skills, context, and effort required to understand how to verify and how to be helpful.
Before AI, these juniors would have to spend so much time and effort learning how to build the repo, learning how the code works, and actually working on the problem before they could post. Then, when they were incorrect, or had poor assumptions, maintainers could help them figure out what they got wrong and actually they would learn. This took effort, but a lot of open source maintainers legitimately enjoy helping juniors learn. Importantly, this was a proportional effort where juniors would generally spend more time than maintainers.
Now, they are not learning anything at all by contributing, and their contributions take almost no human time to submit. There is no point in trying to help someone who is just using an LLM to post, in the same way, you wouldn’t try to teach a bot in a FPS game on how to aim. Although many human contributors are just as incapable of learning as a LLM, it was certainly not the majority of people. This is a losing battle that maintainers cannot win. Instead, they have to shut the door to maintain their sanity.
I do not know what the future of moderation will be like for open source. I suspect that people will build bots that automatically hide or flag contributions that are above a certain percentage of AI generated content. I think that has issues, since this inevitably it will cost a lot to run detection on every comment and PR that your repo receives. Additionally, people should be able to tweak their AI output to get past any automatic detection. However, it should still be enough for a while.
More importantly, we need to educate junior engineers, and let them know what LLMs should be used for, and what they should not be used for. Responsible AI use should be a basic piece of tact that junior engineers learn before they can contribute to open source.
What is responsible AI usage? Well, something like this:
Always disclose the extent of use of AI when contributing to a project, whether that is at work or in open source.
Respect any code of conduct or rules about AI generated content. If an AI maximalist project has no rules, then go wild!
When submitting AI generated comments / PRs, one must manually verify that the content is correct and not misleading.
If there are parts of AI generated content that you are unsure of, then you should remove them, even if this breaks the functionality - you should know why your code works and what your comments are explaining.
Again, I think this should become a basic norm of working with AI. Not everyone follows norms, but those that do not get excluded from communities. I’m not sure if there’s an automated way to enforce this, but that’s a separate issue that will become more obvious when the community begins to establish a shared definition of what the norm is.
I’m curious what the community of AI maximalists think especially, since they tend to insist that reviewing AI generated code manually is unnecessary. I think some projects are a great fit for completely hands off vibe coding, but long lived popular open source projects firmly do not fall into that category, since they have many users, are always lacking in testing, and often have battle hardened code that is not aesthetically obvious or logical.