AI chatbots were tasked to run a tech company
businessinsider.comSummary: Researchers in a new study tasked an AI-powered tech company (nicknamed "ChatDev") with developing 70 different programs. They found AI could develop software in under seven minutes for less than $1 in costs, on average. AI bots were assigned roles and were able to talk, make logical decisions, and troubleshoot bugs.
Original paper: https://arxiv.org/pdf/2307.07924v3.pdf
"The study said 86.66% of the generated software systems were "executed flawlessly.""
What's that susupposed to mean ?
Unit tests were written to have this percentage ? By human or AI itself ?
86.66% of how many LOC ?
How long will it take for a human (because AI can badly do) to debug the code ?
What was the purpose of generated code ?
Lack of useful context here. Maybe I missed information fast-reading the article.
IMHO, it looks like just another rant on how "good" LLMs can "write code".
EDIT : Sorry OP, I didn't see the arxiv link.
I'm on mobile and read a 10 Mo pdf isn't worth it. I'll try to read it on a computer though, looks interesting.
I don't want to undermine the work of the researchers because they did a good one with a solid protocol.
But the problem to me is the specification, basically one of the tasks was : "design a basic Gomoku game". Which implies a set of pre-existing rules, something already known and well documented on the web. For example on github you can find 1.2k repos in python with the tag 'Gomoku'.
In a company you will be less likely asked to code a Gomoku from scratch, maybe for an interview, at most.
So in my POV, it's just code synthesis from already existing stuff. And reviewed.
I think my job is safe.