A new study shows that computer systems like OpenAI’s ChatGPT can run a software company effectively and cheaply without relying on humans. Researchers from Brown University and Chinese unis did a test for the study. They wanted to see if a computer program could create software without any additional training using ChatGPT version 3.5. To test this, they created a fictitious software company called ChatDev. They divided the task into four parts: planning, coding, testing, and writing instructions.
They then instructed the computer programs what to do by giving them “important information”. This information included instructions on what tasks to perform, how to communicate with each other, when to finish, and rules. Each computer program was responsible for a specific task. Some were “CEO” and “CTO” in the “planning” section, while others were “programmers” and “art designers” in the “writing code” section. They communicated with each other without much human help to complete the software, from choosing a programming language to detecting errors in the code.
ChatDev: The CEO Who Never Sleeps
The scientists ran experiments with different types of software, noting how long it took ChatDev to complete each one and how much it cost.
Gizchina News of the week
For example, they asked ChatDev to “make a simple backgammon game”, which is a board game. During the planning phase, the CEO asked the CTO to recommend a programming language, and the CTO recommended Python. The CEO said this was a wise decision because Python is easy for novice and expert programmers. They then moved on to coding, where the CTO instructed the programmer to create a file, and the programmer instructed the designer to create a visually appealing interface. This process was repeated until the software was complete.
After assigning ChatDev 70 tasks, the research found that it could complete software development in less than seven minutes and for less than a dollar. It was also able to detect and fix problems on its own. According to the research, around 86.66% of the software it created was “perfectly done”.
The scholars reported that using ChatDev to create software is efficient and cost-effective.
So it turns out that powerful computer systems such as ChatGPT can perform certain tasks in a variety of ways. However, the study did have some shortcomings. The researchers found that language models such as ChatGPT can make mistakes or have biases. These difficulties can pose challenges when developing software. But, scientists think their discoveries might help new programmers or engineers in real life.