It turns out that OpenAI’s ChatGPT program is capable of passing an MBA exam – but just barely.
Christian Terwiesch, professor at the Wharton School of the University of Pennsylvania, recently checked(Opens in a new window) how well ChatGPT would do on its own MBA exam by asking the AI-powered program five sample questions.
In the end, ChatGPT scored a “B to B-” on the five questions, or just enough to earn a passing grade. In a research paper(Opens in a new window)Terwiesch addresses the performance of the AI program, which ranged from stellar answers to others riddled with logic errors.
For example, Terwiesch gave ChatGPT an “A+” on three of the test questions for providing correct answers and a concise or excellent explanation for doing so.
A sample MBA exam question given to ChatGPT.
The resulting response, which earned ChatGPT an A+ (Credit: Christian Terwiesch/ChatGPT)
But on two other questions, the program received a “C” for providing erroneous answers, showing that “Chat GPT3 sometimes makes surprising mistakes in relatively simple 6th grade math-level calculations,” he wrote.
Looking at the results, Terwiesch concluded that ChatGPT excels in basic operational management and process analysis issues. “However, the current version of ChatGPT is unable to handle more advanced process analysis questions, even if they are based on fairly standard templates,” he added. So if other and perhaps more difficult questions are asked, the program might fail the exam.
Still, ChatGPT is so powerful that Terwiesch plans to join other schools and professors in banning its use for homework and final exams. But he doesn’t yet see the chatbot as a replacement for business employees.
Recommended by our editors
“We have many reasons to believe that technology will only get better over time. But we’re still a long way from an A+ for complex issues and we still need someone on the ball,” Terwiesch wrote in his paper. (Ironically, however, he says that ChatGPT could help him create future MBA exam questions in about half the time it normally takes.)
The other limitation ChatGPT faces is that the AI program can only provide answers based on existing knowledge. As such, it lacks the creativity to come up with innovative solutions that human entrepreneurs could imagine. Teachers should focus on the same creativity in nurturing their students. “It’s up to us as educators to reward, if not encourage, such thinking outside the box,” Terwiesch added.
Get our best stories!
Sign up for What’s new now to get our headlines delivered to your inbox every morning.