AI vs Human Reasoning

AI vs Human Reasoning

• Category: Artificial Intelligence
• Date 07-08-2023

Summary: In an eye-opening study, researcher revealed GPT-3, a popular artificial intelligence language model, performs comparably to college undergraduates in solving reasoning problems that typically appear on intelligence tests and SATs. However, the study’s authors question if GPT-3 is merely mimicking human reasoning due to its training dataset, or if it’s utilizing a novel cognitive process.

The researchers caution that despite its impressive results, GPT-3 has its limitations and fails spectacularly at certain tasks. They hope to delve deeper into the underlying cognitive processes used by such AI models in the future.

Key Facts:

1. UCLA psychologists’ study reveals that AI language model GPT-3 performs similarly to college undergraduates when solving certain reasoning problems.
2. Despite its performance, GPT-3 still fails significantly at tasks that are simple for humans, such as using tools to solve a physical task.
3. The researchers aim to investigate whether AI language models are starting to ‘think’ like humans or if they are using a completely different method that imitates human thought.

Source: UCLA

People solve new problems readily without any special training or practice by comparing them to familiar problems and extending the solution to the new problem. That process, known as analogical reasoning, has long been thought to be a uniquely human ability.

But now people might have to make room for a new kid on the block.

Research by UCLA psychologists shows that, astonishingly, the artificial intelligence language model GPT-3 performs about as well as college undergraduates when asked to solve the sort of reasoning problems that typically appear on intelligence tests and standardized tests such as the SAT.

The study is published in Nature Human Behaviour.

But the paper’s authors write that the study raises the question: Is GPT-3 mimicking human reasoning as a byproduct of its massive language training dataset or it is using a fundamentally new kind of cognitive process?

Without access to GPT-3’s inner workings — which are guarded by OpenAI, the company that created it — the UCLA scientists can’t say for sure how its reasoning abilities work. They also write that although GPT-3 performs far better than they expected at some reasoning tasks, the popular AI tool still fails spectacularly at others.

“No matter how impressive our results, it’s important to emphasize that this system has major limitations,” said Taylor Webb, a UCLA postdoctoral researcher in psychology and the study’s first author.

“It can do analogical reasoning, but it can’t do things that are very easy for people, such as using tools to solve a physical task. When we gave it those sorts of problems — some of which children can solve quickly — the things it suggested were nonsensical.”