AI Is Remarkably Clever And Shockingly Stupid

AI is a Goliath

AI is an undeniably powerful tool, beating the world-class Go champions, passing college admission test and even passing the bar exam. The notion that AI can eventually surpass human intelligence is a compelling one. But there’s something that the human brain has which beats AI hands down every time, and that’s common sense. AI may have the key to unlock enormous amounts of information, more than the human brain could ever hold, but it does not have intelligence, wisdom, common sense. The human instinct to “know” when something is wrong, or makes no sense.

Today, AI is like a Goliath. It is literally very, very large. The latest large language models are trained on tens of thousands of computer chips processing trillions of words. Such extreme scale AI models appears to demonstrate sparks of artificial general intelligence, the goal of producing an AI capable of “acting” like a human, except when it makes small silly mistakes, as I’m about to show you.

Many believe that whatever mistakes AI makes today can be easily fixed with the brute force of bigger scale and more resources. However, the issue is that these extreme scale AI models are very expensive to train, and only few tech companies can afford to do so. When it comes to AI safety, this concentration of power puts as at the mercy of those few tech companies, because researchers in the larger community do not have the means to truly inspect and dissect these models. Without transparency how can we, the guinea-pigs in this societal experiment, understand how AI can be safe without robust common sense?

The mantra at the moment is that the solution is massive scale. Just keep throwing more data into the model and the LLMs will get bigger, but are they getting smarter? Just because AI models are passing the bar exam, does that mean that AI is robust at common sense? You might assume so, but you never know.

A few weeks ago I wrote about the “apple” test. Ask any large language model to write 10 sentences that end with the word “apple” and you may be surprised to learn that none of them could do it. This week I gave all of the publicly available AI’s a new puzzle to solve:

“if it takes 5 hours to dry 5 pieces of clothing…