Chapter 1: Humanity’s Special Power · Extended Discussion

The Shallowness of Current AIs

In the chapter, we wrote that you can “see a shallowness” in the intelligence of current AIs (as of mid-to-late 2025), if you know where to look. If you haven’t seen it yourself yet, here are a few places you might look:

• Anthropic’s Claude 3.7 Sonnet got stuck in repetitive loops while trying to beat a simple Pokémon video game.
• In November 2022, one of the best Go-playing entities in the world was an AI called KataGo. At least, right up until researchers found a way to defeat it using a predictable series of moves that triggered a sort of “blind spot” and caused KataGo to blunder in a way that even amateurs would not. Two years later, engineers still could not render it robust against attacks like this.
• Current “multimodal” LLMs (the kind that can work with text, pictures, and other media rather than just text) struggle to interpret analog clocks and calendars in problems that most human 4th graders can handle.
• Current LLMs notoriously flub simple variations of a classic doctor riddle with straightforward non-trick answers, seemingly unable to resist calling out the trick answer the riddle has in its usual form.

(The online resources for Chapter 4 offer a more technical look at where this shallowness might be coming from.)

None of this is to say that AIs are stupid across the board. Modern AIs can also achieve gold medals in the International Mathematical Olympiad, which is a noteworthy feat of mathematical prowess. Modern AIs can do an incredible variety of things, often matching or exceeding human performance.

Their skillset is strange. Human strengths and weaknesses are a poor guide to what AIs will find easier or harder, because AIs radically and fundamentally differ from humans in many ways.

We are not saying that ChatGPT is going to kill you tomorrow (as we previously mentioned). There’s still a shallowness, of sorts, to modern AIs. Rather, we observe that the field is making progress, and it’s not clear how long this shallowness will last.