everyonedies

Archive 2026-09-24 · 185 entries · 14 chapters

Chapters

Chapter 2

Grown, Not Crafted

This is the online resource for Chapter 2 of If Anyone Builds It, Everyone Dies. Below, we’ll discuss topics related to how modern AIs work and why it’s not “just another machine” or “just another tool.” Although AIs are code running on computers, they are unlike traditional handcrafted software, and they violate many assumptions that people usually take for granted when working with human inventions.

Some questions we don’t cover below, because they’re addressed in the book itself, include:

Questions

  1. Why does gradient descent matter?
  2. Do experts understand what’s going on inside AIs?
  3. Is intelligence understandable in principle?
  4. But some AIs partly think in English — doesn’t that help?
  5. Aren’t AIs “just math”?
  6. Aren’t AIs just predicting the next token?
  7. Aren’t AIs only able to parrot back what humans say?
  8. Won’t AIs inevitably be cold and logical, or otherwise missing some crucial spark?
  9. Won’t LLMs be like the humans in the data they’re trained on?
  10. How could an AI trained only on human data surpass humans?
  11. What makes you think people can build superhuman AI when they don’t even understand intelligence?
  12. Don’t hallucinations show that modern AIs are weak?
  13. But won’t we run out of data before AI goes all the way? Or electrical power? Or funding?
  14. Could LLMs advance all the way to superintelligence?

Extended Discussion

  1. What Good Does Knowledge of LLMs Do?
  2. “Obvious” Insights Take Time
  3. A Full Description of an LLM
  4. Intelligence Isn’t Ineffable ✎

Source: ifanyonebuildsit.com/2