Chapter 5
Its Favorite Things
This is the online resource for Chapter 5 of If Anyone Builds It, Everyone Dies. Some topics we don’t cover on this page, because they’re addressed in the book, include:
- What motive would AI have for wiping us out?
- Won’t sufficiently intelligent AIs discover that the right thing to do is to help us all flourish together?
- Won’t humans still be valuable to superintelligent AIs, e.g., as trading partners?
- It’s a big universe. Why wouldn’t AI just leave us alone?
- Would it be a meaningful end to humanity, to let ourselves be replaced by something smarter?
In the book, we said that we’ve heard a long list of hopes and copes about superintelligence might benefit humanity in spite of the issues described in Chapter 4. This is the place where we summarize and respond to a long list of them. Many of the answers overlap; two relatively central points we’ll be making are that humans aren’t the most efficient solution to almost any problem, and that AI is unlikely to care about us even a small amount. This is also where we’ll dive into a discussion of AI consciousness and morality.