A Draft of a Treaty, with Annotations
Below, we provide an annotated example draft language for the sort of treaty that could be implemented by major governments around the world, if they recognized the dangers from artificial superintelligence (ASI) and sought to prevent anyone from building ASI.[1]
We are not policymakers and we are not well-versed in international law. We present this as an illustrative example of some potentially valuable treaty provisions to have in view, using mechanisms tailored to the situation at hand and grounded in historical precedent.
This draft text covers many different mechanisms that we think would be required to prevent AI developers from seriously endangering humanity. In practice, we would expect different aspects to likely be covered by different treaties.[2] And of course, in reality, the international community should carefully draft the whole treaty, subject to negotiation and review by relevant experts.
For each article in the example treaty below, we’ve provided a commentary section explaining why we made key decisions, and a section discussing some relevant precedent.
A real treaty would involve many details. We’ve included some example details, but most are relegated to “annexes” (which we do not flesh out in their entirety). Many of the quantities and numerical thresholds we use in our draft constitute our best guess, but they should still be treated only as guesses. Many of those numbers would require further study and revision before being finalized. These sorts of details plausibly wouldn’t be included in the treaty itself, analogous to how, in the case of the Treaty on the Non-Proliferation of Nuclear Weapons (NPT), specific details of inspections and so-called “safeguards” programs were decided between each country and the IAEA, rather than being included in the NPT itself. However, for clarity, we have kept our best-guess numbers directly in the treaty text, to help it feel more concrete.
It might be that nation-states concerned about artificial superintelligence would prefer to take smaller steps first — e.g., steps that don’t shut down AI research and development just yet, but that keep the option open to shut down AI R&D in the future. We don’t recommend that course of action, because we think the situation is already clearly out-of-hand and we are not confident the situation will get much clearer before it’s too late. Nevertheless, the MIRI technical governance team is working on proposals for those scenarios, in case they are helpful. You can follow their work here. ↩︎
This is the case with nuclear weapons agreements, where separate treaties establish the IAEA (1956, by the Conference on the Statute of the International Atomic Energy Agency, hosted at the Headquarters of the United Nations), the NPT (1970, through negotiations in the United Nations Eighteen Nation Committee on Disarmament), and the arms control agreements like the START treaty (1991, following nine years of intermittent negotiation between the U.S. and the Soviet Union). ↩︎