A Draft of a Treaty, with Annotations

Article I: Primary Purpose

Each Party to this Treaty shall not develop, deploy, or seek to develop or deploy artificial superintelligence (“ASI”) by any means. Each Party shall prohibit and prevent all such development within their borders and jurisdictions, and, due to the uncertainty as to when further progress would produce ASI, shall not engage in or permit activities that materially advance toward ASI as described in this Treaty. Each Party shall assist, or not impede, reasonable measures by other Parties to dissuade and prevent such development by and within non-Party states and jurisdictions. Each Party shall implement and carry out all other obligations, measures, and verification arrangements set forth in this Treaty.

Where some classes of AI infrastructure and capabilities staying far from ASI may be deemed acceptable but only under conditions of international supervision, only Parties to the Treaty may carry out such activities, or own or operate AI chips and manufacturing capabilities that could potentially lead to the development of ASI if unsupervised. Non-Parties are denied such access for the safety of the Parties and of all life on Earth.(Article V, Article VI, Article VII).

Parties commit to a dispute resolution process (Article XI) to minimize unnecessary Protective Actions (Article XII).

Precedent

Article I of the NPT, as in many treaties, states the high-level commitment parties are making — in this case, to not share their nuclear weapons or help others obtain them:

Each nuclear-weapon State Party to the Treaty undertakes not to transfer to any recipient whatsoever nuclear weapons or other nuclear explosive devices or control over such weapons or explosive devices directly, or indirectly; and not in any way to assist, encourage, or induce any non-nuclear-weapon State to manufacture or otherwise acquire nuclear weapons or other nuclear explosive devices, or control over such weapons or explosive devices.

The commitment summarized in Article I of our draft agreement is stronger than this because an ASI breakout by anyone, anywhere, cannot be allowed to happen even once.^[1] It would not be enough to not “assist, encourage, or induce” others to build it. We have therefore included a commitment to “assist, or not impede, reasonable measures” by parties to dissuade and prevent such development anywhere.

The NPT works to contain an existing threat (nuclear weapons), while our draft agreement is working to prevent a threat from existing at all (ASI). Precedent for preventing the development of dangerous new technology can be found in the Protocol on Blinding Laser Weapons, part of the Convention on Certain Conventional Weapons.^[2] Its Article I reads:

It is prohibited to employ laser weapons specifically designed, as their sole combat function or as one of their combat functions, to cause permanent blindness to unenhanced vision, that is to the naked eye or to the eye with corrective eyesight devices. The High Contracting Parties shall not transfer such weapons to any State or non-State entity.

That language doesn’t try to keep anyone anywhere from ever testing or accidentally making such a system, however. Our agreement must be strong enough to prevent ASI from being made accidentally. Because it’s not clear where the point-of-no-return might be, our Article I includes a commitment to “not engage in or permit activities that materially advance toward ASI.”