A Draft of a Treaty, with Annotations

Article IX: Research Restriction Verification

1. Each Party shall create or empower a domestic agency with the following responsibilities:

   1. Maintain awareness of and relationships with domestic researchers and organizations working on areas adjacent to Restricted Research, in order to communicate the categories of Restricted Research established in Article VIII.
   2. Impose penalties to deter domestic researchers and organizations from conducting Restricted Research. These penalties shall be proportionate to the severity of the violation and should be designed to act as a sufficient deterrent. Each Party shall enact or amend legal statutes as necessary to enable the imposition of these penalties.
   3. Establish secure infrastructure for reporting and containment of inadvertent discoveries meeting the conditions for Restricted Research. These reports will be shared with the Research Controls division.

2. To aid in the international verification of research bans, the Research Controls division will develop and implement verification mechanisms.

   1. These mechanisms could include but are not limited to:
      1. ISIA interviews of researchers who have previously worked in Restricted Research topics, or are presently working in adjacent areas.
      2. Monitoring of the employment status and whereabouts of researchers who have previously worked in Restricted Research topics, or are presently working in adjacent areas.
      3. Maintaining embedded auditors from the ISIA in selected high-risk organizations (e.g., projects difficult to distinguish from Restricted Research, organizations that were previously AI research organizations).
   2. Parties will assist in the implementation of these verification mechanisms.
   3. The information gained through these verification mechanisms will be compiled into reports for the Executive Council, keeping as much sensitive information confidential as possible to protect the privacy and secrets of individuals and Parties.

Precedent

Existing agencies empowered to “maintain awareness of and relationships with domestic researchers and organizations” at risk of developing restricted information, as called for by our Article IX (1.a.) include the DOE and NNSA, discussed in the precedent section for Article VIII.

Precedent for “monitoring of the employment status and whereabouts of researchers” in high-risk fields, as we suggest in Paragraph 2.(a).(ii), can be found in the International Science and Technology Center (ISTC).^[1] Established in 1994, the ISTC was specifically created to reduce nuclear proliferation risks by keeping Soviet nuclear researchers gainfully employed in peaceful activities and connected to the international scientific community. The ISTC also shows the potential of incentives as a complement to penalties for keeping technical experts (who may find themselves unemployed as a result of this treaty) from engaging in Restricted Research.

To the extent that penalties may need to be severe to provide the deterrence indicated in our Article IX.1.(b), a template may be found with the Enforcement chapter (18) of the 1946 Atomic Energy Act, under which the unauthorized sharing of Restricted Data can be punished by death or imprisonment if the disclosures were made with treasonous intent.^[2]

When developing secure ISIA “infrastructure for reporting and containing inadvertent discoveries of Restricted Research,” precedent and potentially usable templates may be found in the extensive DOE procedures for handling different categories of sensitive data. The DOE’s Occurrence Reporting and Processing System, as well as the Committee on National Security Systems’s^[3] instructions for classified information spillage, may also be of use.

Our treaty’s Research Controls division might look to existing practices by the IAEA when developing inspection protocols. Under the framework of the Model Additional Protocol approved in 1997 by the IAEA Board of Governors, states that have made comprehensive safeguard agreements^[4] allow complementary access inspections that look for undeclared nuclear material. As part of such visits, inspectors may interview operators, analogous to our proposal in Paragraph IX.2.a.i.

We also propose “maintaining embedded auditors from the ISIA in select high-risk organizations,” much the way DOE and NNSA field offices are physically located at contractor-operated national nuclear labs and production plants today.

To “protect the privacy and secrets of individuals and parties” when performing verifications, as required by this article’s Paragraph 2.(c), the ISIA Research Controls division might adapt compartmentalization practices of parties’ existing intelligence agencies and multilateral intelligence-sharing agreements. For example, under the “third party rule” or “originator control principle” understood to be commonplace in such arrangements, it is prohibited to disclose shared information to third parties (potentially even oversight bodies) without permission from the originating agency.

Notes

To help verify that there is no prohibited AI research happening, Article IX tasks parties with demarcating “areas adjacent to Restricted Research” and then establishing relationships with the researchers working in these adjacent areas. There are sufficiently few top AI researchers in the world that it may be feasible to track the activities of a significant fraction of them. The technical staff of top AI companies numbers on the order of 5,000 researchers, and it is commonly believed that a much smaller group is critical to frontier AI development, likely numbering in the hundreds.^[5] The number of attendees of top AI conferences is estimated to be about 70,000. States could interview researchers about their activities and offer asylum and financial incentives for any whistleblowers (see Article X).

While much about current AI development practices happens in the public view, we think legal restrictions would dramatically hamper the efforts of rogue actors to create superintelligent machines.

Monitoring could be extended to researchers and engineers involved in semiconductor design and manufacture if states are willing to incur the extra costs. A more affordable alternative might be to monitor semiconductor manufacturing companies rather than individuals, taking advantage of complex dependencies within the industry which ensure that small groups of rogue individuals would have trouble creating their own chip fabricators.

Parties may be concerned that other parties will violate domestic research bans and hide research efforts from foreign intelligence. Most likely, large efforts involving many researchers and AI-relevant chips would be noticed by a determined intelligence community. But smaller efforts, like developing alternative machine intelligence paradigms, might only involve a few researchers and commonly available hardware. Verifying a research ban is a complex and sensitive undertaking requiring ongoing effort and iteration. To facilitate that end, Article X (below) institutes a variety of tools to facilitate intelligence gathering and to protect whistleblowers.