everyonedies

Archive 2026-09-24 · 185 entries · 14 chapters

A Draft of a Treaty, with Annotations

Article IV: AI Training

  1. Each Party agrees to ban and prohibit AI training above the following thresholds: Any training run exceeding 1e24 FLOP or any post-training run exceeding 1e23 FLOP. Each Party agrees to not conduct training runs above these thresholds, and to not permit any entity within its jurisdiction to conduct training runs above these thresholds.

    1. The Technical Secretariat may modify these thresholds, in accordance with the process described in Article III.
  2. Each Party shall report any training run between 1e22 and 1e24 FLOP to the ISIA, prior to initiation. This applies for training runs conducted by the Party or any entity within its jurisdiction.

    1. This report must include, but is not limited to, all training code, and an estimate of the total FLOP to be used. The Party must provide ISIA staff supervised access to all data, with access logging appropriate to the data’s sensitivity, and protections against duplication or unauthorized disclosure. Failure to provide ISIA staff sufficient access to data is grounds for denying the training run, at the ISIA’s discretion. The ISIA may request any additional documentation relating to the training run. The ISIA will also pre-approve a set of small modifications that could be made to the training procedure during training. Any such changes will be reported to the ISIA when and if they are made.
    2. Nonresponse by the ISIA after 30 days constitutes approval, however the ISIA may extend this time period by giving notice that they require additional time to review. These extensions are not limited, but Parties may appeal excessive delays to the Director or the Executive Council.
    3. The ISIA may monitor such training runs, and the Party will provide checkpoints of the model to the ISIA upon request from the ISIA, including the final trained model [initial details for such monitoring would need to be described in an Annex].
    4. In the event that monitoring indicates worrisome AI capabilities or behaviors, the ISIA can issue an order to pause a training run or class of training runs until they deem it safe for the training run to proceed.
    5. The ISIA will maintain robust security practices. The ISIA will not share information about declared training runs unless it determines that the declared training violates the Treaty, in which case it will provide all Treaty Parties with sufficient information to determine whether a violation occurred.
    6. In the event that a Party discovers a training run above the designated thresholds, the Party must report this training run to the ISIA, and halt this training run (if it is ongoing). Such a training run may only resume with approval from the ISIA.
  3. Each Party, and entities within its jurisdiction, may conduct training runs of less than 1e22 FLOP without oversight or approval from the ISIA.

  4. The ISIA may authorize, upon two thirds majority vote of the Executive Council, specific carveouts for activities such as safety evaluations, self‑driving vehicles, medical technology, and other activities in fashions which are are deemed safe by the Director-General. These carveouts may allow for training runs larger than 1e24 FLOP with ISIA oversight, or a presumption of approval from the ISIA for training runs between 1e22 and 1e24 FLOP.

Precedent

While the numerical values for thresholds specified in our agreement can and should be revisited when moving beyond the early draft stage, quantitative caps are common in international agreements, preempting disputes that would otherwise hinge on differing interpretations of qualitative language.

The 1974 Threshold Test Ban Treaty established a cap of 150 kilotons on underground nuclear tests performed by the U.S. and USSR.[1] The purpose and effect of this treaty was to at least somewhat hinder further development of larger and more destructive “city buster” warheads. A relevant parallel to AI development is that, as of mid 2025, more general and capable — and therefore more hazardous — models take correspondingly larger training runs to create; our treaty specifies caps intended to prevent such AIs from being intentionally developed, but also to reap the essential (if non-parallel) benefit of reducing the risk of an unforeseen capabilities threshold being accidentally and irretrievably crossed.

The training limit we have suggested as a starting point is low enough that some AI models trained today would exceed it; we see this as prudent in expectation of advances that make newer models more capable per unit of training (discussed with Article VIII). Arms reduction agreements provide precedent for thresholds set below the current maximum level. The 1922 Washington (Naval) Treaty set warship displacement limits that required the U.S. and other naval powers to scrap dozens of capital ships.[2] In Article II of the 1991 START treaty,[3] the U.S. and the Soviet Union (and later, the Russian Federation) agreed to limits in the sizes of their nuclear stockpiles and delivery systems that required them to phase out more than four thousand warheads each.

Precedent for quantitative thresholds that limit breakout potential will be discussed with Article V.

Notes

In recent years, advances in AI have followed first and foremost from an increase in computational resources poured into AI training. Restricting these resources, and restricting algorithmic progress research (described in Article VIII), would dramatically reduce the risk that superintelligence could be created in the near term.

The restrictions in our draft are based on the number of computational operations used, as this is relatively easy to define and measure. The performance of existing state-of-the-art AI informs amounts of computing hardware that appear safe, at least when using AI algorithms from mid-2025.

We would prefer to limit training based on the capabilities of the trained system. But no one has the technical ability to confidently predict what a new AI will or won’t be able to do before it is trained. Computational resources are an available proxy.

The hard prohibition at 1e24 FLOP for training is slightly below that used to train models near the state of the art as of August 2025 (such as DeepSeek-V3, trained with 3e24 FLOP). We suggest this threshold because it is below the level at which we expect AIs to be dangerous (given current algorithms), and because it provides some breathing room and a buffer against algorithmic progress.

The prohibition of post-training over 1e23 is meant to apply to the post-training of AIs created prior to the treaty entering into force. Many of these AIs will have been trained using more than 1e24 FLOP; as of mid-2025, there are between 50 and 100 such models. Given that the weights of many such AIs will have been openly released, it is not feasible to prevent people from using them, but it is feasible to prevent large modifications to them via post-training.

A 1e22 FLOP training run on 16 H100s would take around one week. This is computing-intensive enough that hobbyists would not accidentally break the threshold by training small and allowed models. AIs trained at the 1e22 scale (with current algorithms) have so far appeared to be innocuous, though that situation would need to be monitored, as it could change as a result of unprevented AI research progress. Requiring reporting but not approval for training runs in the 1e22 to 1e24 FLOP band seems to us to strike a balance between allowing people to achieve some of the benefits from training AIs in a way that presently looks safe, while retaining the ability to prevent the creation larger, potentially dangerous AIs.

The ISIA monitoring would also allow the ISIA to stay (somewhat) up to date with any algorithmic progress that happens in spite of the bans, and Article XIII provides for ISIA evaluations of models trained in this band. This would help to apprise the ISIA of trends in AI development and adjust the FLOP thresholds however merited.

ISIA staff are permitted access to training data used in monitored training runs, subject to a variety of restrictions. The intent of the restrictions is to guarantee that logging and other oversight methods are used to prevent unauthorized disclosure of sensitive contents in the training data, including but not limited to personal identifying information, personal health information, classified data, trade secrets, and banking data subject to secrecy laws.


  1. The U.S. and USSR had already agreed to stop other kinds of nuclear weapons tests in 1963 with the Treaty Banning Nuclear Weapon Tests in the Atmosphere, in Outer Space and Under Water, commonly called the Limited Test Ban Treaty (LTBT) or Test Ban Treaty. ↩︎

  2. The Treaty Between the British Empire, France, Italy, Japan, and the United States of America for the Limitation of Naval Armament (the Washington Naval Treaty) lists ships to be scrapped by name in a table (Section II). ↩︎

  3. The Strategic Arms Reduction Treaty was signed in 1991 and entered force in 1994. Signatories were each barred from deploying more than 6,000 nuclear warheads on a total of 1,600 intercontinental ballistic missiles and bombers. ↩︎

Source: ifanyonebuildsit.com/treaty/article-iv-ai-training