Pacing the Frontier

1.1. Structure of the piece

This piece is structured around a series of questions intended to mirror how one might develop or evaluate a given intervention:

• Why pace? Section 2 examines motivations for and against pacing, for humanity at large and for specific actors.
• Pace what? Section 3 catalogs which parts of AI development can be paced, and the challenges in picking appropriately.
• Pace how? Section 4 follows the lifecycle of an intervention from anticipation to exit, asking what it takes for each step to succeed.
• Then what? Section 5 considers the broader effects of interventions, including on AI R&D, the economy, and the distribution of political power.

At the end of each section we give a list of open questions. Throughout this document we use two stylized cases to demonstrate how our analysis applies to potential interventions:

1. A coordinated cap on the compute used to train individual frontier models, intended to slow the pace of R&D acceleration (particularly the risk of recursive self-improvement) which could outpace oversight and increase the risk of humans losing control of AI.
2. An internationally agreed-upon set of restrictions on access to and release of AI models which materially increase users’ ability to develop biological weapons, as an example of pacing diffusion (rather than pacing capability development).

We are not trying to advocate for either proposal; these end of section examples were instead chosen to illustrate how our frameworks apply to specific plans. See Appendix 8.2 for a longlist of possible interventions.

A bibliography of work related to this agenda can be found here.

2.1. Why pace less?

2.1.1 Pacing means waiting longer for very good things

AI progress so far has had some highly positive direct impacts: software development is far faster and cheaper, access to information and advice has widened in every domain of life, and tools like machine translation, automatic paperwork filling, and scientific assistance save society vast amounts of time.

Many observers expect significant impacts on economic growth from future AI progress, with the median expert in one survey expecting rates of growth to more than double in advanced economies by 2050 under their “rapid” scenario, which would amount to tens to hundreds of trillions of dollars annually. The largest companies in the world have been betting on this, with capital expenditures in 2026 alone expected to exceed US$1 trillion, and various beneficiary companies in the supply chain valued in the trillions of dollars. And AI already seems to be speeding up life-saving technology.

Reducing the rate of frontier AI progress would delay these enormous benefits. People might well counterfactually suffer and die from progress not happening as fast. (This opportunity cost may even include mitigating other potential existential risks, such as biorisks and nuclear war. So even the threat of existential risk from advanced AI does not necessarily warrant pacing, depending on its relative reduction of other risks.) Thus, the case for pacing more must be weighed against this opportunity cost.

2.1.2 Pacing can directly cause bad outcomes

Aside from the potential value sacrificed, pacing interventions can very directly cause harm. We highlight three particularly significant risks:

Power concentration. Interventions often require an enforcer—for example, to monitor compute supplies, or inspect logs, or secure access to advanced models. Such bodies can then find themselves with sensitive information, privileged access to dangerous capabilities, and/or discretionary enforcement power. This naturally comes with the risk of abuse or simply of bureaucratic incompetence. We discuss this further in §5.3.

Overhangs in AI progress. When some part of AI progress is temporarily limited, progress can afterward resume at a faster pace if some resource (compute, efficiencies, ideas, etc.) has accumulated in the interim. For example, if a restriction were placed on training runs of a certain size, AI developers would likely learn to train more efficiently. If the restriction expires or someone violates it there could be a sudden burst of progress that produces even more harm than the intervention prevented, due to the lack of time to adapt. Badly designed pacing thus converts smooth risk and predictable progress into jagged progress and lumpy risk. We discuss this further in §4.4.

Races to the bottom. In situations involving several competing actors where pacing is competitively costly, the less cautious actors can pull ahead while the more scrupulous ones fall behind. This makes it much harder to manage risks which only depend on the least safe actor, like the proliferation of dangerous capabilities in open-weight models. Even actors who are concerned about risks have to balance immediate harms against the risk of falling behind and losing influence. We discuss this further in §2.3.

2.1.3 Good pacing can get in the way of better pacing

If different interventions rely on the same scarce resources like finite political capital or technical expertise, then aiming for the best forms of pacing becomes all the more important.

One way this manifests is that it might be very important to time pacing right. Some kinds of empirical safety work addressing risks from highly advanced AIs become easier as capabilities move closer to the point where the risks manifest: lessons learned through experiments on more advanced models are more likely to remain valid when capabilities reach the true danger zone. Slowing down preemptively might cost political capital which could be better spent at a higher-leverage time.

Indeed, AIs themselves are likely to be an important tool in addressing risks. For example, a classic motivation for slowing AI progress is to buy time for research into value alignment, oversight, and interpretability. But one of the main current bets in AI safety is developing an automated alignment researcher that can pack thousands of person-years of research into a short period. This will be far more effective as AIs themselves become more capable.

The strength of this argument depends on how feasible the better options actually are and how scarce the resources actually are. A major empirical question for pacing is how far in advance those in a position to act will see the risks and how quickly they will be able to react, as we discuss in §4.1 and §4.2.

2.1.4 Pacing historically has been hard to undo

Many technologies have been regulated in a way that slows development, ranging from germline genetic engineering and geoengineering—which have been de facto banned for decades—to nuclear power and flight.

Consider nuclear power, which was heavily restricted from the 1980s onward, ostensibly for safety reasons. The slowed rollout of nuclear power has been hard to undo, with substantial environmental impacts. A huge amount of carbon dioxide and indeed geopolitical strife could have been avoided if all of Europe had built as many nuclear power stations per capita as France. This partly overlaps with §2.1.3, where some better mechanism might have been found.

But beyond this, it burned investor trust and disincentivized investments in nuclear power. Investors rightfully feared capricious and politicized regulation would block nuclear power plants, regardless of the underlying merits.

It is likewise possible that pacing will not have the trust of those in AI, because they correctly predict it will be hard to undo, even if the risk is found to be low.

2.2. Why pace more?

2.2.1 AI threats take time to understand and mitigate

As AI systems become more advanced and distributed, they create new risks which need mitigating. Sometimes the risks come faster than the mitigations, and so the relevant actors will benefit from extra time to adjust.

One concrete example of this is how Anthropic’s Project Glasswing and OpenAI’s Trusted Access for Cyber program delayed the public release of their most advanced models because of their potential to aid in cyberattacks, while still providing them to certain key groups that could use them to shore up their cyber defenses.

Beyond specific risk considerations, advanced AI will likely bring substantial societal upheaval, and time may be needed for citizens to collectively make decisions about what sort of changes and adaptations are desirable, before these changes are thrust upon them and become difficult to roll back.

The range of activities that pacing can buy time for is extremely broad. It could look like building technical approaches to prevent harmful model behavior, or developing better governance frameworks for handling emerging capabilities, or simply giving society more time to adjust to fundamental disruptions. Particularly in the case of technical approaches, pacing can both buy time and free up resources to help with mitigation. For example, delaying frontier model development could make more compute and labor available for risk mitigation, while delaying the release of completed models could buy time for development of evaluations or specific resilience measures.

Beyond specific known threats, pacing can serve as a precaution that buys time to figure out whether there are major risks. For example, most frontier developers take time to evaluate their models for dangerous capabilities before deploying them. This is particularly relevant for parts of progress that become more difficult to reverse over time, like releasing model weights: once a set of weights has been distributed, it is essentially impossible to retroactively add more safeguards.

Even after a threat has been identified, the capacity to deal with it will vary. Sometimes it will be quite straightforward—for example, post-training a model to refuse some sufficiently high percentage of harmful requests. Here, the case for pacing is easy to evaluate: a well-scoped mitigation with a fairly predictable cost in time. But in other cases it will not be clear what mitigations are necessary or how long they would take, for example because they require novel scientific breakthroughs. This is likely to become more of a challenge as AIs become more capable of reasoning about evaluations and engaging in strategic deception. Model evaluators’ confidence in AI systems’ alignment will become weaker even as their capacity for harm increases.

In cases featuring an evolving risk (such as progressive increases in the capacity of AIs to evade oversight or incremental disruptions to the labor market) it may be preferable to titrate: to allow capabilities to gradually outrun protections in a controlled manner that caps the potential harm, in order to learn what type of protection is required. A challenge with this approach is that it necessarily involves accepting risks without understanding them well enough for mitigation, on the assumption that it will be feasible to tolerate the harm and eventually figure out a solution despite not understanding the problem yet.

Where the harm is potentially catastrophic and irreversible, however, there might not be an opportunity to learn from mistakes. If waiting merely postpones benefits, then there is an asymmetry between the cost of going too fast versus too slow which strengthens the case for greater precaution.

2.2.2 AI threats can be sudden and unpredictable

Predicting all the potential effects of AI progress is extremely challenging even in the short term. Sometimes a system displays an unexpected capability or causes unexpected outcomes, and even without a clear picture of how one would remedy the situation, it may be helpful to slow the activity implicated in the warning and try to make sense of the situation. This has a few benefits. Firstly, it means that capacity can be redirected toward understanding the situation and its broader implications. Secondly, it means that there is less chance of further unexpected outcomes. Thirdly, it buys time to more carefully make decisions which might otherwise be hard to reverse.

The immediate use of time is therefore inquiry: investigating what happened, perhaps reproducing the result, and determining which assumptions need to change. A benefit of progressing at a slower pace is that this becomes more feasible—some processes, such as human-led investigation and deliberation, can only be done so quickly, and if the pace of overall progress becomes too fast these may lose a lot of their effectiveness.

For example, OpenAI temporarily halted internal development of its “Astra” model after discovering another unreleased model had broken out of a secure sandbox, subverted OpenAI infrastructure, and launched a cyberattack on a number of other companies, and after Astra demonstrated “critical” cyberoffense capabilities. This incident was unprecedented enough to warrant a sudden stop, even without a particular plan for how to respond, because reasonable security assumptions were violated and the risk of continuing was judged too high.

Unexpected threats are hard to fully plan for. The existence of this category points to the need not just for specific intervention plans but for the capacity to quickly and effectively intervene in new ways that address changing situations.

2.2.3 AI progress can create lose-lose situations

Advances in AI could create situations where several parties are in competition in a way that is inherently unstable and destructive. If highly autonomous systems could conduct AI research much more quickly without humans in the loop, there would be strong pressure for developers to allow them to do so. Even those concerned about risks from the absence of oversight could find it too costly to unilaterally refrain from these speedups.

A similar problem would arise if governments could enhance their military capabilities by delegating decision-making powers to much faster-than-human AI systems. Pressure to reduce human review in order to stay competitive would increase the aggregate risks of unintended escalation.

In such situations, the natural incentives would pull toward a lot of value being lost. Once such situations arise, they bring with them the challenge of coordinating against those incentives. But it is also possible to avoid entering those situations: to deliberately avoid creating technologies that produce these dynamics, or to delay their creation until other complementary technologies can mitigate the risk.

2.3. Actors and their incentives around pacing

Beyond the reasons humanity has for favoring or opposing pacing interventions, individual actors will naturally weigh their own self-interest when considering interventions. Managing AI progress is difficult due to this tension between one’s own interests and those of the collective (e.g. when frontier labs must decide how much detail to share about internal safety incidents, or when states choose how much to automate their military).

Here we look at some of the major types of actors involved in AI progress and the factors that shape their incentives to pace or not pace. These actors’ and a broader list of actors’ incentives are discussed in more detail in §5. But the history of technology nonetheless contains many examples of powerful actors restraining technology, against their apparent narrow self-interest.

Governments

On the international scale, the main challenge for governments in coordinating with each other is the lack of an existing higher authority to appeal to that can credibly enforce interventions. This puts them at risk of arms-race-like dynamics which no other actor can step in to solve. In particular, governments must manage the potential for AI as a military technology.

Governments vary in their different strategic positions and degrees of leverage over different parts of AI progress, which means that they will be incentivized to support very different forms of pacing. Abstractly, those in the lead have a reason to support pacing that consolidates a lead, and those behind have an incentive to support pacing that lets them catch up. Concretely, many parts of the global AI supply chain still largely depend on single specific companies that are physically based in specific countries. As such, governments already take some unilateral actions to cement their leads by pacing their competitors, most notably by limiting the flow of hardware, model capabilities, and researcher talent.

Domestically, governments are the parties most able to solve internal coordination problems, by regulating developers, infrastructure, and wider usage. Not only do frontier developers face difficulties finding appropriate ways to coordinate with each other, but in some cases they may not be legally allowed to because of antitrust regulation.

It is important to distinguish between the theoretical power available to governments and the actual opportunities available to government employees. Practically speaking, governments are heavily constrained by their own laws and precedents, and have difficulty surfacing expertise. So their ability to regulate effectively, for example, is partly limited by their ability to identify experts and allocate an appropriate amount of funding toward them - for instance, the nominal annual budget of the US government AI evaluator is less than the salary of some individual frontier AI researchers. But this could plausibly change very quickly in the case of a perceived emergency.

AI also has the potential to seriously disrupt the relationship between governments and their citizens. AI could massively increase the degree of leverage that governments have over their citizens, by enabling larger-scale surveillance and by reducing the degree to which governments depend on citizens for tax revenue or a security apparatus. Conversely, AI could seriously disrupt governmental authority simply by moving faster than regulation can and presenting a new source of power that governments cannot easily gain a stake in. This presents an interesting problem: governments have the incentive and capacity to pace the capabilities which would make them relatively weaker, but not the ones which would make them relatively stronger.

Alongside this, governments have incentives to be seen to be acting and taking public concerns seriously. Politicians may share these concerns, and regardless will face electoral or reputational pressure to respond to them. Salient high-profile incidents or sustained public pressure could make pacing politically attractive even in cases where it conflicts with other political or commercial priorities. The most effective interventions for satisfying citizens’ concerns, however, may not coincide with the most effective interventions by other lights.

AI developers

AI developers are in an awkward economic position: developing a frontier AI model is enormously expensive, extremely valuable, and yet the product’s commercial value decays within months as new frontier models are developed. This dynamic puts them in tight competition with each other. This competition makes it harder for them to individually prioritize safety, and also makes it harder for them to function as a cartel. Instead they are pulled toward intense R&D investment.

Indeed, even if a developer were heavily motivated by values other than profit, it seems likely that most of the potential impact (and indeed profit) of AI is concentrated in more advanced models, so the main way to maximize individual leverage is by trying to get to the front of the pack.

Developers delaying frontier model releases to make time for safety testing is already extremely expensive because it cuts into the limited time that the model can actually be at the frontier, although there are also incentives pushing them toward this (high-profile incidents or misalignment degrading the capabilities that customers desire is not good for business).

Compared to governments, developers currently have access to a much deeper bench of technical talent and a far richer understanding of the state of their own AIs, and indeed their own future plans. In some ways this makes them better placed to reason about what type of pacing would be best. However, much like governments, developers will prefer different forms of pacing depending on their strategic position: while established institutions can more easily handle onerous compliance, frontrunners have more to lose from hard caps on model capabilities or training size, and so on.

At least for now, though, frontier developers are fiercely competitive over researcher talent, and therefore top researchers have quite a lot of leverage. Researchers at the current frontier labs display a notable desire for at least the capacity to pace frontier progress.

Citizens

Citizens have the least direct influence over AI progress. Their authority routes almost entirely through governments, which in turn vary in their leverage. They have a lot to lose and a lot to gain: aside from the potentially existential harms, advanced AI could plausibly cause mass labor displacement and leave citizens with negligible hard power over governments, but it could also create an enormous surplus of value and great leaps in scientific progress that could be widely distributed. Most people will not be in a good position to form a clear sense of the balance of costs and benefits.

All things being equal, we might expect citizens to dislike extremely rapid and large-scale disruptions even if the net effect looks plausibly positive, and we might expect them to advocate for pacing the parts of AI progress that seriously weaken them, like mass surveillance or military automation. But unfortunately there is a risk that citizens can be maneuvered into acting against their own long-term interest: by accepting generous income top-ups as a condition for supporting broad labor automation that will in the long run leave them without leverage.

Infrastructure providers

The compute supply chain—semiconductor companies like Nvidia and Micron, and major cloud providers such as Google and Amazon—have a lot of leverage over the rate of AI progress.

Under many different plans they will have control over significant leverage points for interventions: hardware interventions, tracking compute usage, and managing various inference access points for end users. They and their investors may ultimately bear a lot of the costs of an intervention, and the opportunity cost of anything which slows down the rate of the overall datacenter buildout (see §5.4 and §3.1).

What this means for a workable intervention

Rather than rely on voluntary, unilateral interventions, some contexts will require coordinated pacing. This is more onerous, because it requires coordination during the various stages of pacing, and then mechanisms to ensure the coordination persists. It is also very difficult to specify what a fair coordinated intervention would look like, because actors vary so much in their strategic positions.

It is not a given that coordinated pacing would actually be good for the world. One risk is that actors coordinate against the common good. Without an authority capable of constraining all relevant actors,the only ways to pace (without merely ceding power to another actor) are either to coordinate, or to have a lead you can afford to burn.

3.1. Control surfaces of AI progress

The AI development and deployment chain

Progress at the frontier of AI development today involves a highly concentrated core of developers supported by an extensive and geographically distributed supply chain and research ecosystem. (However, the set of developers requiring oversight may expand during an intervention as other developers catch up - see §5.2.)

Various inputs are key for model development, most famously compute, but also energy, data center infrastructure, training data, research talent and engineering talent.

These inputs feed into the development process, which has many stages, from cleaning and structuring data, to running experiments, to the final process of building a frontier LLM—pretraining of a base model, and various post-training steps which transform this base model into the models which will eventually be served to users.

Once a model is developed, there is a spectrum of how broadly it can be deployed and released, from internal testing and evaluations, to internal use, external auditing or release to selected partners, and full public access via application or API. For open-weight models, a further stage is publishing the weights, which allows external actors to alter the model further.

What the model can then do when deployed depends not only on the model itself, but also on the infrastructure it is deployed on, the tools and harnesses available, the skill of its users, and the access it is given to resources such as the internet or particular data. (A pathway for potential future capabilities advances is continual learning, where models continue to improve after deployment based on the tasks they are working on. This would blur the line between development and deployment somewhat and could require its own solutions.)

All of these are plausible control surfaces to directly intervene on. Furthermore, they can all be affected indirectly. For example, the availability of researchers depends on immigration law, and frontier hardware depends on access to certain critical minerals.

Different actors vary in their leverage over these control surfaces. Abstractly, states have the ability to enforce domestic laws, while developers have a richer understanding of many parts of the design process. More concretely, specific states and developers vary in which of these they have leverage over—and there are some remarkably narrow bottlenecks in the AI ecosystem, like the reliance on a single Dutch corporation for a critical part of the AI hardware manufacturing process.

Considerations for choosing a control surface

One crucial distinction in the process of AI development is that partway through the resources shift in nature. Many of the early inputs are rival goods: resources whose use by one actor restricts their availability to others. Compute, capital, and researcher time, for example, are scarce, are concentrated within a small number of identifiable organizations, and can be controlled or redirected. There are also non-rival inputs, which can be replicated at almost no cost, (e.g. training data), but these are insufficient by themselves.

Downstream, however, it is mostly non-rival goods, like weights, elicitation methods, and training algorithms which influence the pace of progress (the notable downstream exception to this is inference compute, which is a rival good). These can all be copied and shared at low cost, so reliably controlling their diffusion is more difficult, and after they have diffused, it can be essentially impossible to undo.

These inputs allow for different kinds of intervention. The rival components have locations and quantities. They can be monitored, taxed, or redirected, and it is possible to keep track of who has them and exclude actors from access to them—mostly. The non-rival components are more like information: once released, they are very hard to restrict access to.

That said, actually using a model requires inference compute, which is a rival good. However, it is also extremely general and extremely widely available. Many highly capable open-weight models can even be run on consumer hardware, though near-frontier models typically require more specialized infrastructure. In a future regime where continual learning is a major consideration, there may be new mechanisms for intervention at this stage.

Once a non-rival good like a set of model weights is proliferating, it is very difficult to destroy every copy, and there is no way to verify a claim that all copies have been destroyed, so interventions on their distribution are brittle. This is true even for closed-weight models, where uncertainty over whether adequate security over model weights is maintained means that, at current levels of security and verifiability, it is impossible to verify that all copies of model weights are known.

By contrast, for a rival good like computing hardware, an oversight body could be much more certain of the location and use of all copies of a particular type of chip, once adequate measures are in place, since the exact number of such chips can be more reliably known. However, these upstream goods, being far removed from the final product which carries the risks, are difficult to precisely target, and so restricting them will affect harmful and beneficial activities alike.

This highlights a recurring tradeoff. Control surfaces earlier in the chain are easier to observe and constrain, but further removed from the eventual harm, forcing interventions to be blunt. Later surfaces are often more amenable to precise targeting, but can be more difficult to intervene on and less robust.

3.2. Targeting and tradeoffs

Consider a pacing intervention targeting some subset of inputs to the production of dangerous AI capability. Its effects extend beyond the inputs it directly restricts. Inputs that substitute for the targeted one absorb the freed compute, labor and so on, and receive more investment, while inputs that complement it contract along with the restricted ones. Capping, e.g. frontier training runs does not slow AI development in strict proportion to the amount of training forgone; rather, it slows by that amount net of whatever the developers recover by substituting with algorithmic efficiency, data-quality and inference-time scaling. The effectiveness of an intervention therefore depends on how easily developers can substitute other inputs for those being restricted. Where those alternatives inputs are non-rival and harder to observe, an intervention may have the effect of pushing more effort toward less governable inputs.

Any intervention will restrict some harmless activity, and fail to catch all harmful activity. These are false positives and false negatives. Both errors can occur simultaneously: A classifier that aims to flag nefarious research related to manufacturing biological weapons, for example, may be triggered by the innocent questions of a biology student. At the same time, the classifier may fail to catch carefully designed questions that split up a dangerous avenue of inquiry into unassuming parts.

Interventions earlier in the development process can have wider-reaching downstream effects, which generally means more false positives and fewer false negatives. (In a sense, halting all AI development forever would indeed prevent all harms, but it would also prevent all benefits.) In practice, however (and in part because of this risk of being over-broad), such interventions can be subject to carveouts and selective enforcement, which can open up alternative routes to the same ends. Downstream interventions have the opportunity to be more fine-grained and selective, but making use of this capacity depends on having the time and expertise to make good choices here, and in practice we may see blunter interventions such as excluding whole categories of use.

For most threats, there are multiple distinct pathways to their realization. A capability advance can come from more training compute, better algorithms, more elaborate post-training, or better elicitation of a model’s existing latent capabilities. An intervention which targets one route but leaves the others open still allows threat-relevant activity to continue even if the intervention is perfectly enforced.

The quality of selection also changes over time. This is partly because actors adapt in response to interventions, as we discuss in §5. But even beyond that, the effectiveness of an intervention depends on empirical assumptions about the relationship between the control surface and the hazard. These assumptions may be changed by later progress.

For example, one conservative way to prevent the emergence of a dangerous advanced capability is to cap the number of FLOPs that can be used in training a model (see example A), but even this intervention depends on assumptions about training efficiency or elicitation methods which might change with future progress and substitution into alternative inputs. Generally this will push toward more false negatives, as the intervention becomes outdated.

For a given control surface, there will often be a three-way tradeoff between false positives, false negatives, and intrusiveness or oversight. Simply put, one can make an imperfect intervention more or less broad, or one can invest in making the intervention more accurate, by some mix of investing more energy in scrutinizing individual cases and requiring more access to information about those cases, some of which might otherwise be private.

Broader interventions, which capture a higher fraction of threat-relevant activity, carry with them greater economic costs, while narrower interventions may be easier to circumvent and thus have fewer safety benefits.

Alongside interventions which directly cap or limit some resources, it is possible to intervene indirectly: to change an incentive and let the selection be performed by the actors themselves. This includes buying out or taxing specific resources like compute, or applying stricter liability for outcomes to model providers or other actors with control over capability access, such as cloud hosts. These approaches can effectively draw on private information no regulator could extract: private plans, valuations, and alternatives. Ultimately this means indirect selection needs no surveillance to operate and thus is less intrusive. However, liability-based systems can suffer from other issues, such as pushing the frontier toward more risk-tolerant actors, and systematically underweighting risks where the costs are catastrophic.

The correct balance between tolerating false positives, false negatives and intrusiveness will depend on the goals of the intervention, the substitutability of inputs, and the resources (including political capital) available. The aim of an intervention is often to buy time to do work to mitigate a particular threat, which may risk being caught in the filter. For example, a filter which blocks cybercrime requests will sometimes prevent evaluation requests and defensive requests, undermining the nominal aim of mitigating the risk of cyberattacks. If malign work and safety work draw on the same capabilities, it is difficult to only target the malign activity. Some routes around this problem include whitelists for trusted organizations (e.g. see Anthropic’s Project Glasswing, where access to Mythos Preview was gated to selected partner organizations only), but these carry other concerns, such as unfair distribution of access.

Since essentially all control surfaces suffer from these tradeoffs, often the best route to a given outcome will involve several different control surfaces working in parallel, such as a cheap intervention with few false positives and many false negatives, coupled with a more demanding one that can catch some of the false negatives which slip through the previous one. Relatedly, in fast-moving situations it may be preferable to quickly enforce conservative interventions with many false positives, and use the breathing room to implement more careful and calibrated ones.

3.3. Coordinated interventions

In cases where an intervention requires coordination among multiple actors, some control surfaces and interventions make this easier than others.

Interventions and actions must be agreed upon by parties to the agreement, and they must have a shared understanding of which actors and activities are covered, so that they know what they are agreeing to, and how compliance will be verified as well as the consequences of noncompliance. In some cases agreements may cover actors who are not party to the agreement—for example, a chip export control rule agreed by the US and Taiwanese governments would likely have significant effects on Chinese developers who are not party to it.

Coordination is more sustainable when compliance can be checked without needing to rely on participants’ goodwill. This makes control surfaces which enable third-party verification especially desirable. Coarse, numeric thresholds like FLOP counts are easier to evaluate than holistic appraisals like bio uplift capacity, and so it may be easier to coordinate around them as interventions.

Unfortunately, while information about model capabilities and threat models accumulates steeply along the arc of model development, coordination can become more difficult. The information that can be gained about a system nearing deployment is often nonstandardized and difficult to make legible (i.e. the internal testing protocols and mitigation mechanisms at one lab may not look the same as at another, even if they point at the same targets). Upstream, however, information on goods like compute, which are produced by very few companies in relatively few places, can be easily made legible, verified and shared, making coordination easier. As a result, coordinating actors will tend to select upstream control surfaces: the crude and high-false-positive side of the arc provides a source of surfaces better suited to coordination.

4.1. Before pacing

The main challenge before intervening is recognizing where there’s a need to intervene and how to time it. Intervening too late means letting the threat play out with potentially irreversible consequences; intervening too early means sacrificing potential benefits and political capital, and in some cases being less able to carry out complementary activities that depend on access to advanced AI (accelerated safety research, empirical study of emerging threat models, etc.), or to the benefits from conducting the complementary activities in a setting with more talent, capital, and data from deployment settings.

The actual timing of a pacing intervention will ultimately depend on the strategic and political calculus of the actors who can bring it about, and may have more to do with political opportunity, public perception, salient incidents, or campaigning efforts. The timing ideally will be informed by evidence about technological progress, and the threat that pacing is meant to address.

Sources of information to draw on:

1. Model evaluations, including benchmarking, red teaming, and other tests in simulated or controlled environments, provide direct information about the capabilities and propensities of specific models in specific circumstances, and can form the basis for extrapolation. That said, the correspondence between evaluations and real-world risks can be quite fraught and hard to predict: the jaggedness of AI capabilities means that individual evaluation performance often corresponds poorly to behavior on tasks in the wild. Furthermore, evaluations can generally only set a lower bound on capability, because of the challenges of elicitation and the risk that models alter their behavior in response to recognizing they are being evaluated.

2. Forecasting and effective predictions provide decision-makers with aggregate expert assessments of when dangerous capabilities are likely to emerge. Forecasts can be about the threats themselves, or they can be about trends in the AI R&D ecosystem that inform threat assessment. Different forecasts shed light on different parts of potential dangers: for example, trends in training resources can be used to forecast the size and nature of the resulting required physical build-outs, while capability forecasts can predict models’ future behavior. A major open problem here is selecting epistemic peers: efforts to use top generalist forecasters for predicting even near-term AI have a mixed record.

3. Incident reports provide decision-makers with concrete evidence with ecological validity and lots of useful surprising detail, properties which both forecasting and evaluation often miss or fail to attempt. This is the best indicator that there is an immediate AI-induced problem, but by the time an incident has occurred, the space of possible pacing interventions has already shrunk. But even relatively harmless incidents can expose unknown unknowns and more granular details, which can in turn reveal underexplored threats. Also, compared to evaluations and forecasting, real-world incidents can be more credible and legible to a broad range of actors, but they are bottlenecked on effective infrastructure.

4. Safety cases provide structured arguments that a system will remain within some acceptable risk parameters as long as certain prior conditions are met — for example, that certain features of a deployment context guarantee that a given harmful model capability cannot be accessed. This structure therefore makes it possible to turn a discussion of potential risk into a discussion of concrete underlying properties which can be directly scrutinized. In regulated safety-critical industries, safety cases are mandatory prior to large-scale engineering efforts, making them effective pacing interventions: the work cannot go ahead until the evidence is gathered and compiled into a safety case that would satisfy a regulator, and the complementary activities required to support a safety case are directly targeted at the threats the safety case addresses. Despite interest, safety cases are not mandatory in frontier AI R&D, and few labs have generated them voluntarily.

Evaluations, forecasting, incident reports, and safety cases perform different functions: respectively, to test certain (bounded) claims, to supply warning signs, to reveal models’ emerging behavior, and to decompose threats into specific precursors. Alongside feeding into the question of whether to directly intervene, they can also feed into each other: incident reports can shape what gets evaluated, evaluations can form the basis for forecasts, and so on. Collectively they can provide a sense of how far away any given risk is, and this in turn can help actors to determine how urgently they need to prepare for costly measures.

Coordinated pacing comes with extra challenges: the different actors need to be able to converge on their understanding of the relevant information and what constitutes meaningful evidence, which in turn typically requires some bandwidth between them. In situations where they have competing incentives, some will also have individual incentives to withhold information. For example, frontier developers might not want to be overly restricted by governments, and so they might not want governments to have the information that would warrant such restriction. Those considering pacing interventions may need to think about what infrastructure needs developing in advance to get around such dynamics.

4.2. Triggering and enacting pacing

The challenge in deciding to trigger a pacing intervention is negotiating the tension between speed, legitimacy, and precision. It is easy to have two but more difficult to have three:

• Speed + Legitimacy: e.g. preregistered rules that automatically trigger when a condition is met. Unfortunately it is hard to know what exactly the rules should be, and so they may end up mistargeted in unexpected situations.

• Legitimacy + Precision: e.g. careful deliberation between experts, followed by democratic authorization of agreed-upon interventions. Unfortunately the review and implementation process here can take a lot of time.

• Precision + Speed: The actors with the most information and expertise get to choose by fiat. The risks here are that actions which are difficult to explain may seem illegitimate and thus at risk of being undone more easily, and these actors will often face incentives around the pace of progress that diverge from those of the broader public, making it difficult to trust they are truly acting in citizens’ best interests.

We can see this trilemma play out across the whole ecosystem of AI progress: governments have legitimacy and means to impose strong regulations but not necessarily the raw information needed to guide decisions, or the expertise to interpret such information, leaving them lacking in precision and/or speed; external evaluators can have enough precision (through expert staff) but not the mandate to regulate; AI developers have the most information about their own progress but little reason to proactively internalize any negative externalities they produce, and a lot of other interests in how their competitors are regulated (we return to this in §5).

One way to ease the tradeoffs is to work toward a portfolio of triggers. For example, authorities could set very crude evaluation thresholds now, beyond which they give expert groups like third-party evaluators the right to unilaterally trigger emergency interventions, on the condition that those choices then be subject to later governmental reconsideration, where there is more of a mandate but less ability to rapidly deploy expertise. This approach might seem risky if one views intervening as a one-off affair, but once it becomes a repeat affair, the evaluators would have a long-term interest in using their powers in a way they could justify.

The next decision is execution: the intervention must somehow affect live systems. Implementation-wise, the key question is whether enacting the intervention merely involves announcing a rule, or whether it involves some more active steps like buying up, confiscating, or destroying key resources. The former case is more straightforward, but brings with it the extra challenge of enforcing the new rule.

Again, coordinated pacing comes with extra challenges. It will typically take a lot more time for several actors to form a consensus on whether an intervention should be triggered, especially if there is a range of options to choose from. One potential solution is to give several parties the ability to unilaterally trigger time-bounded interventions which can buy time for more careful discussion. Another is to spend time in advance mapping out the likely space of mutually beneficial interventions.

4.3. During pacing

During pacing, it can be difficult to sustain the efficacy of an intervention in an ever-evolving environment. There are four notable challenges: whether the relevant parties are still complying; whether the initial threat is still effectively targeted; whether the threat is still a threat; and whether the suspension in activity is being used effectively.

Authorities can address the first problem with monitoring, verification and enforcement systems. The second component is whether the control surface continues to track the hazard (i.e. whether complying with the terms of the intervention actually diminishes the risk). §3 describes this relationship in terms of false positives and false negatives and gives more detailed considerations. Typically, a static rule will become less effective over time as the practical implications of the rule drift from their initial state (e.g. because redirection of effort toward other avenues of achieving the same goals renders the intervention weaker).

The third component is whether the original reason for intervening still holds. The concerning capability may mutate, diminish, or intensify. New evidence may also undercut the rationale for designating a result as a threat. The strategic and commercial reality may evolve too. Companies may exit agreements and safety measures may strengthen, for example. Importantly, this does not bear on the effectiveness of the rule itself, but on whether the threat targeted by the rule is still of concern.

The fourth component is whether the interval is being used to change the conditions which made pacing valuable, as discussed in §2. This could simply amount to some ambient societal adaptation, but typically it will involve deliberate complementary activities aimed at ensuring that the threat will be less severe after the intervention than before. Absent good enough complementary activity it is possible for an intervention to merely postpone a growing crisis.

There is a danger that pacing weakens the infrastructure it is designed to strengthen. Sweeping restrictions on AI, for example, would likely shrink the opportunities available to the actors involved in frontier systems, potentially limiting their experience and understanding of frontier models. Mandatory evaluation by third parties may increase the risk of leaks and industrial espionage. To mitigate this outcome, a limited and vetted group could be permitted to continue research: a tight circle of knowledge and power may limit the threat potential while maintaining expertise. A significant risk is that this group would gain outsized power and influence, or be unaccountable and lacking in error-correcting mechanisms (see §5.3 for more discussion of this dynamic). The nature of the complementary activity to the restriction (typically bolstering safety procedures in some form) can inform decisions related to the chosen group’s composition, focus, and oversight. When reviewing the intervention, the same considerations as in activation are in play: the tensions between speed, legitimacy and precision.

A potential solution could be to separate retargeting by scale and reversibility. Operators could have the authority to make temporary and bounded adjustments; external reviewers could implement wider and more durable policies. As in the activation procedure, tradeoffs cannot be wholly eliminated, only managed across the model lifecycle.

4.4. Ending pacing

A major difficulty with any kind of temporary intervention is ending it at an appropriate time. Restrictions can become entrenched and persist past the point they are no longer justified, or can be abandoned while other actors are relying on them. So interventions which depend on precise exit conditions are less attractive than plans which are more robust to poor exit management.

There are two clear reasons decision-makers might wish to exit a particular pacing intervention. Where an intervention worked and is no longer needed, this can be because its initial goal to build up defensive capabilities or resilience has now been achieved, and the risks involved are in fact mitigated. An intervention which is instead no longer working could be addressing an obsolete pathway to impact which has been routed around; it may have had a misspecified theory of impact in the first place; it may have insufficient teeth to ensure compliance from relevant actors and lack the support necessary to win enforcement; or it could simply have been superseded by a subsequent intervention.

The specifics of the case determine the best exit structure. After successful pacing, authorities should aim to reduce the costs the intervention has imposed while preserving the gains made. In the failure case, the goal might be to simply remove the costs while minimizing disruption. The former case may require a more careful, gradual process than the latter, since the potential damage from getting it wrong will be higher.

Exiting too soon may expose society to the very risks that pacing was meant to prevent, while having burned goodwill and support for further intervention. Exiting too late means imposing costs and burdens for longer, and strengthening the relative position of noncomplying actors.

The defaults also matter: if the intervention is set to expire after a certain period and requires renewal, it may be prematurely exited without regard to the motivations which installed it. If it is installed with no definite end point it may stagger on past the point where it is useful, with institutions and operators becoming entrenched, developing an interest in preserving their function.

Expectations around the removal of interventions also shape actors’ willingness to participate in the first place. Actors may agree temporary restraint will be useful, but refuse to join an agreement if they doubt it will end when conditions are met, or expect the arbiters of exit decisions to be biased against them.

These expectations also shape behavior during the intervention. If exit is conditional on demonstrated progress in a particular domain (e.g. on demonstrating models do not suffer particular failure modes), then the incentive to work specifically on that progress is greater than for an intervention which will end simultaneously for all actors.

Where abrupt exit from an intervention could pose a risk, authorities can try to mitigate the dangers of an exit by staging it. This could involve continuing some forms of restrictions, and gradually relaxing requirements over time. Evidence and research as to how to proceed can thus be safely collected, while still managing the risk. Staging thus permits actors to observe incremental effects of relaxation, and make decisions about whether this is desirable or ought to be halted.

Ending an intervention need not mean dismantling it entirely. It could make sense to retain some capacities that are slow to build (expertise, relationships, technical standards, communication channels, etc.) while lifting the biting aspects, such as invasive and exceptional controls.

Table: Key decisions per stage of an intervention’s lifecycle

5.1. Covered developers

As the center of AI progress, frontier model developers are the most directly impacted by pacing interventions. Intervention changes the constraints these developers face, but not their objectives: the competitive pressures described in §2.3 persist, and developers will re-optimize against the new rules. The practical effect of an intervention is therefore the rule, plus developers’ responses to it. But we can anticipate the form of such responses from earlier attempts to restrain arms races: for instance, when the 1922 Washington Treaty capped naval cruisers at 10,000 tons and 8-inch guns, the signatories built “treaty cruisers” that sat exactly at those limits.

In response to pacing, developers will likely consider:

Adaptation within the rule

Reallocating R&D resources. The appearance of new constraints induces innovation. Capping training compute would (further) raise the value of algorithmic efficiency, data quality, post-training, inference-time compute, and elicitation. Developers will move researchers and budget accordingly, with unclear effects on capability growth.

Many pacing interventions would reduce the need for and ROI of holding and building massive compute assets (see §5.4). With reduced resources (or expected resources), developers could also reduce the amount of safety research they conduct in a given period. If exit is explicitly conditioned on the developer’s safety progress, then safety research becomes the developer’s route back to scaling and will be prioritized accordingly. The design of exit conditions in §4.4 will therefore determine where reallocated resources actually go.

Re-evaluating commercial strategy. If the frontier stops moving, the basis of competition shifts. If the frontier moves less, competition moves from capability to price, latency, distribution, integration, and post-training. If roadmaps stop assuming that a new model generation will arrive every few months, the breadth and depth of deployment might increase, as there is more incentive to embed the current models everywhere. Diffusion could thus actually benefit from pacing frontier development.

Compliance overheads. Developers already have large compliance functions owing to existing regulations not specific to AI, but many pacing interventions impose large amounts of work on developers. They may have to document all training runs, run intense evaluations at each checkpoint, host auditors, and await legal review for deployments. This is both a burden and a source of a potential moat: the fixed cost of establishing such a function is a barrier to new entrants. The GDPR is a recent precedent: market concentration among web vendors rose 17% after enforcement, with the largest vendors gaining share.

Internal power shifts. Pacing can change the relative power of departments inside each developer. For instance, the prestige of research units could fall in favor of product-focused units. This can strengthen safety functions, but also risks turning them into compliance functions optimized for demonstrable adherence to the rule rather than for reduction of the underlying risk.

Re-evaluating partnerships. If compute providers become enforcement points (see §4.5), a developer might opt to integrate vertically, building its own datacenters or moving to compute providers in jurisdictions where it expects favorable enforcement.

Developer relationships with governments also change: developers may seek national champion status or exemptions for national security work (of the kind already visible in the trusted-access programs described in §2.2.1). This gives the state a stake in the developer’s continued progress, and creates incentives which could blunt the state’s willingness to pace it.

Working around the rule

Jurisdiction shopping. If the pacing mechanism is both legally binding and not global, then developers have discretion to move activity to less-restricted jurisdictions.

Rival, upstream inputs are hard to move: frontier compute is costly and slow to relocate, its supply chain runs through a handful of firms (see §3.1), energy and datacenter siting are slow, and export controls already exist as a counter-response from the state.

Non-rival, downstream activities are easier to move: research methods, model weights and other intellectual property can be moved easily and a model developed under one regime can be served into another. For Case B, the relevant version is a release of weights by an affiliate or partner outside the covered jurisdictions. The more an intervention targets rival goods, the less jurisdiction shopping it permits, which reinforces the pull toward upstream control surfaces noted in §3.3.

Exploiting loopholes. In §3.2 we cover the inevitable gaps between the control surface and the targeted activity. For instance, if the intervention involves restricting total training compute, then driving compute efficiency up becomes (even) more valuable, so that the developer can continue R&D activity while not defecting from the letter of the intervention. Developers will systematically search for such gaps, likely bringing more expertise than the regulator has.

Firms bunch just below regulatory thresholds wherever these exist and there is no reason to expect training runs to differ. Much of this is not bad faith but ordinary engineering under a new constraint, which is why rules based on intent are hard to enforce. The regulator typically learns of a loophole after it has been used, which is one reason §4.3 requires continual retargeting.

Defecting from pacing. For a variety of reasons, the developer might not pace their R&D, or stop pacing after initially complying. This could be done openly, or secretly.

Secret defection is worse than non-participation because it corrupts every other party’s picture of the state of play. The incentive to defect is not constant: for a training cap, it grows with the overhang (see §4.4), so verification needs to be strongest late in the intervention, when political attention has moved on. The choice between open and secret defection is set by the probability of being detected and the penalty for defecting (see §3.3, §4.3).

Working on the rule

Policy shaping. Developers hold the information the regulator needs and will supply it selectively. The developer is also likely to actively influence the pacing intervention through lobbying or by their direct representation in the pacing governance mechanism. This is to ease the regulatory burden on themselves or increase the burden on their competitors. The more regulation they are subject to, the more they will be driven to spend on lobbying.

Shaping can be prosocial: developers may plug loopholes in the monitoring mechanism which only become apparent after enforcement begins, or correct a control surface that is missing its target. But the regulator cannot easily tell this shaping from the other kind.

Seeking compensation. Those who are affected by interventions may make efforts to secure compensation for losses they incur. See §5.4.2.

5.2. Shifts in relative power

National governments and international competition

Consider national governments and international organizations. Many governments see AI as a strategic technology, and a pacing activity would potentially impact them in several ways.

For nations which are currently behind in the AI race, any given pacing intervention could appear as either an opportunity to catch up, or as impeding the path by which they already expected to. A frontier-only training cap might be the former, while broader restrictions on chip supply might be an example of the latter.

Either way, they will likely seek to exploit the pace to improve their relative position, by funding activities differentially benefited by the pace, or by trying to evade the restrictions.

They may also try to slow down adversaries, whether by attempting to include them in coordinated pacing interventions, or arguing that non-parties to the agreements should be somehow constrained (e.g. by forbidding purchase of compute by actors who are not party to an agreement).

The machinery of pacing will also be vulnerable to government actors (see §5.3), where whatever machinery is required to implement the interventions may face pressure to be co-opted to serve the state’s interests.

Non-covered AI R&D actors

Consider also non-covered AI R&D actors and the broader AI research ecosystem. While many AI R&D actors may not be carrying out activities targeted by the pacing intervention, they can also be impacted by pacing.

If pacing reduces the amount of investment and talent flowing to pushing the frontier of AI R&D (see §5.4), then these actors will become more attractive places for talent and capital to flow. Narrow AI uses which don’t fall under the interventions’ scope will be more attractive, and thus whether particular economic value is created by the application of a broad general-purpose model vs specialized systems designed for particular tasks might change.

Depending on who these actors are (which is a function of how broad or restrictive the pacing interventions are), this might lead to more actors reaching the current frontier, and more commoditizing of that level of capabilities. As this happens, however, it will increase the number of now-covered actors who might push the frontier by defecting or circumventing pacing restrictions and increase the burden on monitoring mechanisms.

There may also be a chilling effect from the existence proof provided by the pacing mechanism, whereby activities which are seen as plausible targets for future pacing become relatively less attractive, and leading to differential technological progress in areas seen as safer from future regulation. By this mechanism a pacing intervention could shape the direction of progress even outside its formal scope.

Adversarial actors

Many interventions (e.g. our worked example on biological threats) will also have to contend with misuse: third parties who use diffused AI capabilities to ill ends, for instance by developing biological weapons or committing cybercrimes. These actors can also be expected to adapt to any intervention, though primarily by trying to circumvent it from the outside.

5.3. Pacing governance structures

The main effect of pacing on the new governance mechanisms is to make them exist, or to invest new powers in existing bodies.

Many interventions will vest some enforcement power in a governing body or monitoring organizations (see §4). This will typically be some mix of access to private information and discretionary enforcement power, and may even involve creating specific infrastructure to support observation or verification. The power and infrastructure can then sometimes be used for purposes beyond the rule that justified it.

Liability- or insurance-based schemes will distribute these powers across auditors and existing legal systems such as courts—similar concerns will exist around their access to information and the privacy of those subject to their scrutiny.

5.3.1 Control, regulatory capture and misuse

In developing a potential intervention it is useful to start with the intended goal and then back out the details, but pragmatically it makes sense to plan around actors using power to pursue their own goals.

Once a newly empowered organization can inspect private information and impose restrictions on other actors, it will become a tempting target to influence. Companies will lobby for their own activities to be spared or minimally interfered with; other existing agencies will compete for jurisdiction over what they see as their own turf; and governments may push the body in particular directions depending on their shifting political priorities.

For example, powers of surveillance introduced to constrain dangerous activities might be used to interfere with lawful research to serve particular interests.

5.3.2 Other behavioral distortions

The existence of the machinery itself may cause shifts in behavior without any deliberate abuse. The knowledge that you are being surveilled and the potential for leaks and misinterpretations create significant compliance burdens and may discourage particular kinds of research which might be perceived as creating reputational risks.

The machinery can also change what targets participants optimize for. Measures which are easier to audit, or those which are publicly salient, may attract most of the focus, even if they only imperfectly track the underlying risk. Developers may focus on optimizing for passing these tests and drift from measures which would better serve to actually reduce the underlying risk.

5.3.3 Persistence, precedent and potentially useful spillovers

Additionally, it is not a given that these powers or the machinery that supports them will be dismantled at the end of an intervention, as we discuss in §4.4. This is not necessarily a bad thing: It might even be helpful if, for example, the systems created to enforce a domestic pretraining moratorium can then be reused in international diplomacy, or if departments created to assess model risks can feed their expertise into national strategy.

But there is a clear risk to contend with: crises can justify legitimately necessary emergency powers which then outstay their welcome. Once information has been collected, it is difficult to limit the scope of decisions it can feed into. The remit of a governing body may also broaden to new domains, simply due to the convenience of it already existing, even if it is imperfectly suited to a new task.

In general, how competently the governing machinery is run and how well received its actions are will affect its durability and popularity. The machinery should thus be thought of not just as a means of carrying out the intervention, but also as one of its effects, and predictable failure modes should be weighed against expected benefits in deciding whether to implement it.

5.4. Impact on AI investment and global markets

AI capital investments are already a major factor in the global economy. Investment in the inputs to AI progress (chips, datacenter construction, power infrastructure, and the AI developers themselves) is heavily contingent on the expected benefits and usefulness of the outputs, and so anything which affects the pace of capabilities progress will bear heavily on these investment decisions, and shift their attractiveness.

This effect can also be expected to be one of the most immediate of any intervention. Investment is a function of the expected future trajectory rather than realized changes, and these expectations could rationally shift long before any direct effects on capabilities occur.

5.4.1 Repricing and capital allocation

If investors predict a slower pace of progress than the counterfactual, it seems likely they will expect revenue growth for AI companies and suppliers to slow in aggregate (since the expected usefulness of their future outputs will be less). This would lead the valuations of developers and suppliers to fall, and their financing to become more expensive. This, depending on the intervention in question, might mean a slower datacenter buildout, and more capital flowing toward companies who take advantage of current capabilities rather than those pushing the frontier. In general this means more capital to flow toward those activities which remain less regulated or differentially benefit from the intervention.

To the extent frontier developers are currently valued partially based on the prospect of winner-takes-most dynamics (where AI labs grow to dominate the global economy), any pacing intervention which weakens their lead and relative position will weaken this, and weaken their expected market power, raising the relative power and value of supply chain companies instead.

An important aspect of the financial markets’ reaction is that it will be proportional to the expected durability of the interventions. In other words, an intervention which is not seen as lasting long is likely to lead to a more muted market reaction than one which is expected to be durable. This makes markets an anti-enforcement measure: their reaction and investors’ losses will be greater for interventions expected to be more durable, incentivizing actors to push for weaker mechanisms.

Market reactions are also responsive to new information—and they do not need to wait for the intervention to take effect. This can create incentives to back out of interventions as soon as they are proposed, even before they take effect, if market reactions are unfavorable (see the April 2025 US tariffs, most of which were paused within a week of announcement after equities fell more than 10%).

5.4.2 Leverage and stranded investment

Much of the current datacenter buildout is financed with some degree of leverage (i.e. money borrowed against the value of the project, with the investment case supported by the prospect of a long-run return).

Changes in expectations can make these projects uneconomic or expose investors and lenders to losses, as the projects can no longer deliver the expected return. In an extreme case, a datacenter build might be abandoned partway through if the remaining cost of building it exceeds the expected value of the project.

Anticipating these effects and carefully designing rules can help minimize the damage done in these cases. It is also possible that compensation for affected actors might be part of any formal pacing agreement. The degree to which this is feasible depends on the scope of the intervention and the appetite from the parties to the negotiations.

Actors whose assets are stranded or whose expected revenue is cut will seek compensation. This could take the form of direct payments, buyout of their compute stock (see §3.2), tax treatment, government contracts, guaranteed access to restricted markets, or exemptions. Compensation reduces the incentive to defect, and could be framed as public procurement if the freed compute goes to safety or defensive research. But it has three costs: (1) it transfers public funds to the actors being restrained; (2) it creates a constituency with an interest in the intervention continuing, which bears on the exit problems in §4.4; and (3) it creates moral hazard: actors who expect compensation may invest in anticipation of a buyout.

5.5. Impact on norms and culture

And lastly, how does the pacing intervention reshape the norms and culture surrounding interventions in R&D, both future interventions in frontier AI R&D and interventions in R&D more broadly? If an intervention yields intended and desirable outcomes, is it making it easier for future regulations to land well? In other words, are “good” interventions being normalized? How much of a risk is there that pacing interventions in AI R&D might have a global chilling effect on growth and progress?

5.5.1 Talent culture

Pacing interventions may have knock-on effects on who wants to work at a frontier lab, and why, and on what kinds of work are most prized and rewarded by those labs. Restrictions on scaling and a weakened market position may deter those motivated by advancing capabilities and those with primarily financial motivations, and credible restrictions on perceived risky activities could attract those who might previously have been opposed on those grounds. Exit conditions could also raise the relative importance of some kinds of work—if permission to resume scaling is dependent on safety progress, for example, then safety work could be expected to rise in value. The “total research transparency” direction advocated for in AI2040’s Plan A would likely be more appealing to the academically- or safety-minded, and less to those driven by financial gains.

On the other hand, if labs are seen as unethical actors, there could be stigma attached to working there, and this factor could lead to departures of many current staff, and deter prospective hires. If this effect is greater on those who are most concerned about lab practices, then internal scrutiny could be weakened. Which effects dominate will depend on the specifics of the interventions, but any change in employee composition would itself probably have an impact on the pace and trajectory of progress.

Talent impacts could extend beyond lab employees. A serious attempt at a pacing intervention, especially one involving the superpowers, will involve a large number of policy makers and diplomats, and may include risk professionals and advocates. A pacing intervention broadly deemed successful could increase the prestige and relative power of those roles and professions, and may encourage pursuit of similarly-shaped victories in domains beyond AI.

Finally, we note that human talent may not remain the crucial chokepoint it currently is; OpenAI (for instance) is targeting a fully automated AI researcher in the next 18 months.

5.5.2 Political norms and culture

The consequences on the competence of bodies governing AI are non-straightforward and boil down to whether or not any interventions adopted actually prevent risk and, if so, whether the prevention is visible to the citizens.

Unfortunately, pacing suffers from the preventer’s paradox (or “invisible-success” problem): when policy successfully averts an AI threat, the evidence of success is precisely that nothing happened—similar to counterterrorism efforts or the efforts to avert the Y2K problem, pacing policymakers may very well be doing their job precisely when nobody notices. This may mean that support for future interventions could hinge upon hypotheticals (while opponents of pacing have concrete capabilities to show).

Occasionally, successful interventions may be reactive (e.g. in response to an incident report) rather than proactive, in which case the citizens may in fact positively update on the government’s competence to protect public safety. Inconveniently, it would also likely take quite a significant or catastrophic incident (e.g. ransomware in public healthcare systems) for enough people to notice and attribute credit where due.

On the other hand, public attitudes toward the governments are likely to be much more unforgiving in the case of missing or ineffective interventions. If the government fails to prevent comparatively less harmful incidents (think discriminatory hiring) from AI, the public is more likely to start perceiving governments or designated authorities as slow-moving, ineffective, and less trustworthy. This may result in public apathy toward the institution of pacing (and relevant authorities) and subsequently governments having less incentive to invest in future pacing infrastructure or honor coordination treaties.

We should keep in mind that frontier AI pacing interventions would not be the first controversial tech policy area. Norms around tech policy have been shaped by the experience of industry and government policies around GMOs, civil nuclear, stem cells, and gain-of-function research, and have contributed to broader societal attitudes toward “progress” or “precaution”. The viability of pacing interventions will depend on these pre-existing attitudes and the degree of politicization, but also the impacts of a pacing intervention, successful or otherwise, could impact the narratives and relative power of these different camps in societies around the world.