AI Risk Debates Intensify Calls for Pause and US-China Pact

The recent U.S.-China summit placed artificial intelligence squarely on the diplomatic agenda, with both sides acknowledging its rapid evolution without reaching concrete agreements. President Donald Trump and Chinese President Xi Jinping are slated for additional in-person meetings before year's end, creating a window for talks that could shape how advanced systems are developed and constrained. Discussions so far have centered on preventing uncontrolled escalation, though experts note that bridging differing regulatory approaches remains difficult.

Geoffrey Irving, who has held senior roles at OpenAI, DeepMind and the UK AI Security Institute, has argued that fundamental uncertainties about superintelligent AI make waiting for clarity untenable. He assigns roughly even odds that such systems could cause human extinction within the coming decade, driven by capabilities in hacking, biological design, social manipulation and long-term planning. Irving contends that these risks cannot be fully resolved through debate alone and that development pauses deserve consideration now, even as technical and governance questions persist. His assessment aligns with broader concerns that once systems surpass human-level performance, corrective interventions may arrive too late.

At the same time, near-term infrastructure pressures are surfacing. Reports from McKinsey and Boston Consulting Group indicate that declining costs for AI inference are spurring higher usage volumes across industries. As token prices fall, companies find it economically viable to integrate models into routine operations rather than limiting them to pilot projects. This shift is projected to increase electricity demand from data centers, potentially straining regional grids already managing rising loads from electrification trends. The firms emphasize that efficiency gains in hardware and software have not offset the overall growth in computational activity.

These threads intersect in policy debates over international standards. Proponents of bilateral agreements between Washington and Beijing point to historical precedents where rival powers negotiated arms limitations despite mutual suspicion. Yet translating that model to AI involves challenges distinct from traditional weapons systems, including the dual-use nature of the technology and the speed of private-sector innovation. Energy infrastructure adds another layer, as any agreed limits on training runs or deployment scales would need to account for varying national capacities to expand power generation.

Analysts stress that effective oversight requires institutions capable of monitoring compliance and sharing safety research across borders. Uncertainties around model behavior and alignment persist, complicating efforts to set verifiable thresholds. At the domestic level, utilities and regulators are beginning to model scenarios where AI-driven demand accelerates, prompting questions about permitting timelines for new generation and transmission projects. The interplay between these factors suggests that incremental steps, such as joint research protocols or shared evaluation benchmarks, may prove more feasible than sweeping treaties in the immediate term.

Progress on these fronts will depend on sustained political attention rather than isolated summits. Both the existential and operational dimensions of AI development underscore the value of mechanisms that can adapt as capabilities advance.