The Machine Intelligence Research Institute (MIRI), a prominent AI safety organization, has undergone a significant strategic pivot, moving away from its long-standing technical AI alignment research to concentrate on informing policymakers and the public about the severe risks posed by advanced artificial intelligence. This shift has drawn attention, with early MIRI funder Peter Thiel reportedly commenting on the organization's perceived decline.
MIRI's 2024 End-of-Year Update revealed a conclusion that technical research in AI alignment has progressed "far too slowly to prevent disaster," leading them to believe that an "extinction-level catastrophe is extremely likely" without international government intervention to suspend frontier AI development. The institute, which has focused for 24 years on building safe smarter-than-human AI, began scaling back its alignment research in late 2020 and 2021, fully pivoting to awareness-raising and governance research in 2023.
Despite the strategic change, MIRI reported a strong financial position with over two years of reserves, totaling $16 million by the end of 2024, and is not in urgent need of funds. However, the organization expressed uncertainty about securing future funding for its new communications and policy-focused strategy. This shift marks a departure from its historical role, which included organizing the Singularity Summit with funding from Thiel.
The move has sparked discussions within the AI safety community, with some interpreting the strategic pivot as a sign of MIRI's diminishing influence in technical alignment. MIRI's "The Problem" document further emphasizes their belief that current AI development paradigms are ill-suited to robustly instill human-compatible goals, making human extinction the "default outcome" if governments do not intervene. The organization now advocates for an "off switch" for frontier AI development, requiring tight control and monitoring of advanced AI work.
