The National Institute of Standards and Technology (NIST) is currently grappling with an internal crisis as reports indicate that both staff members and scientists are considering resigning due to the potential appointment of Paul Christiano to a critical role at the newly established US AI Safety Institute (AISI). This information comes from confidential sources familiar with the situation.
Paul Christiano, associated with the effective altruism (EA) movement and its derivative, longtermism, has reportedly been expedited through the hiring process without prior knowledge until today, as per one insider. The decision to appoint Christiano, allegedly endorsed by Secretary of Commerce Gina Raimondo (NIST falls under the US Department of Commerce), has triggered discontent among NIST personnel who are concerned that Christiano’s affiliations may jeopardize the institute’s impartiality and credibility.
The EA movement, described by the Center for Effective Altruism as an “intellectual endeavor utilizing evidence and logic to maximize benefits for others,” has been criticized for evolving into a group with cult-like characteristics, comprising influential and affluent followers, notably exemplified by FTX founder Sam Bankman-Fried. Their primary focus on averting a potential AI catastrophe, termed “x-risk,” has drawn criticism for overshadowing the essential attention required for existing, quantifiable AI threats such as bias, misinformation, high-risk applications, and conventional cybersecurity concerns.
Establishment of the US AI Safety Institute in November 2023
The US AI Safety Institute was founded in November 2023 to assist the Department of Commerce in fulfilling its obligations under the AI Executive Order. Recently, US Senate Majority Leader Chuck Schumer (D-NY) announced an allocation of up to $10 million to enable NIST to establish the US AI Safety Institute.
In a previous report by VentureBeat, concerns were raised regarding NIST’s lack of transparency concerning the AISI. Lawmakers from the House Science Committee had criticized NIST for its opacity and failure to disclose a competitive process for forthcoming research grants associated with the new US AI Safety Institute. This lack of transparency was particularly focused on a potential grant to the RAND Corporation, a significant think tank associated with tech magnates, the AI industry, and effective altruism.
Emphasis on Preserving NIST’s Core Mission
Divyansh Kaushik, the associate director for emerging technologies and national security at the Federation of American Scientists, emphasized the necessity of safeguarding NIST’s core mission regardless of the individual appointed to the AISI. He stressed the importance of upholding NIST’s tradition of meticulous measurement research firmly rooted in scientific principles. Kaushik expressed understanding for the concerns of NIST scientists venturing into uncharted territories in evaluating hypothetical risks from general-purpose models. He highlighted the responsibility of Commerce leadership to ensure that appointments align with scientific integrity and NIST’s established methodologies.
Kaushik cautioned against the potential loss of exceptional NIST scientists due to controversial appointments, urging Secretary Raimondo to reconsider choices that may lead to such repercussions. He underscored the vigilance of congressional committees monitoring the AISI’s activities and hoped for the recruitment of proficient scientists who can leverage NIST’s strengths to advance the institute’s mission effectively.
Efforts to obtain comments from Secretary of Commerce Gina Raimondo, AISI director Elizabeth Kelly, and Paul Christiano are ongoing, and updates will be provided accordingly.
VentureBeat’s mission is to serve as a digital hub for technical decision-makers seeking insights on transformative enterprise technology and engaging in transactions. Explore our Briefings for more information.
