In early January, headlines were straightforward, all conveying the same message: researchers suggest a 5% probability that artificial intelligence could pose a threat to humanity.
A recent report published on the draft server arXiv unveiled this alarming revelation. The paper detailed the outcomes of a comprehensive study involving 2,778 researchers who had presented their work at esteemed AI conferences and journals. This study, the most extensive of its kind at the time, unexpectedly delved into profound inquiries about the future of humanity. Katja Grace, the co-lead author of the report and the lead scientist at AI Effect, emphasized the significance of understanding AI researchers’ perspectives on these matters, as they play a pivotal role in shaping the discourse surrounding AI.
Nevertheless, some AI experts have raised concerns about the potential bias in the survey results towards a pessimistic outlook. Various organizations, including Open Philanthropy, which endorse effective altruism—a burgeoning intellectual movement popular in Silicon Valley known for its apocalyptic stance on AI’s future interactions with humanity—have financially supported AI Impacts. Criticisms have been voiced regarding the limitations of drawing conclusions on AI threats solely based on speculative survey outcomes influenced by funding ties and question formulation.
Effective altruism, or EA, is depicted by its advocates as a strategic initiative aimed at maximizing resources to enhance human well-being. The movement has shifted its focus to AI as a philosophical dilemma confronting humanity, akin to the challenges posed by nuclear weapons. However, detractors argue that fixating on hypothetical scenarios diverts attention from addressing the immediate risks associated with AI, such as issues of discrimination, privacy violations, and labor rights.
In the latest survey conducted by AI Impacts, researchers were asked to assess the likelihood of AI leading to humanity’s “extinction” or “similarly permanent and severe disempowerment.” Half of the respondents predicted a probability of 5% or higher.
Thomas G. Dietterich, a former president of the Association for the Advancement of Artificial Intelligence (AAAI), contends that framing survey questions in this manner inherently instills the idea of AI posing an existential threat. Dietterich, one of the 20,000 experts invited to participate, declined after reviewing the questionnaire, citing its biased, doom-laden perspective. He highlights that the survey questions, particularly those related to the development of superintelligent systems surpassing human capabilities, presuppose a specific viewpoint not universally shared in the AI community.
While acknowledging the survey’s merit, Dietterich stresses the importance of shifting the focus from alarmist narratives to a more nuanced analysis of AI risks and the formulation of risk mitigation strategies.
Some researchers, like Tim van Erven from the University of Amsterdam, participated in the survey but later expressed regret, criticizing the speculative nature of the questions regarding human extinction without clear mechanisms or timelines for such events. Van Erven warns against sensationalized beliefs that detract from addressing pressing real-world issues posed by AI.
Grace, the lead scientist at AI Impacts, underscores the necessity of understanding the prevailing viewpoints among AI researchers regarding existential threats. She acknowledges the diversity of opinions within the AI community and the importance of approaching these complex issues with caution.
In response to criticisms, Grace and her colleagues have endeavored to address previous shortcomings in AI Impacts’ studies by expanding the respondent pool and reaching out to a broader range of conferences.
However, critics like Margaret Mitchell, chief ethics scientist at Hugging Face, argue that the survey’s scope remains limited, failing to include key conferences focused on ethics and AI. Mitchell highlights the importance of incorporating diverse perspectives to obtain a comprehensive understanding of AI risks.
The debate continues on the efficacy of surveys that prompt participants to speculate on distant future scenarios without requiring evidence or rigorous risk analysis. Dietterich emphasizes the need for research funding to facilitate in-depth assessments of AI risks and benefits, essential for informed decision-making in this evolving field.
