Extremist factions have initiated the utilization of artificial intelligence, particularly generative AI, to produce a deluge of fresh propaganda. Concerns are mounting among experts that the increasing adoption of generative AI tools by these factions could potentially undo the efforts made by major tech companies to eradicate their content from the web.
Adam Hadley, the executive director of Tech Against Terrorism, expressed apprehension about the potential consequences of terrorists employing generative AI to manipulate images on a large scale. This manipulation could jeopardize the effectiveness of hash-sharing as a solution, a system established by tech giants to swiftly eliminate known violent extremist content from the internet. Hadley highlighted the alarming trend of approximately 5,000 instances of AI-generated content being identified weekly, including imagery disseminated by groups associated with Hezbollah and Hamas, aimed at shaping the narrative surrounding the Israel-Hamas conflict.
The rapid proliferation of AI-generated content poses a significant challenge to the existing automated technologies developed by the tech sector to combat extremist propaganda. Hadley warned that terrorists might exploit generative AI to circumvent the current preventive measures in place.
Tech Against Terrorism researchers have uncovered various instances where extremist groups have leveraged generative AI tools for malicious purposes. These include neo-Nazi channels sharing AI-generated imagery with racist and antisemitic themes, far-right influencers providing guidance on using AI to create extremist memes, and the Islamic State offering technical support on secure generative AI tool usage. Additionally, there have been reports of AI-generated posters by pro-al-Qaeda outlets and the Islamic State utilizing AI-based automatic speech recognition for propaganda transcription.
In response to the escalating threat posed by generative AI tools, Tech Against Terrorism released a report outlining the diverse ways in which these tools can aid extremist activities, such as autotranslation for multilingual propaganda dissemination and personalized messaging to enhance online recruitment efforts. However, Hadley emphasized the potential for utilizing AI proactively to counter extremist content by collaborating with Microsoft to develop a gen AI detection system.
The announcement of this partnership coincided with the Christchurch Call Leaders’ Summit in Paris, dedicated to combating terrorism and extremist content online. Microsoft’s vice chair and president, Brad Smith, underscored the importance of leveraging AI in partnership with Tech Against Terrorism to create a safer digital environment.
While major tech companies like Microsoft, Google, and Facebook are actively engaged in combating this issue through their AI research divisions, the collaborative initiative aims to support smaller platforms lacking the resources to address such challenges independently.
The threat posed by AI-generated content extends beyond extremist groups, as evidenced by the Internet Watch Foundation’s report on the proliferation of AI-generated child sexual abuse material (CSAM) on the dark web. The report highlighted the alarming volume of AI-generated images posted on CSAM forums, emphasizing the challenge of distinguishing these fabricated images from real ones.
