Politicians like Imran Khan of Pakistan have faced situations where artificial intelligence has mimicked their voices, sometimes even orchestrated by their own political factions.
A video recording captured the former Pakistani Prime Minister Imran Khan addressing his supporters during a virtual rally.
On December 17, 2023, a virtual rally drew over a million supporters of Imran Khan, the former prime minister of Pakistan. The event, organized by Khan’s party, the Pakistan Tehreek-e-Insaaf (PTI), unfolded amidst intense pressure from the country’s powerful military establishment. This pressure stemmed from a prior protest where Khan’s supporters encircled military installations in protest against his arrest in May. As the nearly five-hour virtual gathering neared its end—mandated by restrictions on public assemblies—the audience was presented with a keynote speech purportedly delivered by Khan himself, despite his confinement in Adiala Jail. “You may be wondering about my well-being in confinement,” he expressed. “I want to clarify that if it serves the cause of Pakistan’s true liberation, being imprisoned is a form of devotion for me.”
In actuality, the voice did not belong to Imran Khan. The PTI’s digital media team had programmed an artificial intelligence system to replicate Khan’s speech patterns using samples of his previous speeches. This AI replica then delivered a speech based on notes smuggled out of the prison by Khan’s legal representatives.
Jibran Ilyas, a Khan supporter based in Chicago who spearheaded this endeavor, elucidated, “Our objective was to create the impression that Imran Khan was physically present, with a clear disclaimer that it was an AI-generated emulation of his voice derived from notes obtained in jail.” He stressed, “The primary aim was to effectively convey his message, as he is seen as a guiding figure not only within PTI but for the entire nation of Pakistan.”
In the upcoming year 2024, expected to witness over 2 billion individuals globally engaging in elections, policymakers are deliberating on the potential influence of artificial intelligence on shaping global democracy. Kat Duffy from the Council on Foreign Relations characterized the current scenario as a “post-market, pre-norms space,” where emerging technologies have entered the arena without established regulations to govern their impact.
In South Asia, machine learning and AI technologies are increasingly impacting democratic processes. For instance, Indian Prime Minister Narendra Modi’s AI-generated voice has been utilized to sing songs in various regional languages he doesn’t speak, hinting at the possibility of leveraging similar technology to make his speeches accessible in regions where Hindi is not prevalent.
In Pakistan, observers and digital media experts have commended PTI’s strategy as a significant breakthrough. Usama Khilji, a digital rights activist in Islamabad, commented, “I found it to be a strategic way to navigate the challenges faced by the political party.” He added, “Now, an individual in confinement is communicating with the public through AI, showcasing the innovative application of technology.”
Current Issue
However, the recent election in Bangladesh exemplified how artificial intelligence can propagate misinformation, especially among voters unable to differentiate between authentic content and deepfake replicas. An AI-generated video on Facebook depicted exiled opposition leader Tarique Rahman suggesting that his party, the BNP, should appease the United States by remaining silent on the Israeli offensive in Gaza—a stance that could be detrimental in a predominantly Muslim country.
A report by the Financial Times exposed a disinformation campaign in Bangladesh, where pro-government media outlets employed deepfakes and other AI-generated content to manipulate public opinion before the January 7 general election. Sayeed Al-Zaman, a Bangladeshi academic specializing in digital information, cautioned that technologies like deepfakes and AI bots could become potent tools for political propaganda on a global scale.
Al-Zaman highlighted that over 60% of individuals exposed to online misinformation are susceptible to believing it, underscoring the challenges posed by AI in verification due to the absence of effective tools and the sophistication of AI-generated misinformation.
Even in the United States, AI-generated content has been utilized to influence voters. NBC News reported the use of a robocall featuring an AI voice clone of President Biden to discourage New Hampshire residents from participating in the presidential primary. Instances of deepfake images circulating on social media, depicting Donald Trump in misleading scenarios, have also been observed.
Meta announced a policy mandating political campaigns to disclose their use of AI in advertisements on its platforms, with implementation slated for this month. However, amidst the escalating need for content moderation, the tech industry has witnessed widespread layoffs. Major tech companies, including Twitter, Meta, and Amazon, downsized their trust and safety teams in 2023, resulting in an estimated 250,000 job cuts.
The ramifications of AI manipulation are profound, especially in countries like Pakistan with a significant illiterate population susceptible to manipulation. Advanced AI technologies in affluent nations pose a significant risk of election interference. The year 2024, dubbed as the “year of the election,” may instead be remembered as the “year of misinformation.”
Hasan Ali
Hasan Ali is a journalist specializing in US foreign policy and South Asian politics.
