According to 404 Media, Civitai, an online platform for sharing AI designs, recently introduced a new feature known as “bounties” to motivate its users to create realistic deepfakes of real individuals. The top performer in developing these AI designs is rewarded with a digital currency called “Buzz,” which can be used by users for real-money transactions.
In many of the bounties posted on the platform, users are tasked with replicating the appearances of celebrities and social media influencers, a majority of whom are female. These creations are often labeled as “nonconsensual physical photos” by observers. While this practice has been prevalent on the internet for years, advancements in artificial intelligence have made it more achievable. The situation is further complicated by reports of requests from anonymous individuals discovered by 404 Media.
Michele Alves, an Instagram influencer featured in a bounty on Civitai, expressed her concerns to 404 Media, stating, “I am truly apprehensive about this. The online realm appears lawless, leaving me unsure of the actions I can take.”
Currently ranked as the sixth most popular generative AI platform by market research firm Andreessen Horowitz, Civitai’s handling of these incentivized requests is under scrutiny. Staff members at 404 Media Moments easily traced images requested for bounties involving private individuals with significant social media followings. Despite the requester claiming to be associated with the individual, discrepancies were found in their social media profiles, raising suspicions.
One particular bounty was flagged by a Civitai user due to potential legal implications in the future. Virginia recently updated its legislation on revenge porn, imposing penalties of up to a year in prison for deepfake developers. Despite this, the specific request was approved, and several images were uploaded to the platform, despite lacking any sexual content.
It is worth noting that while most bounty requests do not explicitly seek sexual content, some individuals push boundaries by using suggestive language and making inappropriate remarks. Civitai maintains that these rewards are not intended for generating AI-generated sexual content of real individuals.
The platform allows for the creation of physical images of public figures and non-sexual depictions of ordinary individuals, with the potential for combining the two. Using their text-to-image tool, 404 Media swiftly produced non-consensual physical images of a real individual within seconds.
