Meta’s “supreme court” has ordered the tech company to take down deepfake videos of a UK politician and a young Muslim woman from Facebook and do more to tackle AI-generated fake imagery.
A fake video showing a Labour party councillor in Scotland making inflammatory comments about refugees should not have been left up by Facebook, the board said. It also ruled that an AI-generated video of a Muslim campaign volunteer should have been removed after it falsely depicted her offering health advice while carrying out absurd exercises or eating junk food.
The board said Meta’s safeguards were “consistently and fundamentally inadequate” to address the rapid rise of AI deepfakes.
“From politicians to private citizens, AI-generated deepfakes are increasingly being used to harass and silence women from engaging in public discourse,” said Pamela San Martin, an oversight board co-chair.
“These cases demonstrate a broader, troubling pattern in which women who engage publicly on issues are disproportionately subjected to harassment and misinformation. Meta and other social media platforms need more robust policies to address the proliferation of deepfakes.”
In the Scotland clip, the councillor is falsely represented as saying: “Refugees are welcome here, even if they rape our women, because white people do that too.”
The oversight board, a quasi-independent body whose decisions are binding, said the video of the councillor appeared to be AI-generated, as indicated by the audio not being fully synchronised to the unnamed councillor’s facial movements.
Even when raised directly with the company by the board, Meta decided the video did not violate its content policies and did not merit an AI label.
In a blogpost published on Thursday, the board said the post should have been removed because it violated Meta’s rules on hateful conduct by alleging criminal and predatory sexual behaviour by refugees as an entire group and not as individuals. The video should also have received a “high risk AI” label, said the board, as it called for tougher measures on deepfakes.
“The majority [of the board] finds that Meta needs more robust policies on deepfakes, including expanding the situations when ‘high risk’ labels can be applied, more measures to reduce the spread of deceptive AI content, increasing the penalties for accounts that repeatedly share it, and more transparency on data around when AI labels are applied,” it said.
The board recommended nine policy changes at Meta, including ensuring algorithms demote content labelled “high risk”, making such posts less likely to appear in users’ feeds. It also recommended that Meta makes it more difficult for users to view AI-generated content by, for instance, introducing a warning screen that requires a click-through before viewing the content.
In a separate ruling, the board also ordered Meta to take down from Facebook an AI-generated video mocking a young Muslim woman in Europe taking part in a campaign to improve menstrual health education and reduce stigma around health conditions for women and girls from ethnic minority backgrounds. AI-manipulated videos and images mocking the woman had tens of millions of views online, including on Meta platforms.
The board singled out one of the videos – showing the woman exercising absurdly and eating junk food – as a breach of the company’s bullying and harassment policy. It recommended the post be removed and made three policy recommendations, including changing its definition of “unwanted manipulated imagery” to cover deepfakes of a private individual saying or doing things they did not say or do.
The board was set up by Meta in 2020 to serve as a referee for content on its platforms. Meta has been approached for comment.
