Microsoft is piloting the use of the anti-nonconsensual deepfake porn platform Stop Non-Consensual Intimate Image Abuse (StopNCII) on its search engine Bing, as tech companies continue to crack down on non-consensual deepfakes.
Non-consensual deepfake porn, often made using AI, has become a huge problem globally in recent years, and is often used nefariously for extortion and revenge porn. Governments and tech companies have scrambled to update their laws and services as they attempt to curb their creation, use and accessibility.
Deepfake porn falsely depicting female celebrities naked and in sexual scenarios has made headlines recently, but other deepfakes to make the news have allegedly depicted fake images of people including schoolchildren.
Microsoft signing up to work with the StopNCII platform will mean that if someone has registered data linked to a deepfake image of themself on StopNCII, it will be searched for on Bing then potentially removed, if found.
The StopNCII platform relies on people who believe themselves to be victims of specific deepfake or revenge porn images or videos reporting them to the platform. People can do this by creating and submitting hashes of the images or video content to StopNCII’s website.
A hash (also known as a ‘digital fingerprint’) is created on the user’s device through an algorithm that assigns a unique hash value to an image. These are then added to StopNCII’s hash bank. To protect privacy and mitigate the risk of an image or video spreading further, the original content is never sent to StopNCII – just the hashes.
If an image or video corresponding to a hash in the StopNCII bank is uploaded to a platform or site signed up to the scheme, the file is moderated by StopNCII then potentially removed and blocked.
As well as Microsoft Bing, other major platforms and sites signed up to the StopNCII platform include TikTok, Bumble, Instagram, Reddit, Pornhub, Facebook and OnlyFans. Meta, which owns Facebook and Instagram, helped build the StopNCII platform. Earlier in 2024 Microsoft donated a new PhotoDNA capability to StopNCII.
Microsoft already has a reporting hub where you can directly submit reports about alleged non-consensual deepfake and revenge porn, to seek to have it blocked and removed. Announcing the StopNCII link-up, Courtney Gregoire, Microsoft’s chief digital safety officer, said that direct reporting like this was not seen to be enough to curb the problem.
De-ranking Search Results
“We have heard concerns from victims, experts, and other stakeholders that user reporting alone may not scale effectively for impact or adequately address the risk that imagery can be accessed via search,” Gergoire said.
Gergoire added that Microsoft was continuing to “take a range of measures to demote low quality content and to elevate authoritative sources”. This mirror’s recent search changes made by Google, in an attempt to prevent deepfake porn popping up in Google search results, even when people search specifically for it.
Now Google Search is designed to present “high quality” content, such as news stories about the dangers of deepfake porn, instead of showing actual deepfake porn in such search results. The move seems to have been largely successful so far, based on tests on Google Search. Google has not signed up to the StopNCII platform.
Microsoft’s endeavor to similarly demote deepfake porn in Bing search results seems to be working well in image searches. When on September 10, 2024 SEXTECHGUIDE attempted a Bing search for deepfake porn depicting a celebrity known to have been a victim of it, no deepfake or explicit content showed in the image search results.
However, on the main Bing search results page, the top result was a well-known deepfake porn site. Multiple deepfake porn image results also appeared high up on the main search page, albeit blurred behind the SafeSearch function.
While non-consensual deepfake porn crackdowns are ocurring in many countries, the issue seems particularly stark in South Korea. A “digital sex crime epidemic” has been described there.
Recently chiefs at messaging app Telegram apologized to authorities in South Korea, after police in the country accused Telegram of abetting the distribution of deepfake porn. Many chatrooms within Telegram, some reportedly frequented by teenagers, have allegedly hosted the distribution of deepfake porn. Telegram said the situation was “unfortunate”.
Police in South Korea have also been investigating alleged deepfake porn rings in universities.
In South Korea you can be jailed for up to five years for creating deepfake porn. South Korean President Yoon Suk Yeol recently told authorities in the country to “eradicate” digital sex crime.
