Instagram has blamed European privacy laws for its failure to suppress self-harm content after an investigation by the Daily Telegraph found that such images are still running rampant on its service almost two years after vowing to stamp them out.
Adam Mosseri, Instagram’s boss, pledged in February 2019 to "do everything we can" to stop such content from spreading in the wake of the suicide of teenager Molly Russell.
Instead, the Facebook-owned app has been locked in a privacy stand-off with the Irish Data Protection Commission (DPC) since March 2019 that has left key parts of its promised crackdown in limbo.
As a result, Instagram’s own recommendation algorithms continue to push users towards accounts and content which clearly break its rules against promoting and glamourising self-injury or eating disorders.
Pictures of open wounds, instructions for starvation, and artful photos of protruding bones intended as "inspiration" for anorexia were among the disturbing images recommended to the Telegraph after following only a handful of related accounts.
After being contacted by the Telegraph, Instagram removed more than 100 accounts and blocked five hashtags from appearing in search results. A spokeswoman said it would soon also begin blocking all searches for certain self-harm and eating disorder terms, and impose a warning notice similar to those use to suppress anti-vaccine content.
But Tara Hopkins, Instagram’s head of policy for Europe and the Middle East, claimed the company’s ability to deploy new technology to catch self-injury content in the UK and the European Union had been crippled by GDPR, which remains in force in Britain.
Instagram’s rules on suicide and self-harm
Ms Hopkins said: "In the EU and the UK, we use image-based technology to find graphic self-harm… outside the EU, we’re able to use a more sophisticated range of technology.
"There are questions about whether this more sophisticated technology is allowed under GDPR, because it’s considered to be potentially making a judgement on someone’s mental health.
"We have a different view on this, and believe it is absolutely within the public interest that we are able to run it so we can look for this kind of content and remove it or take action on it."
A spokeswoman added that Facebook is now discussing a compromise proposal with the DPC, which is the social media giant’s chief data regulator in Europe.
The DPC confirmed that it had "strongly cautioned" Facebook against its initial proposal, citing concerns over "adequate safeguards" for sensitive data and "a lack of engagement with public health authorities".
However, it appeared to blame Facebook for the delays, saying: "Facebook was informed that we are open to discussing the matter further with a view to exploring other viable solutions… the DPC has remained open."
The legal wrangling does not affect eating disorder images, which remain endemic despite Instagram telling the British Government last April that it would remove them.
Instagram vowed to clean up its service after being blamed for the death of Molly Russell, 17
Credit: Russell family/PA
Instagram attributed that failure to delays in building the necessary technology, saying it had prioritised self-injury first because it was more common and carried greater risks of imminent harm.
Andy Burrows, head of online child safety at the NSPCC, said: “In the face of bad headlines Adam Mosseri committed to preventing this harmful content being pushed at vulnerable young people, but this promise isn’t being kept.
“By not enforcing their own rules and having algorithms that target users with this damaging material Instagram continues to put children at risk of avoidable harm."
The Telegraph has been campaigning for a legal duty of care to be imposed on social media companies to protect under-18s who use their services.
Instagram’s plans blocked by Irish privacy watchdog
Facebook’s dispute with the DPC hinges on GDPR’s "special category" rules, which give extra protections to any data that could "reveal" sensitive characteristics such as a person’s health.
Outside the EU, Facebook uses artificial intelligence (AI) to scan for signs of suicidal intent in order to enforce its rules, monitoring both images and any associated text.
If a human moderator believes there is a risk of imminent harm, Facebook may also direct the user towards mental health support organisations or even call emergency services.
A spokeswoman insisted that Facebook’s AI does not make or store any assessment of users’ mental health. But the DPC told the Telegraph that the system "goes much further than simply the removal and moderation of content".
"The expansion which Facebook proposed to the DPC goes much further than simply the removal and moderation of content," a spokeswoman for the DPC told the Telegraph.
"However, Facebook has recently approached the DPC proposing a much more limited use of this tool for removal of content which contravenes [rules]. The DPC is currently reviewing this and expects to respond shortly.”
Mental health | Where to find help and support
Compromises being discussed include removing the mental health support messages or letting Facebook’s AI nix self-harm content automatically without human involvement.
Ravi Naik, legal director for the data rights agency AWO, expressed skepticism about Facebook’s explanation, saying the company was not being clear about how and why it might be profiling its users beyond simply removing their content.
He argued that if Facebook’s suicide interventions only operate "as far as necessary" to protect users’ "vital interests", then such practices would likely be permissible under GDPR.
"The concern for the DPC is said to be that the platforms are going further. It is not clear why they would need to, or what the justification for those wider practices is.
"If the issue is that the content moderation systems are tied to wider profiling techniques that are not necessary or proportionate to protect those vital interests, then Instagram and the DPC are right to be concerned about such techniques operating in opaque and unanticipated ways."
Cutting and eating disorder content out of control
To check Instagram’s systems, the Telegraph created several test profiles in the UK and the US with minimal biographical information, and searched for obvious mental health keywords.
For each keyword, it took less than a minute to find and follow a handful of profiles that had broken, or skirted close to breaking, Instagram’s rules against promoting and glamourising self-injury and eating disorders.
Immediately, the app’s algorithms began bombarding the Telegraph with "suggested accounts" that had flagrantly broken its rules. In many cases, their usernames, display names or biographies contained well-known terms associated with advocating eating disorders
After following enough of those accounts, new rule-breaking content was injected into the app’s flagship "Explore" screen, which Instagram uses to highlight new content from accounts its users do not yet follow, as well as to promote its own products. The Telegraph found not examples of this happening with cutting content.
A caption accompanying detailed instructions for a dangerous diet. Despite these words of caution, the poster later responded to commenters with further advice about the diet and liked comments that said they would try it out
Credit: via Instagram
The images included graphic pictures and videos of still-bleeding cuts; pictures of exposed ribs presented as "thinspiration"; instructions for dangerous diets with fewer than 300 calories a day; and requests for a "pro-ana [pro-anorexia] coach".
The situation seemed to have changed little since 2018, when a previous Telegraph investigation found Instagram recommending numerous similar accounts.
In fact, one former eating disorder sufferer told the Telegraph that the images were similar to those they had seen on Tumblr as far back as a decade ago, saying: "I don’t think that their community helps them… people usually have to remove themselves from that community to get better."
A spokeswoman that eating disorder advocacy is more ethically and technically challenging to suppress because it is more ambiguous. Experts have warned Instagram that overzealous censorship could deprive vulnerable young people of a crucial space for catharsis and support.
Yet Eileen Carey, a researcher with the Alliance to Counter Crime Online in Washington DC who has spent years tracking illegal drug sales on Instagram, said its failure to tackle self-harm was part of a wider pattern that goes back years.
"Instagram is a crime scene and a nightmare and there is no reason children should be on it," she said. "This is stuff that’s so obvious, [but] there’s just no adults in the room… there is no leadership at Instagram."
She accused Facebook of depriving Instagram of the employees and expertise needed to tackle its problems, describing the smaller app as "severely under-resourced" and citing reports that chief executive Mark Zuckerberg had sought to restrain Instagram after becoming jealous of its rapid growth.
"Facebook just cares about Facebook," she said.
Ms Hopkins also stressed the impact of coronavirus on Facebook’s moderators, most of whom were sent home in March for their safety.Removals of suicide and self-harm content dropped from 1.3m in the first three months of 2020 to just 275,000 in the following span.
Свежие комментарии