Glut of social media posts, political divisiveness a challenge for content moderators
Leigh Adams has seen a steady rise of material for review since she began moderating user comments on websites roughly 14 years ago, but she says the volume only exploded in the last few years as the content's nature became so divisive there's only one word for it: "Bonkers."
Misinformation, trolling and worse has always existed online, but Adams says she saw a shift after the U.S. elected Donald Trump president in 2016 that reached a new height when George Floyd, a Black man in Minneapolis, was killed in police custody in May 2020, fuelling racial tensions just as the world was locked down due to the COVID-19 pandemic.
"It was really the perfect storm ...The internet was already struggling at that time with, 'How do we reconcile anonymity and accountability? How do we make sure to amplify the voices of those who might not be heard?"' said Adams, director of moderation services at Viafoura, a Toronto business reviewing user content for publishers.
"We had not solved for that and we still haven't solved for that, but then you have these (events) on top of it, it just made it a bad situation worse."
Adams noticed Trump being out of office and the return of pre-pandemic activities slightly quelled the "inflamed rhetoric" seen by Viafoura's more than 800 clients, which include media brands CBC, Postmedia and Sportsnet.
But she expects future "swelling" and other content moderation companies say they've detected no significant signs of the onslaught receding. It's likely that keeping up with the volumewill mean tackling an evolving suite of challenges.
Moderators foresee health misinformation continuing to spread rampantly, dubious posters becoming even more sophisticated in their attempts to disrupt platforms and a slew of new regulations targeting online harms in Canada and abroad.
"I don't see the demand declining any time soon, despite all of the talk of recession," said Siobhan Hanna, Telus International's managing director and global vice-president of artificial intelligence.
"For better or worse, this content moderation need will continue to just grow, but the need is going to be for more intelligent, efficient, thoughtful, representative, risk mitigative solutions to handle the increased demand."
Hanna says video is becoming one of the most challenging areas because moderators are no longer only reviewing clips depicting violence, indecency or other harms that may be difficult to watch.
Now there are also so-called deep fakes -- videos where someone's face or body has been digitally spliced into the frame so they appear to be doing or saying things they never did.
The technology has cropped up prominently on TikTok, when visual effects artist Chris Ume spread clips purporting to be of actor Tom Cruise playing card tricks, eating a gum-filled lollipop and performing Dave Matthews Band's song "Crash Into Me."
"I don't think anybody's going to be harmed by ... the videos he's creating, but it's also getting us all used to these deep fakes and maybe drawing our attention away from the more sinister applications, where it could affect the course of an election, or it could affect health care outcomes or decisions made around crimes," Hanna said.
In Ireland for example, videos supposedly depicting political candidates Diane Forsythe and Cara Hunter committing sexual acts were circulated while they ran for office earlier this year.
"I never cease to be surprised," said Adams. "You see the worst thing and then something else comes along, you think, 'what could possibly happen next?"'
Her team recently found a photo that appeared to be a sunset at first glance, but 17 layers back, showed a nude woman.
"If we had not had five people looking at that, it would have been live and up there," she said.
"It's getting more sophisticated and so you have to find new artificial intelligence (AI) tools that are just going to keep digging deeper."
Most companies rely on a blend of human moderators and AI-based systems to review content, but many like Google have conceded machine-based systems "are not always as accurate or granular in their analysis of content as human reviewers."
Adams sees the follies of AI when people invent and popularize new terms -- "seggs" instead of sex, "unalive"instead of dead and "not see" instead of "Nazi" -- to avoid being flagged by moderators, security filters and parental controls.
"In the amount of time it's going to take machines to learn that, that news cycle is over and we're onto something else because they found a new way to say it," Adams said.
But humans also aren't perfect and often can't keep up with the volumes of content alone.
Two Hat, a Kelowna, B.C. moderation company used by gaming brands Nintendo Switch and Rovio and owned by Microsoft, went from processing 30 billion comments and conversations a month before the health crisis to 90 billion by April 2020. Microsoft Canada did not provide more recent numbers, with spokesperson Lisa Gibson saying the company is not able to discuss trends at this time.
Facebook, Instagram, Twitter, YouTube and Google warned users in 2020 they were taking longer to remove harmful posts as the pandemic began and staff retreated home, where viewing sensitive content was tougher and in some cases, forbidden for security reasons.
When asked whether backlogs have been cleared, Twitter declined to comment and Facebook and Instagram did not respond. Google temporarily relied on more technology to remove content violating its guidelines as the pandemic began, which led to an increase in total video removals, spokesperson Zaitoon Murji said. The company expects to see a decline in video removals as it scales back that technology as more moderators return to the office, she added.
As the backlogs formed, countries toughened their stance on harmful content.
The EU recently reached a landmark deal requiring the prompt removal of harmful materials online, while Canada is promising to soon table a bill combating online hate, after a previous iteration was shelved amid a federal election.
Adams says the convergence ofCOVID-19, Trump's rise and the killing of Floyd made publishers more willing to take a stand against problematic content such as hate speech and health misinformation. Legislation, which can vary across countries and often be left up to interpretation, could result in companies having even less tolerance and taking anything that runs the risk of being seen as problematic down, she said.
The stakes are high because letting too much problematic content on a platform can make it unsafe, but removing too much can also interfere with free speech, said Anatoliy Gruzd, a Toronto Metropolitan University professor of information technology management.
"From the user side, that may feel like there's not enough effort to make platforms a welcoming and safe place for everyone, and in part that's because the platforms become so huge, with millions and billions of users at once," he said.
Gruzd doesn't see striking a balance between safety and freedom getting any easier as the policy patchwork evolves, but believes society will move toward considering boundaries and what is acceptable or not to be exposed to.
He said, "Some people will vote with their usage, whether they stop using Facebook or Twitter for certain things, they might decide to go to other platforms with or without too much moderation or they may decide to stop using social media completely."
This report by The Canadian Press was first published May 22, 2022.
MORE SCI-TECH NEWS
Dan Riskin on how climate change pushed a pest northward
Dan Riskin on the ecological impact of invasive earthworms
CTVNews.ca Top Stories
Six police officers are in hospital with gunshot wounds and two suspects have been killed following a shooting at a bank in Saanich, B.C., on Tuesday.
Witnesses recount what they saw after police officers engaged in a shooting with armed suspects at a bank in Saanich, B.C., on Tuesday morning. Two suspects are dead and six officers are in hospital with gunshot wounds.
After five years in the role, John Horgan announced on Tuesday afternoon he plans to step down as premier of British Columbia and has asked his governing party, the NDP, to hold a leadership convention later this year.
Ukraine's President Volodymyr Zelenskyy accused Russia on Tuesday of becoming 'a terrorist' state carrying out 'daily terrorist acts' and urged Russia's expulsion from the United Nations.
Donald Trump rebuffed his own security's warnings about armed protesters in the Jan. 6 rally crowd and made desperate attempts to join his supporters as they marched to the Capitol, according to dramatic new testimony Tuesday before the House committee investigating the 2021 insurrection.
A scathing letter from an RCMP communications manager released Tuesday says RCMP Commissioner Brenda Lucki referred to direct pressure from the federal public safety minister to release firearm details in the days after the Nova Scotia mass shooting.
Airbnb has codified a global policy that prohibits guests from hosting parties or events on all listed properties.
Ghislaine Maxwell, the jet-setting socialite who once consorted with royals, presidents and billionaires, was sentenced Tuesday to 20 years in prison for helping the financier Jeffrey Epstein sexually abuse underage girls.
The federal Liberal government has agreed to provide sensitive cabinet documents to the inquiry examining its use of the Emergencies Act during the "Freedom Convoy" protest.
A Nexus card is supposed to help put low-risk Canadians on the fast track when crossing the U.S. border, but at least 330,000 Canadians aren’t sure when their applications will be processed.
A vigil was held on Vancouver's Downtown Eastside Tuesday for Noelle O'Soup, an Indigenous girl whose body was found there a year after she went missing.
A longtime Toronto legislator has been named interim leader of Ontario's New Democrats as the Official Opposition party gears up for a leadership contest.
The chief elected official in the Missouri county where an Amtrak train slammed into a dump truck said Tuesday that residents and county leaders have been pushing for a safety upgrade at the railroad crossing for nearly three years. Meanwhile, the toll from the accident rose to four deaths and 150 injuries.
The first 10 years of Syria's conflict, which started in 2011, killed more than 300,000 civilians, the United Nations said Tuesday -- the highest official estimate to date of conflict-related civilian deaths in the country.
France's president denounced Russia's fiery airstrike on a crowded shopping mall in Ukraine as a 'new war crime' Tuesday and vowed the West's support for Kyiv would not waver, saying Moscow 'cannot and should not win' the war with its neighbour.
President Joe Biden's top health official said Tuesday that "every option is on the table" when it comes to helping women access abortion in the wake of the Supreme Court decision overturning Roe v. Wade.
A year and a half after the deadly U.S. Capitol insurrection, the most memorable recounting of former president Donald Trump's behaviour that day came from a young woman who had graduated from college just a few years earlier.
At a time of enhanced global uncertainty and growing Chinese influence, the Canadian government faces mounting pressure to appoint a diplomatic representative in Beijing after the post has sat vacant for six months.
Following the U.S. Supreme Court's decision to overturn Roe v. Wade, a number of states south of the border have stopped performing abortions. If you're a Canadian who has had an abortion, CTVNews.ca wants to hear from you.
With the overturning of Roe v. Wade opening the door to abortion bans in the U.S., Canadian Tiktokers are welcoming Americans who are considering travelling north of the border to get an abortion.
The European Medicines Agency says it will begin reviewing data to decide if a smallpox vaccine made by the pharmaceutical company Bavarian Nordic might also be authorized for monkeypox, amid a growing outbreak of the disease across the continent.
Fossils of early human ancestors from a South African cave are 3.4 million to 3.6 million years old -- making them a million years older than previously suspected and shaking up the way researchers understand human origins and evolution.
German officials said Monday that numerous priceless artifacts taken from African nations during colonial times will be permanently returned.
A U.S. Navy destroyer that engaged a superior Japanese fleet in the largest sea battle of the Second World War in the Philippines has become the deepest wreck to be discovered, according to explorers.
Mary Mara, an actress known for roles on 'ER' and 'Ray Donovan,' has died, her manager, Craig Dorfman, said in a statement to CNN. She was 61.
A representative for Johnny Depp has denied a recent report the actor would be returning to the 'Pirates of the Caribbean' franchise.
Montreal-raised comedian Nick Nemeroff died on Monday, leaving comedy fans across Canada grieving. He was 32.
Shares skidded in Asia on Wednesday after another broad decline on Wall Street as markets remain gripped by uncertainty over inflation, rising interest rates and the potential for a recession.
The head of the European Central Bank said Tuesday that it will move gradually to combat soaring consumer prices with interest rate hikes in July and September but will keep its options open to "stamp out" inflation if it surges faster than expected.
The May Garden Chinese Restaurant in Bedford, N.S., has introduced 'Bella' to its team -- a robot that helps deliver customers' orders to their tables.
Railroad company Amtrak has announced that its Maple Leaf Train, in conjunction with VIA Rail, is back in business.
Calgary is now officially home to the world's tallest mural. German graffiti artist Mirko Reisser, known as DAIM, used a swing-stage to paint and spray-paint the tremendously tall piece of art.
Hockey Canada’s handling of an alleged sexual assault is again under fire as one of its highest profile sponsors, Scotiabank, announces they’re pausing their sponsorship with the Canadian hockey organization.
Marlin Briscoe, who became the first Black starting quarterback in the American Football League more than 50 years ago, died Monday. He was 76.
Formula One condemned racist language after a slur reportedly directed at Lewis Hamilton by retired champion Nelson Piquet.
A coroner's report released Tuesday is calling on Quebec to teach new drivers to open their car doors with their right hand in order to prevent dooring incidents with cyclists.