‘We truly are stepping into the unknown’: Carney deepfake sparks AI concern and action

CTV News investigates deepfake advertisements on social media using high-profile Canadians to push investment scams.

Artificial intelligence is moving at a rate that is blurring the line between what’s real and what’s not.

Over the last several months, CTV News has documented advertisements on YouTube that used deepfake technology to sell products.

Canadian Prime Minister Mark Carney was used in several ads pitching an investment platform. He could be seen and heard telling the viewer that “AI Is the future” and that “we’ve built a state-of-the-art program to help Canadians be a part of it,” while urging them to register with a link to sign up.

Except, he never said that. The deepfake video was generated from his Canada Day address on July 1, 2025.

CTV News followed one the links advertised in a deepfake Carney advertisement, and it brought us to what appeared to be a CBC News article about the investment platform guaranteeing high returns in the cryptocurrency market.

The article went on to say that the platform promises earnings of over $35,000 a month from a single investment, and that organizations such as Royal Bank of Canada helped create an automated investment platform that ensures high profits.

CBC News and the Royal Bank of Canada both confirmed to CTV News that this article was fake.

Also, after contacting YouTube about the advertisements, its parent company, Google, told CTV News it has suspended the advertiser account for violating its ad policies.

“We have zero tolerance for scams on our platform,” wrote a Google spokesperson.

We then took it a step further and reached out to the investment platform directly. When pressed about its use of deepfake Carney advertisements, the platform was adamant that it was the prime minister speaking about the product, and that Canadian government is backing it.

In an email, the Canadian Ministry of Innovation, Science and Economic Development wrote, “The Government of Canada does not endorse this advertisement,” before explaining that Ottawa is working to protect Canadians from scams, deepfakes, and malignant AI tools being used to impersonate people for nefarious purposes.

The government wrote that, in November, it launched the Canadian Artificial Intelligence Safety Institute with the intention to advance scientific understanding of risks associated with advanced AI systems, while also developing measures to reduce them.

“The institute has been collaborating with international counterparts on research into risks of synthetic content,” wrote the government.

Legal challenge

Canadian lawyer Jordan Donich has been keeping a close on the development of the technology and its increased use for criminality.

He says deepfakes are being used in variety of unsavoury ways, such as for pornography, child exploitation and to perpetuate frauds and scams.

Donich explained that the law surrounding the actual creation of a deepfake – or an image that depicts someone or something that isn’t true, or isn’t true in its entire character – is still developing.

“I don’t think we’re going to get to a point where a deepfake itself is a crime,” said Donich. “Where do you draw the line?”

He added that laws will need to catch up with the technology, but as of right now, there are protections in place for children.

When it comes to using deepfakes of politicians – or anyone else whose reputation or authority could be used to exploit people’s trust in them – Donich says that’s where the crime could be.

“We can’t have people going around producing deepfakes to fool people into making decisions where they suffer a loss,” said Donich. “The law would already capture that.”

He mentioned it could fetch charges such as fraud, theft, and impersonation.

An uncertain future

The Responsible AI Institute is working to help bring awareness and insight to the general public, policymakers, and regulators, urging them to use technologies that improve the social and economic well-being of society and help avoid the danger.

The founder Manoj Saxena told CTV News that the technology has advanced so rapidly that not only can deepfakes replicate a person’s voice and image, they can also convey emotion.

“There are projections that by the end of next year, there will be 100 times more social media generated (by) AI – fake ads, fake products, fake accounts, fake posting – than by humans," said Saxena.

Saxena warned that governments have adopted a mindset of an AI arms race and they’re rushing to build bigger, more accessible and more capable AI models, without understanding the risks of misuse.

He explained that the technology rolling out now is what was thought to be possible decades down the road.

“I used to be an AI pragmatist, an AI utopian. No more,” said Saxena. “I think this is coming at us at a speed and rate which 99.9 per cent of the people don’t know. We truly are stepping into the unknown.”