Quebec AI institute races to make chatbots safer for youth mental health

At a Montreal hackathon, teams train chatbots to handle sensitive conversations as Kids Help Phone sees rising demand.

As young people increasingly turn to chatbots to talk about what they are going through, the Mila – Quebec Artificial Intelligence Institute is organizing a national event to document, test and design technologies aimed at reducing the risk of harm to mental health when artificial intelligence (AI) is involved.

The event is a programming marathon, or “hackathon,” a process frequently used in the digital sector. It consists of bringing experts together to develop programs around a specific theme. Mila’s event focuses on the safe development of conversational AI bots in the context of youth mental health.

Experts taking part will focus in particular on the safeguards used to detect psychological distress. According to preliminary data from a survey by Mental Health Research Canada that has not yet been published, 76 per cent of people believe AI is effective in supporting their mental health.

“There’s a strange dichotomy,” said Simona Gandrabur in an interview. “On the one hand, there are 900 million people who use just ChatGPT on a weekly basis.”

On the other hand, only 14 per cent of people trust AI to provide safe support for their mental health, according to the preliminary data from Mental Health Research Canada. Gandrabur argues that skepticism is positive.

“I would even say it’s healthy. We need to be cautious and aware of the dangers of using this technology, but we shouldn’t deprive ourselves of it either.”

According to Gandrabur, there are two major issues with AI safeguards.

“Either they are not effective enough at detecting distress when it is expressed through euphemisms or language typical of young people, or in unusual contextual situations. Or, if they are effective … there may be a preference to keep the client engaged. So there are other considerations to maximize engagement that take precedence,” the expert explained.

The Tumbler Ridge shooting

The scale of the risks and impacts of AI in mental health is still not well understood, Gandrabur said.

“There hasn’t yet been research with medical-grade rigour, let’s say, to prove the value or the risks of these applications. There is a fundamental need to support what we say with objective analyses that allow us to measure these risks and impacts.”

Preparations for the hackathon began last October.

“It was motivated by the multiple suicide cases that were documented last year, and the tragedy in Tumbler Ridge only shows how timely and relevant this event is,” she said, referring to the mass shooting that took place Feb. 10 at a high school and a home in the northeastern British Columbia community.

Nine people died in the attack, including the shooter. An OpenAI conversational chatbot allegedly helped the gunwoman plan and carry out the killings.

• READ MORE: Tumbler Ridge: Family sues OpenAI over shooting

“A red flag was raised, but perhaps it was raised a little late. We would need to see the conversations that took place before that flag was raised. That’s the first thing. The second is that, yes, the flag was raised, but after that there were internal policies — humans, in fact — who decided the risk was not immediate enough to justify alerting authorities,” Gandrabur said.

The tragic event highlights the importance of safeguards — filters designed to detect distress and more serious mental health issues.

Mila’s programming marathon runs from March 16 to March 23 and is being held in collaboration with Bell Canada, Calcul Québec and Kids Help Phone.

Here is a list of mental health resources in Quebec

If ever you need help, don’t hesitate to reach out to:

• Your family doctor (if you have one)
• Call Info-Social 811 to get help or information regarding your mental health or the mental health of a family member or friend.
• Call the 911 emergency service if you fear for your safety or the safety of a family member or friend or to obtain immediate assistance.
• Call or text the 988 suicide crisis helpline when in crisis, or when they know someone is in a crisis, 24 hours a day and seven days a week, free of charge.

Find a crisis centre (in French only) near you to obtain free specialized crisis intervention services for you or a loved one.

• AMI-Quebec: 514-486-1448
• Canadian Mental Health Association (CMHA): 1-866-277-3553 in Quebec (24/7)
• Dunham House: 450-263-3434
• Centre Interligne Inc.: (for issues related to sexual orientation): 514-866-0103 call or text
• Depressives Anonymous
• Groupe d’entraide pour un mieux-être: (GEME): 450-332-4463
• Quebec Obsessive Compulsive Disorder Foundation: 514-727-0012
• Société québécoise de la schizophrénie: 514 251-4125 | 1-866 888-2323
• Suicide Prevention Centre of Montreal: 1-866 277-3553 or text 535353
• Veterans Affairs Canada: 1 800 268-7708

If you need immediate assistance, call 911 or go to the nearest hospital.

—

Health coverage from The Canadian Press is supported through a partnership with the Canadian Medical Association. The Canadian Press is solely responsible for this journalistic content.

This report by The Canadian Press was first published in French on March 16, 2026.