AI in health care: Hype and red flags to watch for

Some easy-to-understand advice for using A.I, tools safely and knowing when it’s time to pick up the phone and call your doctor.

With tens of millions of downloads, generative AI apps like ChatGPT are increasingly being used to seek out medical advice.

A recent study by researchers at the University of Waterloo found that only a fraction of ChatGPT’s responses to open-ended medical questions were considered correct or clear.

Experts say large language models (LLMs) should be used with caution.

• Sign up now for daily CTV News Ottawa newsletters

Dr. Vera Kohut, national medical director at Serefin Health, spoke to CTV Morning Live on Thursday to discuss what’s helpful, what’s hype, and what red flags to look out for.

“It’s here to stay and we’ve just got to understand that it needs to become more fulsome in its accuracy,” she said.

Kohut says LLMs are designed to mimic conversations.

“When you understand that AI is really a machine that has been fed information by a human and it has learned from those human interactions, what happens is that it is not personal,” she said. “It does not know anything about you, your personal history, your family history, your genetics. It doesn’t know anything about the geography that you live in, your gender, your race, so as a result, it’s giving you blanket information; it’s not giving you personal information.”

Generative AI tools have no capacity to assess risk or get into the nuances of your personal health situation, Kohut added.

“Therefore, it’s likely to give you generalized information. It may be inaccurate,” she said.

Red flags to watch for when using AI

Kohut says if you should be skeptical whenever a chatbot gives you a diagnosis.

“If it suddenly gives you one diagnosis, or it starts giving you these vague diagnoses, or it says more research is needed, be careful,” she said.

“If it suddenly does not recognize, and it may not, the importance of what you are saying and therefore does not guide you, in a situation where your symptoms are increasing, to go and get further help, or frankly, doesn’t talk about you doing that, beware. And if, in situations where you’re talking to it and it suddenly starts giving you information about supplements or strange therapeutic interventions, beware.”

• The information you need to know, sent directly to you: Download the CTV News App

The Ontario Medical Association recently raised concerns about a growing trend of self-diagnosis using online tools and said that while it’s not always necessarily a problem, it can be risky and should be part of a broader conversation with a medical professional.

Can AI be helpful?

Kohut says there are ways AI tools can help.

“It allows you, for example, to track your symptoms. It allows you to take a look at what you’re going through and ask AI what that might mean. It allows you to take medical information and make it more palatable,” she said.

She said you can also use it to track medical appointments and medications.

Kohut said, however, that it is important to consider privacy whenever you use AI tools.

Earlier this year, OpenAI CEO Sam Altman warned that there is no legal confidentiality when feeding personal information into ChatGPT.

“We need to be more aware of the need for privacy in the information that is being given,” she said.