People ‘try to cheat’ female AI agents, exposing gender bias in user behaviour: study

A new study shows humans bring gender bias to their interactions with Artificial Intelligence. (Photo credit: Trinity College Dublin and Ludwig-Maximilians Universität (LMU) Munich)

A new study says people carry their gender biases into interactions with artificial intelligence.

Researchers from Trinity College Dublin and Ludwig-Maximilians Universität (LMU) of Munich used a classic game-theory setup known as the Prisoner’s Dilemma to examine how users co-operate, compete and exploit automated partners.

They paired 402 participants with partners labelled as either “human” or “AI,” and described as male, female, non-binary or gender-neutral.

“In this game, the two players have choices to make,” Taha Yasseri, co-author of the study and director of the Centre for Sociology of Humans and Machines at Trinity College Dublin, said in an interview with CTVNews.ca. “They can make the choice to co-operate with one another, or they can make the choice of defecting against each other.”

The research found that once a gender identity was assigned to an AI agent, participants’ behaviour shifted in ways that mirrored, and sometimes amplified, real-world gender bias.

Design choices ‘not by accident’

Participants were asked why they chose to defect, whether they lacked trust in their partner, or whether they knowingly exploited someone they believed would cooperate. Yasseri said exploitation became especially clear with female AI partners.

“What we saw was that the level of cooperation does not really depend (on whether) people are playing against other humans or machines,” Yasseri said. “But when we start giving gender identities to these machines, the patterns start to change.”

He said this design allows researchers to observe how far trust, risk or self-interest drive a participant’s decisions.

“We noticed that people … try to cheat. They try to rob the partner, particularly when it’s a female AI,” Yasseri said. “Despite the fact that when we ask them what do you think the partner is going to do, the answer often was: the partner is going to be nice, yet I decide to choose against them.”

Yasseri says the findings line up with what already happens in the commercial AI marketplace, where many digital assistants default to female voices or names.

“This is not by accident,” he said. “Our research shows that these decisions are made either consciously or unconsciously to maximize satisfaction of the customers, because they feel ‘I can order this assistant to do this or that. I can be rude.’”

By contrast, he said, people “might not feel the same power dynamics” with a male-presenting AI agent.

Even though the team expected discrimination to vary based on the participant’s gender, Yasseri said that wasn’t the case.

“We saw that the behaviour was independent,” he said. “In fact, the patterns that are described are more exaggerated when we focus on female participants.”

‘They were just more cute’

Yasseri said companies frequently underestimate how consequential gendered design choices can be.

He recalled speaking to a product owner developing a new AI agent.

“She was saying, ‘We just created the list of the names that we might pick up for our product, and after listening to you, I checked and almost all of them were female,’” he said. “We didn’t pay attention, but they were just more cute.”

Yasseri warned that decisions that seem trivial can shape user behaviour in unintended ways.

“This could have big consequences on the way that your clients and your customers perceive these agents and interact with them,” he said.

For Yasseri, the most important implication is behavioural: what people learn about how to treat others when AI becomes the target.

“It might not be a huge ethical issue if I exploit my female AI assistant,” he said. “The problem is that this might become a habit, and it might spill over back to our human-to-human interaction.”

“If I feel more comfortable or rewarded or if it’s OK for me to be abusive to my AI assistant, then sometimes we might step over the boundaries in human interactions,” he said.

“We, over the course of evolution and history, (have learnt) to fix these issues in the workplace and other situations. But then the big risk is that AI pushes us back to having discriminatory behaviour towards others.”