State officials say AI system didn’t discriminate based on color when mistaking students’ chip bag for gun

Baltimore County Public School AI cleared of bias after chips bag triggers gun alert. (CNN)

BALTIMORE (WMAR) -- After an AI weapon detection system at a BCPS school mistook a Black student’s bag of chips for a gun, many people, including the boy’s family, claimed it was racially biased.

But a report from the state Inspector General for Education says that wasn’t the case.

An AI weapon detection system that Baltimore County Public Schools uses, called Omnilert, mistook 16-year-old Kenwood High School student Taki Allen’s crumpled-up chip bag for a gun.

At the time, Allen didn’t know what to do.

“Police showed up, like eight cop cars, and then they all came out with guns pointed at me talking about getting on the ground. I was putting my hands up like, ‘What’s going on?’ He told me to get on my knees and arrested me and put me in cuffs,” Allen said.

The Maryland Office of the Inspector General for Education investigated the incident.

A newly released report says the AI system flagged Taki’s hand position while holding an item as a potential weapon threat.

The office investigated claims that the system discriminated against students of color but found no evidence of bias.

However, Taki Allen’s grandfather, Lamont Davis, still thinks the AI system is designed to automatically pick out Black people.

“Say a white woman with a dog walks up there on a Sunday morning and sits down, the AI system is not going to give a false positive. But I think if a Black person walks up there and walks their dog up on the property, it’s gonna give a false positive,” Davis said.

Although the OIGE found no bias, it did point out flaws in the situation.

The report states the principal at Kenwood High School didn’t see the cancellation notice from other staff and contacted police about an already-canceled threat, showing system flaws.

And once law enforcement is involved, the report says BCPS loses control over alert notifications, potentially creating dangerous situations.

The OIGE recommends that the BCPS Department of School Safety review its protocol about the number of individuals who receive alert notifications, and also calls for BCPS to conduct bi-annual training for school executive staff and members of the Baltimore County Police Department on the use of and established notification protocols when a threat is verified using Omnilert.

In a past statement, BCPS says the district is committed to taking additional steps to ensure all staff fully understand and follow established protocols to prevent such incidents from occurring again.

Raven Payne WMAR