An operator of Canada’s new mobile emergency alert system is acknowledging its limitations after countrywide tests yielded mixed results this week.

The Alert Ready system is designed to deliver critical and potentially life-saving information from federal, provincial and territorial governments on everything from fires to terror attacks. Officials are looking into why many devices remained silent while others buzzed during scheduled test roll-outs in different provinces and territories.

The Canadian Radio-television and Telecommunications Commission (CRTC) mandated annual public alerting tests by wireless service providers and broadcasters every May as part of Emergency Preparedness Week. According to Alert Ready website, the wireless alerts begin with a distinct sound, known as the Canadian Alert Attention Signal, and display an “EMERGENCY ALERT/ALERTE D’URGENCE” banner, followed by instructions relating to the emergency situation.

Tests were attempted in Ontario and Quebec on Monday. Eight other provinces and two territories sent the test alerts Wednesday. A coding error prevented the message from being sent out in Quebec. The results were hit and miss across the country.

In Alberta, an estimated 3.7 million cellphones were expected to receive the alert during Wednesday’s 1:55 p.m. test.

Tim Tryteen of the Alberta Emergency Alert Program called the tests a “big milestone,” in spite of the patchwork of results. However, he suggests more needs to be done to refine the system. He also sees the addition of mobile alerts as complementary to traditional television and radio emergency broadcasts, rather than a replacement.

“There is a lot of work that has gone into this for a lot of years,” he told CTV Edmonton on Wednesday. “It’s going to be very difficult to get everyone using any single way. It would be like saying, ‘Can we reach everybody on radio and TV?’ Well, no. People are at work. They don’t have the radio on. They don’t have the TV on. The same is going to go with wireless.”

Tryteen explained that there are a number of reasons why a device may not receive or relay an alert, including whether a phone is turned off or in “do not disturb” mode.

“The decision was made that we are not going to override the user’s settings,” he said.

Tryteen said measuring the effectiveness of the alert system will take time. Officials are turning to social media to help gauge the success of this week’s test.

In the future, more devices may be compatible with receiving the alerts, such as smartwatches and internet-enabled appliances, and there may be a way for members of the public in affected areas to respond with information about the emergency.

Alert Ready has a compatibility guide on its website to help determine which devices can currently receive the alerts.

With a report from CTV Edmonton’s Nicole Weisberg