Telus email outage a 'catastrophic failure': experts

Telus to credit customers for email problems Thousands of Telus customers are frustrated after an email outage continues to impact them.

As thousands of customers mark the fifth day without email service, Telus said it is now offering affected customers bill credits in an apology as it works “round the clock” to restore the service.

Email service across the Telus network went down Thursday, with some customers receiving temporary webmail service starting Saturday, but there is still a significant portion of people relying on the service who say they’re still incommunicado.

Many are deeply frustrated by what they feel was insufficient information and explanation from the company, a situation that stretched for days.

“We are contacting residential and small business customers directly with details of their bill credits within the next 48 hours,” said Chief Customer Officer Tony Geheran in an email late Monday afternoon. “Restored customers will see the email in their TELUS.net inbox and customers without full functionality will receive an email in their TELUS webmail account.”

The company didn’t address the Telus.net email service issues until Saturday, offering a video apology well after frustration had begun to boil over.

Dozens of customers have contacted CTV News since the email outage began, describing lost revenue from home businesses, confusion between family members and customers, and even missed medical appointments.

“I’ve relied on the confidentiality of my Telus address and missed an appointment with my cardiologist,” said Angela Clark, a Vancouver resident. “I value loyalty and I’ve been a loyal customer, but mediocre is no longer good enough and I think we need to have more [service] options in British Columbia.”

Ken Coach runs his crisis management company, Media Coach, from his home and calls the service disruption and ensuing lack of information from Telus, unacceptable.

“If you're working from home, email is critical and it can't go down -- not even for an hour,” he said. “Why didn't they have a plan in place in case something like this happened and why didn't they know you must respond very quickly when something’s gone wrong? You've got to reassure people you're going to fix it; you've got to reassure people that you're on it."

In an online notice, Telus said the outage happened during a server update.

"The issue occurred during an overnight update to our services in the early hours of Thursday, Aug. 15 when a flawed repair procedure, in partnership with our vendor Dell EMC took the TELUS.net email system offline," it read.

Veteran IT professional and Director of Technology for Uniserve, Jake Stennett, called the situation a “one-off catastrophic failure that should've never happened.”

Stennett said big companies typically have a series of backups and redundancies to ensure that when they have an issue, the backup seamlessly takes over and customers never even know a maintenance event has taken place.

“There are a thousand different reasons this could happen," he explained. “It was probably a cascading failure – fail-safes do fail and one thing led to another and here they are."

Stennett can’t recall an email service issue this far-reaching in Canada and said the amount of time it’s taken to restore service is exceptional.

He does agree with Telus’s approach to provide interim service, allowing customers to send and receive email while it restores full email archives and contacts from a backup.

“Everybody should be worrying about backups more than we do, we rely on the cloud a lot and the cloud is just someone else's data centre," he pointed out, calling this a cautionary tale for email users who rely on Telus to store important information. “Your data is never safe unless it's in more than one location."

Telus still hasn’t replied to CTV’s request for more information on a timeline for full-service restoration or how much of a credit affected customers can expect on their bill. That lack of transparency and communication is baffling to Telus customer Coach.

”In any crisis, people want to hear from a human being that everything's going to be ok, even if you don't know yet it's going to be ok. They want to know you're working on it; they want to know what you're doing to try and fix the problem,” he said.

"It's very unacceptable to wait two days and then put out a video tape of someone reading off a TelePrompTer –words that have obviously been written by the PR department and the law department. We wanted to hear a real human being saying, Gosh, we messed up, we're going to fix it as soon as we can.’”