‘Gremlin within the software’ to blame for halting of all trains Lines 2 and 4: TTC CEO

CP24’s Melissa Duggan spoke with TTC CEO Mandeep Lali about the software issues that halted traffic on two subway lines Thursday morning.

The TTC has launched an investigation after a software issue resulted in the halting of all trains along two major transit lines on Thursday morning.

Speaking with CP24 Live at 5, TTC CEO Mandeep Lali said the agency had an issue with its server that holds all control systems for signalling, schedules and trips for the trains.

“What appears to have happened, there was a gremlin within the software, not the hardware, which caused the system the server A to fail,” said Lali, sharing that the issue was observed at 9:53 a.m.

Another server failed a minute later, he said, which prompted the TTC to suspend service along the entirety of Line 2 and Line 4.

Regular service resumed about 25 minutes later. The TTC said that approximately 200 shuttle buses were previously dispatched along the lines to supplement service.

“As a result of that, we looked through our history in terms of software upgrades,” the TTC CEO said.

TTC staff found that there was a software upgrade on April 19, and Lali said that while testing was done before it was uploaded onto the servers, they identified the “gremlin” that caused this morning’s failure.

“They have now isolated it. They’re now working with the supplier to replicate a patch, so that element will not be activated by the server,” Lali said.

He added that the agency has deployed teams within the network to manage the service while waiting for the patch.

“From my previous experience in New York, when you have digital systems, they sometimes have, after software updates, gremlins which come through. I just want to reiterate my thanks to the customers for the patience and also the teams that rectified this within 25 minutes to identifying the glitch, the gremlin, within less than five hours,” Lali said.