The search for five fishermen whose boat capsized off the coast of Nova Scotia continues into its second day, but the military has scaled back its efforts, saying the chance of finding the men alive has “diminished significantly.”

At a church in Woods Harbour, a small fishing community where four of the lost men are from, a life preserver and five candles remind parishioners of the missing.

“Faith and hope have been tested, but they’re still holding on,” said Pastor Phil Williams.

The missing fishermen -- Capt. Katlin Nickerson, Steve Cole Nickerson, Joel Hopkins, Tyson Townsend and Billy Hatfield -- left last Tuesday to fish in "the gully" -- a spot near Sable Island where the ocean floor drops off.

They were scheduled to return to Woods Harbour on Monday, but it’s believed the boat ran into snow, hurricane strength winds and large waves.

A distress call from the 13-metre boat went out late Sunday night, during hurricane-force winds.

The missing fishermen are all in their early 20s to early 30s.

George Hopkins, father of Joel Hopkins, said his son was not too worried about heading to the gully to fish.

“He was concerned, but I didn’t feel any nervousness in his voice. He wasn’t overly worried at the time,” he said. “The last time I talked to him he said he had damage aboard the boat -- just slight damage, as far as I know. It wasn’t anything big, and that didn’t seem to concern him either.”

The desperate search for the men continued Tuesday afternoon, more than 36 hours after the distress call from the boat, named Miss Ally, went out.

Officials with the rescue team said the search area continues to grow as strong winds are causing the ocean to drift, though visibility in the area has improved since the operation began.

During a news conference Tuesday afternoon, a search and rescue team spokesperson estimated that survivability in the rough, frigid waters was 24 hours.

“We always try and go beyond that time just to make sure we do everything we can to find them, if there’s a chance they’re still alive,” he told reporters.

Asked about the search being scaled back, the spokesperson said rescue operations will continue through the day, but he wouldn’t comment when the search would be called off.

“As long as we’re searching, we have to search as if there’s a chance somebody might still be out there.”

Capt. Doug Keirstead, of the joint search team, said ongoing high winds and rough waters are hampering rescue efforts.

"The weather conditions are still quite challenging," he told CTV’s Canada AM on Tuesday. "When we originally received the emergency locator single from the vessel on Sunday evening, we were dealing with eight- to 10-metre seas, zero visibility and hurricane force winds. So we’re still dealing with -- over the night hours and into this morning -- six-metre seas, which is still quite rough."

Keirstead said two Canadian Coast Guard ships along with two aircrafts, one from the Canadian Armed Forces and another from the Department of Fisheries and Oceans, are helping in the search efforts.

The U.S. Coast Guard spotted the overturned boat and a life raft shortly after the distress call went out on Sunday, but rescuers have yet to locate the men using infrared and heat-sourcing technology.

With a report from CTV Atlantic's Kelland Sundahl