Justice Minister Peter McKay said Friday that his decision to bow out of federal politics came during a moment he shared with his two-year-old son after returning home from a cross-Canada trip.

“I literally walked in the door with my bags in hand and my little guy came rushing out to meet me,” MacKay said during an interview Friday with CTV Atlantic.

“He stopped in his tracks and he said, ‘Bye-bye, Dada,’ because he thought I was leaving.”

MacKay said “something kind of broke” and he thought about his future.

“I decided this was the right time to step back,” he said.

MacKay announced on May 29 that he would not seek re-election in the fall, just three weeks after he had originally announced he would be running. The minister cited family reasons and denied any claims that the move was politically motivated.

“It was a very personal decision,” MacKay said. “This was a decision made between my family and myself and what I thought was in the best interest of both.”

Prime Minister Stephen Harper was understanding about the decision, MacKay said.

“I told him that this was the process I was going through,” MacKay said. “His reaction was extremely generous and humane.”

Harper, MacKay said, did say MacKay’s departure would hurt the Conservatives.

MacKay, who has been a Member of Parliament for 18 years, joins a growing list of high-profile Conservative candidates to back out of the upcoming election. Conservative Alberta MP James Rajotte added his name to the list of departures late last week, when he declared he would not run again. Most recently, Industry Minister James Moore said he would not seek re-election.

Moore, too, cited family reasons.

But MacKay, 49, has not ruled out a return to politics in the future.

“Looking back on those 18 years, what I hope and pray that I’ve been able to do is stay true to the values of my community and deliver for my community,” he said. “If I’m ever in a position in the future to contribute in other ways, I will look forward to doing so.”

“My future is going to be full. I intend to keep working.”

MacKay will continue in his role as Justice Minister and MP for his Nova Scotia riding, Central Nova, until October.

With files from CTV Atlantic