Veterans Affairs Minister Julian Fantino has suggested that the union representing federal workers is behind the recent strife between the Conservative government and Canada’s veterans.

Speaking to Newstalk 1010 on Sunday, Fantino said that he believes the Public Service Alliance of Canada has been spreading "misinformation" that has "agitated" the veterans' community.

A group of veterans travelled to Ottawa last week to discuss the closure of eight regional Veterans Affairs offices with Fantino. But the minister was more than an hour late for the scheduled meeting, angering many of the veterans.

Hundreds of veterans have recently rallied against the office closures, saying the changes will leave them without easy access to the services they need.

The office closures also come on the heels of a string of recent soldier suicides, which have some criticizing the government for failing current and former soldiers suffering from mental illness.

But Fantino insists that in spite of the office closures, veterans will still be able to access the services they need.

"It all goes back to a huge amount of misinformation, miscommunication and also mischief-making that I attribute to very directly to the Public Service Alliance of Canada," he told Newstalk 1010.

"I would be agitated too if I heard some of these lines that they've been putting out," Fantino added. "I feel badly for the veterans. They've served their country; they don't need this added aggravation."

Fantino says veterans services will be made available online and at Service Canada offices, of which there are more than 650 across the country. For those not comfortable accessing services online, case workers are available for home visits.

Fantino also again rejected calls for his resignation Sunday, insisting he's "done nothing wrong."

"Why should I resign?" he said. "For Mr. Mulcair to demand in such an angry fashion my resignation, I take that as a badge of honour," he added.

A number of veterans had called for Fantino to step down last week after his tardy arrival, saying they felt disrespected by the minister.

That call was later echoed by NDP Leader Thomas Mulcair, who said in the Commons last week, "Will the prime minister do the right thing: apologize himself and fire that incompetent."

Fantino apologized to the veterans both in a written statement and with remarks made in the House of Commons.

"I have been committed to having an open dialogue with the men and women who served Canada in uniform, but I realize that (Tuesday’s) regrettable delay has brought that into question," he wrote in his apology.

But one of the veterans calling for Fantino's resignation says that by attempting to shift the blame to the union, the minister is suggesting veterans are "stupid."

"I think he's trying to deflect from the problems he has," retired Sgt. Ronald Clarke told CTV's Question Period in an interview that aired Sunday. "We want him out of there now. We don't like being called stupid by anyone. Especially not by the minister who's supposed to be looking after us."

Clarke was one of seven veterans in Ottawa last week to meet with Fantino. He said that following the meeting, one of Fantino's political aids called his Nova Scotia home to apologize, but he was still in Ottawa at the time.

Clarke said neither Fantino nor anyone from his office has called back since.

"I believe he was forced to make that apology in the House," Clarke said. "I don't think he himself was up for apologizing. If he was, he wouldn't have one of his flunkies call me at the house. He would have done it himself."

Veterans' advocate calls closures "short-sighted"

Retired Sgt.-Maj. Barry Westholm told CTV News Channel that the planned closure of the regional Veterans' Affairs offices is "short-sighted."

"They're doing a very short-sighted job," Westholm said. "We've been told time and time again that there is a wave of injured coming, especially from Afghanistan, and to get prepared for them. To me it's counterintuitive to start closing facilities when this wave is known to be coming our way."

Westholm had previously worked for the Eastern Ontario Region of the Joint Personnel Support Unit – a unit created to help the flood of injured soldiers who were returning from Afghanistan. He now works with the Director of Canadian Armed Forces Engagement for the Canadian Veterans' Advocacy.

Westholm said while working with the JPSU he began to see some of the "shortfalls" of the structure. He brought up his concerns to his MP Cheryl Gallant in a number of emails.

"As time progressed and the situation worsened, my emails to her became more and more terse and more and more urgent that we could really lose people, and we have lost people," he said, referring to the recent soldiers' suicides.

Westholm says he has since severed his association with the Conservative Party.