The Senate expense scandal, a complicated political controversy that mired the Upper Chamber for the past four years, ended quietly Thursday as a Senate committee chose not to pursue lawsuits against seven retired senators who refused to pay back more than $500,000 in disputed expenses.

The decision was based on legal advice that suggested following through with the seven lawsuits would be too costly.

Lawyer Brenda Hollingsworth, who led the review, said there is little chance a court would uphold most of the claims against the seven senators.

"In each case, when you factor in that it wouldn't be one lawsuit it would seven lawsuits, the justification for -- the ability to bring a cost-effective lawsuit for recovery of what's left over -- just wasn't there," Hollingsworth said.

The disputed expenses included travel, hospitality and housing claims that the other 23 senators named in the 2015 report from Auditor General Michael Ferguson subsequently paid back.

The decision marks the final chapter of the Senate scandal, which began in 2012 over questionable housing and travel expenses.

The case eventually escalated to former prime minister Stephen Harper’s office and saw criminal charges laid against three senators. Mike Duffy was found not guilty of 31 charges in April, and charges against Patrick Brazeau and Mac Harb were dropped in the wake of the decision. Pamela Wallin was investigated over her travel expenses but the RCMP never laid any charges.

The Senate has since taken steps to tighten its expense rules, including requiring proof of primary residence, such as a provincial health card and driver's licence, and a federal notice of tax assessment.

Sen. Tkachuk: ‘Serious mistakes’ made by auditor

But some politicians were angered by the auditor general’s $24-million audit, which recovered just over $300,000 in expenses.

"I have questions to him on value for money that we got from the Auditor General and that's not including all the time," said Larry Campbell, an independent senator from British Columbia.

Senator David Tkachuk echoed a similar frustration.

"I thought there were some serious mistakes made by the Auditor General," Tkachuk said.

For some critics, the decision to forego legal action is more evidence that the Senate isn’t accountable for its actions.

"To not pursue this case, says to other Senators who are contemplating ripping off taxpayers, you just gotta drag it out, there'll never be a consequence, you'll never have to pay up," said British Columbia MP Nathan Cullen.

With a report from CTV’s Glen McGregor and The Canadian Press