Canada's tax deadline is quickly approaching, and if you haven't started to think about getting all the necessary paperwork for your tax return together, you're not alone.

Most Canadians typically have until April 30 to file their tax returns. But since that date falls on a Saturday this year, the deadline isn't until the next business day – Monday, May 2.

Canadians who are self-employed have until June 15, 2016 to file their tax return.

Tax lawyer Jonathan Garbutt says there are tens of thousands of Canadians who haven't filed their taxes in years.

But for those who fear hefty penalties and possibly even jail time over their tax return tardiness, Garbutt says there may not be much to worry about.

Does filing late affect your tax refund?

Garbutt said for the majority of Canadians who have a regular income and receive a T4 slip from their employers, filing their tax return late doesn't typically result in any penalties.

"If they're not filing, all they're really doing is loaning the government their money and not getting much in the way of interest back," Garbutt told CTV's Canada AM on Thursday. "For those people it's a shame. They really should be filing and getting those refunds."

Canada Revenue Agency notes that if you file your return after the deadline, your GST/HST credit, any related provincial credit, Canada child tax benefit payments and old age security benefit payments may be delayed.

What about if you owe the government money?

If you owe the government money, Garbutt said consequences for filing late vary depending on the circumstances.

"If they catch you it's one thing," he said. "If you decide to come in on your own and make a voluntary disclosure, that's completely different."

With a voluntary disclosure, Canadians can disclose their income to the CRA for the previous 10 years.

Garbutt said in these cases, Canadians would have to pay the taxes and interest owing, but he noted that there’s often "a pretty good discount" on the interest owed.

"All of the penalties, all of the potential jail time, everything that could be potentially adverse -- all of that goes away if you come in on your own free will and pay the tax and interest owing," he said.

What kinds of penalties do late filers face?

The penalties for filing your tax return late also depend on the circumstances, Garbutt said.

"Sometimes (CRA) will  give you a pass," he said. "If it’s the first time, there may be a 10 per cent administrative penalty. If it’s the second time, there may be a 20 per cent penalty," he said.

Garbutt noted, however, that with some documents, the penalties for not filing are accrued per day.

"So the sooner you get (your return) in, the better," he said.

Do Canadians face prison time for unpaid taxes?

A conviction for tax evasion, which includes not filing tax returns and hiding income, can result in up to five years in prison.

But Garbutt noted: "They don't generally go after people and throw them in jail unless they go out of their way to make themselves a target."