Joe Oliver was quietly sworn in as finance minister in a low-key ceremony on Wednesday that was closed to the media.

The move follows Jim Flaherty's surprise announcement on Tuesday that he was resigning from the position ahead of an eventual move into the private sector.

Reporters in Ottawa were not allowed inside Rideau Hall for the ceremony, and many members of the media learned on their own that the event was taking place.

Prime Minister Stephen Harper confirmed the swearing in on Twitter, posting: "I just named Joe Oliver Canada's new Finance Minister. He will continue to strengthen the economy & balance the budget by 2015."

The Prime Minister's Office posted a video interview with Oliver, billed as exclusive, on its website as part of a taxpayer-funded news service that promotes the government.

NDP Leader Tom Mulcair said he found it "amusing" that Oliver's swearing in was so secretive.

"It's like it's a cult," he said.

The Opposition leader said he doesn't expect Oliver to question Harper.

"Stephen Harper likes his hand puppets to be obedient, and with Joe Oliver he'll get exactly what he wants."

Harper also confirmed on Twitter that Minister of State for Science and Technology, Greg Rickford, will take over Oliver’s former natural resources portfolio. The Kenora MP will now be the government’s advocate for the Keystone XL pipeline project, which is awaiting approval from U.S. President Barack Obama.

Rickford, who was first elected in 2008, retains his post as the minister responsible for economic development in Northern Ontario.

Completing the mini shuffle, MP for London West Ed Holder will take over Rickford’s former portfolio.

Speaking to CTV News on Wednesday, Oliver said his decades-long career on Bay Street has prepared him for his new role.

The 73-year-old Toronto MP had previously worked in investment banking at Merrill Lynch. He had also held the post of executive director of the Ontario Securities Commission, as well as the CEO of the Investment Dealers Association of Canada.

Oliver said he's looking forward to a budgetary surplus in 2015, and added that he won't be that different from Flaherty.

"I'm not focused on differences," he said. "What I want to do is maintain the government's focus, which is a low tax plan for jobs and growth right across this country."

While Flaherty had voiced public concerns about the fairness of income splitting, a lucrative tax cut for families that particularly targets households with stay-at-home spouses, Oliver said he's committed to the platform promise.

"My view is that this is something that we are committed to – it's part of our platform," he said, adding that the policy won't come into effect until next year.

"We'll address that issue at the appropriate time," he added.

Mulcair calls appointment a 'mistake'

Mulcair had strong words of criticism for Oliver, saying Wednesday that his appointment as finance minister is a “mistake.”

Mulcair said Oliver’s “statements on global warming are an embarrassment to Canada,” and criticized the former natural resources minister’s hard-line stance against critics of the federal government’s environmental record.

“Joe Oliver as a minister was frankly an embarrassment, and that he should be named minister of finance for Canada is a real shame,” Mulcair told reporters in London, Ont.

Mulcair said he hopes Rickford can show “more openness and understanding of the natural resources file.

“Canada is blessed to have such natural resources, there are a lot of countries that would love to have what we’ve got,” Mulcair said. “But we’ve got to learn how to do it sustainably, respecting the rights of future generations and not leaving them a massive ecologic debt that will be for them to clean up.”

With files from Sonja Puzic and Marlene Leung and a report from CTV's Ottawa Bureau Chief Robert Fife