LONDON -- Britain's Treasury chief pledged Wednesday to crack down on tax avoidance and evasion as he delivered the first budget put forward by an all-Conservative government in nearly two decades.

Freed from the constraints of his previous government's Liberal Democrat coalition partners, Treasury chief George Osborne also announced major cuts to welfare spending in a budget that pledged to "put economic security first," in a country still struggling with the aftermath of the 2008 financial crisis.

"The greatest mistake this country could make would be to think all our problems are solved." he said. "You only have to look at the crisis unfolding in Greece, as I speak, to realize that if a country's not in control of its borrowing, the borrowing takes control of the country."

Osborne pledged that British workers should get a pay raise and he created a new minimum wage, dubbed the National Living Wage, for working people aged 25 and over. He rewarded Conservative voters with tax relief for those inheriting family homes and pledged to bolster plans for a so-called Northern Powerhouse, pushing for the creation of another major economic centre besides London.

But it was on tax policy that some of the boldest moves were made. Osborne abolished special privileges for so-called permanent non-doms, or people who live in the country but don't claim legal residence. The highly criticized practice has allowed some of the world's wealthiest people to live in Britain and pay no tax on earnings and capital gains outside the country.

Osborne said the changes might cost the country money, but an issue of fairness was at hand.

"It is not fair that non-doms with residential property here in the U.K. can put it in an offshore company and avoid inheritance tax," he said. "From now on they will pay the same tax as everyone else."

He said, after a consultation, anyone who has been a resident in the U.K. for more than 15 of the past 20 years will now pay full British taxes on all worldwide income.

"Most fundamentally, it is not fair that people live in this country for very long periods of their lives, benefit from our public services, and yet operate under different tax rules from everyone else," he said.

While the decision is a significant change it's not complete surprise, said Jeremy Franks, Head of Wealth Planning Advisory, UBS Wealth Management. Franks said he did not expect a vast exodus of the super-rich from Britain because other concerns come into play, such as the quality of life, good schools and stability.

"They are considering the broader picture," he said.

Osborne also said a levy on banks would be gradually reduced over the next six years, while a new 8 per cent surcharge on bank profits will be introduced beginning in January 2016.

Starting in January 2021, the bank levy will only be charged on the U.K. balance sheets of lenders based in the U.K. -- as opposed to their worldwide operations.

That seemed directed at HSBC, which has announced it is considering a move out of London. The global banking giant has cited increased regulatory concerns and the potential that Britain will leave the European Union as a cause for concern.

Prime Minister David Cameron has promised to hold a referendum within two years on whether Britain should leave the 28-nation bloc.

In a move certain to cheer the U.S. military, Osborne also made a commitment to meet the NATO pledge of spending 2 per cent of national income on defence spending for every year of this decade.