David Miller found a moment in his busy schedule to tackle a stack of letters sitting on his office desk late last week. He was writing thank you notes to those who had bid him farewell.

"They all seem to come at once," the outgoing mayor said with a grin last Thursday. It was a rare second of down time during a chaotic day of exit interviews with media.

His messy desk sat in the middle of an otherwise barren office. Nails poked nakedly out from the walls where pictures once hung.

A corner of the room piled with personal effects, boxes of files, empty bookcases and a mass of furniture in disarray were all that was left.

On Wednesday, Miller turns the office over to incoming mayor Rob Ford when he officially takes the reins of City Hall. Miller's vision of growth is set to be replaced by Ford's calls for fiscal prudence.

"My only wish is that people remember me as a progressive mayor who kept his word and did everything he possible could to make Toronto a better place where everybody had a real chance," Miller told CTV News last Thursday, in one of his last official interviews as Toronto's mayor.

After 16 years in politics, seven of those as the mayor of Canada's largest city, Miller says he is ready to return to life as a private citizen. He will work with the World Bank as an advisor on city issues and the environment and, he says, pursue his passion of promoting green jobs and technology.

Miller was first elected mayor seven years ago, clutching a broom and promising to clean up City Hall. With the city's reputation blackened by a computer leasing scandal, voters turned in droves to the then 44-year-old Harvard graduate to bring about change.

"There was an incredible energy, people may have forgotten about the passion in Toronto," he said. "I think I got elected because people wanted a mayor who was younger and forward thinking and progressive. Obviously they did, otherwise I wouldn't have won."

In 2006, nearly 60 per cent of the electorate asked him to return to the position. Miller says his first term was about getting City Hall in order, while the second was about securing money to invest in people's priorities, including public transit.

Miller considers the modernization of the transit system and the Transit City plan for a network of light rail lines across the city among his greatest accomplishments in office.

On June 15, 2007, Miller spent his wedding anniversary in a Mississauga bus garage, where Premier Dalton McGuinty announced the province would fund Transit City, a network of light rail transit lines that would connect the entire city. He called it one of his best moments as mayor.

"That program is so important to Toronto. It was the centerpiece to what I campaigned on in 2003 and 2006," Miller said. "Public transit helps lifts people out of poverty. It literally changes peoples' lives if they go from buses that get stuck in traffic to rapid transit that doesn't."

But two years later, McGuinty would pull the funding, a decision that still confuses Miller. It frustrates him so much he says it is the only reason he regrets not running for a third term in office. Had he run, he would have pressured McGuinty to pay what he promised.

"The Premier gutted the funding for Transit City in the last provincial budget. If I was running again he could not possibly have done that. I would have been campaigning on that all across Toronto," Miller said.

His work with Transit City could be further derailed by Ford, whose election platform included a promise to scrap Transit City and build a subway to Scarborough.

Reports leading up to Wednesday suggested Ford was prepared to put the brakes on the light rail construction. A meeting with TTC's chief general manager was scheduled for Wednesday.

A fully-realized Transit City plan could have become Miller's legacy after leaving office. Instead, it could be an incoming mayor and council whose priorities contrast drastically with his own.

Patrice Dutil, an associate professor at Ryerson University's Department of Politics and Public Administration, describes the shift as a dramatic pendulum swing. Before Miller's moderate left government was a comparably moderate right one led by Mel Lastman. Ford's election marks a hard right turn that indicates a level of dissatisfaction with Miller's time in office.

In October, Ford stampeded to an election victory largely by attacking wasteful spending under Miller's reign. A disastrous garbage strike in 2009 led to a chink in Miller's armour and may have been the final indication to the right-leaning councillor that the pendulum was swinging.

Ford announced his executive council earlier this week, filling the positions with right-of-centre councillors who played opposition during Miller's regime.

While he declines to discuss Ford's election in detail, Miller does say that he is concerned that lower-income "priority" neighbourhoods could be punished by Ford's focus on cutting spending.

"It is no surprise that I don't share Coun. Ford's views of the city administration. The one thing that concerns me is people in our priority neighbourhoods," Miller said. "Every single time that you elect a government that says falsely there is a lot of waste and promises to cut taxes, they don't cut services with the politically powerful neighbourhoods. They cut services to those who have the least. And if you do that in Toronto today, you are going to create a really serious social problem."

In the end, however, Miller is satisfied with his decision not to run for a third term. He says he plans to spend more time with his family and watch his two teenaged children play hockey and soccer.

"The only thing that interests me in politics is how our national government treats cities. I really hope somebody else comes along and takes up that fight because I just can't imagine running again."