OTTAWA - The Conservative government has turfed party luminary Hugh Segal from the helm of a high-profile Senate committee, sparking allegations from the Liberals that he was punished for speaking his mind on policy issues.

The Tories said the departure was nothing more than an administrative change.

Segal, appointed by the Liberals to the Senate in 2005, is a highly regarded Conservative thinker with long ties to the party. He was chief of staff to prime minister Brian Mulroney and has acted as an adviser to Prime Minister Stephen Harper.

He wrote to fellow senators on the foreign affairs committee Tuesday, telling them he was forced to resign.

"I am tendering my resignation reluctantly, at the request of my leadership who feel it necessary to accommodate pressures of seniority in the government caucus," Segal wrote.

"As I am the newest and youngest Conservative senator, who is neither a privy councillor nor minister, there is no seniority contest in my caucus I can other than lose."

Conservative Senate Leader Marjory LeBreton informed her Liberal counterpart the same day that her preferred replacement for Segal was Senator Donald Oliver, who was the chairman of the legal and constitutional affairs committee -- the committee that had reviewed the Federal Accountability Act.

Tory Senator Michael Meighen has also been removed as vice-chair of the national defence and security committee, a group that came under fire last year for an expensive stop in Dubai during a study of the Middle East and Afghanistan. Meighen had defended the trip and his colleagues, despite criticism from within the party.

Segal's departure became the main topic during the Senate's question period. LeBreton said she would not discuss private caucus decisions.

"The problem with a small caucus is once a change is made for a particular reason then it creates a domino effect and results in the need for other changes," LeBreton said.

"However, it was in no way a reflection on the Senate or the Senate committees. We simply acted in good faith."

Privately, Liberals speculated that Segal had been on the outs with the Conservative leadership over different positions he had taken. For example, Segal's committee put out a highly critical report last week on the government's aid strategy in Africa.

He also recently urged his government to investigate the treatment and the demands of terror suspects being held at Millhaven penitentiary in Kingston, Ont. And during debate on the Federal Accountability Act, Harper's marquee legislation, Segal voted against the government on one of the measures.

Publicly, opposition senators said they would miss Segal.

"Hugh Segal has been a terrific chairman, he's a great communicator and I was completely astonished to find that only a few days after we tabled our major report he was forced to resign. You'll have to ask him. I guess he was fired," said Liberal Senator and committee vice-chairman Peter Stollery.

In an interview, Segal said he had no indication the decision was anything but administrative.

"I'm not aware of any policy or kind of left-right spectrum issue as between myself and the government or the leadership," Segal said. "I deal with the prime minister's office on a regular basis. It's always constructive and cordial. So if there is a problem, I'm not aware of it."

The signal that the Conservatives wanted changes to committees set off flurry of activity in the corridors around the red chamber Wednesday. Liberals were perplexed there had been no previous negotiation on the changes, despite the fact they have a majority in the upper chamber.

By the end of the day, the two sides settled on Stollery acting as Liberal chairman of the committee for the day, until further talks could be held.