A Cape Breton doctor is threatening to leave if construction doesn’t start on a health centre.