A business owner in downtown Victoria says he was assaulted while closing up shop, leaving him with a broken arm.