Being that you didn't handle the situation immediatly, (nothing wrong with that for this situation) you will need to approach the situation differenlty. Kinda like yelling at the dog for pooping on the carpet 7 days after the fact. He may end up calling a lawyer (far fetched, but it could happen) and sue you for wrongful termination. (being that he was able to work since the incident.)
Does/did this person bad mouth his previous employer?
Talk bad about his previous company?
If so, put aside your anger and let him go easily! Make it come across as "cutting costs" If he is the "low man" on the totem pole, this should be easy. If not, make it known that you can no-longer afford to pay his salary. He can either agree to take a pay cut, or leave on his own....
Maybe cut his hours back a little at a time and force him to leave due to no work.
There are many ways to cut him loose. How you do it may impact whether he bad mouths your company.