It's simple. We've told him to quit it. Tell him now that it's not our responsibility to make him quit. If it's been a problem for him in the past, he needs to deal with it. Feeling guilty doesn't mean anything without a sincere desire to change. If he does this shit again, we move onto ban.
We have a system in place. It works, as long as we use it.
We can ban for immaturity. We can ban for being a dick. The power is ours.
Doesn't mean we shouldn't continue to document things, issue warnings for first offenses, or any of that. But once a warning is issued for specific behavior, we can move onto ban. He has been warned. We know he's a potential problem. We watch, and if he fucks up, we ban, moving through the process.
The only place where things get at all murky is when a user makes multiple, unrelated fuck-ups. Then you have to stop, see if he's at least learning from his mistakes, and make the call. But when someone has been warned for a specific behavior, and continues that behavior, there's no reason we shouldn't take things to the next level.
Again, in this case, he's been issued a warning. Fucks up again, he gets a ban. His simpering little "the guilt! the wretched guilt!" act, while annoying, does not give us reason to ban. It does serve to give context to his future actions, which may make a ban more likely.