Victim mentality, and now with a whole new angle on it, that's hard to deal with. HE has a problem and is projecting it on you. And generally it won't ever change. Either you do or you don't, either way you'll have to live with your decision. Do you want to enable him and end up with poor performance, so you feel bad, or you don't, and you still feel bad. (Even though you shouldn't, but it doesn't work that way.)
After going through this on occasion over the years, in my old age I finally stopped tolerating people doing that to me. I stopped being their victim's victim. It's far better to have a little lingering angst than to have to deal with it every day. Make the decision on how it affects you, not him. IMHO, YMMV, good luck.