Insist on time to take care of your thing, or more to the point, to take care of you.
There are always going to be reasons to put this off, but I'm going to urge you not to do that.
You have to take the time, or someone else is going to take it from you. When they do, it's a pretty sure bet that they will be asking you to do something in opposition to your interest.
So, don't give them the chance. Go out for a beer. Go eat. Go see a movie. Go somewhere quiet. Hell, just drive up to a shopping center parking lot if you need to and just sit and listen to a podcast, or some music, or write a post (like this one, which was written in a 7-11 parking lot.)
You can afford to take a little time for you, regardless of what the circumstances are. If the people in your life can't or won't allow you to do that, then it's time to do a little soul searching about the people you have in your life.