I'll give voice to the "it's ok to pull back" viewpoint - I played bass from age 12-25, always in a band, fortunate to play with people better than me much of the time. Then I discovered Buddhism, and over the course of a few years got more and more into reading and studying about Buddhism than load in-soundcheck-wait/unhealthy band relationships/etc., so I finally quit and left the area at age 26. Age 30, my mom died, then my dad the year later. So I moved back home to take care of things, and in the process hooked back up with some of my former band mates. Almost quit the Buddhism thing (I was on track to become a minister), but circumstances had me selling the house, saying goodbye to my bandmates again, and moving to Japan for 4 years for more school and then ordination.
But I was back into music - did some recording at a new friend's home studio, got a laptop and M-Box with ProTools, and I was on my way! Even though I was alone I could manipulate stuff we had recorded, plus after the time off (6 years) I had a new outlook on the bass and music in general, finally composing and soloing. Been playing since then, even bringing the worlds of minister and bassist closer, though it can still be tricky to navigate.
So for me, the time off was beneficial - I needed to grow as a person to grow as a musician. Taking some time off is fine, if that's what you need to do. The drummer I was working with since the '90s had been playing since the '80s - even went back to Berklee College of Music and got his degree, only to see the music industry go through a massive change. Just last year he finally decided to "quit music" - he still teaches, but at almost age 50 he finally decided to stop chasing the dream. Hoping to jam with him and record next month when I make a rare trip back home!
The other thing I will add is that I did hook up with a drummer here (in California) and start a band - it was great, the guitarist was on the same page as far as time commitment - he had a family, I can't commit to a lot of gigs due to the nature of my "job" (on call 24-7 if a member of my sangha/congregation passes away) - it was perfect! But just as things were taking off, the drummer freaked out and went back home. End of band. He came back, started another one with a different guitarist that was going great - freakout again, band #2 done. Those times playing with a band really re-energized me. But this drummer was special, and I wasn't going to go nuts trying to find someone like him - instead, I accepted that we had a nice musical relationship and if he makes it back out here we will play again, if not I'm content not to play with a band for now. I still play though! Working on a solo set with 8-string bass, Taurus bass pedals and voice...