On the subject of you either have it or you don't ... I definitely didn't have it when I started bass around 1986 at 13. I'd been playing violin in school for five years and I knew a little about reading the wrong clef. When I started bass I literally couldn't hear the bass, had no rhythm, and didn't understand harmony or theory. But I've been at it for 35 years off and on, had a few years of legit training in school and lessons, spent time playing with people and records, and learned to routinely critically assess my work. As a result I have a heck of a lot more of "it" than I started with. Maybe the most important thing is hearing what you don't have and thinking carefully about what you have to do to get it. I know there are abundant naturally gifted musicians that have everything I have now and more from the get go. More power to them. But through grinding away, listening, owning my deficits, and occasionally working really hard to prep for gigs I wasn't good enough for, I've gotten to a place where I can do what I need to to play the music I want to with people I like (good people, talented musicians, creatives). If you want to get it, and you are willing to work hard and stick with it, you can do it. And if you're willing to work harder at it than I have, it shouldn't take you 35 years.
On "use it or lose it" ... this certainly applies to chops, but I don't think it applies to listening, understanding, and feel. Sure, muscles atrophy if unused. But wisdom and judgment, once earned, are yours forever, and those matter far more than chops.