No doubt it goes away fast, and as we get older, it's even tougher to get it back.
As an avid weightlifter from age 21 well into my mid-40s, I developed a fair amount of muscle and had some decent poundages I could throw around. But as I headed into my 50s, minor aches and pains, along with just lack of motivation lead to me lifting less weight and less consistently. I found any old excuse to skip a workout. Needless to say my poundages were going down quicker than I realized.
At age 57 I began concentrating on the cardio and weight loss aspect of exercising, and in the last 3.5 years I did lose a lot of weight (±70 lbs), but I also lost a lot of muscle mass in the process. Just recently started cutting back on the cardio and adding more lifting sessions and although I'm regaining some strength, I'm nowhere near my old totals, and likely never will be. On some exercises, the weights I used to warm-up with back in my 20s/30s are now the weights I use for my heaviest sets now at age 60.
These past few weeks have been encouraging, though, as I have increased the weight and/or reps on many of the staple exercises...overhead press, pull-ups, etc. I'm careful not to go too heavy, and keep my reps between 20 and 12 on most lifts. Here's hoping my joints allow me to keep on pumping iron.