Fact of life: Women are jealous of guys' time. Nothing is going to change that, and you have to live with it unless you prefer to be Mr. Playa banging skanky hoes with no commitments. Let's assume that's not you.
It's a lot easier for a guy to walk away from a relationship than a girl when there's a kid involved. No matter what happens, there's the kid, and she's the one who's going to have to take responsibility for it in 99% of situations. This scares women when they worry that their guy is starting to drift off.
Now, with those as givens, there are two kinds of women; women who understand that a guy needs some guy time to recharge, and then he'll come back, and women who don't understand that and don't see why he doesn't spend all his time with her and the baby 24/7, 365 days a year.
So you need to put it to her; you love her, you're committed to her, you're committed to the baby, but you need some guy time or you're going to burn out. Your guy-time thing is playing in a band. For other guys it might be bowling or fishing or whatever, doesn't matter. You need to work out how you can get your guy time to recharge, without sacrificing what she and the baby actually need from you.
If she gets it, you work it out and go on to have a great relationship. If she doesn't get it, she turns the relationship into a living hell till you can't take it any more and run away with a barista named Samantha. (You probably shouldn't tell her that part when you have The Talk. Women take it badly. Find a diplomatic way to get the idea across).
If there are outside figures she respects - parents, a pastor, older married friends or co-workers - it can help if they're the ones who give her the hint to let her guy have a little guy time.