I just read "The Secret Life of the Grown-up Brain: The Surprising Talents of the Middle-Aged Mind", and while it doesn't talk about ear training specifically, it does offer some insight about how the brain changes as we age. At the risk of oversimplifying, as we age, we focus less on details (like, did I turn off the coffee maker, and where did I leave my keys?) and more on the big picture.
My response to this is to make sure to put ear training in context. While working on hearing particular intervals via trainers or other exercises can help, I think it's important to also do things like practice playing parts by ear, or transposing parts. It's not just that doing this is more complex, but it's a different exercise. This is just my best effort to deal with my middle-aged brain, but I feel much more energized working on more integrated activities that happen to rely on the ear than just working on training for single intervals.
I also agree with Anthony that the more I do it, the better I get. I think the cumulative benefit of using listening as part of your playing more than offsets the changes in our brains as we age.
Best wishes.