I enjoyed Musicophilia, along the lines of Sacks's other books. The thing that fascinated me is that there are people who simply lack major functions that seem essential for music. For instance, some people have no sense of pitch at all, some can't perceive music in any form. [0] And Sacks is careful to point out that these are people who have no apparent physical lesions or other noticeable abnormalities in their brains.Of what I've read, this was the better overview: This Is Your Brain on Music by Daniel J. Levitin: 9780452288522 | PenguinRandomHouse.com: Books
This one is also fascinating, but it's more of an oddball collection of music-related brain conditions: Musicophilia | Oliver Sacks, M.D. | Author, Neurologist | On The Move, Hallucinations, Musicophilia, Awakenings, The Man Who Mistook His Wife for a Hat
I lean towards your view that musical "talent" is at the very least a collection of multiple factors, that would make it hard to pin down scientifically. Yet we all have a good hunch that talent exists: A world without it would be hard to explain away, given widespread examples of musical prodigies and geniuses.
[0] This thought came to mind when reading the thread about people who join jam sessions but apparently can't perceive tuning or make coherent music. It may be that they have learned the social behaviors of musicians and audience members without any way of understanding what those behaviors actually relate to.