To learn theory, in additon to the above stuff, get a copy of Edly's Music Theory for Practical People. Look it up here...
http://www.edly.com/mtfpp.html
I like the book because it doesn't assume you know piano, it doesn't assume you can read standard notation (but he makes a cogent case that it's hard to talk about music wihtout knowing the language of music), and he doesn't assume you know the history of either western classical music or jazz. His examples are from a lot of different kinds of music- classical, jazz, folk music (not just Appalachian folk music), rock, blues, etc. His diagrams are very easy to understand and in many cases are like more legible versions of what I used to draw for my students back in the '80s.
Then learn basic chord theory. I think it's utterly pointless to get a chart that tells you where to put your fingers for arpeggio studies. Learn what the chord is, find those notes everywhere on the neck and play them while singing the notes. That way you really learn the arpeggio because you're learning its sound, its construction, and its location. Those books and charts of arpeggios only teach you the location, the least important part (least important because its useless knowledge without the other two).
John
