I'm presently awaiting delivery on this Behringer 1622FX Mixer
http://www.musiciansbuy.com/behringer-x1622usb-.html
$179 16 Channels, Effects, Mic pre-amps, USB connections to get you started and when you want direct inputs to your PC just get a Sound card that will handle it.
I presently using my Korg Pandora as my DI into my PC but the 1622FX is what I'm going with because it gets me up and running and I have plenty of room for growth in the future. This lets you connect everything (Bass, guitar. Microphone, MIDI controller, etc..) and not have to swap cables every time. Next purchases will be a good quality sound card and monitors.
I use Audacity for quick and dirty edits on MP3's, but you should check out Reaper. It's free to try for 30 days and only $60 after that and they have it on the honor system for paying. The software is fully functional and extremely capable for a beginner or hobbyist.
Additionally, if you use Reaper you can get online and use Ninjam for online jamming. The client is built in. and your jams are saved on your machine so you can mix them later if you so desire. But you need to delete regularly as they eat up a lot of HD space.
Anyway, that's my 2 cents worth. I sure others may have some better solutions for you.