You will NEED an interface. There are many good suggestions above. The interface will come with the cable to hook it into your computer (USB, FireWire, etc.). Some interfaces will include a DI in them; these are labeled different ways: the front of my Apollo says "HI-Z". But be sure that it is an instrument-level input, and not line level; both can use what look like 1/4" inputs but the line level inputs are balanced. If your interface does not have this, then you will need a DI or preamp of some kind with XLR or line level outputs. For now, I recommend finding an interface that can accept instrument level (HI-Z) inputs.
You will NEED a DAW. The DAW is the software that allows you to record. Many interfaces will come with trial versions of DAWs so you can get your feet wet and see what works for you. There are good suggestions above, but I will through Ableton Live out there as a recommendation. There are a couple of different ways to use it, it's fairly intuitive, and if you don't know what a button does it will tell you when you hover your mouse over it. It's really a great DAW.
You will NEED some way to listen back to the recordings you make. Your interface will have outputs for monitors, headphones, or both. What you choose to use is up to you, but make sure that it can reproduce low-frequencies accurately. I love recording and mixing on my monitors, but headphones can be helpful if you're trying not to disturb other people in your vicinity. I will make a strong recommendation for Beyerdynamic DT770 headphones; they are not cheap, but you'll only need to buy them once, and they sound great for tracking bass. Your drummer will love them too, but he will sweat all over them
That's all you need.
If you want to input any other sounds then you'll need to look into what those sounds are and what kind of outputs they have. A mic is never a bad purchase (especially a Shure SM57 or SM58) but it won't be necessary until you decide to record sounds in the environment.
There are tons of great tutorials on YouTube for ever DAW under the sun. I'd recommend watching the tutorials to see which one seems like it will mesh well with you. Similarly, there are a ton of videos for recording every instrument.