If you need a mic, I recommend the Samson CO1U USB condenser mic. It plugs straight into a USB port on your computer so you don't have to worry about funny mic plugs or phantom power. I bought one the other day to record vocals on and it's quite good. Cheap too. It cost me $130 Canadian. The only thing is, it stinks so badly of foam-rubber that singing into it is like huffing. :(
For software, you can't go past Acid Music for bargain PC recording. It has loops, too.
If you want to go straight into the computer, I think Acid Music has virtual amps as well.