I use GarageBand a lot on my G4 iBook (a bit underpowered for GB, but if you've got a new iMac you'll be fine). I think GB is brilliant for most home-recording purposes. It's easy to get recordings that sound quite good; the simple controls are in plain view and you can dig quite a bit deeper for finer tuning as you learn more. The big pile of loops it comes with is very handy when sketching out songs. Overall, I think GB is extremely powerful and user-friendly.
That said, while the GB amp sims are usable, an offboard unit like a POD or whatever can easily improve on them (though sacrificing the ability to change the amp on an already recorded section whenever you like). Also, using lots of effects in GB (which amp sims basically are) will quickly eat up CPU & RAM! I would tend to use the built-in amp sims when sketching out a composition, and perhaps later re-record with an offboard amp sim (or proper mic'd amp, if you've got it!) once I had a better idea of the sound I wanted.
Logic Express is unquestionably more powerful and more complex. IMO, its most useful advantages over GB are the abilities to have songs with tempo and key changes. (You can kind of fake the latter in the GB, the former is more difficult). I've tried with the Logic Express demo, but haven't gone for the real deal -- though I also hadn't realized it could be had so cheaply off eBay. ;) But there's still a lot I can get out of GarageBand, and Logic Express happily imports GB projects, so perhaps when I upgrade to a snazzy new Intel Mac in the next year or so, I'll score Logic Express then too. :) In the meantime, GarageBand gets me a long way.