Songs About Fucking is really the one you need to get---their best work.
As an aside, I met Steve Albini when I did some maintenance on his DSL circuit at Electric Audio. This was a handful of years ago. He's a great guy and if you ever get a chance to read his diatribes on the music industry you might think twice about ever signing a contract. Oh...here it is.
He gave me a free copy of 10,000 hz.
williac--I was at that concert too! I was just wishing beyond hope that the Butthole Surfers would show up.
BTW, if you're in Chicago you can probably record with the guy for around $2000. He's got the best mics in the business--hands down.
also... this is kind of stupid, but the minute I thought about Big Black I thought about their cover of "The Model" on SAF. That led me to think about awesome versions of Kraftwerk songs which reminded me of Senor Coconut. If you get a chance check these guys out for sure!
Most of what I would tell you has been said, but I wouldn't write off the early EP's. There's some fun stuff there, and if you get Lungs on vinyl and play Steelworker on 33rpm, it actually sounds pretty cool in a warped way. The vocals aren't even pitched too strangely since Albini's voice was so thin back then. But I'm a bit of a vinyl junky, and not everyone has a record player these days.
Also, this site is great for reading up on bands/muscians and seeing what they've done http://www.allmusic.com
(of course, if you do want some Big Black on vinyl rather cheap, you could try here http://www.insound.com/)
I assume you're already familiar with Shellac (his current band), but if not, do what you can to see them live.