I've gone Geocaching a few times with the wife. I'll throw a couple points on my laptop that has a built in GPS and take it with when we go camping. It's nice because it gets us to see areas that are off the beaten path and usually pretty places.
Geocaching is pretty cool - I had a crack at it while researching an article on urban sports. It's a lot of fun, though you have to be in the right mood for it. Anything that turns city streets into a vast pac-man grid as you home in on a location on a handset has to be cool.
What's with all the "lonely" jabs? For all the geocachers I met it was a very social activity, going hunting in pairs (even dates) or groups. Then there's the cache & trash meets, where geocachers from all over will congregate on a GPS point at a certain time and just tidy the place up.
I tried this out a few years ago and eventually decided it wasn't really for me. I think if I had known a few other people who were into it and could have gone out in groups it would have been a lot more fun. As it was I was usually in a group of me and one or two other uninterested parties I dragged along without their own GPSr.