There's a common misconception about playing to a click. A click does not equate to a lack of groove or feel. It gives you a reference point to drop the beat on top of, right on, or on the back side of. It's the drumbers job to place that beat in the appropriate place, which they commonly screw up.
I agree. There's nothing wrong with playing to click. If you're recording a good live band, then record it with no-click. If you're recording yourself and doing all the parts yourself, recording to a click is the only way to go. It makes everything so much easier. Don't be a groove-snob. If you need a click, use it.