Yeah, I've seen many comments on that.
As far as my programmed drums, I've gotten a lot of compliments on how realistic the programmed drums sound. In fact, a lot of them didn't even know they were programmed. But it's because I am a drummer - I've been playing for 15 years. I programmed the drums to my album to closely mimic my own playing. I spent a TON of time, doing very tedious work detailing all the accents, fills, and everything else you can imagine. So yes I agree - it all boils down to the composer. A lot of solo guitarists (or whatever other instrument) who program drums don't really know how to change, or even think about the small details for a more realistic effect. People who make those ignorant comments on how fake the drum sounds don't think of what goes into drum programming, and the composer's actual level of drum ability (or lack of it) and knowledge.
When I say I would rather just record them, I say that because 1) recording drums is easier than doing tedious MIDI programming, 2) playing drums is fun as fuck, and 3) recording drums is just as fun. Obviously, programming has its advantages as well, like changing a drum fill easily, instead of having to re-record it. But, real drums are real drums. Either way, whichever approach I make to my music, I will make it sound as best as I can, whether I program the drums or actually record them.