I know what you're saying, but with many headphone amps, and some plug-ins, you can crossfeed signal with a frequency rolloff and .7 or so millisecond delay to simulate the experience of hearing in a room.
I've mixed like this, and if you are careful, you can get a very good mix without ever hearing it on your nearfields.
Grace makes an amp with the crossfeed feature.