I've always heard that mixing with headphones is dicey at best due to the fact that the stereo field is completely different than with speakers.
The stereo field is certainly part of it. Listening with speakers, each speaker is actually heard with both ears but with a tiny delay between one and the other. With headphones though, each ear gets only the sound from
the transducer clamped beside it. The alters the perception of the stereo field.
(As an aside, if you ever get a chance, have a listen to a binaural recording--one made with two mics spaced as if they were ears and with a simulated head between them. Listened to on headphones, THIS technique gives a stunning impression of "being there" albeit not really a technique suitable for most music.)
However, besides the stereo field, using headphones also interferes with the frequency response. In normal hearing, some of the sound travelling through the air is focused directly on the ear drum by the shape of the ear--but other sounds (particularly low frequencies) are partially received by causing minute vibrations in your actual
bones. Your brain is used to piecing all this together to make the sound you consider
natural. However, headphones clamped to the side of your head interact with your ears in a different way (and ear buds are even worse). Good manufacturers try to compensate for this in the frequency response of their headphones but this can only ever be an approximation since every head and ear is different.
There are lots of papers out there about the physics of hearing and the psycho acoustics involved--I won't pretend to be
expert but some make fascinating reading.
Anyway, all this is at least partly why many people think everything sounds better on headphones--and it often does. However, for mixing you don't want things to sound artificially good. You want the "real thing" warts and all.
As often discussed, you can teach yourself to mix on headphones but it's never as satisfactory and a decent set of monitor speakers.