Yes, you will hear a difference.
Most home stereo speaker systems are engineered in such a way as to have an increased bass response. That's what the market wants, so thats what manufactures provide. Nothing wrong with that. And there are some damn nice sounding stereo speakers out there.
But what that means to you is that you are probably making adjustments in your mixing for that hyped low end from the stereo speaker. Then, when you play your recorded and mixed music back on a different stereo, it probably sounds "tinny" and slightly exagerated in the high end. In other words, your mixes sound fine in your "control room" but they don't translate well to other systems. A good audio monitoring system will have a relatively flat response curve.