Match the speaker impedance to the output impedance of the amp. If you have an amp that outputs each channel at 8 Ohms (typical for a stereo) then
be very careful about connecting 4 Ohm speakers to it. The amp will work extremely hard to drive them, the wires can get hot and the amp can get damaged. If a non-tube amp is rated down to 4 Ohms, you're probably ok driving anything; you can drive higher impedance speakers without a problem.
If this is an old tube amp, better talk with someone who knows tube amps, because the reverse could be true. Weird stuff.
I have a large amp that outputs at 8 Ohms and will be driving a pair of 4 ohm personal monitors (
Bose 101) with it. Because the thing is huge it might be OK to drive them - but it really should have an 8 ohm load. I will probably end up connecting the monitors in series and running on one channel only, which will create an 8 Ohm load and keep the amp happy. But that kills my personal monitor mix. Oh well.
An older, better home stereo amp may have speaker connections marked 4, 8 and 16 Ohms. That would be pretty cool, as you could be sure you're matching impedance no matter what you're driving.
Do a Google search on "speaker impedance matching" and you'll find all the information you need to know.