Dynamics are the difference between loud and soft. The dynamic range of an instrument or voice or song is the difference between the loudest and quietest parts.
The dynamic range of noises and your ear's ability to hear them exceeds the range of a lot of recording and playback gear. Your ear has about 120db of range between silence and pain. Until recently, the range of recording gear and playback equipment was under 100db, and radio for instance is still well below that. Lots of music wouldn't "fit" in it, so the dynamic range need to be controlled.
Compressors and limiters reduce the dynamic range of a signal. A level threshold is set via a control, and any signal which goes above that that threshold is reduced by a ratio, also set via a control. Instead of stopping the signal exactly at the threshold, a compressor reduces it only by a certain amount. This tends to sound more natural than limiting at agressive levels, as limiting basically stops the signal dead at the threshold.
A limiter does the same thing, but at higher ratios, up to a point where the signal will not cross the threshold at all. Most compressors can be set to limit, but generally don't perform as well as a dedicated limiter at those settings.
Compression and limiting have a few basic uses. They can control the dynamics of a single instrument, making for instance a bass track sound more even. Say your bass player lays down a perfect track, except for a few notes that stick out. A compressor can reduce the volume of those few loud notes, saving the track, making it usable.
They can control a whole song, limiting it's dynamic range. And they can do the same for a whole album, and also increase it's apparent volume. Limiting, an extreme form of compression, is often used on whole projects, and to protect against a peak in a track or song clipping a recorder.
Besides their practical uses, compression and limiting have artistic uses as well, as many compressors and limiters have certain characteristic sounds such as distortion and smearing when driven hard. That's a whole other story, though.
When well-used, they are indispensable. When poorly used, they will ruin a project or mix or gig like few other things.
As far as when to use each, compression is basically used for making things sound better, and limiting is used to keep a very firm grip on how hot a signal can get. Both can be used on the same track, for instance. Nothing is hard and fast, though.
An expander, on the other hand, expands the dynamic range. The lower end of the dynamic range is usually determined by what is called the "noise floor". The noise floor is where the signal to noise ratio gets to a point basically where the noise starts to interfere with the desired signal. The noise floor can be determined by tape noise, electrical hum, environmental noise, etc. An expander shuts the signal off when it drops below a certain threshold.
Say you are recording a vocalist, and he sings this song, and gets very quiet at one point. That's fine, but when he pauses, you can hear an air conditioner outside in the background.
You can get him to track it again, telling him to do the quiet part louder, so his voice is louder than the A/C by enough so you don't hear it. You can mute his voice when he isn't singing (this is really manual expansion). Or you can set an expander to shut the signal off when he isn't singing.