Basically a MIDI synth is anything that can respond to MIDI messages by playing notes and sounds... and a MIDI controller is anything that can be used to send MIDI messages to a device that wants 'em (like a MIDI synth)...
A MIDI synth doesn't necessarily have a keyboard, it can be just a box -- or even just a software program.
A MIDI controller can also take other forms like the Roland V-Drums or the Starr Labs guitar controller... or a MIDI footswitch...