I haven't been keeping track these past few months I've begun recording, but are you not supposed to take out the mic while the phantom power's still on? The proper procedure is to power it on after plugging in and powering off before, right? Have I been doing any serious damage if I've done it the wrong way a few times?
Electrically speaking, taking out a powered mic can damage the mic IF when you pull it out, one side is still powered up. Same with plugging in. The diaphram normally does not see any voltage potential between the 2 pins UNLESS one pin is grounded and the other pin is +48vDC. Then there is +48vDC accross the diaphram.
Normally, if you pull the mic out with PP on, one side would be floating and the other side could have +48vDC connected. Technically, that would not harm the mic. The one scenario that will harm the mic is if the cable had pin 2 or 3 shorted to ground. The way I see it, if this were the case, you would harm the mic when you turn PP on anyway, so make sure your cable is good BEFORE you use it.
On to dynamic mics, there is the possibility of magnatising the transformer if pin 2 or 3 were shorted to ground, again, make sure the cable is good.
So, my take on this is: Turn off PP before extracting and inserting. Why take the chance of harming the mic if all it takes is a simple power up/power down routine? And more important, test your cables and keep them in good condition. If you like to take chances, by all means there are plenty of recording engineers that have, and do plug in mics with PP on. Most electrical engineers would not.
I forgot to mention that virtually all modern Phantom Power circuits have self-limiting resistors that will not allow a shorted pin 2 or 3 to kill the mic, but there are plenty of older consoles and mic pre-amps that are not limited. You are much safer with the limiting resistors. This still will not save a ribbon mic much.