The problem is a ground loop. You can die from this, believe it or not, so take it seriously. A missing ground in the system is only one of the reasons this can happen. The problem also occurs when the guitar amp and the PA have different paths to ground, or one has the hot and neutral legs reversed. Remember, electricity follows the shortest path to ground, so if the shortest path is through you and your guitar, that is the path it will follow. You want to make sure this does not happen. Both systems (guitar and PA) need to be on the same power service. Ideally, audio has a dedicated service, and the guitar amps need to be plugged into that service. On large tours they carry their own power distribution, which supplies both the PA and the backline. A well designed music venue will have a similar, dedicated, audio power system.
It is always a good idea to have a plug tester on hand, and I always bring a ground lift in case the wall outlet is two pronged. But this is NOT enough. You also need to get a decent (not great, but decent) multi-meter. I think it is a good idea to get one which is self leveling, as you don't have to search for a problem. Once the show is set up, you set the multi-meter to test for voltage A/C, touch one probe to the mic, and the other to the (plugged in and powered up) guitar. If you see ANY voltage, STOP and find the problem before you do anything else.
I actually knew a blues guitar player once who actually liked it if we could get about 12 V between his mic and his guitar. He said it gave him a little edginess to his vocals. He did not get it most nights, because it is usually 120V or 0V, and nothing in between, but every now and then he got lucky.
Then there was the guy a friend of mine was working for. He was the bass player for a group who was touring clubs in support of their first major label album back in the early 1980's. This guy had just bought himself a brand new live bass rig with some of his advance money, and my friend (who was his tour manager and FOH engineer) said something like, "great, leave it here and we'll set it up for you." This guy said he wanted to learn how to do it himself, just so he would know. My friend led him through the whole thing, and then said, "But we have to make sure there is no voltage running between the rig and your vocal mic." He took out his meter, and showed the guy what it was supposed to read. This guy then said, but what would it look like if it was wrong, so my friend reversed the polarity of the amp, and show the guy. My friend then said, "But if you don't have a meter, just hold your bass without touching the strings, and touch the strings to the mic. This guy gets all excited by this, and grabs his bass and, before anyone can stop him, touches his bass strings to the mic, with the rigs polarity still reversed. There is a very big bang, the mic stand goes flying, and smoke fills the air. This bass player puts down his bass, and says, “You can take care of this from now on." He leaves, and never tries to set up his own gear again.
Buy a multi-meter, and ALWAYS use it.
Light
"Cowards can never be moral."
M.K. Gandhi