Think back to school electronics - a circuit needs two conductors, out and back if you like. One of these could be a 'tube' of wire, surrounding the other. Call it a screen, or a shield if you like, trying to prevent electrical noise or interference being heard. Guitars are the most common musical bit of kit that uses this system. It's unbalanced because the screen is different to the centre conductor. Hi fif connections use the same system with different connectors - the usual smaller phono(RCA) connectors. It also means the shiled/screen is doing double duty because often it is connected to the ground connection - to make it even better at shielding.
The other system, used mainly in microphone circuits is to have the audio on two small conductors and then surround BOTH with the shield/screen connected to ground. This is a balanced connection. The basic idea is that external noise/interference lands on both identical conductors and fingers crossed, it cancels out like magic. You have in essence, 3 conductors in balanced connections (like XLR cables for mics) but only 2 in other connections. Here lays a problem. If two bits of kit are connected together with a balanced cable, even if, because of your electrical wiring the grounds are slightly different mains derived hum gets onto the screen, audio wise, nothing happens. The two audio carrying conductors are separate. However, if at some place there are unbalanced devices - this hum is being carried on half of the audio circuit and it gets in. 50Hz here in the UK and 60Hz hum in the US. A consistent, continuous low frequency hum. Nowadays those nasty cheap wall wart type power supplies make this worse by giving you potentially dozens of noise sources sticking rubbish on the audio ground. It leaks everywhere, carried by the audio cable shield/screen.
In your system, you have some devices with balanced connections and others without - if you have made up XLR to phono/jack type connectors, there's a chance you created the perfect path to let the noise in.
Worth remembering that we think of hum as that very low continuous hummmmmmmm. If you hear changing noise, this is not strictly speaking, hum. Screen displays, and even mouse movement sometimes cause different noise, that get wrongly described as hum - this is a similar process, but the noise gets carried on the screen of USB or Firewire cables generated inside the computer, but leaking into the audio devices we have connected.
The cures when going from balanced to unbalanced can be amzingly simply - often just disconnecting the connection to pin 1 in a XLR plug! It breaks the ground circuit.
For those living in the UK with our 240V mains power, we also bang on about safety. Lifting the ground wire in a 3 pin mains connector also often removes hum. It also removes the life saving purpose of the electrical ground, so here, the practice is considered stupid and dangerous and we just don't do it.
Grounding is the US of equipment is much less common as your standard two flat pin connectors don't include grounding. In a way - when you then DO have some kit grounded and others not, it's quite easy to have just a few items actually grounded to real earth, making it more difficult to work out where it comes from.