You're really going to have to get us a sample. Audio people rarely EVER experience white noise, our lives revolve around reducing hiss, which is usually an HF wide band noise. We use filtered pink noise to ring out rooms with flat mics to analyse the resultant sound, but white noise is the unfiltered sound you hear between FM radio stations. Similar, but spectrally different from pink noise which is filtered to more closely relate to our hearing.
All devices produce noise - even a resistor, and this noise is usually imprecise in it's makeup and just wideband randomly produced noise - originally 'white' but the electronics after it tends to remove stuff above our hearing limits, and sometimes below it too. It's hiss. If you are plugging your amps, headphones and stuff into a battery powered device and they hiss, this could actually be perfectly normal. The key feature is the signal to noise. There will always be noise, the critical thing is how sensitive you are too it. I'm guessing that if you turn the headphone volume down, the hiss goes down too? Assuming this is the case, your problem could simply be you are turning the headphone level up too much, but at the same time have very low audio levels. There will be a noise floor to every system, and hopefully when you have a signal approaching full scale on the meters, the hiss is so low in the background that you don't hear it. Could this be the problem? Your average levels of the wanted audio are so low that turning the volume up reveals the hiss? Generate a tone at full scale - 0dB That's loud. Any hiss? I suspect not. Do it again recording the tone at -10dB, and try again, WITHOUT ADJUSTING YOUR VOLUME CONTROL FEEDING THE HEADPHONES. Report back the recorded tone level at the point you can detect the hiss. It will be quite low. Next step - repeat the same test but with gaps. full scale for two seconds, then two seconds of no tone and see if you can hear the hiss in the gaps. Reduce the level and try again. At some point in your levels, the tone will be the same level as the noise and hearing the gaps difficult. If you plot this all out on the graph you will be able to see your system's noise floor limits - and I bet there's no problem, just a poor signal to noise in your recording process that is preventing you exploiting the entire range. If you can give us an example, we'll have a listen.