The thing is, you don't need to record at 192khz (though most interfaces offer it nowadays, though it really is just a marketing ploy)... I've owned several audio interfaces that can run up to 192khz, and what sample rate do I always chose to record at? 44.1khz. This isn't because my computer cannot handle it or I don't enough room on my hard drives, it is simply because there is no benefit.
Note the word
theoretically in gecko's post. What does this mean?
Well, according Nyquist–Shannon sampling theory, the sampling frequency needs to be just over double that of the highest frequency in the signal (often called the Nyquist frequency). If this is the case, then the original signal can be completely, accurately and faithfully reproduced from the digital signal through the clever though mathematically-solid interpolation that goes on in all digital-analog convertors (I don't quite understand the exact mechanisms of the signal reconstruction myself, but it works!)...
Therefore, a signal sampled at 44.1khz will be able to be perfectly reproduced, assuming the original signal was bandlimited with nothing over they nyquist frequency. Human hearing is generally quoted as being in the range of 20hz to 20khz, though you lose the high frequencies with age, and therefore a 44.1khz signal should be all you need. Increasing the sample rate just allows higher frequencies to be sampled, but you can't hear these anyway so why did manufacturers start pushing higher and higher sample rates?
Well, the sampling theory is exactly that - theory - and we all know that putting that theory into practise is never quite as easy as it seems on paper
As I said, the original signal being sampled cannot have any high-frequency content over the nyquist frequency... if it does, then the signal will not be sampled frequently enough to record these high frequency sounds and they will become 'aliased' and reconstructed as audible lower-frequencies. This diagram stolen from wikimedia illustrates this quite well...
So low-pass filters have to be incorporated into the analog-digital converters in order to filter out any frequency content over the nyquist frequency before it is sampled. This is where the problem lies. It is very difficult/impossible to design and build an 'ideal' analog filter that would act as a brick wall to anything above the nyquist frequency, so in reality these filters had to more gently roll off the high frequencies.
The designs of these filters were 'less efficient' than they are nowadays and had a noticable effect / introduced artifacts for very high frequencies. For this reason, converters with higher sample rates were developed that pushed all this filtering mumbo-jumbo into the inaudible range. For example, if you sampled at 96khz the nyquist frequency would be 48khz, so you could have a filter that gently rolled off the highs up to this point without having to worry about intruding on the audible signal.
...but then came along oversampling!

I think its safe to say that oversampling is used by most ADCs nowadays and it really is a neat idea... the original signal is sampled at a very high sample rate so that the analog filter affects frequencies way above the range of human hearing, but then an ideal digital filter is used before the signal is downsampled to a more sensible rate (e.g. 44.1khz).
You'll find many converters in interfaces nowadays use 128x oversampling... a 44.1khz signal might actually be obtained by sampling the original at ~5.6mhz before filtering and downsampling! The previous technical limitations requiring a higher sample rate are gone. As an offshoot of this, it is all too easy to offer interfaces with sample rates such as 192khz, usually simply as a marketing ploy.
Some people argue that a higher sample rate preserves really high frequency content that we can't hear but we can feel... well, we can't hear it, but I doubt we can feel it as well - that point is debatable. At the most I would say you are just more likely to annoy some nearby dogs! And remember that most mics have a frequency response that starts to tail off long before 20khz, as do most speakers.
I agree that it is possible there might be some benefit to, lets say, running a soft synth at a higher sample rate then downsampling - kind of a DIY oversampling. I'm sure a null-test could easily verify this - truth or otherwise - and it has probably been done many times, but its areas like this that people just don't seem to be able to consistently agree on.
So forgetting all that debatable stuff, what obvious and guaranteed benefits do you get from recording at 192khz?
Well, mainly its just much larger files and much more strain on you computer / lower possible track count
Joking aside, the only benefit I can see is to obtain much lower latencies. If the maths in my head is correct, at 44.1khz and having your ASIO buffers set at 512 samples will give you a roundtrip latency of roughly 10ms. At 192khz this is reduced to around 2.5ms. Not a great advantage considering you experience larger delays standing 4m from a speaker, but at least it is an advantage!
Basically, don't get too hung up on sample rate. You'll probably end up running it at 44.1khz anyway...