Higher sample rate = higher possible badwidth. You can record higher frequencies with a higher sampling rate, assuming the analog electronics before the convertor will pass them.
Sound has to be bandwidth limited before being converted to digital, otherwise alaising will occur and sound bad. Higher sampling rates also allow for less drastic filters to be used for limiting the bandwidth. Audio recorded at 44.1k has to be filtered really close to the audible range, which is tuff to do transparently. Normal audio recorded at something like 88.2k has the filters well above the audible range, so there will be less detrimental effects from the conversion. To me, there is more difference between 16 bit and 24 bit, than there is between 44.1k and 96k.