Timmy -
Something is not right with your system. The Layla is reputed to be a pretty good sound card. If you are recording through the Layla at 24 bit and 96.0 sample rate, then either there is something wrong in your setup, or your "upstream" stuff is a problem.
The quality of your recorded sound is going to be essentially a function of your mics, pre-amps, and a/d converters (leaving aside talent for the moment). Since I'm led to believe the Layla has decent a/d converters, why don't you tell us what mics and pre-amps you are using. That is likely to be where your problem lies.
Approaching the problem from the standpoint of trying to bring a digital signal into the Layla is not the way to go. You should be fine with the Layla and an analog signal. Let's work on the quality of the analog signal your are providing.
Also, when you say you are lacking the "crisp" sound of other recordings, what exactly do you mean?