Rounding errors in DSP

  • Thread starter Thread starter Bulls Hit
  • Start date Start date
B

Bulls Hit

Well-known member
In his Mastering Audio book Bob Katz has a great way of describing the difficulties in explaining the audible differences between similarly spec'd audio gear from different manufacturers. He likens a piece of gear to a building with lots of windows but no doors. Each window we peer through will give us a slightly different view of what goes on inside, but because we can't walk in, we're never going to get the full picture. To me it seems a very analagous situation to quantum indeterminacy, the Heisenbery Uncertainty principle and the fact that we can never measure all the properties of a subatomic particle.

Anyway one of the things he seems particularly hot about is the area of signal degradation as a result of digital processing. Everything I do in my DAW while mixing, from a simple volume envelope to inserting a reverb plugin on a track are achieved by algorithims and calculations that result in rounding errors. Because of the many thousands of calculations that are occurring, these errors accumulate.

Different manufacturers have different ways of dealing with rounding errors, longer internal word lengths, temporary registers etc. resulting in an output signal that's affected to a greater or lesser degree depending on how 'carefully' these errors are treated. This means that all audio processing software would leave a kind of 'digital footprint' characteristic of their particular error handling regime that could have an audible effect on the output signal.

Has there been any attempt to compare the likes of Sonar, Pro Tools, Nuendo, Cubase etc. in terms of how much their native processing corrupts, deforms or otherwise affects the signal? Or is this even possible?
 
I've often posted on the problems of round-off errors for in-the-box applications. It should be possible to use the same raw WAV files in different s/w packages and complete a mix in each one, then compare the results.

Hardly scientific (since many variables will come into play other than DSP error), but it would be interesting to hear the results.

That being said, I think your ears should be the determining factor.... if the final result - whether DSP'd or not - sounds good, it IS good!
 
Good Point, and some years ago now there was a thread, that I believe was posted on this board, in regards to a Drumtest. An identical set of files played through both Logic and Digital Performer, to demonstrate the difference in the Bus summing algorithms. Not very scientific, but clearly an astonishing difference between how the 2 programs handled the same task (Logic winning out IMHO).
 
None that I can think of....


I suppose what it comes down to is just good ol fashioned experience with different equipment. I've been fortunate enough to work with a buttload of different formats over the years, so you tend to pick what you like best from each, but it's completely subjective.

What might be noise or a defficiency to me, might be an advantage for you.

If only we had a JD and Power Associates for the audio world.
 
Bulls Hit said:
...This means that all audio processing software would leave a kind of 'digital footprint' characteristic of their particular [error handling] regime that could have an audible effect on the output signal.
Don't forget in Analog there are issues with distortion also and they call that character. Generally the character and coloring seems a bit different though doesn't it. In the long run I think the BB idea to let the ears judge is a good one both in analog and digital. As DAW bit resolution gets deeper I think the nasty quantanization distortion will fade into memory. Since Bob Katz started out with 16bits I imagine his teeth are still chatterin from some of those pioneering experiences. The software apps I use sum at 32bit float (about 120dB dynamic range) but the effects buss is where the cumulative effect adds up for me - that's where I do most of the DSP stuff which uses double precision 64bit float (about 312 dB dynamic range) and I saw a plug this morning that uses 128bit - if that's all integer then that's 768dB dynamic range. So it gets harder for the brain to make out that kind of distortion I think when you have that kind of resolution.

What I usually end up hearing more than anything elae are mic, pre-amp, and performance artifacts :D
 
i thought 24 bit was 144db dynamic range and 32 bit float had a ridiculous amount of dynamic range :confused:
 
Teacher said:
i thought 24 bit was 144db dynamic range and 32 bit float had a ridiculous amount of dynamic range :confused:
Yes I've read that 32bit float had like 1500dB dynamic range on some forums too...The integer part of 32bit float is 23 bits though...23x6dB per bit = 138 actually. Then there's the dither amount (to get it out your 24bit soundcard) which varies and adds more perceivable dynamic range I don't know how much that adds in all cases, best case another 6dB worth.

I don't do DSP development and am just trying to work it out so if you have some corrections that's cool too... ;)
PS Here's the word lengths I was looking at:
http://en.wikipedia.org/wiki/IEEE_floating-point_standard
 
OK Teacher - I'm wrong here. After checking into this some more it seems there's more to the DSP implementation than just the simple integer portion (allowing for dither) of the 32 bit float data type as well as the 64 bit floats...

Anyway I'm looking at this till I get more time to get into it some more - it explains the approx 1500dB dynamic range while in the 32bit float format - and explains why it's really hard to clip in that format :D

http://www.bores.com/courses/intro/chips/6_precis.htm
 
kylen said:
What I usually end up hearing more than anything elae are mic, pre-amp, and performance artifacts :D

Yeah same here. When I stick a mic in front of my amp, the meter's bubbling away at around -50db. So much for SNR
 
Bulls Hit said:
Yeah same here. When I stick a mic in front of my amp, the meter's bubbling away at around -50db. So much for SNR

:confused:

Dynamic range is about how quiet a signal can be without hitting the noise floor, not how loud it gets. No matter how many bits you've got, the reference voltage isn't going to be different.

Anyway I can't believe even 24 bit mixing has problems with audible rounding errors. Aren't they all washed away in dither to 16 bit? What is the mathematical difference between rounding error and dither, except for the position of the least significant bit?
 
mshilarious said:
:confused:

Dynamic range is about how quiet a signal can be without hitting the noise floor, not how loud it gets. No matter how many bits you've got, the reference voltage isn't going to be different.
I guess I'd throw this in - you're talking about dynamic range of an amplifier, on the lower limit the noise floor, on the upper limit some kind of distortion factor maybe (just guessing - I'd have to research that stuff some more). I'm talking about dynamic range of a digital mix bus which is derectly related to the # of bits. I think Bulls Hit is saying his noise floor is at a miserable -50dB, hehe - I get that too somedays...

Anyway I can't believe even 24 bit mixing has problems with audible rounding errors.
You'd have to take the listening test on that one - if you here it it's there - if you can't it's not! :D
 
kylen said:
I guess I'd throw this in - you're talking about dynamic range of an amplifier, on the lower limit the noise floor, on the upper limit some kind of distortion factor maybe (just guessing - I'd have to research that stuff some more). I'm talking about dynamic range of a digital mix bus which is derectly related to the # of bits. I think Bulls Hit is saying his noise floor is at a miserable -50dB, hehe - I get that too somedays...


You'd have to take the listening test on that one - if you here it it's there - if you can't it's not! :D

No, I was talking digital, but I see what Bulls was talking about now. I thought he was PLAYING his guitar when it was -50dB. All is now clear.

As for the mix test, I've read the results, and I doubt I have better ears (actually I know I don't--I have significant hearing loss in one ear) or a better chain than the guys who kicked it around. But the lack of significant differences in the digital files seemed fairly conclusive to me.
 
this might be stupid or obvious to most. But i wanted to make sure that people understand that 32 bit floating is only very hard to clip on the file format side of things. Clipping will still take place in the conversion process. So dont think you can sit there and hit you converters above 0 DBs at 32 bit float without any distortion :D. It wont be like pumping tape.

Just so no one gets the wrong idea.

So back on the subject. This is one reason why there is a difference between different softwares. Nuendo 2 is always using 32 bit floating with its summing algorithms. Infact, this is why you should almost always have a ditherer on the post inserts on the master outputs. Even if you are recording at 16 bit.

Some software might actually have better algorithms, while some of the higher end software might just be different.

At least thats the way i understand it. I might be wrong on how the system works though so please correct me if im off. I dont tend to worry about it too much. I think obviously its going to be different if i put the same files through a different software. But i think overall its irrelevant because when your recording it, your recording things based on how your hearing it with the software your using. The minor differences i think get settled out even while your just tracking. I would be worried if the algorithms just plain out sucked though.

danny
 
darnold said:
... But i wanted to make sure that people understand that 32 bit floating is only very hard to clip on the file format side of things. Clipping will still take place in the conversion process. So dont think you can sit there and hit you converters above 0 DBs at 32 bit float without any distortion :D. It wont be like pumping tape.
It's good to re-iterate this ! Kablooey if the amount of overhead that 32bit float is cabable of hits a converter! We need a sign 'FOR INTERNAL USE ONLY' :D
 
I think some people are misunderstanding dynamic range.
Simplified definition:

Dynamic range- the difference between the highest and lowest volume level in any given sound wave.

Two examples:

Modern rock- dynamic range of maybe 3-4db (very light)
Classical music- dynamic range can reach 100db at times
 
LRosario said:
I think some people are misunderstanding dynamic range.
What makes you say that ?

Here's another example of dynamic range - that of the dynamic range of a type of media. 16 bit CDs have a potential dynamic range of about 96dB, tape about 65dB, etc.
 
kylen said:
You got a blocked middle ear - or worse? Sorry...
Mine's blocked right now (cold season) but I found this article the other day which explains a few of my seasonal problems:

http://www.houseearclinic.com/eustachiantube.htm

No, permanent damage from childhood I'm afraid. Lately I've been having intermittent trouble with tinnitus, partly from wax and partly from my screaming one month old.

I really like it when I get the ears cleaned though--it's like I have dog hearing for a couple of days until I get used to it again.

Well, in one ear anyway. I think I probably have no hearing above 10kHz in my right ear.
 
mshilarious said:
No, permanent damage from childhood I'm afraid. Lately I've been having intermittent trouble with tinnitus, partly from wax and partly from my screaming one month old.

I really like it when I get the ears cleaned though--it's like I have dog hearing for a couple of days until I get used to it again.

Well, in one ear anyway. I think I probably have no hearing above 10kHz in my right ear.
Oh well that's too bad but it seems you're getting around OK just the same. I like an ear cleaning too and that sounds right about the dog hearing for a few days - I just stumbled on this inner ear idea and my eustacian tube blockage so it's time to do another sinus flush (sorry for the biology) and Valsalva maneuver to clear my tinnitus! Take care... :cool:

Where were we - digital sound degradation ? I think I heard it once after piling on a bunch of plugins at 32bit float and burning a 16bit CD without dither - now that sounded ruff!!!
 
kylen said:
Oh well that's too bad but it seems you're getting around OK just the same. I like an ear cleaning too and that sounds right about the dog hearing for a few days - I just stumbled on this inner ear idea and my eustacian tube blockage so it's time to do another sinus flush (sorry for the biology) and Valsalva maneuver to clear my tinnitus! Take care... :cool:

Where were we - digital sound degradation ? I think I heard it once after piling on a bunch of plugins at 32bit float and burning a 16bit CD without dither - now that sounded ruff!!!

Here's an interesting thread on the topic: follow all the links too, especially to the DAW SUMM project CD:

http://forum.nuendo.com/forum/Forum12/HTML/001138.html
 
Back
Top