24bit vs 16bit and Hz

[gecko zzed]

And again, these focused listening tests smack of control freakism.

Maybe, but then again, any rigorous scientific test of anything could be labelled as such. All scientific tests endeavour to remove as much subjectivity as possible, and as a result, considerable control of the test is needed.

You might just as well conduct a controlled taste test and then tell people what they like to eat, despite their protests that they don't like what you say they like and they in fact taste a difference.

This is drawing a conclusion that I don't think is there. The purpose of an audio test is not to tell people what they like to hear. We've already established many times that listening pleasure is a subjective thing not bound by the rules of science.

 

[Bobbsy]

And again, these focused listening tests smack of control freakism. You might just as well conduct a controlled taste test and then tell people what they like to eat, despite their protests that they don't like what you say they like and they in fact taste a difference.

This carries your argument into the realms of silliness. We've already discussed that there are lots of things in audio that DO sound different and where the choice is a personal preference. I don't need ABX testing to know that an RE20 sounds different from a C451EB and I make my mic choice based on what I like and what's suitable for the recording at hand. It's just like the food you mention. I hate hard boiled eggs and love asparagus--that's my choice and no ABX testing is needed to prove that.

However, things like $300 mains leads--and the choice of 16 vs 24 bit and lots of other things--are a very different kettle of fish. There's a very real possibility--based on science--that any differences people claim to hear are down to autosuggestion rather than anything real. In these cases, blind testing is the only way to sort this out. To go back to your food analogy, my wife claims she can tell the difference between whether I use a hand whisk or an electric whisk when making whipped cream. My own simplified version of blind testing proves otherwise.

Finally, as Ethan points out, Meyer-Moran is not the only testing of bit depth differences. Even if their method was flawed, it doesn't necessarily make the results...or the results of others...wrong.

 

[Beck]

This carries your argument into the realms of silliness. We've already discussed that there are lots of things in audio that DO sound different and where the choice is a personal preference. I don't need ABX testing to know that an RE20 sounds different from a C451EB and I make my mic choice based on what I like and what's suitable for the recording at hand. It's just like the food you mention. I hate hard boiled eggs and love asparagus--that's my choice and no ABX testing is needed to prove that.

However, things like $300 mains leads--and the choice of 16 vs 24 bit and lots of other things--are a very different kettle of fish. There's a very real possibility--based on science--that any differences people claim to hear are down to autosuggestion rather than anything real. In these cases, blind testing is the only way to sort this out. To go back to your food analogy, my wife claims she can tell the difference between whether I use a hand whisk or an electric whisk when making whipped cream. My own simplified version of blind testing proves otherwise.

Finally, as Ethan points out, Meyer-Moran is not the only testing of bit depth differences. Even if their method was flawed, it doesn't necessarily make the results...or the results of others...wrong.

Silliness to whom? My analogy was spot on based on human nature, which plays a far bigger role in tests and surveys than you know. Autosuggestion is a social phenomenon, but as I alluded to before there's a lot more in the realm of social science as opposed to hard science in the aforementioned studies. Perception of expertise and authority is a big one; not to mention the ambitions of those conducting and participating in the experiments. It influences the audience as well.

I hear a lot of talk about these mythical controlled experiments, but after all is said and done the proponents rely more on social pressure than they do pure science. Case in point... your willingness to blindly accept Ethan's backup example in the event someone refutes Meyer-Moran. Knowing Ethan, when his backup example is also refuted he'll continue to dodge and weave like a politician, never admitting a mistake. Here's a few examples of how socio-psychological factors play the primary role in giving these studies life.

1) Some authority is established giving the experiment cred and believability. Meyer-Moran "Audibility of a CD-Standard A/DA/A Loop Inserted into High-Resolution Audio Playback" is regularly referred to in popular press as an AES study. But that's wrong. Meyer and Moran are simply members of the Audio Engineering Society. Their views are not endorsed by AES. All it takes to become a member of AES is to join and pay the dues. And then you too can conduct and submit a study for peer review.

2) Buzz words like, "Double-blind test" are thrown around, but that is also claimed on late night TV infomercials for boner pills, so...

3) There's an elitist air to the whole thing, which intimidates laymen and wannabes... makes them want to be a part of it to sound intelligent. The rest of us aren't so easily schmoozed.

I could go on and write a book about what can go wrong with these studies. Not everyone knows how to conduct a proper survey or experiment. The devil is in the details.

 

[Bobbsy]

I wonder if you're possibly guilty of the elitism you accuse others of--just in reverse.

Just because not everyone knows how to perform a perfectly valid test, it doesn't mean that a valid experiment can't be done. IF the Meyer-Moran test was flawed (and I don't know the details so I won't comment) it doesn't mean that there can't have been other tests which were carried out properly.

A couple of things:

First, I know that I am very much subject to autosuggestion or the placebo effect or whatever you want to call it. Even if I'm aware of it and try to fight it--it's there and I think I hear things just because I believe they will be there. FYI, I've discussed this with my wife (who is a doctor of psychology) and she confirms that such a thing absolutely exists and is pretty much universal. Not totally relevant to audio, but she used to do drug counselling and treatment and would often "prescribe" glucose tablets to addicts to help with their withdrawal. The placebo has a 100% success rate in easing the physical effects of heroin withdrawal and the same thing applies to audio.

Second, although it wasn't peer reviewed, I was involved in an ABX test of 16 vs 24 bit back in the early 1990s. The TV station had just invested in a digital post production audio suite and we needed to set up our own in-house standard with a choice of bit depth. One of our engineers ran the test, switching between sample rates and also throwing in the original analogue material (which came from the FM audio tracks on Beta SP). The rest of us listened on decent monitors in a room where we couldn't see anything to do with the playback. The result? Nobody could reliably detect the difference.

Now, I'm sure you can find flaws with our methodology but it was a fair as we could make it--and certainly accurate enough for us to take a decision on what settings to use and also to persuade me that there are a lot of myths out there.

 

[miroslav]

Honestly..I find this whole double-blind testing thing to be rather irrelevant.

Many things that we do and choices that we make are VERY dependent on our perceptions and NOT mathematical calculations and testing.
The point being (and one I was making earlier) is that if someone perceives something to be better/different, and they make a choice based on that perception...it can very well act as a positive catalyst on subsequent decisions, so IMHO it's the perception that is more relevant than anything else.

I'm not saying measurements and testing should be ignored and never used...but the level to which they have become important in this one area of audio on forums for some reason, I find rather puzzling...when there's so many other things we don't measure and test, yet make decisions about purely on our perceptions, and it's ALL VALID if it drives the process in a way we desire.

Example:
A woman has a pair of brown pants and black pants....they are IDENTICAL in size and fit...yet the perception is that she looks thinner and therefore better when she puts on the black pants.
Now....someone here will probably want to break out a measuring device and do all kinds of double blind tests to prove to her that they are exactly the same and it doesn't matter which ones she wears...but the question has to be asked, why is it so important to prove to the woman that they are...????
Let her go with her perception and she will enjoy her day.

So...like I said earlier....you can prove to me that device XYZ is more________ (whatever)....but you know, I really like the one with the cool knobs and buttons, and if I get that one, I will feel better about it and more inclined to use it with a certain conviction and pleasure...therefor to me it IS the better device.

It is about perceptions...most of the time with most things.

 

[Bobbsy]

Real perceptions, I completely agree with you. I've already discussed that things like mic and pre amp choice, effects choice, etc. all involve real differences and your perceptions are the ONLY way to make a choice.

There are other things though where there are no differences and, if you think you perceive one, you're conning yourself--or at least being conned by autosuggestion. The proverbial $300 mains lead is an example. For this sort of thing, the only way to sure is ABX testing and, in the case of the $300 mains lead, it'll also save you $300. I'm sorry, but I'm not one to say "It doesn't matter if you're conning yourself...if you think it's better go ahead and waste your money (or your disk space or whatever).

Your example of the black and brown pants isn't a fair analogy for what I'm try to put across. Like with different mics there is a definite difference between black and brown and therefore the perception is genuine.

The proper analogy for what I'm talking about is if the same woman has two identical pair of black trousers and yet thinks one of them makes her look different than the other. In this case the woman's perception is faulty and, if I was her, I'd want to know the truth.

As with your gear with cool knobs and buttons (personally I'd want to add flashing lights) if you choose it because you like the knobs and buttons, that's fine and dandy. However, if you extend this to say the knobs and buttons make it sound better, that's a step too far. (I'm ignoring the possibility that maybe the knobs and buttons cause you to use it better--but that's a different argument. You can't use a $300 mains lead better.)

Anyway, I'm not shooting down the importance of perceptions, just saying that I want to be careful that my perceptions are accurate.

 

[gecko zzed]

Honestly..I find this whole double-blind testing thing to be rather irrelevant.

It is indeed irrelevant if people operate on the basis of their preferences.

It is not irrelevant if people claim their subjective preference is objective.

So it is fine for people to say I like this because it is bright and shiny and I think it sounds better.

It is not ok for people to say I like this because it is bright and shiny and sounds better, therefore it is better, and anyone who disagrees is wrong.

 

[Beck]

The proper analogy for what I'm talking about is if the same woman has two identical pair of black trousers and yet thinks one of them makes her look different than the other. In this case the woman's perception is faulty and, if I was her, I'd want to know the truth.

But I know that woman, and I agree she looks hotter in the second of the two pairs of identical black trousers. Probably because they're even more identical than the first.

P.S. Never tell a woman her perception is faulty concerning her wardrobe... even if she says to be honest and tell her the truth. (It's a trick!)

 

[bouldersoundguy]

Double blind tests are useful to find out if your preferences are based on something or nothing. If your preferences are based on nothing then they are not worth pursuing. I find it to be a time and money saver to know the difference.

 

[miroslav]

It is not ok for people to say I like this because it is bright and shiny and sounds better, therefore it is better, and anyone who disagrees is wrong.

But how much of that really happens....even around here?

It's more about someone saying something like, "Hey, I tried XYZ with my setup, and I really think it sounds better than when I was using ABC".....and then all of a sudden you get people trying very hard to convince that person how wrong HE is in his perceptions.
It's not the other way around where that person is trying to convince everyone else that they must have the same perceptions.
Seems like there's always a need for some folks to prove that math is right, and as soon as someone claims a "preference", here comes the math arguments.

I say....LET them have their preference if it drives the rest of their process...even if it is just a placebo effect.

Double blind tests are useful to find out if your preferences are based on something or nothing. If your preferences are based on nothing then they are not worth pursuing. I find it to be a time and money saver to know the difference.

OK...but tell honeslty....how many actual double-blind studies have you done in your daily confrontations with your own personal preferences about things...or do you often just go with your gut about a lot of choices?

I know you're an audio sound guy, and you probably have done a lot of spec checks/tests with gear in your work as many of us have, but the topics of audio and music (and here on these forums) are full of subjective opinions/choices and we use subjective opinions to make a LOT of decisions not just creatively, but even technically....that I don't get why with this one area of audio (digital conversion) there is this obsession with the math and with double-blind null tests...???

Example:
One might think that choosing a mic on personal preference is a different thing, like Bobbsy was saying.....but at some point, I bet the initial decision for the mic we picked was driven purely by subjective preference and hype, and there was NO testing or double-blind study done by us....ever...to disceren how/why we were using our own preferences.

"This is a Neumann and that's an Apex.....which would you like use?"
...or how about...
"This is a Gibson and that's an Epiphone....which would you like to use?"
...or another...
"This has vintage tubes and that has transisitors....which would you like to use?"

There are dozens and dozens of these kinds of decisions made every day, and I seriously doubt anyone is doing an initial double-blind test on anything, before their brains are subjectively "poisoned" by the opinions of others or their own auto-suggestion.
Sooooooo.....I'm just wondering why the conversion of digital audio has gotten so much focus when it comes to testing and proving something...while so many other areas are still working off of subjective perceptions, and it's not a problem.
And it's not just the digital conversion...it also seems to be now happening with digital emulation of real/hardware stuff...a need to prove that there's no mathematical difference. Why?
(Personally, I think it's becuase there's still an ongoing "battle" in some pro-digital camps and a need to win...but that's another topic/thread.)

While I do a lot of number crunching at times in my own SOP...I still say that even IF a perception is based on nothing, it is still a perception and can have a direct impact on a lot of other decisions (both good and bad)....and if the person's perceptions push them in a positive direction....IMHO, it validates any/all perceptions.
We're talking music and audio here....not a space shot to Mars...

 

[Lt. Bob]

But how much of that really happens....even around here?

It's more about someone saying something like, "Hey, I tried XYZ with my setup, and I really think it sounds better than when I was using ABC".....and then all of a sudden you get people trying very hard to convince that person how wrong HE is in his perceptions.
It's not the other way around where that person is trying to convince everyone else that they must have the same perceptions.
Seems like there's always a need for some folks to prove that math is right, and as soon as someone claims a "preference", here comes the math arguments.

I say....LET them have their preference if it drives the rest of their process...even if it is just a placebo effect.




OK...but tell honeslty....how many actual double-blind studies have you done in your daily confrontations with your own personal preferences about things...or do you often just go with your gut about a lot of choices?

I know you're an audio sound guy, and you probably have done a lot of spec checks/tests with gear in your work as many of us have, but the topics of audio and music (and here on these forums) are full of subjective opinions/choices and we use subjective opinions to make a LOT of decisions not just creatively, but even technically....that I don't get why with this one area of audio (digital conversion) there is this obsession with the math and with double-blind null tests...???

Example:
One might think that choosing a mic on personal preference is a different thing, like Bobbsy was saying.....but at some point, I bet the initial decision for the mic we picked was driven purely by subjective preference and hype, and there was NO testing or double-blind study done by us....ever...to disceren how/why we were using our own preferences.

"This is a Neumann and that's an Apex.....which would you like use?"
...or how about...
"This is a Gibson and that's an Epiphone....which would you like to use?"
...or another...
"This has vintage tubes and that has transisitors....which would you like to use?"

There are dozens and dozens of these kinds of decisions made every day, and I seriously doubt anyone is doing an initial double-blind test on anything, before their brains are subjectively "poisoned" by the opinions of others or their own auto-suggestion.
Sooooooo.....I'm just wondering why the conversion of digital audio has gotten so much focus when it comes to testing and proving something...while so many other areas are still working off of subjective perceptions, and it's not a problem.
And it's not just the digital conversion...it also seems to be now happening with digital emulation of real/hardware stuff...a need to prove that there's no mathematical difference. Why?
(Personally, I think it's becuase there's still an ongoing "battle" in some pro-digital camps and a need to win...but that's another topic/thread.)

While I do a lot of number crunching at times in my own SOP...I still say that even IF a perception is based on nothing, it is still a perception and can have a direct impact on a lot of other decisions (both good and bad)....and if the person's perceptions push them in a positive direction....IMHO, it validates any/all perceptions.
We're talking music and audio here....not a space shot to Mars...

^^^^ all of this ^^^^

Excellent post.

It is true that a lotta the time when someone says something really sounds better to them a ton of people immediately tell them they're wrong. It doesn't just happen in digital discussions .... it happens in guitar and gear discussions.

It used to drive me crazy how often some newb would show up and express happiness over getting their first inexpensive bit of some kind of gear and immediately get told, "That's a piece of crap!"

One guy who had a hatred for behringer would do that ANY time some newb bought some Behringer stuff and I thought it sucked and fought with him about it several times.
Why take the fun outta someone else's learning process?
It's just shitty and rude and uneccessary IMO.

Sure .... when someone makes direct claims and promotes them like the KLOPS guy that requires setting them straight but if someone just pops in and likes something better than something else why trash 'em?

 

[miroslav]

It doesn't just happen in digital discussions .... it happens in guitar and gear discussions.

Yeah...it does happen with other things too, I just notice that digital discussions seem to really bring out the math.

I use to get subconsciously defensive about my own choices by sometimes going against someone else's...but now I just defend my own choices as my personal preferences, and I don't much care what someone else uses.

I mean....there's a time when people are asking for opinions about "XYZ"...and then it's fair game for everyone to put something on the table....but when like you say, someone is gushing about the new thing they just picked up, then it's poor form to go in and point-blank tell them their perceptions were all wrong, and you got the math to prove it, and they got taken for a schmo and wasted their money...etc...etc....

 

[bouldersoundguy]

While I do a lot of number crunching at times in my own SOP...I still say that even IF a perception is based on nothing, it is still a perception and can have a direct impact on a lot of other decisions (both good and bad)....and if the person's perceptions push them in a positive direction....IMHO, it validates any/all perceptions.
We're talking music and audio here....not a space shot to Mars...

I would say it's more a matter of psychology than music or audio when perceptions are being altered by beliefs.

 

[Ethan Winer]

I say....LET them have their preference if it drives the rest of their process...even if it is just a placebo effect.

Sure. The only time I object is when someone asks honestly if he'll benefit from spending his limited funds on [whatever], and someone says Yes even though there's no objective evidence that Yes is the correct answer.

--Ethan

 

[Ethan Winer]

It used to drive me crazy how often some newb would show up and express happiness over getting their first inexpensive bit of some kind of gear and immediately get told, "That's a piece of crap!"

It appears that you and I are in fact on the same page. That's exactly the same motivation I have for wanting to stamp out audio BS and elitism. Just replace "Behringer" with "16/44".

--Ethan

 

[Ethan Winer]

as Ethan points out, Meyer-Moran is not the only testing of bit depth differences. Even if their method was flawed, it doesn't necessarily make the results...or the results of others...wrong.

Exactly. In their original AES article Meyer and Moran reference three prior tests that also concluded 44.1/16 is transparent enough, and those three are in addition to the 1984 test I linked to earlier.

I just emailed E. Brad Meyer to ask him how the HHB "bottleneck" was connected, and hopefully he'll answer. It's not clear to me how you'd even do that, since most recording devices just echo their analog input to the analog line output jack.

--Ethan

 

[miroslav]

I would say it's more a matter of psychology than music or audio when perceptions are being altered by beliefs.

I agree...but it's not always all bad and necessary to change one's view, just because someone has a math formula for an alternative perception.

IOW...if it makes you feel good, it's a valid perception...for you.

 

[Lt. Bob]

people can trot out all the math in the world but the decoding of what comes in thru the ears is done in the brain.
Therefore if you believe something sounds different then it really can regardless of any measurements.

This actually argues favorably for BOTH points of view.
First it argues for the measurement geeks that want to point out that what you're hearing is only different because you believe it is. It absolutely can be.
But it ALSO argues for those that want to insist they actually do hear a difference regardless of the reason because they do regardless of the reason.

Check out people with synesthesia ..... clearly there aren't really musical notes floating in the air before their eyes ...... but they see them and all the proving there aren't really any floating notes in the world won't take those things away when what their brains have decided they're there.
They see them just as surely as I see this keyboard.

There are many other forms of synesthesia and clearly these are extreme cases but the fact is that when our brains decide that something is different then it IS different to us regardless of any math you may perform.

 

[Ethan Winer]

But it ALSO argues for those that want to insist they actually do hear a difference regardless of the reason because they do regardless of the reason.

Sure, and that's fine. If looking at a fancy front panel make you feel better even if the sound quality is no better, that's fine with me. Again, I object only when people tell others they'll never get pro results unless they drop a lot of money on mic pres and converters. That simply isn't true.

--Ethan

 

[TripleM]

There are many other forms of synesthesia

I haven't read much about the subject, but I'm told I have a form of it. I associate letters and colors. For example, if you ask me to think of the letter "A", I will simultaneously think of the color red. Ask me to think of "H" and I associate grey. I never knew there was a term for it until we got to talking about it one day. Someone told me I had synesthesia.

But as I'm reading, I don't pick out individual letters and associate colors. I'm concentrating on what I'm reading. Oh well, back to the thread. I believe 24 bit sounds 8 times better than 16 bit.