Blind Test - Recorded Tube Amps vs Amp Sims

Aramis-V

Aramis-V

New member
Hey guys!

I'm doing a master's thesis comparing recorded real Tube Amps with Amp Sims.
I've ran some tests and recorded some samples, now I'm doing a blind test to see if people can tell them apart.

I've got some answers from guitarist but I really need some inputs from mixing/recording engineers, so if anyone want's to try it, I would really appreciate.

Thanks a lot!

Here is the link to it:
https://forms.gle/E55VrqKFaxhrFLMR6
 
You can get a usable sound or an unusable sound out of either.
Also, the audience doesn't care, only the player would.

I've done a bunch of albums with Sims. They turned out well. No one knows the difference.

The good thing about Sims is the consistency. Something that can be tough with tube amps and microphones.
 
Aram, have you checked that your proposed MO is valid for the thesis?

One factor that might derail it is that there is no such thing as "A" valve amplifier, at least not in the guitar amp world.
Each V amp would need to be "specced out' to ensure it met its rated power output (keeping mains input constant at local nominal V) and the frequency response within a dB or two of the manufacturers specification (and good luck finding that out!)
When comparing V amp A with modelled amp B you would need to ensure that the baseline response was the same within 3dB at worst. You will, also need a decent C weighted SPL meter to keep sound levels the same (they won't ever be!)

Dave.
 
That's half an hour of my life wasted.
I spent quite a few years working for two exam boards in the UK, and as Dave says, there are substantial flaws in your methodology here. You have asked random people on a forum a question - but it's sort of asked a question. The tube amp sound is clearly a hot topic with people very split on their opinions and viewpoints, which is good. Music is always polarised. However, you've slipped into a mindset which comes across in the questions. I genuinely tried to do the survey, but I failed. You fell into a trap. If you study communication skills and the psych aspects of questioning you should have realised that intricate and hard questions that demand real effort need some placebo guidance to get people up to speed before the real test. Ignoring the fact that you want people to contribute and chose a stupidly awkward system, I'll explain how it went for me.
Went and dug up the best headphones I have, filled in the personal questions, then went back and spotted the link - just text. sighed when I realised I needed to click on each one and download the folders - many people will not download random files from the net to their music systems, by the way.

Tiny snippets. So little to actually listen to, so you get a choice. If you believe that sims are better, then the least thin example is a sim. If you believe sims are poor, then the best sounding one is the amp. Flawed I am afraid. Even worse, if the clip is not to your taste, you tend to not even listen properly, and certainly not twice. You get presented with three options and guess. After doing two, I gave up. I could not detect why A was thinner, B was more distorted or C was tonally different. The entire test is flawed and will not generate accurate responses that will give any meaningful results.

In fact, the entire premise is based on smoke oil. It matters little if you believe in so called tube distortion or simulations of it because in practice each player, guitar and entire system is unique. Perhaps even genre dependent. The only thing that matters is do you like the sound. A thesis on this is a ridiculous and unanswerable waste of educational time and effort.

If you really wanted to get lots of respondents, then you set up a cheap, simple website. You don't mention tubes till later, and let people click on a sound file and then comment - no effort. A bit of coding and you collate their data. You ask innocent question to establish experience and age, plus maybe location. You give them complete phrases and get them involved in real choices as to what they think is best. Maybe give them the clip in a track, then separate - make it interesting, and ask them settle in questions that you don't need but make the experience better. You could extract the responses relevant to your research and they would be framed. You could perhaps even use examples from well known guitar riffs over a huge time period and ask opinions. "Jeff is trying to recreate Deep Purple's famous track XXX - click here to play a clip. He has plenty of gear and recorded these versions. click to listen. Which do you feel is closest to the sound?" It's the same net effect as yours but involves no downloading, and the people feel involved. Tubes vs a sim or two and with a genre specific clip, people would get the steer as to what to listen to. You don't need to even tell them the test is amp or sim - they will hear examples and see if they fit. If you do the same with punk, then the same response might happen - essentially the sound that is best this time should be sim? Maybe?

Sorry I couldn't complete the test - but I would simply have had to guess, based on my belief that I CAN spot the difference, when perhaps I can't. PS - it's also standard practice in thesis research like this to include the university quality department where respondents can get in touch to check validity, this indicates a level of security and the validity.

Sorry I can't complete it - I did try but what a faff. Now I have to go and delete the files I've downloaded, remove them from Apple Music that loaded them up and put things back.
 
Hi Rob, I did not download and listen to the clips but I would like to pickup on one point you mentioned.

In very well respected audio and recording magazine one reviewer used try to do A/B comparisons with an AI under review and his own very good system. He would tell us that he matched levels to better than 1dB but in most cases he really could not tell base system from the one on test. When he did find a tiny preference it almost always shifted about with the music genre.
The magazine no longer does such tests.

To try to get any kind of hard data about valve v simulated amplifiers is largely an exercise in futility.

Dave.
 
It’s far too subjective for academic study. A bit like asking a dancer which is best tap or modern. The question makes no sense
 
rob aylestone said:
It’s far too subjective for academic study. A bit like asking a dancer which is best tap or modern. The question makes no sense
Click to expand...
I agree. If OP wants a vastly simpler study but still in keeping with guitar sound, might I suggest he does some well controlled comparisons on pre amp valves?

In the electric guitar community there is quite a debate as to the sonic 'qualities' of pre amp valves. For simplicity let us nominate just one type? The 12AX7/ECC83 it is afterall THE most common 'git valve'.

People get very exercised about "bright" "dull" "hard" "brittle" and other adjectives and apportion those characteristics to certain brands (without one iota of scientific fact!) Such a test must hide the brand names and each valve checked for its DC conditions because if say one valve's anode was 25V lower than most others it is out of specification.

Still not easy to do well but a shedload fewer variables than comparing amplifiers!

Dave.
 
If you stand in front of a big stack on guitar or bass it’s a feel thing. Plus the impact of that sound on the strings. Parisian walkway in a sun really wouldn’t work.
 
Aramis-V said:
Hey guys!

I'm doing a master's thesis comparing recorded real Tube Amps with Amp Sims.
I've ran some tests and recorded some samples, now I'm doing a blind test to see if people can tell them apart.
Click to expand...
The Test is pointless - for the most part people can't hear the differences - no matter how bat eared they happen to be - I can tell the difference between Tube Amp,Solid State and Modelers when playing the devices - but it's not a difference that matters -
 
Agreed, for a recording, does it really make a difference which you use? Maybe you can't make a sim sound exactly like an amp, but that doesn't mean it isn't just as good or even better for the recording. Recordings NEVER sound just like the amp in a room. Even in a reamping setup, the amp won't vibrate the strings of the guitar that was recorded. Then the recording is coming through a different type of speaker, probably in a different room and at a different volume.

As you said, the test is pointless.
 
TalismanRich said:
Agreed, for a recording, does it really make a difference which you use? Maybe you can't make a sim sound exactly like an amp, but that doesn't mean it isn't just as good or even better for the recording. Recordings NEVER sound just like the amp in a room. Even in a reamping setup, the amp won't vibrate the strings of the guitar that was recorded. Then the recording is coming through a different type of speaker, probably in a different room and at a different volume.

As you said, the test is pointless.
Click to expand...

I did some guitar tracks on a song years ago with a Soldano amp in a great room. Sound was exactly what I wanted but performance on one track was not up to par.

When years later I wanted to re-track, I was faced with a dilemma. How to get that sound seeing how I’d long since sold that amp. And I surely didn’t have access to that room. Hmmmm

For the hell of it I plugged into my Avid Eleven Rack, which by today’s standards isn’t a cutting edge modeler.

But I was surprised. Within a minute or two of fiddling with the knobs, I dialed up my Soldano Tone.
I couldn’t tell the difference between the Soldano Track and the Eleven Rack track, except for the new track being in time.

Live, I like using a tube amp. In the studio whatever works. Plus, it’s nice tracking guitars at 2am without pissing anyone off. :)
 
The shocking thing is that a university allowed such flawed research to go ahead. Even at the entry university level, forget Masters and Doctorates, you get given a supervisor who poo-poos your first half a dozen ideas sort of automatically, on the basis they cannot be researched, evidenced and verified properly. That's why they're usually dull, boring and 'acedemic' - I cannot imagine how such a first meeting went? One comment with computer connected research is to do with lifespan - keeping in mind current D/A, A/D data formats, bit depth, sample rates are constantly changing, the life span of the research might be a couple of years before it's outdated. Instead of being current, it becomes history - which is fine, but a different focus totally. It's kind of fine for a high school projct where it's to do with getting a pass, but at uni? How would it survive peer review when it's so plainly subjective - UNLESS - the purpose is hidden and he's really doing a study on human perception - where the source files have been tweaked to simulate what people think they will sound like, and bolster the opinion that old farts like me hate new stuff and have picked the thinnest sounding file because that is what we expect the sim to sound like - when the sim could be the best ones? We simply are fodder and kept from the real project - reinforced by the lack of uni quality statement. If the supervisor is Professor Smith of the Department of Pyschology and not Professor Smith from the Academy of Music, we'd have a clue?

If this is a genuine research project the poster is a bit clueless. Downloading files from strangers is pretty stupid - I did, now my machines are stacked with protection and slower - a bit of a test of how my new post ransomware attack works.

For those that didn't download them, they're all very short, quite unimpressive guitar things - but odd. I couldn't decide if they were the same files parallel recorded or different versions, too short to tell, outside of a DAW, and the systemj he picked has you flitting from the playing app on your machine to the text, then back to the folders - would a dedicated website where you could be nice to the people you are using as source material be nicer as an experience. I bet many gave up like I did.
 
On the subject of "research"? As a keen follower of QI they often come up with the weirdest, most apparently useless things that scientists have been getting money to investigate!

There was a similar piece in this weeks New Scientists. It seems elephant seals with larger harems die sooner than those with smaller ones? Anyone surprised they are shagged out?

Dave.
 
You must log in or register to reply here.
Back
Top