That's a good point. Sometimes it is useful to compare two completely different things. What is also important though is the way things are compared and even more importantly, how that comparison applies to real life. I think the VTB-1 fills a great niche and is definately a bargain given what you get out of it compared to what you pay for it. The test CD's are actually a pretty cool tool, but in the real world are very limited in their uses.
First off, when you listen to one of those CD's, what you are listening on are definately a factor in how "accurate" or "useful" the test is. If your monitors aren't capable of reproducing tiny intricate nuances and emotion, then that greatly evens the playing field between the different tests.
Secondly, a solo'ed track done with different preamps is really only partly useful in showing just how much a preamp is really capable of. T me it is much more important how a preamp sounds once that track is placed in a mix. How does it handle EQ? How much smearing is there in the extreme frequencies? Those things are often overlooked when people choose equipment. Thats why so many people ( I am one of them) reccomend testing everything you can in your own working environment. Sometime when you get a really good mix going, go through and solo all your tracks one at a time. More often than not the solo'ed track may not sound quite like what you thought, but in the mix is exactly what is needed. Preamps are even more complicated. With a solo'ed signal they may sound much more similar to another than you originally thought. However in the mix, the preamp can really help a track find a space, energy, and light of it's own. I believe that this is the reason that so many people seem to overlook just how much a preamp cna really change things. I think they go down to their local mega music store and use the same mic to test all the different preamps. The results to them sound pretty close so they fail to see what the nicer preamp really is capable of.
I haven't even strated talking about how a lot of the nicer preamps sound so different with different mics. A lot of preamps out there sound so different because they are more accurate and are more capable of really reproducing the differences between the different mics where a cheaper preamp may kind make each different mic sound a little more similar. I also haven't talked about how good preamps can react so differently with different settings, and different gain stages. There are so many things to test that the only real way to test them is in your own environment and really putting a piece of equipment through it's paces.
Thats one of the really nice things about a shop like Mercenary. They don't just hook up a piece of equipment, listen for a second, and then turn around and "endorse" it to the general public. They ruggedly test things. They try to find what a piece of equipoment is NOT good for. They use it how "you" might use it. To me, knowing what a piece of equipment won't do well tells me just as much about it as what it will do well. I hope some of that made sense