MP3 Encoder shootout: Help!!! Need test material!

[regebro]

I'm having trouble finding material that will show a quality difference in 128K encoding. So far I have only found one, which is a song called "Lidocaine & aspirins" by "Melanies". It's a straight and pretty normal rock-song, but with excellent production.

I found the song on a reference CD in a HiFi magazine, but non of the other recordings (and I tried pianos, symphony orchestras, ethnic, electronic, blablabla) made ANY difference between most of the encoders (Blade not included. Boy does it suck).

I had some songs which 56K or 64K encoding with Lame had produced interesting phasing effects. So I tried these. They all sound marvelous in 128, no matter what encoder. I'm going to check out some more high-energy, slick production rock'n'roll, but to make life easier, I'd like you do send me material that YOU have noticed encoding artifacts in with 128K encoding. If you have something you know are problematic, rip it from the CD to a WAV and send to me!

Contact me, on lennart@liberal.se, and YES, I'm in a hurry. I need this before the weekend.

 

[Slackmaster2K]

Dude, if you can't hear the encoding at 128kbps then you need to listen more closely. Is that what you're saying? That 128kbps MP3's sound as good as the originals, or that all encoders produce good 128kbps MP3's?

I have yet to hear a song encoded at 128kbps where I couldn't hear some negative effects from the encoding. You should prove this to yourself by doing a real A/B comparison. That is, take a nice dynamic song (maybe a slow blues number) and rip it from a CD to an MP3. Then play the CD and the MP3 at the same time (as closely as you can get them to synch) and switch back and forth between the two. You'll have to do this through an amp with multiple preamp inputs (like a home stereo receiver).

Pay VERY close attention to the cymbals...especially the ride. This is where you'll hear some crazy flange-like crap. It will be especially apparent if you use BladeEnc which has got to be the worst 128kbps encoder out there. Compare the same song encoded with a better encoder like SCMPX and you should hear the difference right away.

Other than the obvious problems with MP3 encoding, you should also listen for the stuff that people can't describe just right....you know, using stupid words like "sparkle" "depth" "openness" "warmth" "punch" ... blah. The *goodness*!

I guess that if you can't do an A/B comparison, the only thing to do is to listen to familiar music in MP3 format for a great deal of time....say a couple weeks...let yourself get familiar with it. THEN pull out a CD or a record of the same music and prepare to be impressed.

Listening to 64kbps MP3 encodings and then listening to 128kbps ecodings is not a good thing to do. It's kinda like the old "why do pretty women always have ugly friends" thing....

Slackmaster 2000

 

[pglewis]

In general, I'd look for material that would expose the weaknesses of MP3's methods. Slack mentioned cymbals: they're a great place to find how well the high frequencies are reproduced. Something with crisp hi-hats and cymbals will easily expose the enocders' weaknesses in that area. Since low frequencies are not particularly directional, many encoders will clobber any stereo seperation of the low end (if joint stereo is used). There are some Beatles tunes with the bass all in one channel that might be good to test what the encoder does. Also, dynamic rock material is good to test. Surprisingly, classical is fairly easy for a lot of encoders to deal with. Rock tunes with both quiet and LOUD stuff can expose weaknesses.

You will probably find that the encoders vary even on the same piece. One encoder might be better on one part of a song, and a different one better in another part of the same song. Especially at 96 or 128kbps. Above 128, the playing field seems to be more level.

I'll save my opinions on the individual encoders until after you've finished the tests to avoid tainting the experiment .

[P.S.: make sure you have the latest version of LAME, since they're constantly making improvements: http://www.mp3dev.org/mp3/ ]

[Edited by pglewis on 10-18-2000 at 13:04]

 

[regebro]

Slackmaster: Your ideas on how to test is very good, and of course, this is exactly what I'm doing. Not only do I do A/B teste with the original and the MP3, I also do A/B tests with the different encoders. And nope, I can't hear much of a difference on anything in 128 (BladeEnc excepted).
Lame, Fraunhofer and SCMPX all seem to give perfect results to me. Sometimes I think I can hear differencies, but then they all sound the same again...

At least, I seem to be normal in this. Yesterday, swedens largest newspaper showed a test they had done with AudioActive (Fraunhofer). People were completely unable to tell the difference between CD and 128kps encoded material. So at least I know I'm not alone in this.

Pglewis: Yeah, cymbals I noticed myself. It seems to me that what is hard to encode is sounds with high informational content, i.e. high volume and loads of high frequencies. I actually have made some tests with white and pink noise, and tonesweeps, which were VERY interesting...
If you can give me hints on how to create more of these "measurable" tests I'd be very happy.

The "beatles" test was a good idea. I could definitely hear a loss of stereo definition in the lower speeds of some encoders. I have Seargent pepper on CD I'll use it! And I'll do one with generated data too.