I don't seem to have a problem with dbx nr. It's all about keeping your levels under control. I recommend never sending a level hotter than -1 VU to the noise reduction unit or you'll get artifacting and breathing/gasping.
Sharp jumps in level, too, will cause errors. dbx NR is not for tracking drums or hand percussion.
Since I play drums, it should come as no surprise why I don't like dbx. We are on the same page regarding drums.
I disagree. My 34b sounds miserable without dbx nr. With ATR tape and dbx engaged, I get zero artifacts and dramatically less hiss. Almost no hiss at all, actually.
"Miserable"? As in noisy? As I stated, analog w/o NR cannot be dead silent like digital. But unless you listen to music at high volume levels, the hiss is not a problem. For high volume listening, then I agree that NR provides audible benefit. Of course with dbx employed, hiss will disappear. During silent passages the dbx suppresses hiss very well, more so than Dolby. But when there is music playing, such as plucking of strings, or solo vocals, the background noise is quite audible w/ dbx. Each time a string is plucked, the compander rapidly changes the playback gain. The hiss volume rides up & down w/ the string plucking. This, to my ears anyway, is much more annoying than a steady consistent level of background hiss.
Dolby exhibits much less of this modulation effect. When a piano solo, or guitar picking passage is playing back, the background hiss is low & steady. At high volume it is audible for sure. With dbx, the hiss is modulating w/ the music. But in silent passages dbx reduces hiss even more than Dolby. That is my point. It's a matter of which bothers the user more. To me, live music is the reference against which all audio equipment is judged. When listening to a live performance, there is a steady background noise, such as the air conditioning, people rustling their programs, etc. Live music does not have modulation noise riding up & down in volume w/ the music volume. The residual noise w/ Dolby is akin to normal background moise encountered w/ live performances. The noise incurred w/ dbx is not natural.
You're also using a quarter inch half track machine, which is a format that typically has a very low noise floor to begin with. For Tascam's multitrack decks (1/4" four track, 1/2" eight track, 1" 16 track) dbx noise reduction is a godsend, provided you've got your levels right.