I don't understand you here. Doesn't DBX rely on the recording being as transparent as possible?
Input signal ---> DBX NR Encoder ---> Tape
then on playback
Tape ---> DBX NR Decoder ---> Output signal
So if there is distortion added when recording/monitoring from tape, that distortion will be multiplied by 2 (such as THD) because of the DBX NR 2:1 compander on decode. This is my concern and why I think I'll need to calibrate the machine for 499/SM900 before I can use it with any decent sounding results.
There is 2:1 and 1:2 taking place as you described but is is specific to level. dbx does not have the ability multiply low level distortion and we are dealing with exactly that: low level THD. Nor does dbx require a clean signal to operate. It's strictly looking for level changes up and down to do its job which is to keep the program content above the tape's inherent hiss levels in a compressed package that can then be expanded on playback to expand that difference in hiss level and make the hiss that much more inaudible.
Let's look at the calibration process to get a better handle on this topic.
Using an MRL tape to calibrate for 499, we are setting only the repro level in this step of the process and we are turning down those levels because the tape level is that much stronger. So, in order for our meter to still show 0vu, we need to turn down the repro level which ensures that the circuitry is now operating at at a lower THD and because there is a very reasonable amount of headroom in the circuitry before audible clipping occurs we are not concerned one iota that any audio distortion will be detected by our human ears. Once that step is completed, the MRL tape is removed from the process completely. Everything else that follows is now concerned with getting the record circuitry to deliver a signal on that tape to equal what the now adjusted repro circuit is look for. So this means that the record circuit is now going to be raised to meet this elevated standard. Are we needing to raise that level an insane amount? No. We're only raising it by 3db, so we're still below the threshold of where the recording circuitry will clip and produce ugly, audible to the human ear distortion. So in reality, the circuitry is still happy and operating cleanly which also means that the dbx circuitry is also still happy because it's not being grossly slammed by this slightly elevated record level.
Now, if we look at using 499 with 456 settings, we will see a theoretical level drop of 3 db because the repro level is turned higher expecting for the weaker 456 flux level and if the distortion was inaudible before, it will now in fact be lower still because you're stressing that circuit that much less. Now, when we move onto the recording level side of this equation with our 499 tape at 456 settings, we are going to see the same levels because nothing has changed or been adjusted.
This is why I used the 300 HP motor verses the 400 HP motor but still keeping to the speed limit laws "parallel example". Yes, you could go faster but you're not going faster because you've set the cruise control to the posted limit. The cruise control can be thought of as the level controls inside your tape machine. Because no level setting have been changed, your levels stay the same.
I realize this stuff can take a bit of time to absorb and get your head around. I didn't learn it overnight and odds are you won't fully get it completely either via reading a couple of posts here. This is why we encouraged you to try it for yourself to see what if any tangible differences can be heard/seen/experienced via simply trying the 499 at the current 456 settings. You will learn more from practical hands on experiments then by just reading about it here.
Please let us know what your finding were.
Cheers!