The dbx has unbalanced inputs/outputs. Here is one way to do it, assuming that your signal chain is Micpre-compressor-recorder: From the line output of your unit, connect a TRS-RCA connector (1/4 TRS to 2 RCA connector Y-cable) to the left and right input (A, B) on the dbx. Then, using an identical Y-cable (2 RCA to 1/4 TRS) to connect the Line output of the dbx (it has 2 pairs of outputs; the line output is the pair closest to the top) to your recorder/soundcard.
Flip the mode switch on the dbx so it is up (i.e. on 'above threshold', and not on 'linear' mode). The big dial is your ratio. At 1.0, it will let the signal pass unaltered. Counterclockwise will increase the ratio (give more gain reduction). The threshold is the little knob.