Just to be pedantic, "didn't" is probably about equally correct here. It isn't describing you in the past tense. It is describing the action (caring) in the past subjunctive and saying that if that action (caring) had not been true at one time in the past, the current action (defending) would not be continuing to occur now.
That said, from a purely stylistic parallelism POV, I agree with you. It should probably have been the present subjunctive "don't" instead of the past subjunctive. That doesn't mean it is incorrect, though, IMHO.