Steenamaroo
...
Can you turn down guitar volume pot, interface input pot, and all the options (input vol, gain, output vol) on the amp sim, to the point that your signal is clean?
If so, take all other effects out of the loop to make certain there's no other compression or limited involved, and make sure that your guitar track, and master track, in your software are no where near clipping,
then gradually raise everything bit by bit until the problem appears.
I'm still betting that some stage is just too hot for the next stage and it's going to be a case of reducing some level,
most likely input level at the sim or input gain on your interface.
If that doesn't help then I'm really not sure. I'd guess it's just a quirk of the amp sim or amp sims in general,
but that seems unlikely.
If so, take all other effects out of the loop to make certain there's no other compression or limited involved, and make sure that your guitar track, and master track, in your software are no where near clipping,
then gradually raise everything bit by bit until the problem appears.
I'm still betting that some stage is just too hot for the next stage and it's going to be a case of reducing some level,
most likely input level at the sim or input gain on your interface.
If that doesn't help then I'm really not sure. I'd guess it's just a quirk of the amp sim or amp sims in general,
but that seems unlikely.