what I actually wanted to do was to record 4 on top of 3, which would of worked (I think).
It wouldn't have. Certainly not on a 488. The trick to bouncing on the 488 is to make sure you always have at least three open tracks, the two {or more} that are to be bounced and the one that all that bounced stuff ends up on. And if you're going for a stereo pair, then you'll need two tracks that the bounced stuff ends up on. Once you've only got two tracks {ie one open for the bounce and one with the stuff to be bounced} left, you're in soup. That's one of the things I love about virtual tracks. If you've only got one physical track left, you could still record on lots of virtual ones then bounce them all down to that last open track.
Actually, I'm not sure how you did that without monumental feedback warning you it was a bad idea. If you run a track to itself it's a feedback loop and normally would have had you scrambling to shut it off so I suspect that one way or another you didn't actually have track 3 going to track three.
The 488 doesn't give any kind of warning because it can't be done. If there's something on the track that one is bouncing to, it just gets erased as happened here. It's like a policeman that arrests you for breaking the speed limit even though there's nothing to tell you what the limit is.
how could you record a track to itself?
Interestingly, I've only heard of a couple of machines that can do this. Back in 2004/5 when I was looking for a digital standalone, I spent ages researching and because I needed varispeeding, came close to buying the Yamaha MD8. There, you can record on all 8 tracks and then bounce the entire 8 down to a stereo pair. The only drawback was that the two tracks that carry the pair overwrite what was on those two tracks originally so even though they are part of the bounced pair, you lose them as individual tracks which is a killer in Manilla if you don't like the bounce ! The Tascam 564 allowed you to bounce it's 4 tracks to a stereo pair, but retained the original four tracks because the bounced 4 tracks went to a new song location or something like that, so you could always redo it if you didn't like it.
In either event, the 488 has no such luxury !