Getting a new PC is not a bad suggestion . . . worth looking into.
However, what you've got should work okay. Doing audio stuff does not demand a lot of a computer, and I started digital recording on a PC with lower specs than yours, and did so quite successfully. Recoding and playback will be fine, You may run into trouble when trying to use lots of plug-ins, and you may find editing waveforms to be slow. But it will work.
Your latency problems are not likely to be caused by the age or specs of your PC, and are more likely to be caused by the interface you are using. My guess is that you are using the PC's onboard soundcard, and this is the chief cause. If you are using a special audio recording interface, then we would need to look further afield for latency causes.
A potential setback for you, a consequence of the PC's age, could be that the motherboard has no USB functionality, which might limit your options to deal with latency.