USB is not a problem; it's far far faster than original midi serial ports.
Latency is 100% the interaction between OS, soundcard and drivers.
Get a real asio soundcard and you'll be fine.
Not entirely true I'm afraid!
The speed of USB, even version 1.1, is indeed faster than MIDI's 31.25kbps, but it's not just an issue of interface speed but the way the interface functions and its underlying interrupt (or other) mechanism, data packetisation and protocol.
With USB the slave (keyboard) is 'polled' by the master (computer) on a regular basis, unlike MIDI's immediate asynchronous transfer mechanism, often resulting in a phenomenon known as jitter, depending whether the data just missed a poll or not. Not to mention a small latency in the transfer of data and conversion of USB packeted data to MIDI bytes. It often adds another millisecond or two compared to a dedicated MIDI interface connected via an interrupt driven internal interface card.
Of course, as you correctly say, the main factor is the generation of the note by the underlying software and its streaming through the DAC to the analogue audio interface.
Some in depth analysis and detailed tests were done some years ago and published in the UK magazine Sound on Sound.
A pair of articles by Martin Walker titled "The Truth About Latency" from September and October 2002 will tell you all you need to know about the subject.
Nothing has essentially changed very much in the interface functionality since these were published.
Well worth a read IMO.
Regards,
John.