From my own experience with using midi, latency isn't any issue to worry about. Although the best of any audio or midi device may have some finite amount of delay. The largest latency I've personally experienced has been using my midi keyboard/synth or an external midi sound into my DAW and back, sending midi & audio back & forth. Anything sent to the DAW by a midi controller (be it a midi keyboard, midi eDrums, or midi guitar) and then running a VSTi inside the DAW, latency has been pretty much non perceivable. That includes regular midi, or USB midi...and midi, either directly into the computer, or through an audio interface, as well as a dedicated midi router/interface device.
I will say that in my experience, I've had many problems with USB midi...not the latency, but actually getting it to even work! Sometimes I get a midi signal into the DAW, other times I can't get a midi signal for the life of me...sometimes a complete re-boot helps, many times not. For this reason I always use standard midi for everything and avode USB midi. Standard midi always seems to just work for me. I just got tired of chasing after, chasing after and when I'm doing music with midi, I want to just Get R' Done! Others may have had better success with USB midi than I, so take my comments with a grain of salt and try it yourself.
All my audio interfaces have standard midi I/O, and I've not had issues with this. In fact, I much prefer to use the standard midi I/O within my sound cards. I even use my dedicated midi routers (Motu MTP AV 8X8 midi routers) with all my midi controllers/gear, plugged in directly to the midi I/O of my audio interface's.
Just in the last couple of days, I've been reconnecting/reconfiguring some of my midi gear...I'm using a customized Yamaha DTXpress drum set up, which consists of multiple kit's combined together for one SUPER e-kit. (so I can have more trigger pads to shake my sticks at). To do this, I'm using four (4) Yamaha DTXpress trigger modules to accommodate all my trigger pads.
I tried daisy chaining all four trigger modules using the proper settings of 'midi mix' so I can send midi from one module, into the next and have all midi get sent out of the last module. It actually works... but this seems to keep changing the midi note numbers on me. So I'm now using a standard midi cable 'OUT' of each trigger module, into one of my Mout MTP AV midi routers, combine and send out of one of the Motu's outputs, into one of my audio interface's midi IN.
I don't need any midi back into my Yamaha trigger modules because I'm using Toontrack Superior Drum VSTi for my drum sounds within the DAW. Guess what??? NO perceivable latency! I actually heard more latency (perceivable though negligible) when I hooked up a single trigger module directly to my sound cards midi IN & OUT (to & from module) as well as routing the trigger modules audio into my sound card, using it's sounds. I suspect this had to do with the audio's latency, rather than the midi. This was before I was using drum VSTi's for sounds by the way.
PS, I'm not using my Motu's USB connected to my computer, this is because of my above experiences with USB midi not wanting to work too many times...I may try it again at some point...or not, it works 1005 as it is for me now, so...?
I hope my experience has helped in some way.