@redhexagonal I am no expert on midi (I don't use it), and I am not using Linux.
So..... a total guess.....
I believe reading the doc that [midiin] and [midiout] dump raw midi..... so no work is being done in Pd.
[notein] and [noteout] are doing work, and they are synced to the audio thread.
I would expect their latency to be the same as the audio.
But I am surprised if [midiin] and [midiout] are not also scheduled.
If Pd is doing no audio work, and the -noaudio switch is available on the RPI version of Pd, then you could reduce the audio buffer..... Delay (msec) to 1ms (or maybe even 0?) and hopefully have very little latency........
That whole thread might help. There is some talk of using osc translation as well, but I cannot really see how that would help.