Was curious if there's an easy way to send an entire .syx file via midi. When looking through the forums for answers I mostly found questions pertaining to sending individual sysex messages, but I'm looking to simply route a .syx file like MIDIOX or similar software would. Would I have to parse the message somehow or is there a ready made abstraction for this sort of thing?

As always, thanks in advance

Was curious if there's an easy way to send an entire .syx file via midi.

A [file handle] object should give you access to the bytes in the file. Then pass those to [midiout] I guess. I think you'd just dump the bytes sequentially into [midiout]'s left inlet.

hjh

It is possible that a whole sysex dump can be passed in one message (still decimal though) to [midiout], but I have no means for testing that.
David.

@lo94 [midiout] requires decimal....... and a sysex file will be in Hex (almost certainly)....... https://forum.pdpatchrepo.info/topic/377/sysex/32

Well... maybe.

For instance, https://mido.readthedocs.io/en/latest/syx.html has a section for "SYX files -- reading and writing," and another section for "Plain Text Format" sysex files. So, apparently, there are two ways that sysex could be written to disk -- a sequence of bytes (numeric), or as text, where each byte has been encoded as two hex digits.

"read_syx_file() determines which format the file is by looking at the first byte. It raises ValueError if file is plain text and byte is not a 2-digit hex number."

Sysex messages are supposed to be introduced by a byte F0, and terminated by F7. So you could do as this site suggests -- read one byte, and if it == 240, then treat it as a byte stream.

I have no idea whether binary or hex-coded sysex dumps are more common in the wild, only that at least one reference page implies that both exist.

It is possible that a whole sysex dump can be passed in one message (still decimal though) to [midiout], but I have no means for testing that.

I just tested this by having SuperCollider print out sysex messages -- that is, there is no need to have a MIDI device receive sysex.

MIDIClient.init;

// '5' is Pd's MIDI source in the device list
// may vary in your system
MIDIIn.connect(0, 5);

MIDIdef.sysex(\y, { |... args| args.postln });

[240 1 2 3 4 247( --> [midiout] then caused SC to print: [ Int8Array[ -16, 1, 2, 3, 4, -9 ], 8454145 ] = data block, and device UID (where -16 = 0xF0 and -9 = 0xF7, as signed 8-bit ints).

hjh

I think I understand what you're saying but because of the formatting of the template file I'm having a hard time applying it to this use case (as I'm used to relatively short sysex messages like the one shown in the forum thread, where this one is massive). I attached the file in case it would be useful/relevant to see

FirstButtonCC.syx

I think I understand what you're saying but because of the formatting of the template file I'm having a hard time applying it to this use case (as I'm used to relatively short sysex messages like the one shown in the forum thread, where this one is massive). I attached the file in case it would be useful/relevant to see

In theory it would be something like the attached. But, I haven't gotten the entire sysex to be received on the other end. (EDIT: When you click 'dir' at the top, the number box will show 240... not done in this screenshot.)

sysex.pd

The right hand [print] shows that the entire file is being sent to [midiout].

For receiving, I tried two tests:

• SuperCollider, like before. If I send separate messages [240 1 2( and [3 4 247(, then SC's sysex handler does assemble them into a single packet -- so there should be no problem with sending sysex in chunks. But when I send the entire file (in chunks), SC never receives the terminating 247 -- then, if I send 247 on its own, it outputs all of the bytes stored so far!

• Pd. In qjackctl (I'm in Linux), I connected Pd's MIDI output port to its own input port, and try to collect sysex bytes in Pd itself. ([midiin] gives access to raw incoming MIDI bytes, which SC doesn't.) [midiin] gets only 512 bytes.

There are two possible explanations for the 512-byte limit. One is that perhaps [midiin] buffers the input -- but I think we can reject that explanation, because [midiin] outputs the bytes one by one, suggesting no internal processing.

The other would be that [midiout] is trying to collect an entire sysex packet, and that this imposes a limit of 512 bytes. The fact that neither Pd nor SC got the 247 terminator is consistent with this.

Unfortunately, that's bad news for your use case. There appears to be no alternative to [midiout] here, and AFAICS it would require a software change to extend the outgoing packet size.

hjh

hjh

This means [midiout], [midiin] and [sysexin] in Pd seem unable to handle long sysexes.

It's a kludge, but you could run a background SuperCollider process, and send open sound control to it when you need it to send one of these big sysex messages.

hjh

@ddw_music Long sysex messages work just fine if you treat them as what they are and what pd is quite good at dealing with, streams.

I do understand what you're saying, and I began with the same assumption -- that [midiout] should be just a low-level conduit -- byte goes in, byte gets sent out.

Unfortunately it doesn't work that way in practice.

My test scenario looks like this. Launch SuperCollider and Pure Data. It helps in my case that I'm testing in Linux, where I can create and remove MIDI connections arbitrarily in qjackctl's ALSA tab.

SC code:

MIDIClient.init;

MIDIIn.connect(0, MIDIClient.sources.detectIndex { |src| src.device.containsi("pure") });

m = MIDIOut(0).connect(MIDIClient.destinations.detectIndex { |dst| dst.device.containsi("pure") });

m.connect(MIDIClient.destinations.detectIndex { |dst| dst.device.containsi("superc") });

MIDIdef.sysex(\a, { |... args| args.postln });

Then I need to go to qjackctl and connect Pd's MIDI output to Pd's MIDI input.

At this point, I can send MIDI from either Pd or SuperCollider, and both SC and Pd will respond.

Pd patch, implementing "stream of bytes" (btw I tested both [sysexin] and [midiin], no difference in the results):

sysex-stream.pd

(Note also: Because I am collecting the MIDI-in bytes into a list, I also checked whether there is a maximum list length. There may be, but we won't hit any such limit in these tests -- I could go up to 5000 numbers in a list, no problem. So, in case of any message truncation, it must be happening in the MIDI objects, not in list handling.)

The test case will be to send a packet of n 14-bit integers (MSB first). The total packet size will be n*2 + 2, so if we want to produce exactly b bytes, n = (b / 2) - 1.

SuperCollider sending:

(
f = { |n|
    var out = Int8Array[240];
    n.do { |i|
        out = out.add(i >> 7 & 127).add(i & 127);
    };
    out.add(247);
};
)

// small packet
m.sysex(f.value(10));  // SC OK, Pd OK (22 bytes)

// 512 bytes
m.sysex(f.value(255));  // both OK

// 514 bytes
m.sysex(f.value(256));  // SC OK, Pd did not print

And my results for Pd sending:

• 22 bytes (n = 10): Both OK
• 512 bytes (n = 255): Both OK
• 514 bytes (n = 256): SC did not respond. Pd printed only 512 bytes (truncated).

Conclusions:

• When SC sent 514 bytes, SC received 514 bytes, so we know SC is sending a complete packet.
  • Pd did not respond, suggesting that both [sysexin] and [midiin] simply stopped listening after 512 bytes, did not process the ending delimiter, and thus did not output the packet at all.
• When Pd was asked to send 514 bytes, Pd received only 512 bytes. This suggests that [midiout] is buffering the incoming bytes, waiting for a complete packet before sending, but it must have a limit of 512 bytes, causing the message to be truncated. (And SC didn't respond at all, which suggests that the closing delimiter was never sent.)

So again... in principle, you're correct -- it's technically possible for some [midiout] object to relay the data out, byte by byte. But neither [midiin] nor [midiout] are implemented that way.

hjh

s_midi_alsa.c:

//the maximum length of input messages
#ifndef ALSA_MAX_EVENT_SIZE
#define ALSA_MAX_EVENT_SIZE 512
#endif

But I can't quickly find where Pd's source is interfacing with CoreMIDI (macOS) or ... whichever MIDI framework in Windows.

In any case, it's good to be aware of the behavior, in case a patch needs to be cross-platform compatible.

hjh