Hi everyone,
I'm loading some MIDI files (format 0) into [cyclone/seq] and saving them to text, I get a series of lines that are 3 or 4 elements long.
The first element seems to be the time (in ms?) at which to play the line, item 3 is the midi note and item 4 is the velocity. I can't find any reference to what item 2 is. The text goes something like this:
0 192 0;
0 144 36 28;
0 192 0;
0 192 0;
7.12308 176 64 127;
61.055 144 36 0;
61.055 144 48 28;
122.11 144 55 25;
253.06 144 55 0;
253.06 144 60 26;
383.483 144 60 0;
383.483 144 62 24;

etc..

Any help would be greatly appreciated, thanks in advance.

@ingox Yes! Now I get it, no more pd for me after 3am
The only question I have is why do you need to add ||| at the end of each item?
As for the rest I think it's the simplest method if you want a bare bone abstraction, everything else like state saving and building chords from a stream etc.. can be done outside of it.
There is one further simplification I have made which gets rid of a few objects, it's inside [pd make.markov] after text sequence. There was a leftover from the pre-list store days of MarkovGenerator.
Great work! thanks a lot

@weightless Great!

If you have the chord 88 90 in the [text define $0-chords] and a single note 88 comes in, [text search] would only look in the first column and find the line with 88 90, where this is in fact another element.

If you add ||| or any other unique symbol to the right of each note or chord, it would be 88 90 ||| in the [text]. Now [text search] is looking for 88 ||| and cannot find the line with 88 90 |||. So each note and chord is uniquely identifiable

...

@ingox Perfect!
This is your patch with the simplification I mentioned, I also changed the text names so they make more sense (hopefully?).

markov-poly2.zip

I'm wondering if you'd include state saving in this simple abstraction, it's basically the same as using the text define -k flag but multiple abstractions can hold unique chains.

@weightless I think using state saving sounds great. Can this also be disabled or otherwise controlled from the outside? I have to admit i have never looked into it

@ingox State saving works only if one (or more) instance of the abstraction is loaded in a parent patch. When you save the parent patch, all abstractions dump the content of their tables which get saved in the parent .pd file, each linked to the instance of the abstraction that sent it. when you re-open the parent patch, those saved tables are dumped back. Nothing is stored in the abstraction itself, so if you open the abstraction file nothing gets recalled and so the tables are initialised as empty (you mustn't use the -k flag).
I suppose it could be disabled but I can't think of any scenario where that would be advantageous.

I've recently made these abstractions which I use all the time and have made my pd life so much easier I don't even remember what it was like before.
https://forum.pdpatchrepo.info/topic/12066/store-state-patch-less-state-saving-method-for-abstractions/9

The only difference in our markov patch is that the order cannot be set with an argument anymore because if you create the object [markov 3], then send it the message [order 2(, and save the parent patch, the text is saved containing a second order markov chain, but when you open the parent again it loads the argument 3. In fact I think it's not ideal to set the order in the argument.

I hope this makes sense, I'll post a modified patch to show this.

@ingox Here it is with state saving. If you open the help patch and click the two iterate bangs you'll see that the two abstractions are saved with different chains of different orders. If there is no state to load, the order initialises to 1.

markov-poly3.zip

I had to change a few of things connected with the order, the most significant is that rather than automatically clipping the order according to the length of the indices text, I think it makes more sense to give an error if the input source is too short for making an nth order chain. That is, if I send [order 3( but there are only 3 lines of text, I wouldn't like the abstraction to silently make a second order chain instead.

EDIT: updated zip

very nice work

@Jona It's a very useful little abstraction, I'm trying it out with some real data to see if there are any errors still.
I had to add a receive from the stored order object before the moses for [make(, this way when re-making the chain after it has been saved in a parent patch, the number of indices is checked against the correct order that was stored.

markov-poly4.zip

One annoying limitation of the way I've implemented state saving in these abstractions is that there is no way of keeping track of the order in which the saved parameters/arrays/texts are loaded. I have no idea how to do that reliably, otherwise it would be easier to just store and load the indices and memory texts first, then load the stored order and re-make the chain on loading.

@weightless Sorry for answering so late, i didn't find the time earlier.

I implemented a new system to generate the chains by simply iterating over all indices. Not sure if this is more or less efficient, but it is very simple.

Chains are build from left (oldest) to right (newest).

This system has the slight benefit that the order is not limited. An order higher than the length of the material is nonsensical and results in the original progression being played without jumps, but is not considered to be an error by this system.

I also incorporated the state saving into the object, so it doesn't rely on any other external, and made a new help file.

markov.zip

I used some code from weightless patch that uses code from markowGenerator. So i assume that the GPL still applies to this code. The authors can always reissue the code. @weightless, @Jona, would you agree to release the code used in this patch into the public domain?

I tested it and the process of creating the chains is now about 300 times faster

@ingox Great work, thanks for sharing. Great idea to include the state saving mechanism, in fact that subpatch can be taken as a nice starting point for state saving that requires a specific order of execution. As far as I'm concerned, of course you or anyone else can do whatever they want with the code I posted, which in turn was taken from somewhere else as you mentioned.

Out of curiosity, you say it's 300 times faster compared to which patch? Sounds like an amazing improvement.

@weightless This was stupid. I didn't specify the order. My patch is actually a bit slower...

@ingox Oh I see, well at this point it's not likely to get much simpler than this I reckon, and it's probably fast enough even with very large inputs.

@ingox @weightless I would agree to release the abstraction into the public domain (if I am in the position to do that). And I do not mind if the license is GPL or BDS ...

A slight improvement in speed: This version uses [array] to store the indices. To create the chains, the array is combined with the end of the array and chains are taken directly from this combined array. The downside is that the structure of the source material that is in the indices cannot be looked at like it was with [text], but for practical purposes, this should not be an issue. Also some other slight improvements.

markov.zip

This is now in the public domain. Thanks again for the great collaboration!

Edit:
Some more slight improvements. Also i added the clipping of the order again, as i think it contributes positively to the user experience. It is explained in the help file and there is a message when the chains are being created, so it should be reasonable.

This looks awesome. I don't get how the lists get encoded as one item (for chords). Can anyone please enligthen me?

@porres The lists are stored in a text, the markov chains are simple build upon the line numbers of the text, so over the indices.

yeah, now that I had a deeper look into it, I get it nice way to make multidimensional arrays

anyway, I was working on an object based on cyclone/anal + cyclone/prob for ELSE, but I'm gonna try and make something like this now! I just miss the possibility to set a probability transition matrix like in prob, but I may be able to work on this patch and get there.

great work and thanks for this

@porres Well, the [markov] object takes source material and generates something like an implicit probability transition matrix from that. If you already have a probability matrix, you only need a starting point and can play the markov chains immediately. A much more simpler abstraction can do that. Mixing the two approaches seems complicated, since [markov] follows a different philosophy, i.e. it allows adding more source material later in the process.

Interesting could be to have two separate abstractions: One to generate a probability matrix from source material and another one that plays markov chains from that. So you can have both approaches and combine them.

This would also be similar to the combination of [anal] and [prob], but as a generalized approach to have markov chains of arbitrary length.

The question is rather if it is a realistic scenario to have a complex probability matrix for markov chains of higher order. [markov] is built as a basic machine learning tool.