Just noticed this in another thread:

As if on cue: A question popped up that falls outside of the streaming paradigm: https://forum.pdpatchrepo.info/topic/13269/splitting-variable-lists-based-on-symbol

IMO that user is just begging for a simple dictionary in Pd.

I've been working in HTML5 lately, so when I see this which is ad-hoc and annoying to parse in Pd:

ps 0 fft -1 -2 hip 5 0.4 9,3;
fft 0 1 hip 0 1 2;

I immediately see the desire for this kind of data structure:

{ps: 0, fft: [-1, -2], hip: [5, 0.4, 9,3]};
{fft: [0, 1], hip: [0, 1, 2]};

Immediately, you see that just throwing a Javascript object in there is way too rich for Pd-- even for this simple example I've got anonymous arrays and curly brackets. It would be quite a task to add JS objects or JSON objects as a primitive type in Pd.

So the question is-- what would Pd-style key/value pairs look like? What would it feel like to use them?

If someone has a good answer that gets me excited I'll write it as an idea for Purr Data GSoC this year. The only related area I can think of are data structures, but they tend to be too clunky and complicated for most users.

Edit: Even the puredata.info syntax highlighter is hinting at the hidden structure in that flat list!

@jancsika A little test for [text] without semi-colons...... slist.zip
Stepping could be done by line number if one needs that.
Long data sets should not slow it down as (I think?) they are not being searched.

I have no idea what method is used for a search (and that would be good to know), but it is fast enough for my purposes looking up control data by key..... (0.12ms on my machine for 5000th key.... 0.033ms for 3rd key).

Putting back the method for changing data by key..... which will suppress double entries....... makes it more useful but harder to test. There are no failures..... so not an associative array in the strict sense?.. or more likely because I have not tested for iteration?

Here it is in its useful form...... parameter storage and lookup.zip ...... a re-invention I am sure.

The key heads up the data on the right outlet for routing purposes.
Data sets can be saved and loaded from files by $1-float.txt
David.

0.12ms on my machine for 5000th key.... 0.033ms for 3rd key

That's an order of magnitude difference.

Imagine a table full of fortune cookies with 100 different people's names on them. You're asked to retrieve them for the people. In this case your only method is to search all of them. The person whose cookie is at the front of the table is lucky-- if they need to go in a hurry you'll find their cookie immediately. The person whose cookie is at the far end is unlucky-- you'll probably have to search through all the cookies before you find theirs.

Now hang 26 chains from the edge of the table, and label them with the letter A-Z. Store each cookie in a link of the chain that matches the first letter of that name.

Now on average you'll take a little more time to find the cookie that was previously sitting on the near edge of the table. But on average retrieving cookies will be 100 times faster.

However, in the worst possible case everybody's name begins with the letter "d" and you're back where you started. So you need a better rule than simple alphabetical order to try to get the worst case retrieval time closer to the average retrieval time. You'd prefer that "Dan" and "Dane" are in different slots, for example.

If you use the simple hashing algo in the gensym function that registers Pd symbols, it seems to do a decent job of spreading out symbols among all available slots in the table. So as long as the algo to map a name to a slot in the table is quick to run, this gets the worst case lookup time close to the average case lookup time, and that's good enough for a soft realtime system.

If the table has 1000 slots, and the time to turn a key into an index is negligible, then the only big expense is searching through the various items hanging from the chain at that index. In your example of 5000 items, the chain would likely only have five items to search. So the retrieval time on average is going to be a lot close to 0.033, and the worst case is unlikely to be much more than that.

@jancsika Fascinating.
It will take me some time to work through data science development since the 18th century.
gensym seems to be used extensively in externals.
60 keys is likely to be a maximum in any control patch I build, but any increased speed would be helpful.
The biggest so far had 64 in each of about 30 stores, where the patch looked up which store to search. Maybe a similar principle although of course no redundant slots.
David.

@whale-av I tend to be like you. The places I miss key/value pairs the most are the cases where there would typically be less than 10 keys. E.g.:

• doing math on some values. pitch * attenuation_level is way more readable than $f1 * $f2

• small collections of values, like what comes out the right outlet of soundfiler. It's a travesty that you cannot do [get - samplerate] on that output-- instead, you have to open up the help patch and manually map list positions to arguments. (And a reader of that patch must do the same if there happens to be an error somewhere.)

This is the problem of making even an elegant, performant, well-documented abstraction for a key/value store-- you cannot extend it to the other places in Pd where you need it.

@jancsika Just an undeveloped idea..... because I use a small lookup [slist] as part of [slist-master] throughout some patches....
.... a [value x] with a proper hash key "x" table behind it......created when the first [value x] is put, or separately... and so with global variable keys...... so that one could just send the key into its inlet to get the data.
You could add samplerate and other useful keys to it automatically of have a [value pd] that serves up internal parameters.

It "sort of" half exists for [expr].
You can do [expr pitch * attenuation_ level] if you first set [value pitch] and [value attenuation_level]....... but of course at the moment every [value] has one slot.
David.

@jancsika So I have changed [slist-master] so that it uses [value] to distribute globally.
Because I started with lists the second piece of data for foo is assigned to [value foo2] and so can be "got" from anywhere including in [expr]
slist-master.zip

Unfortunately [value] doesn't distribute messages across its inlets so its "name" cannot be changed on the fly without extra objects.
External with # please..... although maybe not for lists...... just key/value......?
David.

So here's what I'm thinking about. I'm not sure if this makes sense in Pd...

In SuperCollider, I can do:

s.boot;

(instrument: \something, midinote: 60, sustain: 2, amp: 0.7).play;

... and the sound comes from SC's own engine, based on a synth template named \something.

And I can also do:

MIDIClient.init;
m = MIDIOut(0).connect(0);

(type: \midi, midiout: m, midinote: 60, sustain: 2, amp: 0.7).play;

... and the same parameters midinote: 60, sustain: 2, amp: 0.7 are translated into MIDI noteon/noteoff messages, and performed by any MIDI software or hardware synth listening on that port.

A concrete benefit is: If I write some compositional algorithm and I want to decide later whether to play it within SC or using an external instrument, I don't have to change the composition logic at all. That part simply stuffs data into Event objects, and the event determines what finally happens.

In Pd, the data and the path the data take through the system are tightly coupled -- positional access to specific values is one way that this manifests, but more significantly, in a way, tight coupling is just the nature of dataflows.

So I'm wondering if this is a way to decouple: you could describe an action as a set of key-value pairs, forward the set to any event-player, and the event player would look up the data that it needs.

To some extent, you can already do this by discrete "key value" messages and [route]ing them into different parts of the player, but this isn't totally transparent: 1. If they are discrete messages, how do you know when the event is complete? Needs a sentinel. 2. [route] takes a little time to ignore irrelevant data (while a dictionary lookup would simply not look up irrelevant data). 3. If the pairs arrive out of order, math ops could be tricky, e.g. the following spits out a spurious 5 before the dog value comes through:

But with this hypothetical, you could control the order of access in the patch, instead of relying on well-behaved data:

This use case suggests that a dictionary should be lightweight, like a list, easy to pass around in the system (so, different from [text] or [array]). That matches up with jancsika's use cases (passing this type of dictionary into [expr] or getting it out of [soundfiler])... but it perhaps doesn't fit easily into the current message-passing scheme.

hjh

but it perhaps doesn't fit easily into the current message-passing scheme.

Funny enough, [get] takes a "gpointer" to a "scalar": essentially a bunch of typed key/value pairs stored in a byte array.

The problem there is too much boilerplate-- you can't just create an ad hoc datum and send it on its way. Instead you must first define the template for that data, then instantiate the data, then grab a reference to it by walking a linked list.

If that paragraph I just wrote could be superceded by a more sensible method for quickly defining and passing such data, that could be a solution. (There's also [set] for setting a key by name.)

@ddw_music "A concrete benefit is: If I write some compositional algorithm and I want to decide later whether to play it within SC or using an external instrument, I don't have to change the composition logic at all. That part simply stuffs data into Event objects, and the event determines what finally happens."

That is exactly the purpose for which I built [slist-master]
Because it was built for lists the data "get" has to look for the place in the list so with a single value for the key........ key/value dog 12 is looked up by [value dog1]
It can be modded easily to avoid that if only single values are ever to be assigned..... see below.
I have changed the setting of data to use the more conventional "set" in place of "add".

When the data set is loaded (by set or loading a file) all [value] objects are loaded with their data, so retrieval should be fast.
Even a dynamically created [value] will already store the data for the key.
cat_n_dog.zip

And here for just key/value (lists truncated)...... key-value.zip so the receive objects are simply [value key]
Of course [value] is already [receive] with a store........ so be careful with key names...
And values can be overwritten but not cleared... that's in the nature of the [value] object....
David.

PS It is possible to access much of Pd media settings and change them from within a patch.
Some work was done recently to cope with reordering midi connections when their order has changed in the os. That could be applied to audio devices as well...... https://forum.pdpatchrepo.info/topic/13217/save-and-recall-midi-settings-in-a-project/2
I didn't follow up to see how complete the project was.....
A little more complex than some other programs of course.