@whale-av

Hi whale-av,

thank you too for your reply.

Your patch present a similar situation to Ingox's one. If I send the note on trough the message box and if the notein is not connecting to the second noteout the patch works well with reaper. But if both noteout are connecting at the same time, note on and off are sent at the same time twice. At least, using no.n, and disconnecting the first noteout (which is linked to message box), the patch is not working.

It seems like noteout is the problem

If you're measuring search, last I looked it's just a simple linear search with [text] classes.

The point is that Pd doesn't have a properly optimized associative table implementation in vanilla.

Well, it does-- it is the global symbol table which t_symbol hooks into.

There are all these leaky parts between the language and the implementation-- I think it's what you are noticing when a solution gets produced from "on high." The user either has pentested potential workarounds enough to know which constructions are likely to be performant, or they've looked under the hood to see how things are implemented.

For example, when I said this in the other thread:

At least when I look at OP's problem, I see global receiver names followed by messages.

It's because I know that the global receiver names are a) created using a simple, effective hashing algo, and their associations are already cached by the time a symbol becomes part of a message. So the cost of forwarding a message to a receiver is just the cost of message evaluation plus the cost of invoking the relevant receiver's function in pd_typedmess. Since the user obviously knows how many arguments each receiver expects, they can skip iterating over the args in order to forward the next message. No lookups or data copying is needed.

TL;DR I would love to see more robust and well-performing data structures in Pd.

Yeah, I'd be curious to know after jumping into Pd if you have any opinions on what they could look like or at what level they could be implemented. IMO a decent associative array implementation ought to fill most niches. The demo solution I wrote that @jona found could be expanded to make things more expressive. (It just reimplements Pd's t_symbol hashing algo in a subpatch.) But of course it would be limited to that particular class/object-- you couldn't use it to pass around an associative array. (I guess gpointers could be used to do this, but that's not a great UX IMO.)

It's also possible to add something at the level of a native data type, at the cost of all the thinking about new syntax and compatibility that requires.

Edit: my letter b above got converted to a sunglasses-smily emoji. I like it.

If you need to store a bunch of key/value(s) pairs as a group (like an associative array does), a [text] object will allow you to do that with semi-colon separated messages.

Performance is abysmal, though -- I had guessed this would perform linearly = O(n), and a benchmark proves it.

• The array y range is 0 to 3000 (= 3 seconds). 10000 lookups per 'pd benchmark'.

• I had to abandon the test because, long before 100,000 elements, each iteration of the test was already taking 4-5 seconds.

• [* 1.2] -- in a search, failure is the worst-case. This is an arbitrary choice to generate 1/6 or about 16% failure cases.

I translated this from a quick benchmark in SuperCollider, using its associative collection IdentityDictionary.

(
f = { |n|
    var d = IdentityDictionary.new;
    n.do { |i| d[i] = i };
    d
};

t = Array.fill(200, { |i|
    var n = (i+1) * 1000;
    var d = f.value(n);
    var top = (n * 1.2).asInteger;  // 16% failure
    bench { 100000.do { d[top.rand] } };
});

t.plot;
)

• SC's implementation is a hash table, which should give logarithmic performance. The graph more or less follows that.

• SC is doing 10 times as many lookups per iteration, but:

  • In this graph, SC's worst performance between 46000 and 56000 elements is 17.4 ms.
  • If I look up in the Pd 'bench' array at 50 (51000 elements), I get 3082.
  • Extrapolating for SC's number of iterations, that's 30820 / 17 = 1771 times slower on average per lookup.
  • Since Pd is linear and SC is logarithmic, this ratio will get worse as n increases.

"This isn't fair, you're comparing apples to oranges" -- true, but that isn't my point at all. The point is that Pd doesn't have a properly optimized associative table implementation in vanilla. It has [text], but... wow, is it ever slow. Wow. Really not-OK slow.

So than people think, "Well, I've been using Pd for years and didn't need it"... but once it's there, then you start to use it and it expands the range of problems that become practical to approach in Pd... as jancsika said:

It's hard to know because I've gotten so used to the limitations of Pd's data types.

That shouldn't be considered acceptable, not really.

TL;DR I would love to see more robust and well-performing data structures in Pd.

hjh

@whale-av said:

@ingox Solving the users problem it seemed to me that Pd is seething with key/value pairs.

To be clear, I'm talking about dictionaries which are collections of key/value pairs. You can use a list, a symbol or even a float as a single makeshift key/value pair, but that's different than a dictionary. (Also known as an associative array.)

The headers/tags float, symbol etc. are used extensively as key/value for message routing.

This is a flat list where the first atom of the list acts as a selector. That's definitely a powerful data structure but it isn't an associative array.

[list] permits longer value strings.

These are variable-length lists, not associative arrays.

The problem for the OP was only that a series of key/value pairs had been stored as a list and that needed splitting.... but it's not a common problem..... and luckily the key was not also a float.

The OP's problem is instructive:

1. If you need to send a single key/value(s) pair somewhere in Pd, a Pd message will suffice.
2. If you need to store a bunch of key/value(s) pairs as a group (like an associative array does), a [text] object will allow you to do that with semi-colon separated messages. The important thing here is that the semi-colon has a special syntactic meaning in Pd, so you don't have to manually parse atoms in order to fetch a "line" of text.
3. If you want to send a group of key/value(s) pairs downstream, or you want to keep a history of key/value(s) pair groups, you have to start building your own solution and manually parsing Pd messages, which is a pain.

After doing a lot of front end work with Javascript in Purr Data, I can say that associative arrays help not only with number 3 but also number 2. For example, you don't have to search a Javascript object for a key-- you just append the key name after a "." and it spits out the value.

It may be that number 3 isn't so common in Pd-- I'm not sure tbh. But the design of the OP's data storage thingy doesn't look unreasonable. It may just be that those of us used to Pd's limitations tend to work around this problem from the outset.

The old [moonlib/slist] shared keys throughout a patch.
I used to use [slist] extensively as a dictionary, loading it from text files as necessary.

I'll have to play around with that one-- I'm not entirely sure what it does yet.

Keys are already a fundamental part of message passing/parsing.
And the correct way to store them as a string in Pd would have been with comma separators.
(I think...!! ...??)

I tend to use [foo bar, bar 1 2 3, bee 1 2 3 4 5 6 7( as a substitute for an associative array. But again, there's a limitation because you stream each message separately. E.g., if you have a situation where you route your "foo... bar... bee" thingy to some other part of the chain based on some condition, it's way easier to do that with a single message. But again, perhaps we're used to these workarounds and plan our object chains to deal with it.

David.

This seems as likely a subcategory as any...

I'd like to connect with teachers using Pure Data in the classroom.

• Where are the stumbling blocks that you see students crash into frequently?
  • (Anyone using Pd could chime in on that one... what are the things that confused you?)
• Are there any techniques/ideas/paradigms that helped the students to understand these difficulties more easily?

For a specific example: https://forum.pdpatchrepo.info/topic/13263/samphold-at-the-control-level -- "I want to get the number to update on the downbeat so it doesn't play anything while the number is being updated by some other process."

Extremely common scenario -- but I'll be danged if I can find anything in the help series that makes it clear.

This happens to be one of the sticking points for me -- which, lately, got me thinking about a paradigm of "feedforward" and "feedback" to cold inlets. It's (relatively) easy to understand chaining through hot inlets. Everything is immediate, and that's where the quoted question comes from -- if the only thing anyone taught you is how to chain immediate operations, then "save this datum for later" is scarcely even thinkable. (The quoted thread goes on to say "I'm having trouble explaining" -- meaning, whatever degree of exposure to Pd this user had, it wasn't enough to provide a vocabulary to talk about this problem.)

• "I have data now, but I don't want to use it until later" --> feed it forward to a storage object, then bang the storage object when you need it.
• "I want the next cycle of a loop to operate on the result of this cycle" --> feed it back to a storage object that's triggered by the loop.

This solves a bunch of problems. The quoted problem -- (data source) feeds forward to [f] right inlet. Or, initializing a counter at the beginning of the loop (feedforward). Or, building a list iteratively, but outputting only the final list (on each loop iteration, feed the list-in-progress forward to list storage, and bang the storage when the list is finished -- for this one in particular, I had struggled in the past with various bizarre usages of [spigot] but this is much easier).

One of the things I was missing over the last year and a half of getting up to speed in Pd is an established vocabulary of usage patterns. Sometimes I think Pd and Max pedagogy tries to stay away from typical computer-science problems -- but sooner or later, you're going to run into problems that have standard solutions. So why not collect them into a unified place? Like, in SC, for the fourth example below I can just write arrayOfMidiNotes.collect { |note| note.midicps } and uses of collect are all over the place in the help system... but in Pd, it took me literally over a year to figure out the best way to collect / map. That's... kinda crazy.

hjh

@forty-2 I don't have this problem and I have not seen anyone else raise an issue.
I would take a look at the file associations for Windows.
In my associations the .pd extension shows the wish application as the default program.
If it shows Pd.exe that could be the problem?
Otherwise (for other programs) a registry problem or a remote user logon have been mentioned.
David.

@Joseph-Mikkelson No, changing the samplerate will not help, except that matching it to the soundcard reduces the load on the cpu......but there are more important issues...... see below.....

I should have expanded on latency.
Using a live input [adc~] and doing some Pd processing effects and then sending that to speakers [dac~] latency is not so much of a problem. Often it involves delays anyway (echo, reverb, etc.) and our ears are used to that.
Latency for playback of an audio track, or generated audio, is no problem...... it is heard when it arrives at the speakers and that is all.
Problems arise when the input goes straight to the output....... as with a monitor mixer built in Pd for example..... with live acoustic instruments...... where the musician hears their own instrument acoustically and the same sound delayed in the monitor. Essentially a chorus effect is produced, caused by the latency, and that will make it harder for a violinist for example to pitch correctly.
Chorus in a reverb effect really upsets violinists..... anything much over 5ms.
Of course a massive latency...... 100ms+..... will cause timing problems even for a guitarist, and even if they are the rare musician that didn't have such issues in the first place.
I have met classical and jazz musicians that have learnt to dissociate what they play from what they hear...... for example playing through a delay, but in sync with everyone else...... so playing say 1 second before everyone else....... but......

SAMPLE RATE
You should match your samplerate to your soundcard, or you will have re-sampling artefacts....... usually a low level high pitched whine when your patch is not producing any sound.
It is slightly more complicated than that.
If you play an audio file that was recorded at 44.1KHz while the Pd samplerate is set to 48KHz it will play back at the wrong speed....... Pd will not resample it and so the pitch will be wrong.
It's a PITA.
So the Pd samplerate must be set to match all the audio files that you use in Pd.
To avoid the artefacts you then need to set your soundcard to match Pd.
Other programs will adjust..... so that is not a problem.
External cards can make that change through their control panel, and internal cards should have the option somewhere.
In windows it is here in the speaker control panel..... https://forum.pdpatchrepo.info/topic/12094/newbie-clipping-on-pure-data-portable-with-mmio/4
and I am pretty sure that windows always sets the on-board soundcard to 48KHz "out of the box".
David.

Hi!
I'm starting to learn Pd Extended and I trying to make a filter able to highpass, lowpass, both things or neither of them, but I'm having problems making a toggle to bypass the filters:
While in the highpass filter everything works fine (I can move the slider to the frequency I want, I can turn it off and I can turn it on again keeping the frequency I chose before), when I try to do the same on the lowpass filter, everytime I turn the toggle off (as a bypass), I stop hearing the sound.
Here is my configuration and my patch:

problem.pd

Thank you in advice!

I come from here: https://forum.pdpatchrepo.info/topic/9461/record-audio-of-any-length-into-array/8
Then, searching more about sound clicks and interruptions I've found this thread so I will continue here.

I'm dealing with the exact same problem as you, especially when resizing tables, which is crucial for dealing with samples and loops in realtime audio processing applications like you said. As far as I know, Pure data was designed in a way that array modifications and object calculations are done in the same thread the audio is flowing, so it's impossible to separate them and audio will be always interrupted. So, after looking for pure data distributions and externals, I've found something that maybe solve part of this problem, the sndfiler external, which is included in the last pd-l2ork distribution:

https://github.com/pd-l2ork/pd/tree/master/externals/tb/sndfiler

FEATURES:

• threaded reading of multichannel soundfiles into arrays
• threaded resize of arrays

I've installed the latest pd-l2ork distribution in windows and the sndfiler isn't installed by default, but it appears in the source code, so I will have to try building it from the source code if nobody has the built external. It also needs libsndfile and libvorbisfile and threadlib external in pd, a little bit complicated but may solve the problem.

For the GUI updating problem, you may solve it by separating the main GUI and the sound processing in differents patches. I've been doing this separation since I started using pd and never run into dsp problems.

Hi everyone,

I've run into an interesting problem with a iPhone rhythm game project I'm working on:

Basically the game has a scrolling score of music that you can play with your instrument at the same time. My patch detects pitches in real-time and then marks notes on the screen as correct or not.

The problem is that there is a built-in guide synth that plays each note as it scrolls (this is toggleable however). Right now I'm having the problem of the built-in guide synth scoring notes correctly without the user providing any input.

Obviously this issue goes away once the user wears headphones, but when you are playing audio on the phone's speaker at anywhere above 50% volume notes will get triggered automatically no matter what. I have tried several techniques (noise gate, some filters, minpower settings on helmholtz~ and sigmund~) that reduce the problem but not eliminate it.

That being said, I'm not hoping to completely eliminate this problem 100% (because it's most likely not possible), but I was wondering if anyone had any suggestions for kinds of techniques that could differentiate a real-life instrument from this built-in synth playing on the iPhone speaker, where both instruments are playing the same note?

Thank you