This seems as likely a subcategory as any...

I'd like to connect with teachers using Pure Data in the classroom.

• Where are the stumbling blocks that you see students crash into frequently?
  • (Anyone using Pd could chime in on that one... what are the things that confused you?)
• Are there any techniques/ideas/paradigms that helped the students to understand these difficulties more easily?

For a specific example: https://forum.pdpatchrepo.info/topic/13263/samphold-at-the-control-level -- "I want to get the number to update on the downbeat so it doesn't play anything while the number is being updated by some other process."

Extremely common scenario -- but I'll be danged if I can find anything in the help series that makes it clear.

This happens to be one of the sticking points for me -- which, lately, got me thinking about a paradigm of "feedforward" and "feedback" to cold inlets. It's (relatively) easy to understand chaining through hot inlets. Everything is immediate, and that's where the quoted question comes from -- if the only thing anyone taught you is how to chain immediate operations, then "save this datum for later" is scarcely even thinkable. (The quoted thread goes on to say "I'm having trouble explaining" -- meaning, whatever degree of exposure to Pd this user had, it wasn't enough to provide a vocabulary to talk about this problem.)

• "I have data now, but I don't want to use it until later" --> feed it forward to a storage object, then bang the storage object when you need it.
• "I want the next cycle of a loop to operate on the result of this cycle" --> feed it back to a storage object that's triggered by the loop.

This solves a bunch of problems. The quoted problem -- (data source) feeds forward to [f] right inlet. Or, initializing a counter at the beginning of the loop (feedforward). Or, building a list iteratively, but outputting only the final list (on each loop iteration, feed the list-in-progress forward to list storage, and bang the storage when the list is finished -- for this one in particular, I had struggled in the past with various bizarre usages of [spigot] but this is much easier).

One of the things I was missing over the last year and a half of getting up to speed in Pd is an established vocabulary of usage patterns. Sometimes I think Pd and Max pedagogy tries to stay away from typical computer-science problems -- but sooner or later, you're going to run into problems that have standard solutions. So why not collect them into a unified place? Like, in SC, for the fourth example below I can just write arrayOfMidiNotes.collect { |note| note.midicps } and uses of collect are all over the place in the help system... but in Pd, it took me literally over a year to figure out the best way to collect / map. That's... kinda crazy.

hjh

Counterintuitively, this turns out to be misleading in Pd.

In Smalltalk, Javascript, etc. your list of seven integers would be a didactic example that applies to any size data structure that can fit into memory of the user's machine. The epiphany for that programmer comes when they realize they can replace those seven integers with the content of Wikipedia in its entirety.

In Pd, the epiphany starts with a stream of data coming into the patch. So you have to turn your message box 90 degrees so that it points down into the patch, with each message being sent at a different logical time.

The epiphany in Pd comes when the user learns how to efficiently analysize and synthesize the stream-- e.g., perhaps they hook into a script that sends Pd a message for every tweet that includes the string "twerk," have Pd do something in response to that message, forever to the end of time.

Iterating in zero logical time is needed, but it has severe limits on it because it happens in the DSP process. For that reason, most of the extant patches I see tend to have somewhere close to the number of list elements in your example. In other words it's no longer a didactic example, but a practical one-- in the case of your last example it's easier just to unpack the elements, test out a chain to do the mapping, then copy/paste that chain for each element and roll the overall results back into a list with [pack].

@jancsika said:

Counterintuitively, this turns out to be misleading in Pd.

Ok... Which part is misleading? The loop, or the feedforward / feedback in general?

I maintain that the usage of storage objects' cold inlets is under-discussed in the documentation.

In Pd, the epiphany starts with a stream of data coming into the patch. So you have to turn your message box 90 degrees so that it points down into the patch, with each message being sent at a different logical time.

That's a fair point... still, I had a case in my class where there was an incoming stream of bytes (from Arduino --> [comport]) and I needed to clump them into lines ([comport] --> clump one line --> [fudiparse] --> get a list). This is not actually that hard to do, but it took me literally almost a year to develop the chops to see an efficient way.

Iterating in zero logical time is needed, but it has severe limits on it because it happens in the DSP process. For that reason, most of the extant patches I see tend to have somewhere close to the number of list elements in your example. In other words it's no longer a didactic example, but a practical one-- in the case of your last example it's easier just to unpack the elements...

It's not at all difficult to think of a case where this would break: say, a chord player where the chords may be 3, 4, 5, or 6 notes, to be played as block chords. If one hasn't got the programming vocabulary to handle variable-length lists, then one is likely to make a rule for the composition that all chords should be the same size -- so a programming-paradigmatic limitation has become an (unnecessary) artistic limitation. (Ok, you could fill out "short" chords with, say, -1 and then ignore -1 for playing... not to my taste but it would be a valid workaround.)

hjh

It's not at all difficult to think of a case where this would break: say, a chord player where the chords may be 3, 4, 5, or 6 notes, to be played as block chords.

In that case the user would just copy-pasta a solution for max = 6 notes, probably with a sentinel value like a negative number to mean "this voice isn't used this time." Ah, I see you mentioned this already. Anyway, the limitation can be abstracted away. That is, the user sends a variable length list <= 6 to the interface, and inside the implementation [unpack] is used to distribute the list to its outlets. The implementation could simply initialize the chord to be all "-1" values, and the incoming list replaces the necessary slots.

Just to spitball-- I think the paradigm of specifying and rigorously testing the max number of "things" is an important and overlooked paradigm for Pd. E.g., if you want to handle up to 6-note polyphony, make an interface that has a slot to specify "6" as the max polyphony in there. Then in the implementation, actually compute the maximum number of things you gave it, and discard the ones you don't need at the last possible moment. (E.g., multiply the signal for that unused voice by "0" before outputting it.)

This way the user will spend all their time testing the maximum amount of time it takes the program to do its work. The user will then get a much better idea of how stable the patch is for realtime use.

If instead you optimize for average use-- say [switch~]ing off unused voices in a piece where you have 4-note polyphony for 15 minutes and 32-note polyphony for 10 seconds-- you do probably 100x fewer tests of the realtime readiness of the 10 seconds of 32-note polyphony. And that's probably the climax of the piece!

Edit: just to be clear-- if you turn on some fancy CPU-hungry filter during the 4-note polyphony and turn it off for the 32-note polyphony, the analysis and design obviously changes.

As if on cue: A question popped up that falls outside of the streaming paradigm: https://forum.pdpatchrepo.info/topic/13269/splitting-variable-lists-based-on-symbol

From a pedagogical perspective, the solution is given from on high without an attempt to explain a thought process by which a user might arrive at the solution independently.

My intent in starting this thread was to work toward some vocabulary by which such magic can be revealed and taught more effectively. The solution in fact exactly reflects "feed forward into storage and bang later" though I need to think of a better, shorter phrase for that.

Instead we're talking about why you should avoid loops -- which I find really funny, as it's orthogonal to the topic title.

hjh

@ddw_music said:

From a pedagogical perspective, the solution is given from on high without an attempt to explain a thought process by which a user might arrive at the solution independently.

Sorry....... a bit rushed yesterday...... I usually try to add an explanation but the forum is a hobby and sometimes life gets in the way......
David.

As if on cue: A question popped up that falls outside of the streaming paradigm: https://forum.pdpatchrepo.info/topic/13269/splitting-variable-lists-based-on-symbol

Hm... this user is just begging for dictionaries. I'll post a new topic about that.

The solution in fact exactly reflects "feed forward into storage and bang later" though I need to think of a better, shorter phrase for that.

Actually the solution purposefully avoids that approach. Look inside [list-drip]: it was purpose-written to do an end-run around the performance problems that come from copying data downstream to be processed later.

The approach of incrementally building the sub-messages does indeed work in this case, but again only because the message payloads are "Pd sized" payloads. Add some configuration state for a "table" key that takes a 128 element payload are you might start to see problems (depending on when and how this configuration state gets set.)

At least when I look at OP's problem, I see global receiver names followed by messages. One can create an abstraction that takes the entire incoming message and sends its arguments to the symbol at the front of the message. Wherever that configuration state needs to be received, put a matching "receive-and-forward" abstraction that specifies the name and number of args (or alternatively symbol names for each arg, which even pdcontrol in Pd Vanilla allows you to do now).

That "receive-and-forward" abstraction can output the args for the relevant objects in that part of the patch. Then for the remaining part of the message, that same abstraction can reuse the message sending abstraction that was built at the beginning.

Because the facilities are abstracted away it's easy to then add error-checking or modify it for more sophisticated needs. And since the receivers are wireless you don't have to add another arg to a central [route] object and hook up more wires. It should also be easy to prepend "$0" in the sender to protect against namespace pollution.

I think there's a broader paradigm about encapsulation here. But even just fetching abstraction arguments is relatively new to Pd Vanilla, so there may not be a lot of general knowledge and vocabulary around it yet.

@jancsika said:

Actually the solution purposefully avoids that approach. Look inside [list-drip]: it was purpose-written to do an end-run around the performance problems that come from copying data downstream to be processed later.

I thought [list store] and a "get" message avoided the need to split the list.

The approach of incrementally building the sub-messages does indeed work in this case, but again only because the message payloads are "Pd sized" payloads. Add some configuration state for a "table" key that takes a 128 element payload are you might start to see problems (depending on when and how this configuration state gets set.)

So: One Pd weakness is that there is no mechanism to initialize a [list] object with no contents but n pre-allocated slots. If you could send a message, say, "alloc 20" to a [list prepend], then you would have a concatenator that could absorb 20 items before incurring the cost of allocating more slots. Currently there is no way to avoid that cost, even for short lists.

Concatenation is really useful. It would be better to support it well, rather than trying to find weird ways to avoid it.

At least when I look at OP's problem, I see global receiver names followed by messages.

That's a good insight.

hjh

FWIW: Three recent threads where for loops and "forward to a float box and bang it later" were crucial.

https://forum.pdpatchrepo.info/topic/13315/how-can-i-get-the-original-order-of-a-accending-decending-resorted-list

https://forum.pdpatchrepo.info/topic/13308/draw-straight-line-into-arrays

https://forum.pdpatchrepo.info/topic/13311/check-if-float-in-list

E.g., from the third,

Hey, anyone know a good way in pd vanilla to check a number stream against a list, and output a bang if the number is in the list?

I've used awful, brute-force solutions to this problem with tons of [sel] objects and message boxes, but these are messy and inflexible. It's a simple challenge logically, so I'm hoping there's a better solution.

I still wonder if these problems would be less baffling if we gave names to some of these idioms and actually talked about them... well, I'm going to try that in my class this semester, see how it goes.

hjh

@ddw_music In most tutorials the underlying idea is explained very early: The difference between cold and hot inlets. Surely cold inlets are used in most cases to store values which are processed later.

Another crucial basic element is [trigger] which is in some sense the counterpart to cold inlets: Compute that first, compute this later. Triggers are also crucial to understand the order of operations (right to left, all the way down).

With these basic principals all the questions above can be solved and they are explained in tutorials. It is just necessary to understand them and to get familiar with them.

I am with you in that these kind of questions are asked repeatedly. So something could be done better in communicating these principles. I just wanted to argue that they are not obscure but the very core of how Pd works.

I thought [list store] and a "get" message avoided the need to split the list.

Why do you desire to avoid splitting a list? The cost is a) the pointer math necessary to index into an array in c, and subtracting the new index from the length of the list..

If the list you want to split somehow already exists inside your [list store] object, then both methods of "dripping" will/should be roughly comparable in speed.

On the other hand, if you are receiving a list from an upstream object chain (e.g., you're inside an abstraction purpose-written to split lists) and want to "drip" it, [list store] requires copying the entire list inside the object before splitting it. For very large lists that operation is not negligible. The [list split] object doesn't require any copies and is why it's used in that insane-looking recursive implementation of [list-drip]

This is why I rhink the [list-drip] implementation is instructive-- it shows clearly the problems that come trying to do recursion in Pd, and it shows clearly how writing performant abstractions often requires drilling down past the language's class boundary and directly reading the implementation. (Or the user has to pentest relentlessly to guess-and-check at the class boundary. Either way, it is unfortunate.)

Edit: clarification

@jancsika Not sure how you get to this. Using [list store] is almost three times faster than the list-abs solution, especially for large lists:

test-list-drip.zip

https://forum.pdpatchrepo.info/topic/11122/faster-list-drip-with-list-store-pd-0-48

@ingox Ha, I guess this highlights the number 0 paradigm for Pd-- always measure first!

It looks to me like message-passing overhead dwarfs the time it takes to copy a list once into [list store]. That single full list-copy operation (a for loop if you look in the source) is still significant-- about 10% of total computation time when testing with a list of 100,000 elements.

However, if you remove all message-passing overhead and just code up [list drip] as a core class, you consistently get an order of magnitude better performance.

So we have:

1. original list-drip: slow, but I can't find the code for it. But I hope it was just copying a list once before iteration, and not on every pass.
2. matju's insane recursive non-copy list-drip: faster than the original implementation
3. list store] implementation of [list-drip]: looks to be consistently 3 times faster than number 2 above, lending credence to the theory that message-passing overhead is the significant factor in performance. (Well, as long as you aren't copying the entire list on every pass...)
4. Add core class [list drip] to x_list.c and use a for loop to output each element from the incoming vector of atoms without copying anything. I quickly coded this up and saw a consistent order of magnitude performance improvement over the [list store] implementation. (Didn't test with gpointers, which will probably slow down every single implementation listed here.)

I think Matt Barber even had a case where removing a [trigger] object and replacing it with bare wires (and just depending on creation order) generated enough of a performance difference in a complex patch to warrant doing it. (Edit: this may have been in an abstraction, in which case the speedup would be multiplied by the total number of abstractions used...)

It's too bad that message-passing overhead makes that much of a difference. And while I can think of a potential speedup for the case of [trigger], I'm not sure there's a simple way to boost message passing performance across the board.

@ingox said:

@ddw_music In most tutorials the underlying idea is explained very early: The difference between cold and hot inlets. Surely cold inlets are used in most cases to store values which are processed later.

Right, but... "these kind of questions are asked repeatedly" so there seems to be a disconnect between the concept of hot/cold inlets and the use cases for them.

The broader context for my thoughts here is the question: What are some effective ways to teach patching (or, what are some ways to teach it that don't work well, in order to avoid them)? How to highlight the importance of these techniques? I'd suggest that their current format in tutorials isn't fully getting the message across.

hjh

@ddw_music Yes, i guess you are right. So, to get constructive here, the examples in your original post above are great. Only the fourth example is unnecessary against the third (and impractical) and the sixth is too complicated, can be simplified (no list loop needed, just append $1 and list store).

I applaud any improvement to knowledge distribution.

@ddw_music Also i like to add that i don't think that naming things after concepts of other languages is particularly helpful (i.e. for loop for an until construction), as many people come to Pd from music, not from programming. Or the course should be called "Pd for programmers" or something. The danger here is that in some aspects Pd just has different concepts and forcing the thinking in other languages onto Pd can also hinder the understanding, like in the function call debate.

@ingox said:

@ddw_music Also i like to add that i don't think that naming things after concepts of other languages is particularly helpful (i.e. for loop for an until construction), as many people come to Pd from music, not from programming.

Here, I think graphical patching is only partially successful. If the intent was to give nonprogrammers access to a level of control over the machine that normally requires programming, but with a more approachable interface, it seems to me that graphical patching ends up being just as fussy and technical as programming... but then, the wish to shield users from "programming" means that a lot of the explanatory material cuts itself off from programming concepts that are useful for solving problems in Pd.

I don't think Pd users would benefit from material that expects them to know in advance what a for loop is (of course most will not). I do think they would benefit from material that shows them what a for loop is (and gives it a name), how to build it in Pd, and when/why you would use it.

Or the course should be called "Pd for programmers" or something.

I'd guess that it's more likely for users of patchers to migrate towards text languages rather than the other way around.

The danger here is that in some aspects Pd just has different concepts and forcing the thinking in other languages onto Pd can also hinder the understanding, like in the function call debate.

I think part of my concern here is about the area of friction when the requirements of a musical algorithm don't line up neatly with the strengths of patching.

hjh

@ddw_music Yes. Just [until] is not really a loop and "for" would reference an "i" that is not there. In Pd language, bangs and other messages are sent down the object tree and [until] sends multiple bangs in sequence.

@ddw_music I think determinism..... the order of operations.... is the concept that you need to hammer into the minds of your students. Once grasped they will be able to "see"..... to read a patch and visualise the data flows in the same way that you (with your knowledge and understanding) are able to visualise code in a programming language.
Almost all the errors that Pd users bring to this forum are caused by their not understanding the data flow..... and more often not even knowing that there is an order to operations.

I have just made a first attempt at teaching the order of operations here....... https://forum.pdpatchrepo.info/topic/13320/welcome-to-the-forum

I have locked that thread, but comments are welcome on this thread...... sorry @ddw_music I just (partly) hijacked it...... but on the same theme I hope.....
I have tried to make it as basic as possible...... while still introducing the concept of value storage.
David.

@ingox said:

@ddw_music Yes. Just [until] is not really a loop and "for" would reference an "i" that is not there.

But when you use it with a float box and an increment, then there is an "i" ... which you did exactly in the "scan a list for a value" thread.

If list is an array of pointers, that's effectively for (i = 0; !list[i]; i++). It happens here that the end condition is not based on a count but that doesn't significantly change the concept.

The Pd name for this construct doesn't have to be "for loop" (agreed that "loop" is somewhat misleading) but if it has a name, then users can recognize it when they see it, and copy it when they need it. Or maybe it could be an abstraction.

It remains true that branching and repetition cover pretty much everything a programming environment needs to do.

I saw the welcome thread -- that's great! Very helpful.

hjh