Something I was thinking about yesterday, while adding to my [text sequence]-based abstractions:

[text sequence] is a stateful object, in that the result of [step( depends on the previous state.

What if you need the result to be handled differently in different cases?

Specifically, I'm rejiggering the data so that it outputs the time delta to the next event alongside the data list, rather than the time delta from the previous event. As a step sequencer, in pseudocode, it looks like this:

coroutine
    t = text sequence blah blah;
    list = t.next;
    #time ... data = list;  // [list split 1]
    time --> time outlet;  // withhold data now
    wait for bang;
    while
        list = t.next;
        list isn't empty
    do
        #time ... data2 = list;
        time --> time outlet;
        data --> data outlet;
        data = data2;
        wait for bang;
    loop;
end

So there's an "initializer" handling of the [text sequence] result, and then a "normal" handling within the loop.

For stateless operations like [+ 1], you can just replicate the operator for the different contexts. But because [text sequence] is stateful, you can't do that.

I could actually think of a nice, clean solution for numeric results -- I'm OK with this as it would scale up to any number of references:

But [value] can hold only a single number (why? wouldn't the equivalent of a 'variant' type be useful?), and [text] is dealing with lists.

I ended up using the [list prepend] --> [route 0 1] trick -- fair enough as the above routine uses the sequence in only two places. It might not scale when things get more complicated (for instance, in https://github.com/jamshark70/ddwChucklib-livecode/blob/master/parsenodes.sc I'm passing a stateful string iterator through dozens of functions, haven't counted them but I guess it's easily 50 or 60 places -- from a computer science perspective, a parser is not an exotic use case -- quite common really -- [route 0 1 2 3 4 5 6 7 8 9 10 11 12] oh, I give up already ).

Wondering if there are other options. It would be really nice if graphical patchers had a concept of a shared object reference, replicating the 't' variable in the pseudocode, but I guess that won't happen soon.

hjh

Thanks for all the suggestions. Using [text] as a list-register is not a bad idea.

The other side of the topic is, however, pondering not only workarounds, but also ways to improve graphical patching itself to reduce the need for workarounds. I sometimes feel when using Pd that I'm spending a nontrivial amount of time working around limitations -- and not only in Pd's core object set, but also in the design of graphical patching in general.

This is a concrete case where text programming languages' concept of a function/method call is richer and more robust than patch cables, because the target that will receive the function's output is not fixed, but rather depends on the place that called it. Multiple references are then entirely idiomatic. I think this might be one reason why graphical patching has not supplanted text languages in computer science generally: while it's very well adapted to trigger responses, and describing signal processing, the lack of a function call that brings the result back here is a pretty hefty obstacle to many types of computing. (And, when something is difficult, users tend to avoid it... so the problem doesn't really get solved.)

I'm using Pd in teaching because I have a few students who have really picked up the graphical-patching ball and run with it. (SuperCollider, despite being more richly featured, has largely intimidated almost every student of mine who has approached it.) So I fully appreciate that graphical patching has advantages... but I can't shake the feeling that there's a complexity wall at some point, and it's hard to climb it. I wonder what it would take to reduce the slope of this wall.

@jameslo

Would you be able to side-step this issue if [text sequence] had a post-wait mode? Or is the pre-wait requirement a consequence of loading data from standard MIDI files?

One, I'm not considering MIDI files anymore. That's off the table. Also, pre-wait is a recommendation of [text], isn't it? (Personally I disagree with the pre-wait design, but it seems baked into [text] for whatever reason.)

Two, I did make an abstraction to output post-deltas -- but I didn't arrive at this solution, which is more elegant -- close to it, but not so clean. I probably got confused at that time by advice from the forum that the wait time should always be "pre." So... I'll rewrite that abstraction sometime.

Thanks!

hjh

@ddw_music Well, with [value], [array] and [text] we at least have nonlocal variables. Nonlocal functions are more difficult, @jameslo built some in the examples above.

This obstacle of course stems from the data flow paradigma. The common solution is to use abstractions: Build your general function as an abstraction and use it in place locally within the data flow.

If you then want shared values or states between abstractions, there are several ways to do it: Update all abstractions via [send] or use central data stores within the main patch.

@ddw_music As to the waiting problem: I always saw [text sequence -w 1] as a highly specialiced solution for only a specific set of use cases. My tendency would always be to build the logic i want outside of [text].

So another solution to the one @jameslo proposed without [text sequence] could be this:

text_post_wait.pd

@ddw_music I've read that spreadsheets are also "data flow languages"--I wonder how similar they are to Pd WRT algorithms that are difficult/inconvenient to implement. When I was a working coder, a colleague who was a database admin showed me how he used spreadsheets to help him write SQL stored procedures--I was amazed.

Wondering if there are other options.

Use [send] and [receive].

You can set up an abstraction which just forwards the message on to your global sequencer along with the abstraction's "$0" value. You use that "$0" value to set the symbol to the right inlet of a [send] that connects to the output of the sequencer-- something like that "$0" value prepended to "_the_output". Now just let that output flow from the sequencer to that [send] object.

Inside your abstraction, you have [receive $0_the_output]. Just hook that to an [outlet] and you're done.

Now that abstraction receives the output for whatever method it called, another abstraction receives its own output, and so on.

@jancsika

You can set up an abstraction which just forwards the message on to your global sequencer along with the abstraction's "$0" value. You use that "$0" value to set the symbol to the right inlet of a [send] that connects to the output of the sequencer-- something like that "$0" value prepended to "_the_output". Now just let that output flow from the sequencer to that [send] object.

Ok, this is promising.

Note that "sequencer" is just an example. I'm referring to a general problem which currently has no general solution in Max or Pd. Maybe other dataflow environments have found a solution.

Inside your abstraction, you have [receive $0_the_output]. Just hook that to an [outlet] and you're done.

Hmm, no, the point is to decouple the output from one and only one path. I think I would just have the wrapper abstraction do the "send" and the caller is responsible to have a receiver for it (and to request that the output be sent to the specific receiver).

hjh

I'm referring to a general problem which currently has no general solution in Max or Pd. Maybe other dataflow environments have found a solution.

As is often the case with Pd, it's not ergonomic. But it does generalize. You can dispatch arbitrary pd messages to and from any stateful object or abstraction using this technique.

You can probably also use abstractions to wrap the sends/receives in order to hide the ugly $0 passing.

Hmm, no, the point is to decouple the output from one and only one path.

It decouples the output of your state machine from one and only one path. You get N paths where N is the number of abstraction instances you use to, say, implement the Supercollider state machine thingy you linked to. (Just to clarify-- each abstraction only generates output when a message gets sent to its own inlet.)

If you're thinking that you now want each abstraction to dynamically output to different paths, either I don't understand what you're trying to do or you are overengineering your solution.

Related-- one mechanism that would be nice is the ability to have a singleton as part of an abstraction-based library. For example-- perhaps you want an abstraction that forwards messages to and from a sequencer, and you want the sequencer itself to be part of the abstraction. There's no easy way to do that.

As a result users generally just put the singleton thing-- array, text object, etc.-- in the main patch, adjacent to the object chain that reads and writes the thing.

@jancsika

It decouples the output of your state machine from one and only one path.

I'm afraid I'm not quite following you.

Returning to a sequencer as an example -- if I want the output of a [text sequence something] to be directed to different places depending on the system's state, the one thing I can't do is to have multiple instances of [text sequence] -- because each instance has its own position that is independent of other instances.

You get N paths where N is the number of abstraction instances you use to, say, implement the Supercollider state machine thingy you linked to. (Just to clarify-- each abstraction only generates output when a message gets sent to its own inlet.)

... which means each abstraction instance must have a separate sequencer instance.

That isn't going to work.

The only way I can think of to make it work is to have one abstraction containing one sequencer, and use the [send] to reroute to the appropriate [receive].

Here, the upper construction is wrong; the lower one gets the result I want.

20-1024-multiseq.pd

hjh

PS Ergonomics do matter. The systems we use exert some influence over the ideas we're willing to consider (or, in extreme cases, the ideas we can even conceive). If something is inconvenient enough to express, then it restricts the range of thoughts that are practical within the system. In part this means we should choose systems that are closer in line with the thoughts we want to express, but it also means there's no harm in extending the system's reach.

@ddw_music I think you are constructing a problem where there is none.

Functions are from the functional programming paradigm, abstractions are the equivalent in the data flow paradigm.

You can even simulate function-like behavior in data flow, as shown above. The method @jancsika mentioned with sending $0 is basically the same @jameslo posted as the third example in his first response.

Although it is possible to simulate function-like behavior in data flow like this, this is very clunky and complicated.

Fortunately it is completely unnecessary.

You can simply use abstractions within the data flow. If you like them to share a state, this is possible not only using a central storage in the main patch, but also by using [value] within the abstractions themselves.

Yes, [text sequence] cannot do this. You would have to replicate the behavior of [text sequence] with simply [text] by storing the current state somewhere else.

Take a look here: shared_state.zip

In this example, i had to put in the content into each [text] individually. This can be automated by loading the text from a file.

I think it is called a paradigm because it takes another perspective to solve the same problem in a different way.

@ingox Yes, control flow is paramount for sub-routines, and where most mistakes are made.
Same same in Pd.
But I find it much harder to fail in Pd, with the determinism afforded by [trigger] and [route].
David.

So, to sum up the strategies presented so far:

• Global storage -- Pro: Super-easy for numbers (with [value]), fairly easy for other entities with [text] as storage. Con: Potential for name collision, but that can be avoided.
  1. Trigger the sequencer/counter/whatever and save the value in a global location.
  2. The caller reads from the global location.
• Changing [send] target -- Pro: Caller directly specifies the return point. Con: A bit "magical."
  1. Caller sends the request with a [receive] name tag.
  2. The pseudo-function updates a [send] with the name, and passes the data there.
• Abstraction, globally-stored index -- Pro: Call and return are all in the same place. Con: Global storage name is hidden in the abstraction -- be careful of collision. Also, in the provided example, redundant memory usage (three copies of the [text], all with the same contents).
  1. Abstraction increments a counter, which is shared across all instances.
  2. Gets from [text] at that index and returns directly.

It seems to me that any or all of these could be useful in different cases -- the globally-stored index might be perfect in one case, while a name-switching [send] might be better in another. (Tbh I don't see this as "clunky" at all -- if you have widely spatially disparate references to the same source of data, as may easily be the case in a large and complex program, a simulated function call might be the cleanest way to represent it. Also, it seems to me that in all of the "shared storage" approaches, there is some risk of name collision, but this risk essentially doesn't exist with the send/receive approach, because the caller is completely in control of the location where the data will be returned. Oh, and I'm seeing a bit late that this is one of the options jameslo suggested.)

As you've probably gathered, I'm much more comfortable with text languages. (With apologies in advance... tbh I think in most cases, you can go farther, faster, with text languages. With the exception of slider --> number box, pretty much the only thing I can think of that Pd can do better than SC is [block~], which SC simply doesn't have. Especially when complexity scales up, I find text to be more concise and less troublesome. E.g., if the synthesis code is structured correctly, I can create 3, or 10, or 100 parallel oscillators just by changing one number in the code, and without saving extra files to use as [clone] abstractions. I would love to see some counterexamples here! What am I missing? Being proven wrong means that I learned something.)

WRT to Pd, after about a year of use in a couple of courses, I feel like I'm starting to get a bit better at it. For me, the most important point of this thread is to learn dataflow vocabulary for the text-programming concept of function calls. This discussion has been really valuable for me.

hjh

@ddw_music Yes, great summary!

To add a few points:

The simulated function via setting [send] to the caller of the "function" is possible, but it somewhat contradicts the data flow paradigm. It is important to see that in Pd, every operation is done sequentially, in a strict linear order of operations. So you are actually always in control over what is happening. Sure, if you like this approach better, go for it. I just think it is not necessary under any circumstance.

Name collision can be avoided by using $0 in every [text], [array], [value], [send] and [receive]. $0 of the main patch can be handed down to abstractions, so they can communicate over this somewhat protected channels. In addition, abstractions can get ids from the main patch, like [abstraction 1], [abstraction 2]. With these two techniques combined, you are flexible while in control.

The example i have given can also be done with a central [text] in the main patch to avoid the redundant memory usage: shared_state2.zip

As of the flexibility of generating many instances of code on the fly, i think you are right. Other than [clone], i think there is nothing. Overall, i do agree that text based coding is more efficient, clearer, more flexible and altogether more powerful. Patching is just more fun.

@ddw_music

I'm afraid I'm not quite following you.

Let me try again.

Imagine you've got a single sequencer in the top level of a patch.

Feeding into its input, you have a global receiver [receive in]. (For the purposes of this example it's global. This is just didactic, and we could fix that later if we wanted.)

At the bottom of your single sequencer, you are connected to a [send] object with no argument.

Now, you are also going to build an abstraction which you can use with that same patch. It is not part of the sequencer, it's something different. Inside this abstraction you have something like this:

[inlet]
|
[list prepend $0]
|
[send in]

[receive $0-my-output]
|
[outlet]

Finally, you just have your sequencer slice off the head of the incoming message to [receive in], prepend it to the string "-my-output", and send it to the right inlet of the [send] with no arguments. Finally, you trim the list selector off the rest of the message and have your sequencer handle that message as it normally would.

This will result in the sequencer sending a message only a single abstraction instance.

Thus, you have a single state machine accessible by an arbitrary number of abstractions which can advance the state by sending and receiving their own arbitrary messages to and from that single sequencer.

but it also means there's no harm in extending the system's reach.

There is harm in extending it the wrong way. For example, the "init" message of the iemguis is often a harmful feature. Imagine tracking down an insidious bug in a program like this:

|
[float]
|
[tgl]
|
[== 0]
|

vs. this:

|
[float]    [preset_node]
|         /
|        /
|       /
|      /
|     /
|    /
[tgl]
|
[== 0]
|

If you get unexpected behavior from the first example and forget (or never knew) that there's an "init" checkbox for that [tgl], you have to do creative debugging in order to find it:

1. Right-click and open the dialog to check init state, probably because you hit such an insidious bug before.
2. Close Pd, run with the "-noloadbang" flag because you've learned that if that fixes the bug there is almost certainly an iemgui init or a [loadbang] deep in a subpatch somewhere.

But the one thing you can't do is simply read the patch! Not only can you simply read the 2nd example, the secondary loadtime data flow is immediately obvious.

I understand the desire to just say, "forward data to this object and return me the results." But depending on how it's implemented it could be a boon or a footgun. The obvious direction of pointing to "this" object using a gpointer don't work very well in a diagram-based language.

For me, the most important point of this thread is to learn dataflow vocabulary for the text-programming concept of function calls.

I don't think Pd is a dataflow language. The signal diagrams might count as that. But the non-DSP objects are a weird kind of Rube Goldberg-machine of imperative programming. (And they can interact with the DSP objects in a diagram.)

It creates an irony where the most difficult conceptual part of Pd-- DSP programming-- gets the simplest diagrams. E.g., all objects are guaranteed to receive their inputs before they compute their outputs. While the simplest conceptual part-- sending sporadic arbitrary data to immediately compute an output-- requires special objects to keep the order of operations correct.

@ddw_music Just for fun i built this general wrapper to use an abstraction like a function. The only requirement is that it has just one inlet and one outlet. The input is sent to the place where the function is defined and the output is sent back to where the function was called. The [value function_state] is only to prevent multiple returns if the function is defined multiple times by accident.

Anyhow, below is the simple solution just using the abstraction directly.

function_demo.zip

@jancsika

Imagine you've got a single sequencer in the top level of a patch.

Somehow, from your first description, I didn't get this idea at all Basically it's the same as jameslo's second example here and my second example here -- the latter of which explicitly strips the [send] name off of a list btw.

@ingox

The simulated function via setting [send] to the caller of the "function" is possible, but it somewhat contradicts the data flow paradigm. It is important to see that in Pd, every operation is done sequentially, in a strict linear order of operations. So you are actually always in control over what is happening.

I do see the point -- I think that concern is somewhat mitigated by the fact that the intention here is to send to one and only one [receive], and (deterministically) this [receive] object will trigger its chain before the [send] yields control.

If there are accidentally two [receive] boxes with the same name, that would indeed be non-deterministic, but also user error.

Patching is just more fun.

I do find it fun in the sense of learning a different way of thinking. I don't know that I will ever really love patching in the same way that I love coding though.

hjh

This thread would have saved me two weeks of work if it had come around two weeks ago. Somehow I muddled my way through it and a pd implementation of Befunge-93/Funge-98 is essentially a reality. We should have a meta thread to collect good threads such as these.

@ddw_music said:

I think that concern is somewhat mitigated by the fact that the intention here is to send to one and only one [receive], and (deterministically) this [receive] object will trigger its chain before the [send] yields control.

If there are accidentally two [receive] boxes with the same name, that would indeed be non-deterministic, but also user error.

Yes, sure. From Pd's point of view the order of operations is still deterministic if there are multiple receives. The one created first will be executed first. From a user point of view it can lead to unexpected results, for example if the patch is opened twice and the receives are not protected by $0.

For me, the most important point of this thread is to learn dataflow vocabulary for the text-programming concept of function calls.

This is a decent starting point, but more and more it will lead you astray.

For example, your initial text pseudo-code leads quite naturally to a notion of passing around references to objects, which is what Pd scalars typically depend on. That's like anathema in a diagram-based language, and it's the reason nearly nobody uses data structures in Pd aside from demonstrating cleverness. But there's nothing in the text-based programming world to clue you in to that.

Additionally, there are useful patterns in Pd that you won't discover by thinking in terms of functions. There's a common pattern of sending a message through an object chain to be filtered, a lot like a vertical version of piping data in a shell script. There's another common pattern where the same message flows through an object chain unchanged so that each abstraction can trigger its own side-effect. Finally, you could leverage ingox's "function" abstraction to create an object chain that steps through a state machine in a quite readable way. (In fact ingox's wrapper may be more abstraction than you need for your state machine since you could just make a custom abstraction with the exact kinds of inputs/outputs that you need for that specific case.)