Following up on some posts from the "Symbols explicit" thread, where there were some questions about concrete use cases.

To be honest, I'm not sure how clear this is going to be, because the best way to make it clear is to build it in Pd, and I don't have time to do something complicated.

Anyway... consider a "data stream" interface. Some part of the code asks the stream for the next value, and -- in text languages -- the value is returned to the caller. There may be dozens of places that need to access the stream, and the rule is, whoever asked gets the result.

I had mentioned parsing. If the input string is held in a stream wrapper (in SC, CollStream), then this object can be passed around the system as a function/method argument, and everywhere that needs the next character can just ask for it.

The challenge, then, in graphical patchers is that there is no concept of returning a value to the caller. A message traversing a patch cable is calling into the next object. "Returning from a function" is, in patching terms, backtracking to a prior object to see if it has more output cables that haven't been handled yet. But data don't travel backward up the stack, only down.

A file handle can be treated as a stream. Here, Pd recognizes the issue and allows shared file handles to be "define"-d: [file define xyz], and then every "caller" can have its own [file handle xyz].

If the data source is in memory, though, then the current position in the stream has to be maintained within the abstraction -- and there can be only one of those, because Pd/Max don't have a concept of a pointer to an abstraction that's loaded somewhere else. E.g., a really simple data queue (didn't implement "rewind" here) -- as an abstraction, everything would have to go through this.

So then there has to be a way to request a value from a specific part of the patch and get back to that part of the patch.

... where oid said, "In that situation I would say that [pd my-stateful-thing] should have a send method" -- seems to me that's what the above is (just not folded into the subpatch).

Regarding your live-coding dialect, would be curious to see it if you shared it anywhere.

https://github.com/jamshark70/ddwChucklib-livecode -- see https://github.com/jamshark70/ddwChucklib-livecode/blob/master/cl-manual.pdf for full documentation.

The parser (with heavy use of CollStream throughout) is at https://github.com/jamshark70/ddwChucklib-livecode/blob/master/parsenodes.sc

Example:

Most of what I do in pd is designing languages

Quite curious about that. Anything public to see?

hjh

@ddw_music said:

If your primary frame of reference is patching, then this is going to seem a bit vague or weird.

Give us a working patch in SuperCollider with an example, not too complicated thoufh

I guess I'll have to build it out in Pd for this to make any sense, but I can't do it right now.

I could build it too, but give us a patch that actually makes sound please. Again, try to be as minimal as you can

If I get it, you got one sequence and 3 independent playing from this sequence. I can envision a simple [clone] with 3 instances...

@ddw_music Right, like I said, I am terrible with contrived examples, I need context, I always confuse myself trying to deal with the unknown context. The only reason you have no iterator is because you are using [list split], [text] or [array] would be more suitable here, could move the headaches somewhere else but in my experience those headaches are from fighting data flow. [list] objects are not meant for sharing data and they are inefficient at it, could end up with loads of needless memory reallocations, [text] is not much better but when it reallocates you are only reallocating one object instead of two in your example abstraction and how many others in the rest of the patch?

I will read your link in the morning in case I am still missing the point, so feel free to ignore explaining it to me if that is the case and your link explains it.

Looking forward to examples -- I can't quite get the gist from this.

You can do stuff like unpack the canvas from the patch window and fill the window with Tk widgets instead, or have the Tk widgets and the canvas tiled or tabbed, make new windows, embed other patches into your current patch, lots of fun stuff. The abstractions do all the hard work and setup the stuff for communicating with your patch.

Edit: That was a silly place to connect the [text get].

@ddw_music Had some free time before bed so read your link, don't see anything there which would not translate to pd/data flow without issue.

each player needs its own [metro], and a way to pull from the stream.

like this?

All of this can easily be worked with each instrument being an abstraction using the right inlet on the array get or arguments or sending the pointer. Skipped some of the details but duration and getting the stream into the individual abstractions is straight forward enough and more of the same. I can't tell if I am missing something, life outside of data flow has become weirdly abstract for me, I am one of those sorts who really took to it. I am very interested in know what I am missing if I am missing something, I have been trying to cure my data flow myopia, it has become a bit of a disability in the last year or two.

Edit: Or is it that you just don't like that each needs its own metro? If so I think your issue is not with patchers/data flow but low level languages, pd is low level. But that is what abstraction and externals are for.

reading this thread, what i understand so far is this is about calling some function, store its output and retrieve it at a given point... Something like this in pd? do i miss something?
`

Seems to me English is not up to the challenge here.

So here is what I mean:

triple-arp.pd -- plugdata, and uses else and/or cyclone for audio, apologies to vanilla-land.

The notes are to play in the same order continuously, but distributed among the three players. If I print out the symbols being accessed and add note indices, it's:

next: symbol instr3 -- play note 0
next: symbol instr2 -- note 1
next: symbol instr1 -- note 2
next: symbol instr1 -- note 3
next: symbol instr2 -- note 4
next: symbol instr1 -- note 5
next: symbol instr3 -- note 6
next: symbol instr2 -- note 7
next: symbol instr1 -- note 8
next: symbol instr1 -- note 9
next: symbol instr2 -- note 10
next: symbol instr3 -- note 11 etc.

@porres 's "one sequence and 3 independent playing from this sequence. I can envision a simple [clone] with 3 instances..." sounds like it will clone the sequence -- that's not the same thing.

@oid 's version will skip notes, because it's advancing through the stream on a fast timing that is independent of the three players' timing (and the order isn't deterministic -- whereas SC's Pshuf repeats the same shuffled ordering). That's also a misinterpretation. The advancement through the stream should be passive, in response to requests from the players.

("Why would you want to do that?" Why not... it's a reasonable algorithm.)

Now, why do I care about this? Because I've done a lot with state-machine parsers and found it to be really nice to have, for instance (and speculating in Pd terms) an abstraction that parses an identifier, and another abstraction that parses a number, and another that parses some kind of bracketed group containing identifiers and numbers, etc. How do these different parts of the patch know where you are in the source string? This is easy if there is a shared object that streams out the characters one by one, and the character goes to the part of the patch that requested it. Distributing a single stream among multiple players is just a different version of the same problem -- I was hoping that putting it in musical terms would make it clearer, but that seems not quite to have worked.

It's mainly out of curiosity to see how patching handles this.

• I'm suggesting [send] with a variable target: the request tells the stream where to send its "next" result.
• Or @solipp has suggested to spray the value everywhere in the patch that it might be needed, and then the caller bangs only the one it's interested in.

I feel a bit like the "spray everywhere" solution would require more object copy/paste -- mine just has "; next symbol instr1" where the other needs a [t], a [s], a [r], and an [f]. Also my [pd stream], if it used a list instead of an array, could be type-agnostic. (I only used an array here to avoid more lectures about the perils of list manipulations, which are not quite relevant I think.) But YMMV.

hjh

@ddw_music

I feel a bit like the "spray everywhere" solution would require more object copy/paste -- mine just has "; next symbol instr1" where the other needs a [t], a [s], a [r], and an [f].

well, you can wrap it into a subpatch and copy/paste that : )
Essentially both methods are not complicated, they just have to be implemented in a low level language as @oid pointed out.

@ddw_music said:

Seems to me English is not up to the challenge here.

We just needed to identify the actual issue here, which is not patching but low level language. To do the sorts of things you want to do means you have to define an interface and a protocol, those things which high level languages like SC take care of for you. Pd has no concept of an instrument and if you want to use such a paradigm it is up to you to implement it. One way would be to send the instruments args to a master abstraction which then takes the necessary actions, a quick and simple example which is not fully fleshed out but should show the logic:

Started adding in my $0s halfway through. You would probably want a [text] in [register] to keep track of all this stuff and it can also keep track of values you can query for in the various abstractions and what not through their send methods. [stream] would need some more logic, needs a [receive] to start/stop it and probably would want a way to change its settings as well as an [iemguts/canvasdelete] so you can clean things up as your patch changes. [synth1] seems pretty complete and a solid instrument, I wouldn't change it.

@ddw_music Does this behave like a shared stateful function that returns to the caller? returnToCaller.pd

@oid said:

We just needed to identify the actual issue here, which is not patching but low level language. ... Pd has no concept of an instrument... it is up to you to implement it.

The instrument isn't the point (and [pd instr1] etc. in my example is an implementation, though a minimally simple one).

But... as I read your design... if I "; register asStream synth1" then synth1 gets a new instance of the stream abstraction, and if I subsequently "; register asStream synth2" then synth2 gets a separate instance of the stream abstraction... which is precisely not what I'm talking about.

In SC, if I define multiple sequences based on patterns, then each sequence makes its own stream:

(
var pattern = Pn(Pseries(0, 1, 15), inf);

var seq = Pfindur(24, Ppar([
    // this can be written with less code,
    // but I'd better expand it out because
    // the shorter version hides the multiple
    // references to 'pattern'
    Pbind(
        \instrument, \instr1,
        \degree, pattern,
        \dur, 0.25
    ),
    Pbind(
        \instrument, \instr2,
        \degree, pattern,
        \dur, 0.25 * 4/3,
    ),
    Pbind(
        \instrument, \instr3,
        \degree, pattern,
        \dur, 0.25 * 4/5
    )
]));

m = SimpleMIDIFile.fromPattern(seq, inTracks: 3, inTempo: 60, trackNames: [\instr1, \instr2, \instr3]);

m.write("~/tmp/parallel-streams2.mid".standardizePath);
)

If, on the other hand, a single stream is made at the top, and shared among the sub-sequences, then the notes are interleaved in time order.

(
var stream = Pn(Pseries(0, 1, 15), inf).asStream;

var seq = Pfindur(24, Ppar([
    Pbind(
        \instrument, \instr1,
        \degree, stream,
        \dur, 0.25
    ),
    Pbind(
        \instrument, \instr2,
        \degree, stream,
        \dur, 0.25 * 4/3,
    ),
    Pbind(
        \instrument, \instr3,
        \degree, stream,
        \dur, 0.25 * 4/5
    )
]));

m = SimpleMIDIFile.fromPattern(seq, inTracks: 3, inTempo: 60, trackNames: [\instr1, \instr2, \instr3]);

m.write("~/tmp/shared-stream.mid".standardizePath);
)

The latter is what I'm after (and is in fact what my example patch does). And it's not even that I care that much about the musical result, but that this is a musical context for a general programming problem.

hjh

@jameslo said:

@ddw_music Does this behave like a shared stateful function that returns to the caller?

It does (and is basically identical to the approach in my example patch above).

So perhaps this really is the best way.

hjh

@ddw_music said:

(and is basically identical to the approach in my example patch above).

Oh sorry, I missed that. TBH I've been barely hanging on I'll go back and read why you were hesitant about endorsing this approach. But don't you think that in the end, all languages have things that are convenient and things that are inconvenient, and so....so what? Before I retired I was studying functional programming languages like Haskell, and I didn't get far enough along to understand how the hell you could program without state! I understood that it was great for multiprocessing, but wow.

@ddw_music said:

The latter is what I'm after

Oh, that is simple enough to do, just replace the [f ] in the stream abstraction with a [v ] shared between the abstractions like the array. Getting there?

Obviously you would want asStream to create the [array define]s as needed and the symbols for the array and value having their own iterators so it can manage multiple streams but that is simple enough. Oh, and register its array and value symbols so you can grab them as needed and append to the array and what not.

Also thought of [v]

@oid the [v] for the index is interesting. I wouldn't have thought of that.

(I guess?) I still lean slightly toward the targeted-send approach, over the spray-everywhere approach. One reason is, if you're using lists, sending a sub-list to one place is likely to be more efficient than sending it to potentially dozens of places. It's a valid concern, about redundant list allocations; one way to deal with that is to be less promiscuous about sending data to places where they won't be used.

I do appreciate everyone's indulgence. I've certainly gotten more comfortable with patching over the years, but I'm never quite sure if what I'm doing is really idiomatic.

hjh

@ddw_music Value has been mentioned multiple times in both threads It has been fun. I took a break from my projects and threw together asStream, it will be awhile before I get back around to it but here it 99% complete, there is a stupid error on the append so it is actually insert and for some reason I just can not figure out it is not resetting the [v] when it completes. You can solve those if you care. Oh, and it spits out an extra repeated number second to last, at least with my simple test, not sure why but worst case a [change] between the [array get] and [s] in stream.pd will fix that. Report back if you figure out why it is not resetting to zero, driving me nuts but I have other things to do. Patch in next post.

Not sure what you mean by "spray everywhere," these are targeted approaches and there is no copying. Does reallocate on resizing the array but that could be avoided easily enough, just use an array bigger than you will ever need and only use what you need.

Realized my errors finally, extra output was because I forgot to add a [- 1] after a [list length]. The [v ] not resetting was order of operations but decided that the logic of automatically resetting to zero was flawed so now it resets only if you send [start( when it is at the end up the array. The stupidest mistakes are always the hardest to see. Fixed and added an optional delay before the metro so they don't have to all start at the same time. Fixed [append( as well and give it a right inlet for appending without the [append list( which is probably how it should be? Still could use some logic to make sure you don't duplicate streams and perhaps the ability to configure individual streams but that is for someone else or some other day. Added some usage notes in the patch.

Improved version below.

@oid said:

Added some usage notes in the patch

don't see it, where are the help files?

@oid said:

@ddw_music Value has been mentioned multiple times in both threads

I know, and I've used value in other places. I hadn't thought to connect it to the current index, though.

Not sure what you mean by "spray everywhere," these are targeted approaches and there is no copying.

Sure, that's a misreading. I see also (on closer inspection) that folding the metro into the stream abstraction makes the stream effectively its own caller, sidestepping the return to caller problem. Though this means that your approach could be used only in self-timed situations, where SC streams and Python iterators are more general.

Haskell people like to say that a programming language isn't worth learning if it doesn't change the way you think. Pd has actually done that for me, to some extent.

hjh

@porres No help file, usage notes in asStream.pd, this thread is the documentation. If I ever get it to the point where I don't think there will be any major changes I will upload it with help files over in Abstractions. But a new version, reworked it as an array object since that is what it really is. Also added looping. Still needs lots of work to be a useful abstraction for more than learning from, it is kind of fun though and eventually I will develop it more.

asStream.pd
stream.pd

@ddw_music said:

Though this means that your approach could be used only in self-timed situations,

Just need to change [obj 10 10 stream $1 $2 $3...( to [obj 10 10 $1 $2 $3...( and add in some simple logic and now asStream can load the abstraction you specify through methods/arguments. As I said earlier, you design the interface and the protocol which is what those sends and receives and values and dollar arguments in asStream are, an interface and protocol for communicating with the stream.pd abstraction or any other abstraction designed to work with that interface. Object names themselves can also be filled in through arguments, [$1 $2] can be either a metro or a receive or anything else but [$1 $2 ....] are actually [list]s so they only give you two inlets and one outlet, if you need more than that you need to resort to editing the abstraction in a text editor, patch it together with any object with suitable io and then open it in a text editor and edit that object into $1 $2... And there is plain old dynamic patching to programmatically create the required patch. Lots of option and they can all be mixed.

@oid said:

@porres No help file, usage notes in asStream.pd, this thread is the documentation.

You see, I'm interested in this thread, I suggested that it would be created. I'm interested in the problem and how to make it work in Pd. I read the thread. This that you're sharing is not enough for me. I get that you may be satisfied and that you worked a lot and just about as much as you care on this, but I just wanted to make this clear. Ok, I know, I could do more of my homework, but I'm born and bred in Pd-land, all this just sounds way too complicated and like explaining 'snow' to me, who's also born and bred in a tropical country (I was at the beach yesterday by the way), so more help would be appreciated.

I have brought myself into such Pd x SC challenge other times, it's not the first time that things don't get too conclusive. I hear that it's impossible to do something "x", or too hard, and I hear about complex programming paradigms, and then it comes down to a simple solution like using [value]. My idea is to maybe try and expand Pd to improve it and open it up to what some people that come from elsewhere struggle with, and maybe offer a new object in my ELSE library... but all I'm getting so far from this thread is that you can share the index of an array with [value] in multiple subpatches or abstractions.

I was interested in studying this abstraction and see if maybe I could add it if I really got what it did and what it is for, but for now it's just quite abstract and over my head.

cheers