@ddw_music I never said pd did not have a stack, I said that I did not think it had a RETURN stack. I don't think I can reasonably respond to your post without risking more confusion.

@FFW I sort of thought you meant something like that but wasn't sure. I don't think message rate stuff is flattened and can't find anything in the source but I don't really know, I just don't see much advantage in doing it with how slow and simple message rate is. Thanks for the clarification.

@FFW said:

By the way, it's also the order the process stack is filled because the full tree is flatten before it computes

Is there a process stack? Can you tell me where to look in the source for it? What is actually on this stack? Is my above reasoning regarding the patch just being a chain of objects incorrect? The patch being a chain of C objects explains why we can't turn off message rate and what happens when pd is running out of cpu but I have not quite grasped how pd's scheduler actually works. I'm guessing my previous post is difficult for a non-native English speaker since my grasp is not great and my explanation is probably not the clearest and I am not sure how well you know pd's internals either. I am almost at the point where I will post on the mailing list to get a response from the devs but I want to get enough of it that my post is not just a very verbose way of saying "I don't get this, explain the entire inner workings of pd for me!"

@ddw_music All that stuff about instruction pointers and the like was me getting turned around and falling back onto Fourth because that is where my mind goes when I think about stacks. In pd there is no instruction pointer so never any need to return or backtrack or goto. In pd's core, every internal and external object is a literal OOP object and the inlets and outlets are methods of that object. When we create a new object pd creates a new C object of that type and makes an entry for it in glist with its pointer and UI details; when we connect a wire to its outlet pd retrieves the pointers to those objects from the glist and then goes through the outlet method. I am not quite sure how it deals with reaching the end of the branch, I suspect that the last object always has its outlet method run to point it at the branch point and when we append a new object pd just passes that to the new object when it updates the previous end of chain object's outlet method to be the new object.

Some details there might be a tad off, I am not quite good enough with C pointers to really follow pd's source yet, but I am fairly sure that is the mechanics of it. I have no idea about how Max handles this stuff but I would assume it is quite similar.

@ddw_music said:

In Pd, wires are procedure calls, not gotos.

Are they actually procedures or just conceptually? I would assume the latter but if they actually are procedures can you point me where to look in the source? I get the depth first stuff, I just turned myself around in those edits, hence the follow up post and the disclaimer added to the original post. I did a bit of digging in the source and I think my original point was correct, pd does not do anything weird with how it uses the stack, it is just that without a return stack we can't actually return which means we have to go about/think about things in a different way.

lisp would write (print (* (random 20) 2)) where the last operation to be performed ("print") is at the head of the tree

But there is no tree in lisp, right? it is just a list data structure for the REPL to evaluate. I can't quite grasp how lisp works under the hood, I poisoned my brain with one of those build your own lisp tutorials which I later learned did not actually function in anyway like lisp proper, it was just an easy way to do a lisp like language and I have been lost ever since. Lisp is one of the languages I really want to learn and understand because it is supposedly great for building languages but there is something about it which I fail to grasp.

Regarding Forth since you keep bringing it up, it is no where near as mind bending as people think, in practice you don't spend much time doing that low level RPN stuff; you define some words and use those words to define other words and before you know it you have a purpose built high level DSL for your task. But you have to learn all that low level RPN pushing and popping stuff first. I am a big fan of Forth and what got me interested in language design/implementation was the whim of implementing a language in pd, so I found a language which would seem suitable for that whim and it was Forth, I learned Forth by implementing it in pd.

I guess I went off on a tangent there, went into implementation and lost focus. The important thing there is that abstractions are not functions which you return from or objects you point to, they are macros which expand in place. We can sort of use them like objects or functions but they are macros from the users perspective and that is what we need to keep in mind.

Probably best to skip this post and just read the next, will leave it here since it could result in useful replies from those who understand the internals of pd and perhaps some people here are as interested in these sorts of details as I am,

@ddw_music I think pd uses the stack in exactly the same way but the difference here is that pd lacks a return stack (and stack/instruction pointer manipulation); abstractions are actually macros which are expanded, they are not functions which are pointed to and returned from. The unique ID given to an abstraction when it is expanded allows us to use them sort of like functions and mimic scope but pd is a very simple globally scoped imperative language.

Edit: I guess pd does have very basic IP manipulation, [send] is essentially a goto. The big difference with pd and dataflow is that the IP carries a bit of our data with it as it moves through the patch. The result of an operation does not go onto the stack but is carried with the IP. Or maybe data is only carried with the IP in a conceptual sense and result actually is TOS? Think I will have to investigate pd's stack when I have the time.

Edit2: I guess wires are also gotos then. I suppose what this really is is that pd is a 2d language, [send] and wires only seem like gotos if we try and think of pd as a 1d text language.

Edit3: (last one) Pd would have a return stack, but the returns always point to branches. A [t b b b b] is essentially push and drop commands for the return stack, rightmost bang pushes, leftmost drops. End of the branch having an explicit pop so it can return to the [trigger] or where ever the code branched. More like other languages than I thought in stacks, assuming my reasoning is sound, the big difference being that pd has no concept of anything remotely function like. IP probably carries the pointer as well as the data, would save popping everytime it return which means no return stack and back to my original point of pd lacking a return stack (subrosa edit4.) But that would mean the IP caries a return stack in a rather funge like way, which it might, the return stack is part of the IP object and not the interpreter proper (subrosa edit5.)

@ddw_music said:

"Return to caller" as in any structured or object-oriented language is nicer

I think this thread may have convinced me that the idea of return to caller is antithetical to dataflow, it goes against the flow of data and my solution which you said side stepped it is just restating the problem into dataflow. Not that this was my intent, didn't even realize I sidestepped it until you mentioned it but it got me thinking. Not sure if it is even possible or practical to avoid but I think I am going to try and be more rigid with the flow of data for awhile and see what happens.

Here, I'm referring specifically to

That is what I originally thought but the previous post confused me, I now realize that you weren't addressing my patch directly but offering a summation of what you got from the thread in your reply to me.

whereas most other solutions posted here seem to assume it will always be a single float):

Just replace [array] with [text] and make a tiny change in stream.pd/add other abstractions asStream can load/change the arguments it is loaded with. I switched from [text] to [array] because you used an array in your SC example and I switched tactics to confront your example head on. Originally I thought your ignoring the suggestion of [value] was that you wanted to use things other than floats and then thought that what you were having issues with is the methodology. This is probably a part of the problem porres mentioned about people having issues transitioning to pd from another language, you have a good grasp on programming and when you ignore a solution which seems the easy/obvious solution to us it is easy to assume you have a good reason for ignoring it. These sorts of assumptions are probably reasonably common in such situations, we all gave up on the suggestion of [value] early on and it came back into the discussion purely on accident.

It has been an interesting thread, the failures in communication ended up surprisingly insightful on many fronts.

@porres I am more than happy to answer any questions you have about using it or how it works. As I said in the other thread, I hate doing documentation, it tends to turn into a rabbit hole for me and eat up a good deal of time; once I start thinking about usage I start seeing how some things are not right in the abstraction and I start fiddling which means a change to the docs and it quickly turns into a feedback loop. That small change to the manual I suggested in the other thread caused/required me to reread much of section 2 and reread a few sections many times even though I had it all typed out after reading two short section once.

I don't think extending pd would solve this issue, the question was not really how to do this in an isolated situation but about the methodology. How do you do something like functionA (functionB (1 2) 849 functionC ($1 $2) asdf) in pd? In a one off situation it is generally a fairly simple problem to solve but to make it so at anytime you can do the equivalent of [abstractionA [abstractionB 1 2] 849 [abstractionC $1 $2] asdf] is a much more complicated issue and one which externals don't solve and often make it seem more complicated than it actually is because they have a tendency to make pd feel like a higher level language than it is, keep the user from ever really learning the nuances and techniques of the low level dataflow paradigm that is pd. The one off solutions which often come up in these threads result in what I think @ddw_music refers to as "spray everywhere," to try and build that solution into every abstraction so you can use it everywhere but to get the equivalent of abstractions calling abstractions would require you to fill your abstractions with [v ]s and [send]s and [receives]s in the vain attempt to cover every possible situation and very complicated/convoluted bit of inter abstraction communication. All my posts since the first asStream screenshot post have been about methodology, how to build higher level paradigms in a low level language like pd and avoid spraying everywhere, this is what asStream is meant to show. Like any well founded low level language pd is quite good about giving you the tools to build out higher level functionality and adapt it into your own language. At least that is what I got from this thread.

asStream has the potential to be a really useful abstraction but still has some issues, I can help you understand it and flesh it out, just ask your questions, they will probably make it a better abstraction than either of us could make on our own. As an exercise I will attempt to repatch it in your heavily subdivided patching style on my next break from my projects, I will probably fail but it will be interesting, I get the logic of the style but it obfuscates the dataflow for me and anytime I want to understand the work of a subdivider I have to remove all that subdivision, I can't even see how it could be subdivided into discrete, well named subpatches of highly singular/specialized function, for me it already is that.

@porres No help file, usage notes in asStream.pd, this thread is the documentation. If I ever get it to the point where I don't think there will be any major changes I will upload it with help files over in Abstractions. But a new version, reworked it as an array object since that is what it really is. Also added looping. Still needs lots of work to be a useful abstraction for more than learning from, it is kind of fun though and eventually I will develop it more.

asStream.pd
stream.pd

@ddw_music said:

Though this means that your approach could be used only in self-timed situations,

Just need to change [obj 10 10 stream $1 $2 $3...( to [obj 10 10 $1 $2 $3...( and add in some simple logic and now asStream can load the abstraction you specify through methods/arguments. As I said earlier, you design the interface and the protocol which is what those sends and receives and values and dollar arguments in asStream are, an interface and protocol for communicating with the stream.pd abstraction or any other abstraction designed to work with that interface. Object names themselves can also be filled in through arguments, [$1 $2] can be either a metro or a receive or anything else but [$1 $2 ....] are actually [list]s so they only give you two inlets and one outlet, if you need more than that you need to resort to editing the abstraction in a text editor, patch it together with any object with suitable io and then open it in a text editor and edit that object into $1 $2... And there is plain old dynamic patching to programmatically create the required patch. Lots of option and they can all be mixed.

Realized my errors finally, extra output was because I forgot to add a [- 1] after a [list length]. The [v ] not resetting was order of operations but decided that the logic of automatically resetting to zero was flawed so now it resets only if you send [start( when it is at the end up the array. The stupidest mistakes are always the hardest to see. Fixed and added an optional delay before the metro so they don't have to all start at the same time. Fixed [append( as well and give it a right inlet for appending without the [append list( which is probably how it should be? Still could use some logic to make sure you don't duplicate streams and perhaps the ability to configure individual streams but that is for someone else or some other day. Added some usage notes in the patch.

Improved version below.

@ddw_music Value has been mentioned multiple times in both threads It has been fun. I took a break from my projects and threw together asStream, it will be awhile before I get back around to it but here it 99% complete, there is a stupid error on the append so it is actually insert and for some reason I just can not figure out it is not resetting the [v] when it completes. You can solve those if you care. Oh, and it spits out an extra repeated number second to last, at least with my simple test, not sure why but worst case a [change] between the [array get] and [s] in stream.pd will fix that. Report back if you figure out why it is not resetting to zero, driving me nuts but I have other things to do. Patch in next post.

Not sure what you mean by "spray everywhere," these are targeted approaches and there is no copying. Does reallocate on resizing the array but that could be avoided easily enough, just use an array bigger than you will ever need and only use what you need.

@ddw_music said:

The latter is what I'm after

Oh, that is simple enough to do, just replace the [f ] in the stream abstraction with a [v ] shared between the abstractions like the array. Getting there?

Obviously you would want asStream to create the [array define]s as needed and the symbols for the array and value having their own iterators so it can manage multiple streams but that is simple enough. Oh, and register its array and value symbols so you can grab them as needed and append to the array and what not.

@ddw_music said:

Seems to me English is not up to the challenge here.

We just needed to identify the actual issue here, which is not patching but low level language. To do the sorts of things you want to do means you have to define an interface and a protocol, those things which high level languages like SC take care of for you. Pd has no concept of an instrument and if you want to use such a paradigm it is up to you to implement it. One way would be to send the instruments args to a master abstraction which then takes the necessary actions, a quick and simple example which is not fully fleshed out but should show the logic:

Started adding in my $0s halfway through. You would probably want a [text] in [register] to keep track of all this stuff and it can also keep track of values you can query for in the various abstractions and what not through their send methods. [stream] would need some more logic, needs a [receive] to start/stop it and probably would want a way to change its settings as well as an [iemguts/canvasdelete] so you can clean things up as your patch changes. [synth1] seems pretty complete and a solid instrument, I wouldn't change it.

@ddw_music Had some free time before bed so read your link, don't see anything there which would not translate to pd/data flow without issue.

each player needs its own [metro], and a way to pull from the stream.

like this?

All of this can easily be worked with each instrument being an abstraction using the right inlet on the array get or arguments or sending the pointer. Skipped some of the details but duration and getting the stream into the individual abstractions is straight forward enough and more of the same. I can't tell if I am missing something, life outside of data flow has become weirdly abstract for me, I am one of those sorts who really took to it. I am very interested in know what I am missing if I am missing something, I have been trying to cure my data flow myopia, it has become a bit of a disability in the last year or two.

Edit: Or is it that you just don't like that each needs its own metro? If so I think your issue is not with patchers/data flow but low level languages, pd is low level. But that is what abstraction and externals are for.

@ddw_music Right, like I said, I am terrible with contrived examples, I need context, I always confuse myself trying to deal with the unknown context. The only reason you have no iterator is because you are using [list split], [text] or [array] would be more suitable here, could move the headaches somewhere else but in my experience those headaches are from fighting data flow. [list] objects are not meant for sharing data and they are inefficient at it, could end up with loads of needless memory reallocations, [text] is not much better but when it reallocates you are only reallocating one object instead of two in your example abstraction and how many others in the rest of the patch?

I will read your link in the morning in case I am still missing the point, so feel free to ignore explaining it to me if that is the case and your link explains it.

Looking forward to examples -- I can't quite get the gist from this.

You can do stuff like unpack the canvas from the patch window and fill the window with Tk widgets instead, or have the Tk widgets and the canvas tiled or tabbed, make new windows, embed other patches into your current patch, lots of fun stuff. The abstractions do all the hard work and setup the stuff for communicating with your patch.

Edit: That was a silly place to connect the [text get].

@ddw_music said:

seems to me that's what the above is (just not folded into the subpatch).

You already have the send method, you just are not using it, [list store] gives it to you for free, as do [float], [int] and [value] , [text] and [array] have the send method to get a pointer to their data. I find it difficult to think of a situation were we can not build our abstractions around these objects and easily get access to the internal state of an abstraction to send data backwards or even "rewind" the patch but you have to plan it that way. Am I missing something? I am good at missing the point when it comes to contrived examples and fail to see beyond the example.

Not sure about pointers to abstractions, think I would need an example of your use case to address that.

Quite curious about that. Anything public to see?

Current project is actually getting a few of them into a shareable state, required me making a set of abstractions which is essentially a dialect of Tk since interacting with a text based language is awkward at best in pd. Believe the last hurdle in those abstractions has been passed and will finish them all up and document/upload them this coming week so I can get onto reworking the languages to use the Tk abstractions. They will get posted in the abstraction section as they become usable, I suspect only the Tk abstractions will have any real interest but I am curious to see if anyone uses my languages or they get people to develop their own languages. Thanks for the links, will dig into them this weekend.

Edit: forgot pic. And I did not quite plan that one out well, could easily cause an infinite loop or overflow depending on where [s backwards] actually sends to earlier in the patch. But I think it shows the logic?

@impression On Bela it is more difficult to use a frame buffer and those sorts of graphics but it is a BeagleBone running linux so you can do it but they probably will not offer much help in how to do it, they seem to try and keep Bela using just the Bela products, they don't want to have to try and offer support for the massive amount of third party peripherals out there and I don't blame them for that. On a pi if you use an OLED which there is a frame buffer driver for it will act like a normal computer screen once the driver has been loaded, the OS/kernel will handle all the details of how to draw stuff to the screen for you, it will draw your desktop when you startup and when you open pd it will draw pd.

I would recommend just getting one of those cheap pis and using it for a simpler project to get your feet wet and learn and get all your beginner mistakes out of the way before diving into the big ambitious project, it will most likely be quicker in the long run. Get a couple of the adafruit quad rotary encoder boards and some buttons/pads, make yourself a simple 303 style bass synth, use the onboard sound and you can probably manage it for $50 total with most parts (if not all) being able to be reused for your big project when the time comes if you want to save money and are willing to sacrifice your first born.

@impression As I said in your other thread, if you use an OLED which has a frame buffer driver for it you will be fine, you just need to load the driver and it will act like a normal display. Here is one for the pi: https://github.com/foldedtoad/raspberry-pi-oled but if you can find an OLED that uses the same chip or one of the others that driver supports and a normal I2C interface instead of the pi bonnet it will probably work just fine with bela. There are probably others but that is just the first I found, did not dig. Could also make the pi bonnet work withh bela without much work, just would have to connect it with wires/jumpers instead of plugging it right in like you can on the pi.

Why do you only have OSC? Just assuming that based off of the OLED you found or something else?

@ddw_music said:

The best I can figure out for Pd is:

In that situation I would say that [pd my-stateful-thing] should have a send method. But [value] is also good if floats, if symbols/lists then an abstraction which works the same as value but holds any sort of data is trivial, probably some externals which do that as well.

Regarding your live-coding dialect, would be curious to see it if you shared it anywhere. Not much for live-coding but it would be interesting/informative to see how you went about things. Most of what I do in pd is designing languages, it is not so bad but you have to go about things differently than you would in other languages, instead of designing a dialect of pd I design a language including the audio generation aspects, SC style server/client or a VM. Pd is a low level language and you need to treat it as such, work to its strengths instead of fight its weaknesses.