@Jona Looks great! Yes it should be possible. But I think you would need to replace [ofTouchListener] with [_locRcv]s to listen to the event from the main patch so the module interface can listen to the mouse click according to the render order. (Just like how it's done with draggableShapes example)

[_locRcv] should be used one level lower from the main patch. This is to use main patch's local variable name and still communicate with other abstractions. Also note if you use [ofMouseListener], it will not handle multitouch on mobile devices. (it will respond to one finger at a time) That's why I used [ofTouchListener] in pdgui abstraction. But if you're targeting desktop only, [ofMouseListener] is enough and easier to handle.

If you really want to build the modular environment using multiple modules, I suggest you first consider how each module should work commonly and try to first build a minimal module that only has common attributes. (e.g. window bar, interface section, render order..) Then it would be easier to maintain and to create other module later on since you only need to add the non-common part on top of your minimal module.

I think creating such large system requires thorough planning and clear idea of how things should work in the first place otherwise it is likely that you will continuously face many unexpected problems and have to rework many times.

P.S.: You probably know this but your patch currently uses left audio channel only.

@cuinjune thanks a lot for the new ofelia update. with the method from @Johnny-Mauser https://forum.pdpatchrepo.info/topic/8698/renaming-abstraction-objects/4 i found a nice way to store the position within the main patch.
drag_v0.3.zip
and here is a very first attempt to draw cords between two points: cords.pd

@Jona Nice trick! I think you should also save the render order btw.
And interesting to see the patch cords demo.

I will consider adding the object that can change abstraction's arguments. It can be called something like [ofSetDollarArgs] I think.

I will inform you if I make any progress.

@cuinjune something like ofSetDollarArgs sounds very useful. it should also save the render order now (i dont know if its the best way, but it seems to work): drag_v0.3b.zip

@Jona Nice work! I just made the following changes to ofelia. (Not uploaded to Deken yet.)

• renamed [ofGetDollarArgs] to [ofGetCanvasArgs]
• added [ofSetCanvasArgs] and [ofRemoveCanvas]

The new [ofGetCanvasArgs] and [ofSetCanvasArgs] allow you to get/set the creation arguments of the subpatch or abstraction. So I renamed it to "canvas args" because it also works with the subpatches too. And with [ofRemoveCanvas], you can remove the subpatch or abstraction by sending 'bang' to the object.

Using these objects, you'll be able to dynamically create/change arguments/remove abstractions more easily. You can do this on a separate empty pd patch so you can save/open patches for your modules.

I'm planning to upload v1.0.8 binary in a week or two with a few more updates.
But you can download and build the external yourself using the lastest master branch from https://github.com/cuinjune/ofxOfelia.
Or just let me know if you want me to quickly build and send you the current v1.0.8 external file for Windows.

@cuinjune the new objects are very helpful. here is a demo how the dynamic patching could work:
drag_v0.3e.zip

@Jona It's really impressive and seems to work pretty well! (except when dragging on the interface section while it's overlapped with other)

And it would be nice if you do dynamic patching on a separate pd patch. then it would be possible to save/load these patches independently. (just like how Pd basically works.)

Anyways, congrats for finding the solution! Looks like it's at least "theoretically" possible to create a Reaktor-like modular environment using ofelia.

@cuinjune thanks. i am sure the overlapping problem is solvable, i think i need to change the mouse behaviour of the gui templates (for me) for that case. i am also quite positive with the results so far, and motivated to try further. somehow i needed to change every loadbang to ofWindowLoadBang to make the dynamic patching work (?). now it makes really sense, like you mentioned earlier, to think about a concept i have some ideas, but what i definitely would like is if everybody could put their own modules easily inside kind of a template and connect them with others. also if there should be something like a main mixer or sequencer is a question for me. or if i want a "reason like" rack or more like a modular interface or something else. also saving is a good question. it would be nice if it is possible to save globally and locally, so that there are presets for a global setting that can also save the local settings of the modules. saving them as patches sounds like a good idea. i think about four module categories: instruments, effects, sequencer and visualizer or something between them. and analog to pure data, it would make sense to have audio and control connections seperated. i think more difficult than creating and destroying modules with dynamic patching is creating cord connections with dynamic patching, but perhaps also solvable because there is a fixed number of objects in every module. so maybe, theoretically one just needs the module id and the module and input output number and then it could work with the connect and disconnect message. i am not sure if i think too far with the concept, the initial idea was more to create some modules, connect them and see what happens

@Jona I think you would need to store the connection state of modules as creation arguments just like how you store position of modules. Since now you can dynamically change creation arguments of abstraction using [ofSetCanvasArgs], it is possible to store as many data as you want per module.
So for example, one module can store 6 arguments that are xpos, ypos, myID, myOutletIndex, targetID, targetInletIndex.
And I'm not sure yet if cord connections should be done with dynamic patching. If you do dynamic connections, you need to track the ID(or creation order) of each abstraction in a canvas whenever creation/deletion happens. I think this can be tracked using a [ofLoadFloat] on a main patch which is basically dynamically resizable float array.
Other solution might be to use send/receive to create connections. Then you don't need to track the pd's internal canvas IDs and use custom IDs which may or may not make the work easier.
I agree that creating connections is pretty tricky but I think there should be a possible solution.
I will think about it a bit more deeply next week.

@cuinjune thanks for your thoughts. i will think about the send and receive method and saving the connections as arguments. here is a different approach with canvasindex from iemguts. depending on the method the object id could be useful.
perhaps i need to create a new abstraction for every connection (a line with an invisible rectangle around, so that its possible to select it with help of ofIsPointInsidePath). That could also mean that its easy to disconnect the pure data cord together with deleting the abstraction (the disconnect values could be saved as arguments). with this method the connections are automatically saved together with the modules. it would be nice if you inside the ofelia window, like in pure data edit mode, first click on a cord, then it gets colored or bold, and then you can delete the cord and disconnect the pure data cord with del, but thats another problem.
connect.pd

@Jona I'm sorry for the delay. I think creating a new abstraction for every connection makes sense.
I'm not sure what would be the best way to detect the cord selection yet. an invisible rectangle around the line could work.
I will try to create a simplest demo this week. I'm not sure yet how hard/easy it would be though.

@cuinjune hi, @ingox worked with me on the one abstraction per cord solution. its still buggy but so you can get an idea if our concept makes sense or not. Mainly the communication between cord and modules seems difficult, especially if i save and reopen the patch, because $0 is renewed.
its without (mouse)cord selection and deletion and without dynamically patching the pure data connections yet, but that should be possible if the communication works well (which seems tricky, if possible at all, with dynamic patching).
the idea of iemguts canvasindex was that only the newest cord gets deleted if i release a cord outside of an input, but i think that should be possible with vanilla too.
edit: we are working on another one abstraction per cord solution.
drag_v0.3i.zip

@Jona Hi, I couldn't finish creating the demo yet. But I think I'm getting there.
Here's what I've done so far, modular.zip. You need to reinstall ofelia v1.0.8 since I made changes to [ofEditRect] today. (I added "assignPos" and "assignDimen" method)

When you open the "main.pd" patch, you will be able to create a box using Control + '1'. And delete the box using the "backspace" key.
Also, you can select/unselect multiple boxes while holding down the "shift" key. Or you can select boxes with the area selection.
And you can zoom in/zoom out the screen using Control + "-"/"+".
I only tested the patch on macOS so far. Let me know if you find any problem.
EDIT : It seems the keycode doesn't output correct value when pressed with Control key on Windows. (or it could be the problem of my macbook keyboard) I will find a way to fix this. Until then, you can try on macOS or modify the patch.

I tried to make it work just like Pd. I will try to implement the patch cord feature this week.
I would also need to implement the copy/cut/duplicate/paste/undo/redo feature later. (after patch cord is ready)
And I think typing the name to create module can be implemented later.
Also, I think it needs to have some kind of window expander later. (similar to the OF addon you showed me last time)

And I'm thinking to make 2 modes (editmode, playmode) just like Pd. On editmode, you can create/delete/move/connect modules but you cannot adjust the GUI. On playmode, you can only adjust the GUIs inside the module. (e.g. knob) It will be just like how Pd works.

Let me know if you like the above mentioned plan and design idea.

@cuinjune that is really great! thank you very much. and i am just starting my mac because of the keycode it seems that the keycode for the numbers on mac and windows isnt the same (control and shift seems correct). i will play with it and write more.

@Jona I just figured out what is causing the problem. I think I would need to create [ofKeyCodeListener] in order to be able to listen to the keycode values regardless of the modifier(Control). I remembered I asked a question about this before. https://forum.openframeworks.cc/t/way-to-detect-original-keycode-while-control-key-is-pressed/28684
I will update ofelia within 24hours and then re-upload the patch.

@cuinjune no hurries i really like the pd style and all of the ideas. so its kind a simulation of a simulation of real electronic instruments copy/cut/duplicate/paste/undo/redo also sounds great, if its not too much work. i think 2 modes would make sense, too. also the scaling is great, but it doesnt work with windows, so maybe there is something wrong with the keycode too? on my macbook i can only use "control + -" and make it smaller. but i have a very old (2009) macbook, perhaps they had a different keyboard layout? i also found that when i double click and then drag a new object, its position jumps to 0 0. everything else seems to work great.
i worked on the lissajous sequencer (which could be a module), its very minimalistic now. in the example you just need to click into the lissajous window to change the shape and then press play. it has 2 "voices" and is connect to 2 seperate synths. drag_v0.3k.zip

@Jona Hi, the new lissajous sequencer module looks nice and seems to work well.
I added [ofKeyCodeListener] which outputs constant keycode values independent from modifier keys.
Please reinstall ofelia v1.0.8 so you can use this object.
Here's a new version modular.zip. Please try to delete boxes using the backspace key(or del key?) on Windows. If it doesn't work, please let me know the output value of [ofKeyCodeListener] when you press the backspace key. Also, Please let me know the output values of left/right Control keys.
Could you also try "Command + '-'/'+'" on your macbook to zoom? if it doesn't work properly please let me know the output value of [ofKeyCodeListener] when you press '-' and '+'.
And I fixed the position jumping issue when double click and drag a new box. Thanks for finding this.
Now I will try to work on the cord connections. Let me know if you find other issues or have suggestions.

@cuinjune hi, thanks. deleting works with backspace on windows. also creating objects with control+1 does work now.
Zoom doesnt work for me on macos and windows.
here are my windows keycodes:
shift = 340, control = 341, backspace = 259, del = 261, +(numpad) = 334, -(numpad) = 333, + = 93, - = 47, left = 263, right = 262, up = 265, down = 264.
on my macbook i dont have a numpad, and like on windows , + = 93, - = 47.
edit: if i replace keycode 45 61 with 47 93 zoom does work.

@Jona Thanks for confirming. I also added 47 and 93 for zooming. modular.zip
Maybe I should change the shortcut for zooming later since -/+ seem to differ by platforms and hardwares.

Or its fine like you did it now, that you assign two or more keycodes to + and -? if the keycodes do not conflict with another key function it shouldnt be a problem.
I found that the render order only changes if you drag a rectangle directly, if you first select the rectangles with drawSelectBox and then drag them the render order remains the same. In Pure Data the render order is the order of creation, but i think it could be nice if the dragged objects are always in the foreground.
Another question not related to the example: If I use [openpanel] it opens a window behind the ofelia window but in front of the pure data window. Is it possible to change that, so that its also in front of the ofelia screen (not important)?