@ddw_music said:

One consequence is that there would have to be a strict division between objects that produce image data (declaring the name), and objects that receive/process image data (require-ing the name)... which I think reflects the real situation.

I might have a solution to do the pixel access between objects without giving a name for reference, I replied to your issue here:

https://github.com/60-hz/Ofelia-Fast-Prototyping/issues/1#issuecomment-750807213

@60hz said:

I might have a solution to do the pixel access between objects without giving a name for reference, I replied to your issue here:

https://github.com/60-hz/Ofelia-Fast-Prototyping/issues/1#issuecomment-750807213

Will have to look next week I think, with holiday tomorrow and exam grading shortly after.

I got a working prototype of a pix_gain function. This may end up as a dead end: it's far too slow in Lua. I got acceptable performance by resizing the original 1008x768 image to almost a tenth of its dimensions.

The only viable solution for pix filters in Ofelia will be C plugins shaders, I think.

In case this helps someone else later: I also found that ofPixels getColor and setColor use different index semantics -- really -- getColor indexes by bytes and setColor indexes by pixels. So, for the second pixel in RGB, you'd getColor(3) but setColor(1, ...) -- I'm at a total loss to understand the rationale (unless somebody fouled up the initial implementation and then they couldn't change it without breaking compatibility). Anyway I crashed Pd several times to figure that out. Edit: Actually I think it's that the new image ended up being grayscale, even though I tried half a dozen different ways to set the colorspace. So a color is 3 bytes in the original but 1 in the processed result. Still mucking around with that. It's impressive when you 'setImageType' and it doesn't set :-/

hjh

Made some progress... this is transferring the image data into a FBO, and applying the shader at that time, and then using the FBO as a texture onto a plane. In principle, it should be possible to rotate/translate the plane without affecting the shader's behavior.

Integrating this into the [of.] abstractions' plumbing may have other challenges, but it looks like it may very well work.

FWIW: This is hard to figure out. Really hard. Open Frameworks doesn't do much to smooth over any rough edges in interfaces. (Even here, if I switch off the fbo, the image flips upside down.) The abstractions being discussed here are really necessary. From time to time, I hear people complain about the intense learning curve for SuperCollider, but in my experience, Open Frameworks is a million times worse. Little gotchas hiding all over the place. Every little step in this process has been literally days to work through. Ofelia is powerful, yes, but usability is not there yet.

hjh

@ddw_music not sure if i am wrong or use the wrong terms. but open frameworks makes it easier to use opengl and c++ (as a c++ framework for opengl).
and ofelia makes it easier to use open frameworks (not really sure about that). at least, it is a great combination of pure data and open frameworks. of course, it is easier to learn super collider, but that is something else...
the learning curve is steep (depending on your background), because you need to learn several languages at once (lua, c++, open frameworks, opengl, shader, emscripten, java script etc.)...
and alone learning to program shaders can take a long time...
what i wanted to say: there are reasons for the complexity. (of course i support the idea of having compatible ofelia abstractions and a better documentation ).

@Jona "the learning curve is steep (depending on your background), because you need to learn several languages at once (lua, c++, open frameworks, opengl, shader, emscripten, java script etc.)..."

For example, if you draw an ofImage and specify a width and height, it will scale the image to the width and height. If you do the same with an fbo, then (according to the documentation) it will not scale. Or, if you draw an image, it's the right way up, but if you texture it onto a geo, it may be rotated 180 degrees. Fine print like this is maddening. It suggests that the development focus is on the implementation internals rather than the user's experience of the object interfaces. There may be good reasons for that but I'd love to see another layer on top of this to make behaviors more consistent.

Granted, I'm coming from an environment where OOPy polymorphism means that different classes should strive to implement the same method name in ways that are basically compatible. Maybe this isn't possible or desirable for graphics in the same way (I can understand shaders' more intimate connection with the hardware), in which case the static is between my expectations and the reality.

Anyway, maybe I'm wrapping my head around it by now...

hjh

@ddw_music i just wanted to say that it is possible to scale an fbo (and strange, that the doc says the opposite)...

@ddw_music Congratulation, I think you already achieved something nice here.
I am in the same case as you, I am just discovering openrameworks and lua at same time so the learning curve is pretty hard. But I see you are making great progress and I am very happy that someone help to make something nice here.

I replied in the github issue, I think it's a good time to re-think about the way the process is done in my abstraction yes, better to do it earlier than later...

@Jona said:

@ddw_music i just wanted to say that it is possible to scale an fbo (and strange, that the doc says the opposite)...

Yeah, strange, but this is what I read:

draw(...)
void ofFbo::draw(float x, float y, float width, float height)

This allows you draw everything that's in your fbo to the screen using any height and width. Any stretching is up to you to handle appropriately.

https://openframeworks.cc/documentation/gl/ofFbo/#show_draw

... where the equivalent method of ofImage says "... with any attendant scaling that may occur from fitting the ofImage into the width and height."

It's moot for these pd abstractions because I'm allocating the fbo to match the image size but it struck me odd.

I'm glad to have some bandwidth between semesters to contribute something to help future users 😁

hjh

Incidentally I've made a lot of progress in the last couple of days -- I've got a working prototype of the simple "gain" shader where the parameter is "soft"-defined in a descriptor file rather than hardcoded in the Lua script. It will support single values, pairs, triplets, and RGBA colors in 4 variants (float and integer, times with alpha vs alpha assumed to be 1).

Need to clean it up but I think it will be pretty cool.

hjh

Hi to all, i'm new in the forum and still a newbie in pure data, i do outing, i come from max msp.
I wanted to thank @cuinjune for creating Ofelia and also @60hz for creating these abstractions. I am currently trying to experiment by creating a GUI with sliders etc. As I am absolutely not familiar with LUA I wanted to ask if it is possible to create multiple OF windows. Because any object I insert (both abstractions and other) is rendered in the same window. Maybe the $ dollar doesn't seem to work? my aim would be to create a clickable button that creates a pop up with another Ofelia render window.
thanks

@cipp0 Hi, that is not possible with Ofelia because you can create only one global OF window per Pd instance. Ofelia is not designed to be used for creating multiple windows applications.

hi @cuinjune, thanks for the reply! I imagined it, launching two instances of PD and making them talk in OSC I managed to do what I meant thanks also to the abstractions of 60hz.
On the one hand the GUI and on the other the manipulation of the visuals. With ofxOfelia would it be possible to create a single standalone with two windows instead? sorry for the ignorance but I have no experience of OpenFrameworks

I was imagining a diy software for audiovisual
2instance slider.mov

@cipp0 You can do multiple windows if you really need it using pd~ object with ofelia, I tried it without problem. Then you can share data between two pd instance using network or share_mem (shmem in deken) external.

@cipp0 It should be technically possible to create multiple windows in a standalone OF app with ofxOfelia addon by using OF's multi-window trick: https://blog.openframeworks.cc/post/133404337264/openframeworks-090-multi-window-and-ofmainloop But I imagine it won't be an easy task. I think implementing the GUIs in OF and C++ (without Ofelia) for the second window would be easier.

If you really want to do this, you can also consider drawing the GUIs using Tcl/TK, or using an external that allows you draw images on a Pd canvas to mimic the knob.

An iterative spiral, with an image, shader and texture mapping, without touching Lua code directly:

21-0210-iteration-recoded.mp4

hjh

Very nice @ddw_music

without touching Lua code directly:

I am curious about the progression, do you think it could be added to the list of abstractions soon?
I am curious about the needed changes required if it is the case?

Cheers!

I have updated many abstractions, and changed a bit the way some works...
It was ok for my last workshop, but still thinking about a way to make it better.

I'm definitely very late to this discussion but want to see if Gem is still considered a good enough tool for visual programming? I have tried to install it on Linux over the past few weeks and in a lot of cases it just doesn't work. But even when it does, it is not able to playback video well, which means I have to now get inside the video backend stuff and maybe compile the library myself....
So now I am seriously wondering if going through this trouble is even worth it if the library is not being maintained or people are moving away from it.