Hi everyone,

Gem was a perfect tool for small workshops: easy to understand for a beginner and fun.
It's now very outdated and have many consistencies problems on various plateforms (fullscreen, key codes pb, camera, media etc...).

So I need a good visual tool for my next pd workshop and Ofelia is the perfect library... but it's also a bit difficult for beginners because :

• Too many objects for fast prototyping
• Sometimes long object names with uppercases etc
• help files lead to OpenFrameworks documentation

So I made a few abstractions in order to think of a kind of "Gem" replacement.
Main ideas are:

• Modular single blocs primitives like Gem, with many methods and arguments + their help file.
• Allows to draw and see something on screen even with unfilled arguments (and expandable default arguments)
• Remove all dependencies but Ofelia (no need for more dependencies with this lib)

Here is a first test... I am learning Lua at the same time so it's a bit experimental.
Somebody have good hints to share? Best lua / Openframeworks practise?

https://github.com/60-hz/Ofelia-Fast-Prototyping

(of course the only lib needed is Ofelia)
Cheers!

May I ask how this worked out for that workshop?

After a year of trying to use GEM in an interactive multimedia course, I've just about hit a wall with it, too many bad surprises. I'm also looking for a way to make Ofelia friendlier to students who do not want to write code.

Thanks,
hjh

Hi @ddw_music,

I did not have time to finish it in a "satisfaying usable state" for my last workshop unfortunately. Making such an exhaustive library pretending replacing Gem from scratch is not an easy task.

Also since then Ofelia v4.0 was released and Zack Lee did what it should to make ofelia a better and cleaner lib: remove all openframeworks pd's objects and keep only one [ofelia] main object as function call.
This make this "Fast Prototyping Ofelia" project even more useful for workshop, since with current ofelia v4.0, students really need to know lua to be able to draw anything on screen!

But I made many improvement on the project recently and totally re-designed a lot of stuffs, specially for the coordinate part because historically Openframeworks used a pixel based coordinate system (point 0,0 is top left of the screen) and we need to have a true normalized one (like Gem where 0,0,0 is at center). So the solution was to create a default camera along with the window context rather than rendering things at the center of the screen). Also I discovered the reversed openGl coordinate for texture etc that need a bit of time to be fixed before going further.

So expect a test update very soon, so I might be ready for my next workshop beginning of december, if the Coronavirus Lockdown in France will finish :/

@60hz I'm quite interested... I'm quite busy for the next month but would be glad to take a look at a prerelease version.

I understand why the OF objects would be removed... but that pretty much rules out Ofelia for my class, unless there are some wrappers. I don't mind the code but these students will simply refuse to engage with it.

hjh

@ddw_music it does include some abstractions which encapsulate an [ofelia d] object with a Lua script, like [ofSetup] for example. Haven't gone through it exhaustively though.

@60hz Hi, I just found this. Thanks for the amazing work!
I tried your patches and they work fine with Ofelia v3 but not with the latest v4.

Do you have one that works with v4? If not, do I need to just replace the listener objects (e.g. [ofDraw]) with the abstractions in v4? Any tips on how I can make your abstractions also work in v4?

@cuinjune Nevermind, I got an answer to my question from @alexandros
He said I simply need to add [declare -path ofelia/abs] to patches so the listener objects can be created as abstractions.

FWIW, I don't think it needs to be a replacement for GEM, or pretend to make the interfaces similar to GEM. As noted at the top of the thread, GEM is outdated.

If an obstacle is converting Ofelia's pixel system into something like GEM's normalized coordinates, then... why not simply abandon the idea of normalized coordinates? Unless it's really necessary for practical use. If the only reason to normalize the coordinates was to make it "similar to GEM," I'd say, don't waste time on it.

FWIW, I'm not going to be able to tolerate GEM for another year. Like today -- it took me over two hours to figure out how to extract the alpha channel from pixes. (Like, how many years has GEM existed, and nobody thought it would be useful to have an object to transfer the alpha channel to the main channel of a grayscale pix? Or, if you don't need an object for it, this still seems to me a pretty basic use case... but of course hard to find in the documentation...)

I realize that a set of abstractions on top of Ofelia is not likely to be fully-featured -- but I could cobble together enough Lua to add abstractions myself where needed (and contribute them back), whereas trying to add objects to Gem is impractical (1, my C is not that good and 2, I could build in Linux but not Mac or Windows). Even a partial set of abstractions might be a starting point to get around the kinds of problems I'm facing now, where the existing objects don't quite do what I want and there's not a good way to add functionality.

Otherwise, for this course, I might not have a choice but to jump ship and go to jitter (which some of the students have already done). It pains me to think of switching away from FLOSS but where I am right now is simply not sustainable.

TL;DR Even if the current state is incomplete, would you mind sharing so I could have a look?

hjh

@cuinjune Thanks to you for realeasing the great ofelia library, it is really a lifechanging library in the puredata world for many of us.
All is working with v4 and now I am using only [ofelia] objets in all the abstractions.

@ddw_music actually the goal is to be able to use the camera function when needed, and keeping everything simple. I agree with you that Gem is obsolete but some ideas are still valid like a normalized coordinate system which is more handy for 3d. Actually now I made [gl.draw] behaviour better: you can send a negative rendering value so the rendering will be done outside of the camera and go back to the traditionnal Openframework coordinate system, maybe more suitable for 2d.

Here is a better place to download the updated abstractions: https://github.com/60-hz/Ofelia-Fast-Prototyping

Feel free to test it, I am not sure it is ready for distribution with students yet but at least I make progress and learn a bit of lua/openframeworks

Any help for making new abstractions are very welcome, what would be nice is a good 3d model importer, but I don't know if it is a good idea to make one from scratch, maybe adding the ofxAssimpModelLoader would help @cuinjune ?

Downloaded.

Your example 1-OFP-Simple_Geo is already a hundred times cooler, and easier to read as a patch, than anything I could manage in Gem. I'm sure it's already enough for an undergraduate class geared towards audio artists who need to learn a bit of video.

hjh

Hi @60hz -- I was just looking into what it would take to add a [gl.multiimage] (analog of Gem's [pix_multiimage]).

I'm thinking...

• Add local images and fill it as an array of ofImage() objects.

• Add a method show i to choose the image to display -- here I'm not completely sure. Should this simply set image$0 = images[i]? Or do we need to copy the image data?

• It seems pretty easy to replicate [pix_multiimage]'s approach of requiring images to be named e.g. /path/to/images_*.jpg where * is replaced by 0, 1, 2... n (formatted without leading zeros). I never really liked that, though (spoiled by SuperCollider's pathMatch globbing function). I saw online that Lua doesn't have globbing built in... so I'm stuck with the sequential numbered approach? Or do you know if there's a way to reference Lua libraries from Ofelia?

Thanks!
hjh

Also, relative paths for [gl.image] are not working.

hjh

Hi @ddw_music,

@ddw_music said:

Also, relative paths for [gl.image] are not working.

Recently, I changed the behavior of loading any media. I am not sure if it's the best way to do, but It's very easy: I set the main (parent) patch path as root path using ofSetDataPathRoot function when calling any "open" method. It seems to work well in both OSX and windows system... so please try to download again from github.

Note: renamed all the [gl.object] into [of.objects] because I am adding more and more global functions that are not related to gl drawing, like [of.soundplayer]. It's more logic that way and they can replace the old [of.abstraction] from ofelia v3.

@ddw_music said:

I was just looking into what it would take to add a [gl.multiimage] (analog of Gem's [pix_multiimage]).

Sure, here is a simple way to do it, I might give a try soon, but if you have some ideas you are welcome
Also, it would be possible to fill many buffers with images?

@60hz said:

Recently, I changed the behavior of loading any media. I am not sure if it's the best way to do, but It's very easy: I set the main (parent) patch path as root path using ofSetDataPathRoot function when calling any "open" method. It seems to work well in both OSX and windows system... so please try to download again from github.

Ah, nice, that will be more intuitive. I had worked around it with pdcontrol but it's better if the abstraction does it.

Sure, here is a simple way to do it, I might give a try soon, but if you have some ideas you are welcome

Nice, thanks. I'm slowly grasping what's going on in there.

I ran into a bit of a snag trying to access pixel data... pick it up another day.

Thanks!
hjh

@ddw_music said:

Nice, thanks. I'm slowly grasping what's going on in there.

@ddw_music, I just added [of.muliimage] that replicate pix_multiimage and add more features like the ability to replace any picture in the array using its index, and get different dimensions picture or filetype in the same array.
As usual it tooks me half an hour to make it roughly work, but half a day to make a clean intergration to pd abstraction with understandable help

@ddw_music said:

I ran into a bit of a snag trying to access pixel data... pick it up another day.

That's an even bigger task, but truelly interesting part! I see you opened an issue in github, I will discussed about it here in the nexts days. Thanks for the help.

@60hz said:

@ddw_music said:

I ran into a bit of a snag trying to access pixel data... pick it up another day.

That's an even bigger task, but truelly interesting part! I see you opened an issue in github, I will discussed about it here in the nexts days. Thanks for the help.

I think I found the problem. One thing is that the [ofelia f] e.g. at the bottom of [of.image] shouldn't be named. That's obliterating the [ofelia d] namespace.

What is the reason for unbinding at the bottom of [of.image] btw?

hjh

@ddw_music said:

One thing is that the [ofelia f] e.g. at the bottom of [of.image] shouldn't be named. That's obliterating the [ofelia d] namespace.

Yes, I think it's my fault, I dont remember why I did this way long tile ago... maybe I wanted to share buffers and add pixel FX but I misunderstood some parts. I will fix it.

@ddw_music said:

What is the reason for unbinding at the bottom of [of.image] btw?

We need to unbind the texture after drawing the last shape otherwise all subsequent shapes will receive the same texture. I did it in a old fashion pd way long time ago, but I discovered that it is possible to do it in main box if I declare a local outlet.

You can have a look at the last [of.muliimage] object to check the correct way to do it. I need to change all objects like that...

Edit: all texture objects are edited in the correct way now

@60hz said:

@ddw_music said:

One thing is that the [ofelia f] e.g. at the bottom of [of.image] shouldn't be named. That's obliterating the [ofelia d] namespace.

Yes, I think it's my fault, I dont remember why I did this way long tile ago... maybe I wanted to share buffers and add pixel FX but I misunderstood some parts. I will fix it.

It seems to be very easy to misunderstand -- I also thought at first that sharing the name would somehow be related to sharing the namespace, but it isn't so.

One consequence is that there would have to be a strict division between objects that produce image data (declaring the name), and objects that receive/process image data (require-ing the name)... which I think reflects the real situation. Maybe helpful to be explicit about that.

@ddw_music said:

What is the reason for unbinding at the bottom of [of.image] btw?

We need to unbind the texture after drawing the last shape otherwise all subsequent shapes will receive the same texture.

Yeah, that's a nice trick -- you found something that's harder to do in code languages: a post-return operation. This case would be awkward in SC.

I was able yesterday to split the image load and image bind into separate objects but didn't yet manage to do anything with the pixel data. Back to the manuals...

Btw I'm estimating it would have taken me a couple weeks of intensive effort to get to this point (intensive meaning working on basically nothing else). You've given me a big leg up. Hugely appreciated!

hjh

@ddw_music said:

One consequence is that there would have to be a strict division between objects that produce image data (declaring the name), and objects that receive/process image data (require-ing the name)... which I think reflects the real situation.

I might have a solution to do the pixel access between objects without giving a name for reference, I replied to your issue here:

https://github.com/60-hz/Ofelia-Fast-Prototyping/issues/1#issuecomment-750807213

@60hz said:

I might have a solution to do the pixel access between objects without giving a name for reference, I replied to your issue here:

https://github.com/60-hz/Ofelia-Fast-Prototyping/issues/1#issuecomment-750807213

Will have to look next week I think, with holiday tomorrow and exam grading shortly after.

I got a working prototype of a pix_gain function. This may end up as a dead end: it's far too slow in Lua. I got acceptable performance by resizing the original 1008x768 image to almost a tenth of its dimensions.

The only viable solution for pix filters in Ofelia will be C plugins shaders, I think.

In case this helps someone else later: I also found that ofPixels getColor and setColor use different index semantics -- really -- getColor indexes by bytes and setColor indexes by pixels. So, for the second pixel in RGB, you'd getColor(3) but setColor(1, ...) -- I'm at a total loss to understand the rationale (unless somebody fouled up the initial implementation and then they couldn't change it without breaking compatibility). Anyway I crashed Pd several times to figure that out. Edit: Actually I think it's that the new image ended up being grayscale, even though I tried half a dozen different ways to set the colorspace. So a color is 3 bytes in the original but 1 in the processed result. Still mucking around with that. It's impressive when you 'setImageType' and it doesn't set :-/

hjh

Made some progress... this is transferring the image data into a FBO, and applying the shader at that time, and then using the FBO as a texture onto a plane. In principle, it should be possible to rotate/translate the plane without affecting the shader's behavior.

Integrating this into the [of.] abstractions' plumbing may have other challenges, but it looks like it may very well work.

FWIW: This is hard to figure out. Really hard. Open Frameworks doesn't do much to smooth over any rough edges in interfaces. (Even here, if I switch off the fbo, the image flips upside down.) The abstractions being discussed here are really necessary. From time to time, I hear people complain about the intense learning curve for SuperCollider, but in my experience, Open Frameworks is a million times worse. Little gotchas hiding all over the place. Every little step in this process has been literally days to work through. Ofelia is powerful, yes, but usability is not there yet.

hjh