Hello everybody,

it's been a long time since i started wondering about getting some advanced visual effects out of pd. I know "advanced visuals" could mean a lot of different things, but let's say I am thinking of pixel stuff like depth of field, bloom, glow, blurring. I kind of tried everything, from basic pix effects to freeframe fx and gridflow convolutions but no matter what I do, since these effects are cpu based the resulting patch is always dead slow.

My first question is: as far as i know pd is born as an audio software, does it make sense to keep pushing it into the domains of visuals?

Don't get me wrong, I love pd and I know the amazing stuff you could get out gem and gridflow. Let's think of all these kind of 3d manipulations, sound visualization, video mixing, opencv stuff, pmpd physics simulation, just to name a few. You could just get some wonderful visuals by only using geos and simple texturing. But, sometimes, I find myself in front of limitations, like the ones about pixel effects I said before, and I wonder if I should just leave pd to what it's good for and move to video driven software like vvvv or "classic" programming environment like Processing.

I know a lot of stuff I've been talking about could be achieved with an irrelevant cpu cost by leaving calculations to the gpu. I think GLSL potential is extremely huge and I got to work some basic blurring, glowing and blooming effects I found on the web, but still seems a little workaroundy for me (especially multipass rendering).

Here is the second question: could opengl and glsl scripting be the solution to my first question? and what do you guys think about having a place where we can host a (hopefully growing) collection of ready to use GLSL effects along with example patches? maybe with a standard framework of objects for multi texture effects and general GLSL handling?

Ok, that's all. Any feedback will be extremely appreciated.
Here follows a simple GLSL blooming effect applied to gem particles (works on macosx 10.5, pd extended, gem 0.92.3)

http://www.pdpatchrepo.info/hurleur/effects.zip

It's cool.
I also meet the same problem as you.
You work show us that we can build any effect from a very basic object in PD. It depend on your knowledge about CG. To me, GLSL is confusing, I have no enough background about it. The GLSL examples is too simple.
I hope pd will be more easy to use and flexible in visual field. At the same time, we need learn more.
Indeed, I learned from your patch. Thanks!

hey guido!
first i want to say that your shader adaption is pretty awsome and i like it alot.

to your questions: i use gem and pd since 3 years for visuals. i like the possibilities pd gives me. i like the connectivity.
i think, as you said, OPENGL scripting and GLSL shaders are a great way to improve performance and make a wide range of new things possible, and also that documentation is very poor in this part.
it would be great to build a collection of shaders, patches and knowledge about these possibilities.
i sometimes feel that the visual making pd community is...let say - very, very small...compared to vvvv and others.
so it will be even more important to get help of the few who know how.

what im missing the most in this GLSL part is documentation about the linkage of pd and opengl/glsl. when i ask someone about shaders the most common answer is "read the red and the orange book ." but these say nothing about the implementation in pd.
like this float K stuff and how [GLSL program] works?
how to get changing parameters into the scrips?
there is nothing about the Geometry Shader at all?
in pd docu just the patches are explained, but not the shaders.

so i would be extremely thankful for some simple shader examples with every line of code explained.

btw.: thanks for the tip with the multitexture patch...i somehow got it working without exactly knowing what im doing.

i played around and got a little collection of some simple shaders i managed to implement. have a look

www.3rd-eye.at/GLSL.zip

i would use GEM_engine for camera navigation and lighting, without it you wont see effects in some.

www.3rd-eye.at/GEM_engine.zip

i also got a geometry shader "working", wich intrests me a lot. unfourtunatly there are some errors regarding the manipulation of the modelviewmatrix in the geometry shader, so the stuff is floating around in space when you move the camera. this is above my skills rightnow.
so it seems pretty easy to implement simple shaders.
i ran into walls when i tried with more complex shaders which use functions or libarys that need to be included in some way. for example in the reflection example the fragment shader used samplerCube, wich i couldnt enable, so i replaced it with sampler2D...if anybody knows how to do this, help would be very much apprechiated.

really good thread-thanks for sharing all- GLSL shading is a good direction to move visuals forward in PD. A place to host GLSL effects/patches would be great too

just had a thought- for hosting how about a google code page ? they are easy to set up and share access?

Hi there,

finally more interest in GLSL-Shaders

i would be really interested in sharing GLSL-Shaders. So far I've only written Fragment-Shaders for Image Filtering and/or Generation. The first one rendering the Mandelbrot-Set and the second one is coloring Image-Areas based on Brightness-Values (more or less a extended cv_threshhold)

My main interest here lies in creating Shaders for Computer Vision Applications, or replacing pd_opencv with Shaders, where possible and reasonable, because i'm stuck with an ATI-Card which doesn't support CUDA and OpenCL seems to be no fun so far

And yes i would be interested in a Google-Code Page.

@ az: this might interest you

https://picoforge.int-evry.fr/cgi-bin/twiki/view/Gpucv/Web/

Thanks, i looked in to that a while ago, but it has hidden somewhere on the requirements page, that you need a CUDA-Thingie...

Back to the Topic:
Based on this article here http://www.gamerendering.com/2008/10/11/gaussian-blur-filter-shader/ i created an abstraction which makes an blur-Effect. Unfortunately it only works with pix_video as input so far. When i try to use it on an pix_image -> pix_draw/pix_texture it doesn't work.

Any ideas how to Blur images,... etc?

http://www.pdpatchrepo.info/hurleur/blur.zip

Another shader coming up

This time it's a Sobel-Edge Detection Abstraction. But its still only working for the camera. Still haven't found out yet, how to apply the Shaders to e.g. pix_image->pix_draw

To use it you should set the resolution properly (currently default at 640x480)

http://www.pdpatchrepo.info/hurleur/sobel.zip

hey az

i get these errors...it doesnt want to link

error: [glsl_fragment]: shader not loaded
linking: link 9.80909e-45 7.00649e-45
linking: link 9.80909e-45 1.12104e-44
[glsl_program]: Info_log:
[glsl_program]: linking with uncompiled shader
[glsl_program]: Link failed!
[glsl_vertex]: Vertex_shader Hardware Info
[glsl_vertex]: ============================
[glsl_vertex]: MAX_VERTEX_ATTRIBS: 16
[glsl_vertex]: MAX_VERTEX_UNIFORM_COMPONENTS_ARB: 4096
[glsl_vertex]: MAX_VARYING_FLOATS: 64
[glsl_vertex]: MAX_COMBINED_TEXTURE_IMAGE_UNITS: 32
[glsl_vertex]: MAX_VERTEX_TEXTURE_IMAGE_UNITS: 16
[glsl_vertex]: MAX_TEXTURE_IMAGE_UNITS: 16
[glsl_vertex]: MAX_TEXTURE_COORDS: 8

Hm, what happens when you click the print message to glsl_fragment?

For me it looks like this:
[glsl_fragment]: glsl_fragment Hardware Info
[glsl_fragment]: ============================
[glsl_fragment]: MAX_FRAGMENT_UNIFORM_COMPONENTS: 2048
[glsl_fragment]: MAX_TEXTURE_COORDS: 8
[glsl_fragment]: MAX_TEXTURE_IMAGE_UNITS: 32

What kind of GPU / Drivers do you use?

i realized that i just installed a new ubuntu and its running without nvidia drivers...i guess thats the reason...will try again tomorrow

Hi

Here is a good resource of shaders, Im going to try them out

http://www.koders.com/noncode/fidC6EAE80EC0339478E2E8C43E643FE63ABAD3C3AA.aspx

Nice thread ! I'm also curious about how to blur specific shapes in G.E.M., my goal is to provide a simple 'depth of field' effect, a kind of blur based on zbuffer.
The 2 solutions I see for now :

• repeat shapes (with [until] or glsl), because once repeated a lot forms could seems to be 'blurred', and more the form is far from the point of view more the shape is repeated
  see Gem/examples/10.glsl/10.geometry.pd or Gem/examples/02.advanced/20.double-gemhead_vs_repeat.pd

• render a scene to a framebuffer, blur specific parts of the frame with a shader, based on zbuffer...

i think this is usually done by multipass rendering.

http://humus.name/index.php?page=3D&&start=56

btw...is anyone skilled with geomtry shaders and can help me get my example working?
i cant set the right modelview matrix...should be a piece of cake for someone who knows what hes doing. the example is in the .zip i posted above.

While searching for something complety different i stumbled over this Thread and felt like reinventing the wheel again...

http://lists.puredata.info/pipermail/pd-list/2007-06/051145.html

Does anyone know what happened to these shaders, did they find their way in to an external-collection?

Hey everybody,

sorry if it took me so long to reply but I've been kind of busy with work recently. I'm happy to see lots of stuff happened here in the meanwhile, that's great!

We should defintely try to put some stuff together. Like testing shader on different platforms and making example patches so that our work could be a little more usable.

Any suggestion?

Today I've spent some time working on shaders. As I am no expert in computer graphic technologies the patch below is a result of a simple trial and error process.

You'll find a pd implementations of two shaders together with a first try of an abstraction to make glsl shaders work just out-of-the-box. There is also an hypothesis of a "standard" for collecting glsl shaders. It's cheap stuff at the moment, but it works (at least it does on my Macbook Pro with 10.5 OSX).

More informations in the patch below, download and run "shader-help.pd".
Any feedback will be extremely appreciated.

edit: [patch deleted, needs some fixes and a lot more thought before being public]

Just finished the first version of a freeframe GL 1.5 host for Pure Data (pdp to be precise). Maybe it would be a good idea now to build these GLSL shaders for freeframe, portability being one of the main considerations.

The host needs some testing though. Any feedback wildly appreciated!

Find it here: http://www.ewocprojects.be/pdp_ffgl.html