Since a while I design Pd patches in such a way that they can be operated from a touch screen. Bangs and toggles with ample size (50x50 pix), and in some cases a large x-y field where standard Tk GUIs can be moved around in runtime when they receive [pos x y( messages. Like a Kaoss pad, but now with dozens of effects and parameters available simultaneously.

Though I'm happy to see that Pd can be operated from a touch screen, layout options with the Tk GUIs are very limited of course. Therefore I now consider using GEM for interactive GUI purposes. Pictograms could be used to indicate controller types, and audio settings could be reflected not only with numbers, colors, or position, but also with motion. For example, a cogwheel represents a sample or loop, and it's rotation speed indicates playback speed. GEM could give more information on controller state than Tk objects, making the interface more intuitive. And certainly more inspiring!

But now comes my question. Where Tk GUIs are made for interaction and visualisation, GEM seems to be made for visualisation mainly. So far, I found no simple method to make clickable objects in a GEM window, replacing toggles and bangs for example. How can you prompt a graphical object in the GEM window to identify itself on click? Does anyone know of patches where GEM is employed for such purposes?

An example of a Tk-based touch interface which I'd like to translate to GEM is described here:

http://www.katjaas.nl/slicejockey/slicejockey.html

Thanks in advance for your suggestions,

Katja

Sonsofsol, I followed your hints.

I got the idea to do a single cross-platform tcl script, because tcl is by definition present on all computers with a Pd or Pd-extended installation. I had help from a script-wiz this weekend.

It works like this: from a running Pd you open a second Pd instance, with optional arguments, by simple click or loadbang.

[bang]
|
[getdir]
|
[open tclsh $1/anotherPd.tcl <argument1> <argument2etc.>(
|
[moonlib/popen]

The tcl code:

# cross-platform script to open a Pd-extended instance

# change working directory to location of this script so we can use relative path names:

set workingdir [file dirname [info script]]
cd $workingdir

# find operating system and executable path, assuming regular location

if { $tcl_platform(os) == "Linux" } {
puts "this operating system = Linux"
set pdpath [exec which pdextended]
} elseif { $tcl_platform(os) == "Darwin" } {
puts "this operating system = OSX"
set pdpath [glob /Applications/Pd-extended*.app/Contents/Resources/bin/pd]
} elseif { $tcl_platform(platform) == "windows" } {
puts "this operating system = Windows"
set pdpath [exec which pd.exe]
}

# in case of more than one Pd-extended executables found, use the first one:

set pdpath [lindex $pdpath 0]

# start Pd-extended with optional arguments:

eval exec $pdpath $argv &

You can automatically load a patch with relative path into the new instance by passing it as argument. We tested this on Linux and OSX and so far it seems to work fine. Would be nice if someone could test this on Windows. Example patch and tcl file are attached.

Maybe there are easier or better ways to perform the same functionality. We did not try Python scripting for example.

Katja

edit: changed the path find function for Windows to [exec which pd.exe]

last edit: sorry folks, this which command for Windows will only work if you have special gnu commands installed. I hope to find a better solution.

http://www.pdpatchrepo.info/hurleur/anotherPd.zip

Whoa!

GEM interactive GUI, evaluation so far

I tried to translate one of my setups, SliceJockey, to a GEM GUI. But by now, when the work is almost done, the setup is full of not-so-ideal workarounds and I must conclude that it worked just more straightforward with Tk.

I won't completely give up on GEM, because it is fairly well possible that audio and control procedures in SliceJockey are somehow 'tuned' to the Tk GUI because they were developed there, and a setup developed with GEM right from the start may work smooth.

Anyway, I've noticed some characteristics of GEM which are sort of incompatible with the notion of a general GUI framework. GEM is optimally suited for motion graphics and video manipulation. It is efficient where Tk is slow, when it comes to updating the screen. But the other way round is also true. Rendering of static information is in some cases much easier with Pd Tk than GEM. Some GEM bottlenecks:

• good quality text rendering (crossplatform, with [text2d]): way too expensive

• plotting of static arrays like sample waveform: cumbersome and expensive

• separate Pd instances required for GEM and audio patches

GEM aspects which I find *very* attractive for GUI purpose:

• GEM window with all it's content can be created at any size, and thus accommodated to screen size

• fairly easy to set up an x-y pad with multiple objects of any shape or size

• fairly easy to animate objects in response to audio data

• option to create pictogrammed control objects

Katja

What would be really nice would be a GEM library of a set of GUI "widgets" such as simple buttons, knobs, sliders, waveforms, etc. Things that are already in Pd as Tk GUI's that also can be easy for GEM novices to build a GUI from. I'm thinking like objects that can be put in a GEM chain that would react to mouse interaction automatically, send out values, and can be customized with properties send to an inlet or as arguments.

I don't know how practical that would be because I've only dabbled here and there in GEM, but one of the things that has kept me from getting into it and making a GUI with it is that there aren't any high level objects like that (that I know of), and thus it would take some serious GEM studying to get results. As much as I would like to, I've just never had the time, and I imagine there are others in the same boat.

@Maelstorm said:

What would be really nice would be a GEM library of a set of GUI "widgets" such as simple buttons, knobs, sliders, waveforms, etc. Things that are already in Pd as Tk GUI's that also can be easy for GEM novices to build a GUI from.

Sonsofsol has beautiful sliders in ZODIAC and they are an abstraction. With lots of arguments to set size, position, send/receive ID etc. There's also bang and toggle. With some documentation they could be the start of such a GEM widgets library.

I am a GEM novice myself. One of the things I had to get used to is the coordinate system, which is so different from Tk. Every object, the GEM window itself included, has an 'origin', and distances are expressed as ratio's, not pixels. Like the complex plane, but with three dimensions. Also, if you want to create a block of objects which belong together, it makes sense to use a 'parent' translate object which sets a local origin for the cluster so you can move them together like a GOP. These things are not in help files. I learnt a lot from looking at pdmtl abstractions and sonsofsol's work. A GEM widgets library should certainly document the GEM coordinates system as well.

Katja

So I shouldn't try to make a GUI out of GEM!?!?!?! You are breaking my heart...won't someone please stop the pain of the Tk?

What I had in mind was basically to create a window system - tabs at the top so that I can switch between instruments, graphs, sequencers, etc. Sounds like my imagination is way ahead of my understanding of GEM -

I must run into this problem with static elements, haven't worked with it enough to notice it but it sounds very discouraging.

Still, it sounds like GEM should be great for feedback like VUs, oscilloscope, etc...

PonZo, I didn't mean to discourage you. Actually, your idea of creating multiple tabs in one window is realizable with GEM, much easier than Tk. I've already experimented with that. It is just a matter of activating / deactivating [gemhead] for groups of objects. Also, you must disable [gemmouse] data for the objects which are not rendered.

Efficient rendering of static data would probably require a couple of new GEM objects. For example, [text2d] is so expensive because it has to compute anti-aliasing anew for every frame. When a text is moving on the screen, this makes sense and the CPU load is inevitable. But you don't want to spend 3% CPU time on a simple BPM indicator for example, which changes once per minute at best. For titles of knobs etcetera you could solve it by creating the text as an image and render it as a texture. For the BPM indicator (and any numbox-equivalent in general) this is impossible, therefore it would be great to have a new GEM object which can convert anti-aliased text to a texture on the fly. And for static plots, a similar reasoning holds. With these two additions, GEM could include all the regular Pd Tk jobs without loss of efficiency, and it would be an all-round GUI framework, albeit still 'low-level' in that sense.

I am not (yet) familiar with openGL programming, but I have the impression that openGL offers the best opportunities to create interactive GUI's which can translate the iPad experience to serious computers. At the moment, very little general-purpose computers have a touch screen, because OS'es and applications do hardly support it. Meaning, we are in the frontline of touch-interface programming. We should not complain about difficulties, but work hard to overcome them. Imagine, when touch screens will become regular part of computer hardware, we are the first to have applications for it!

Katja

I suppose I may give it a shot. Regarding touchscreen technology, mine is older and rather primitive - that is, not consistent and reliable - dragging across the screen is ... "a drag"

Good point about rendering a texture for the static data. It seems complex to me to tell the computer "this is a static plot, do not refresh" and then "but now it's being manipulated, refresh." Maybe GEM can save an array plot as an image, render it to a texture, and as soon as it's being manipulated again revert to the array data, etc....

For now I will experiment. And I'm not so upset by Tk now that I have some hardware acceleration.

Ha, a month from now you'll probably post a stand alone OpenGL GUI.

J.P.

Hello there! I've been trying to develop a way to open a pd subprocess for rendering GEM lists using [pd~] as Katja originally suggested above. There is practically no documentation on this object, and this is what I found if anyone is interested:

Miller's paper on Multiprocessing for pd: http://puredata.info/community/conventions/convention09/puckette.pdf

Someone already asked for help to the pd-list regarding the same issue:
http://lists.puredata.info/pipermail/pd-list/2009-03/068504.html

To my experience, I can only use [pd~] correctly when using the pd~-help.pd file. If I try to replicate the same code within my own patches another pd thread is indeed launched, but not the subprocess patch though Being both the parent patch and the subprocess in the same 'pixelbit' folder, I constantly get this error message in the pd subprocess thread console:

rror: Data/pixelbit: can't open
error: -inchannels: can't open
error: 2: can't open
error: -outchannels: can't open
error: -r: can't open
error: 44100: can't open
error: sub/pd~-subprocess.pd: can't open

This is very odd since I'm mimicking the help's file code and it works fine when run from the help file. Thanks, any help will be appreciated!

I think it would be really cool to have a 3d GUI for a synth, where you could have a triple LFO set by dragging your mouse across a cube and then that cube continues to rotate with the momentum vector that you gave it. Or have a row of 2d sliders that you can drop a bouncing ball on top of to reset their values.