Since a while I design Pd patches in such a way that they can be operated from a touch screen. Bangs and toggles with ample size (50x50 pix), and in some cases a large x-y field where standard Tk GUIs can be moved around in runtime when they receive [pos x y( messages. Like a Kaoss pad, but now with dozens of effects and parameters available simultaneously.

Though I'm happy to see that Pd can be operated from a touch screen, layout options with the Tk GUIs are very limited of course. Therefore I now consider using GEM for interactive GUI purposes. Pictograms could be used to indicate controller types, and audio settings could be reflected not only with numbers, colors, or position, but also with motion. For example, a cogwheel represents a sample or loop, and it's rotation speed indicates playback speed. GEM could give more information on controller state than Tk objects, making the interface more intuitive. And certainly more inspiring!

But now comes my question. Where Tk GUIs are made for interaction and visualisation, GEM seems to be made for visualisation mainly. So far, I found no simple method to make clickable objects in a GEM window, replacing toggles and bangs for example. How can you prompt a graphical object in the GEM window to identify itself on click? Does anyone know of patches where GEM is employed for such purposes?

An example of a Tk-based touch interface which I'd like to translate to GEM is described here:

http://www.katjaas.nl/slicejockey/slicejockey.html

Thanks in advance for your suggestions,

Katja

Hola katja,
you can check mtl library from
http://wiki.dataflow.ws/PdMtlAbstractions

There you can find some example of gem draggable object (gems.i.rect if I'm not wrong)
Or you can use html+javascript+css or other solutions to make different GUIs
cheers
husk

Thanks husk, that's exactly what I'm looking for.

gems.i.circle-help even shows an example of multiple clickable / draggable objects.

Katja

here is a patch of mine that includes gem objects for all standard tk gui elements
bangs, sliders, radios, toggles
http://puredata.hurleur.com/sujet-5550-zodiac-flow-based-sequencer-groovebox-gem-gui
should be all straightforward to implement

sonsofsol takes us on an interstellar trip, fasten your seatbelts!

In the meantime I've assembled what will be the GEM user interface for one of my audio patches, SliceJockey. It has pictogram toggles and multiple animated controls on an xy-field.

The audio stuff is not inserted yet, but I am so happy with GEM's possibilities to replace Tk GUI that I wanted to show some results right now.

Katja

http://www.pdpatchrepo.info/hurleur/WaveRefinery.zip

wow...that looks awsome fun, katja!
some pmpd stuff could go well with it.
cant wait to play with that.

Oh my gosh, that's an excellent GUI, can't believe that's GEM !
cheers,
Alberto

Well I worked a week on that cute GEM GUI, but when connecting the audio processing I stumbled upon the big drawback. Dropouts!

I should have known, because I played around with sonsofsol's ZODIAC which must load the audio in a different Pd instance. Naively, I assumed this is useful for cases when you want to run the processes on different computers. I did not realize that it is absolutely problematic to run GEM and audio in the same Pd (while it's so easy to check that...).

OK then, need to run two processes for one application. On Windows and Linux this is straightforward because if you open a patch by clicking, it is automatically opened in a new Pd. That is how you can have loads of Pd's opened without even being aware of it. On OSX only one Pd is loaded, even if you click to open new patches from the finder. So you need to open a second Pd from the command line. Further, you need to turn audio on and off again in the Pd with the GEM. Why? When you start Pd, dsp is apparently off, but still there is CPU load associated with audio computation. This only stops after turning dsp on and off. If you forget this, some 10% CPU load is waisted on a process doing no audio at all. And another nuisance: two Pd consoles on the screen - which one is which?

I wanted to use GEM to make some patches very user-friendly, with informative GUI elements. But in a two-Pd version, it is no longer so straightforward to even open the patches. For Pd-beginners such an application may be discouraging. Therefore I think, when we want to use GEM as interactive user interface for widely shared patches, the issues should be solved somehow. Maybe a script should be developed to perform all the necessary actions in the correct order.

By the way has anyone tried to use [pd~] for separating processes? I have never used [pd~] so far, maybe this is a very stupid question...

Katja

hey katja!
in win and linux its really simple to make a .bat file or bash script that launches 2 pd's. one with gui, one without, for the audio processing. this shouldnt be an issue in osx either.

it goes like this (at least in win)

start C:\Programme\pd\bin\pd.exe -noaudio D:\koffa_root\camera\gui.pd
start C:\Programme\pd\bin\pd.exe -nogui D:\koffa_root\camera\audioplayer.pd

in a shell script

with this its possible to launch both pd's without tk- guis, so tcl/tk is not even loaded. gem will still work.
so the user would start them without seeing anything of pd at all.

@sonsofsol said:

start C:\Programme\pd\bin\pd.exe -noaudio D:\koffa_root\camera\gui.pd
start C:\Programme\pd\bin\pd.exe -nogui D:\koffa_root\camera\audioplayer.pd

That's a good idea to start a -noaudio Pd for the GEM GUI.

OSX does not like to start two instances of a process, but there is a syntax to do it from a single terminal window. As a script (example):

#!/bin/bash
/Applications/Pd-extended.app/Contents/Resources/bin/pd /Users/katja/pd/Slicejockey/SliceJockey.pd &
/Applications/Pd-extended.app/Contents/Resources/bin/pd -noaudio /Users/katja/pd/GEMdingen/draggable-guis/1WaveRefinery.pd

The '&' option does it.

Now you would need a different script for each OS, and even for each user because paths are always different. I'm thinking of ways how to make this user friendly so the patches can be shared without the user selecting and rewriting the correct script.

Say you first open the audio patch in a Pd with gui and audio. On that patch is [ggee/getdir] to find the working directory and the GUI patch. Now it would be great to open a second Pd with -noaudio and load the GUI patch, from *within* the first Pd. There is a variety of shell objects for Pd:
[motex/system], [ggee/shell], [flatspace/open] and [moonlib/open]. They all work different and it's easier to crash Pd with them than get anything useful done, like starting a second Pd. I have not yet succeeded in this. Further, to make it work for everybody, the path to the Pd executable should be somehow found in runtime. I have the feeling that these things can be solved.

Katja

shouldnt it be possible to put the script in the working directory and use a local path? - to avoid having the user to change anything...
or if that doesnt work - use a function to find the path of the workingdirectory in the script?

also when executing something in the shell from pd, i found it more useful+stable to write a shell-script with the commands and just execute that script from within pd instead of the commands themselves.
well
good success with this anyway.

Sonsofsol, I followed your hints.

I got the idea to do a single cross-platform tcl script, because tcl is by definition present on all computers with a Pd or Pd-extended installation. I had help from a script-wiz this weekend.

It works like this: from a running Pd you open a second Pd instance, with optional arguments, by simple click or loadbang.

[bang]
|
[getdir]
|
[open tclsh $1/anotherPd.tcl <argument1> <argument2etc.>(
|
[moonlib/popen]

The tcl code:

# cross-platform script to open a Pd-extended instance

# change working directory to location of this script so we can use relative path names:

set workingdir [file dirname [info script]]
cd $workingdir

# find operating system and executable path, assuming regular location

if { $tcl_platform(os) == "Linux" } {
puts "this operating system = Linux"
set pdpath [exec which pdextended]
} elseif { $tcl_platform(os) == "Darwin" } {
puts "this operating system = OSX"
set pdpath [glob /Applications/Pd-extended*.app/Contents/Resources/bin/pd]
} elseif { $tcl_platform(platform) == "windows" } {
puts "this operating system = Windows"
set pdpath [exec which pd.exe]
}

# in case of more than one Pd-extended executables found, use the first one:

set pdpath [lindex $pdpath 0]

# start Pd-extended with optional arguments:

eval exec $pdpath $argv &

You can automatically load a patch with relative path into the new instance by passing it as argument. We tested this on Linux and OSX and so far it seems to work fine. Would be nice if someone could test this on Windows. Example patch and tcl file are attached.

Maybe there are easier or better ways to perform the same functionality. We did not try Python scripting for example.

Katja

edit: changed the path find function for Windows to [exec which pd.exe]

last edit: sorry folks, this which command for Windows will only work if you have special gnu commands installed. I hope to find a better solution.

http://www.pdpatchrepo.info/hurleur/anotherPd.zip

Whoa!

GEM interactive GUI, evaluation so far

I tried to translate one of my setups, SliceJockey, to a GEM GUI. But by now, when the work is almost done, the setup is full of not-so-ideal workarounds and I must conclude that it worked just more straightforward with Tk.

I won't completely give up on GEM, because it is fairly well possible that audio and control procedures in SliceJockey are somehow 'tuned' to the Tk GUI because they were developed there, and a setup developed with GEM right from the start may work smooth.

Anyway, I've noticed some characteristics of GEM which are sort of incompatible with the notion of a general GUI framework. GEM is optimally suited for motion graphics and video manipulation. It is efficient where Tk is slow, when it comes to updating the screen. But the other way round is also true. Rendering of static information is in some cases much easier with Pd Tk than GEM. Some GEM bottlenecks:

• good quality text rendering (crossplatform, with [text2d]): way too expensive

• plotting of static arrays like sample waveform: cumbersome and expensive

• separate Pd instances required for GEM and audio patches

GEM aspects which I find *very* attractive for GUI purpose:

• GEM window with all it's content can be created at any size, and thus accommodated to screen size

• fairly easy to set up an x-y pad with multiple objects of any shape or size

• fairly easy to animate objects in response to audio data

• option to create pictogrammed control objects

Katja

What would be really nice would be a GEM library of a set of GUI "widgets" such as simple buttons, knobs, sliders, waveforms, etc. Things that are already in Pd as Tk GUI's that also can be easy for GEM novices to build a GUI from. I'm thinking like objects that can be put in a GEM chain that would react to mouse interaction automatically, send out values, and can be customized with properties send to an inlet or as arguments.

I don't know how practical that would be because I've only dabbled here and there in GEM, but one of the things that has kept me from getting into it and making a GUI with it is that there aren't any high level objects like that (that I know of), and thus it would take some serious GEM studying to get results. As much as I would like to, I've just never had the time, and I imagine there are others in the same boat.

@Maelstorm said:

What would be really nice would be a GEM library of a set of GUI "widgets" such as simple buttons, knobs, sliders, waveforms, etc. Things that are already in Pd as Tk GUI's that also can be easy for GEM novices to build a GUI from.

Sonsofsol has beautiful sliders in ZODIAC and they are an abstraction. With lots of arguments to set size, position, send/receive ID etc. There's also bang and toggle. With some documentation they could be the start of such a GEM widgets library.

I am a GEM novice myself. One of the things I had to get used to is the coordinate system, which is so different from Tk. Every object, the GEM window itself included, has an 'origin', and distances are expressed as ratio's, not pixels. Like the complex plane, but with three dimensions. Also, if you want to create a block of objects which belong together, it makes sense to use a 'parent' translate object which sets a local origin for the cluster so you can move them together like a GOP. These things are not in help files. I learnt a lot from looking at pdmtl abstractions and sonsofsol's work. A GEM widgets library should certainly document the GEM coordinates system as well.

Katja

So I shouldn't try to make a GUI out of GEM!?!?!?! You are breaking my heart...won't someone please stop the pain of the Tk?

What I had in mind was basically to create a window system - tabs at the top so that I can switch between instruments, graphs, sequencers, etc. Sounds like my imagination is way ahead of my understanding of GEM -

I must run into this problem with static elements, haven't worked with it enough to notice it but it sounds very discouraging.

Still, it sounds like GEM should be great for feedback like VUs, oscilloscope, etc...

PonZo, I didn't mean to discourage you. Actually, your idea of creating multiple tabs in one window is realizable with GEM, much easier than Tk. I've already experimented with that. It is just a matter of activating / deactivating [gemhead] for groups of objects. Also, you must disable [gemmouse] data for the objects which are not rendered.

Efficient rendering of static data would probably require a couple of new GEM objects. For example, [text2d] is so expensive because it has to compute anti-aliasing anew for every frame. When a text is moving on the screen, this makes sense and the CPU load is inevitable. But you don't want to spend 3% CPU time on a simple BPM indicator for example, which changes once per minute at best. For titles of knobs etcetera you could solve it by creating the text as an image and render it as a texture. For the BPM indicator (and any numbox-equivalent in general) this is impossible, therefore it would be great to have a new GEM object which can convert anti-aliased text to a texture on the fly. And for static plots, a similar reasoning holds. With these two additions, GEM could include all the regular Pd Tk jobs without loss of efficiency, and it would be an all-round GUI framework, albeit still 'low-level' in that sense.

I am not (yet) familiar with openGL programming, but I have the impression that openGL offers the best opportunities to create interactive GUI's which can translate the iPad experience to serious computers. At the moment, very little general-purpose computers have a touch screen, because OS'es and applications do hardly support it. Meaning, we are in the frontline of touch-interface programming. We should not complain about difficulties, but work hard to overcome them. Imagine, when touch screens will become regular part of computer hardware, we are the first to have applications for it!

Katja

I suppose I may give it a shot. Regarding touchscreen technology, mine is older and rather primitive - that is, not consistent and reliable - dragging across the screen is ... "a drag"

Good point about rendering a texture for the static data. It seems complex to me to tell the computer "this is a static plot, do not refresh" and then "but now it's being manipulated, refresh." Maybe GEM can save an array plot as an image, render it to a texture, and as soon as it's being manipulated again revert to the array data, etc....

For now I will experiment. And I'm not so upset by Tk now that I have some hardware acceleration.

Ha, a month from now you'll probably post a stand alone OpenGL GUI.

J.P.

Hello there! I've been trying to develop a way to open a pd subprocess for rendering GEM lists using [pd~] as Katja originally suggested above. There is practically no documentation on this object, and this is what I found if anyone is interested:

Miller's paper on Multiprocessing for pd: http://puredata.info/community/conventions/convention09/puckette.pdf

Someone already asked for help to the pd-list regarding the same issue:
http://lists.puredata.info/pipermail/pd-list/2009-03/068504.html

To my experience, I can only use [pd~] correctly when using the pd~-help.pd file. If I try to replicate the same code within my own patches another pd thread is indeed launched, but not the subprocess patch though Being both the parent patch and the subprocess in the same 'pixelbit' folder, I constantly get this error message in the pd subprocess thread console:

rror: Data/pixelbit: can't open
error: -inchannels: can't open
error: 2: can't open
error: -outchannels: can't open
error: -r: can't open
error: 44100: can't open
error: sub/pd~-subprocess.pd: can't open

This is very odd since I'm mimicking the help's file code and it works fine when run from the help file. Thanks, any help will be appreciated!

I think it would be really cool to have a 3d GUI for a synth, where you could have a triple LFO set by dragging your mouse across a cube and then that cube continues to rotate with the momentum vector that you gave it. Or have a row of 2d sliders that you can drop a bouncing ball on top of to reset their values.