Hi,

on Linux, how can I create a pd patch where there are 2 number boxes and 1 bang, so I can do this:

1. Positioning the mouse [using (x,y) coordinates in 2 number boxes]

2. Mouse's left click [using a bang]

I use a 1366x768 display.

Thank you,
a.

@atux I think it should also work in Linux.
You can send messages to a patch window...... 0.all_msg.pd
In that example the messages are sent to a sub-patch, but they can also be sent to another patch or even the patch that sends the messages...... [s pd-mypatch.pd] where mypatch is the patch that should receive the message.
David.

Thank you, it seems to work.
But I have some clarifications to ask: I would expect to see the mouse pointer move.
In fact my intent was to "take over" the mouse cursor (by pd) to "click" in the GUI of any other application other than pd, for example a drawing program, opened in parallel on the same computer. An example case: in the right half of the display there is the pd patch, in the left half the GUI of the other application (of any kind). The mouse should go to the left half and click.
It can be done?

Thank you,
a.

@atux I don't think so but I will look into the possibility.
I have controlled other programs, but it relied on an external from the Pd Extended days which could send at least keyboard characters to shortcut characters in the other program, and I think also mouse positions for the screen.
I will look for the external and the patches.
David.

@whale-av It could probably be done with a tcl plugin.
But here is the "input_event" windows keyboard and mouse control from Pd which has the source code...... input_event.zip
I don't know whether it would be useful for Linux as the call names will probably be different.
David.

@atux Using [command] and xdotool would be easiest but will not be portable, but it will work on linux and maybe bsd/osx as well. You will likely need to install xdotool but that is easy enough and it is a very easy program to use. The trick will be that unless it is a full screen window or rigidly fixed in place you will also have to get window position and calculate the absolute position of the number boxes since I do not think xdotool knows about windows, xprop or xwininfo should do the trick for window position. Don't have time for a detailed response with a patch right now but the helpfile for [command] and a couple google searches on xdotool/xprop/xwininfo or their man pages should get you your answers.

@oid said:

Using [command] and xdotool [...]

Yes! It works fine in pd.
This move the mouse cursor to (120, 70):

[exec xdotool mousemove 120 70( ---- [command]

And this make a click:

A double click:

But...there is a problem: the "click" has no effect inside the window (of another app).
As you were saying, the cursor needs to know "which" window needs to be activated.
The command should be this one that I added in the middle:

xdotool_test.pd
Note: 0x4c00006 is the "Window id" I copied from the target window information obtained with xwininfo.
Any tips?
I will do further tests.

Thank you,
a.

If you know your window name you can find its ID

@atux If it does not need to be the literal cursor you could fake a cursor. I just used a unicode arrow as the label for a white canvas and send it some movements I recorded with [iemguts/receivecanvas]. This way you can just use pd's own mouse movement and click messages in tandem with the fake cursor. Possible nice side effect is no collisions between programmed input and user input, you can always return the real cursor to where the user last left it.
cursor.pd

I think I found the problem.

This is my test.
I divide the display into two parts:

-on the left there is Pencil2D (a simple drawing app)
-on the right there is the Pure Data patch

xdotool_test3.pd

By clicking on the bang in pd, a circle is drawn in Pencil2D, at the point of desired (x,y) coordinates.
Very nice.
Note: it also works without the "Windows id" (which isn't there).

So, this happens:

in puredata: it works only introducing a delay > 100ms between "mouse move" and "mouse click". With a delay <100ms it doesn't work.

But...running directly from terminal it also works without delay:

me:~$ xdotool mousemove 600 400 && xdotool click 1

...a circle is drawn istantly, without delay.
How do you explain that in puredata I have to insert a delay > 100ms?

Now, the problem is that...trying with other windows of other "non-drawing" programs, it doesn't always work: the click doesn't always have an effect. I have to understand why. Yet the arrow moves and the arrival window is highlighted (therefore it is active), but the click has no effect.

@oid : about your patch "cursor.pd", I wasn't able to use it for my purpose, but it's so cool... I didn't know it could be done, beautiful.

Bye,
a.

@atux Your terminal example has the &&, second command will not be run unless the first command returns zero. In pd you just execute each command without care if the first has completed, delay gives enough time for it to complete. Generally if I need to do compound commands like these I either put them into scripts and just execute the scripts instead so the shell can handle it all or I use multiple instances of [command].

Scripts are easy and reliable but it is not all right there easy to work with and see in the patch. Multiple instances of [command] gives a bit more flexibility since we can use the output of each individually and well as run it into the next {command].

For your problem, is it always consistent? certain apps always display the behavior or is it a sometimes things? Does it work from the terminal in this case? Have you tried having it click twice? xev could be of help, it will provide a good test window for your mouse manipulation since all it does is print those events to terminal.

Xnee is another program you might find useful for your project, it can record and playback mouse/keyboard events, might be more suitable for your needs than xdotool depending on what you are doing.

@oid said:

[...] I use multiple instances of [command].

I tried modifying my patch above by removing the delay and putting the [command] in series, like in your example:

xdotool_multiple.pd

...but it does not work. Instead from the terminal, with &&, it works.

For your problem, [...] Have you tried having it click twice?

Regarding my problem, I solved it like this:
in some GUI, where something happens when you click inside with the mouse (a simple click), for the click to take effect, in puredata you need to replace "click 1" with "mousedown 1":

[exec xdotool mousedown 1(

Then it's like when I click the mouse (a simple click) inside the window, the event happens. I do not know why, but...it works.

So my opening question is solved. Now I need to see what happens with more complex patches.

Thank you,
a.