Hello.

I'm trying to control the curser position and click of the mouse through pd, but outside of a pd window. Specifically in a browser.
There's a button a website that makes a particular sound when clicked, and I would like to sequence some clicks if possible. Of course it would be easier to sample it, but I would like to see if it's possible using pd.

Is there a way to do this? Position the mouse, bang a click? Or perhaps there is a way to communicate with some kind of browser command line? When you receive this, click this button in the browser window.

Looking through other recent threads I've found it's possible through the use of this solution: https://forum.pdpatchrepo.info/topic/11291/mouse-control-objects
But it is, as far as I can see, limited to the pd window itself.

Any ideas?

Thank you.
Mh.

@Hasselbalch Hi, you didn't specified your OS. On linux I would use xdotool to control the mouse. On windows there's a similar program called autoit. On mac, well I pretty sure you can use xdotool using brew.

But that's one way of doing it, you can also use something like webdriver.io (with selenium).

Cheers~

It's Mac. Webdriver might be useful! Is there a way to have pure data communicating with webdriver?

Thank you. If there's any other suggestion, please let me know.

Some years ago I made a patch for something similar.
It's not very well done, but it works. You can adapt it to your needs.
It works with [shell] and xdotool, like mentioned above.

https://forum.pdpatchrepo.info/topic/9021/mousestate_hid

Thanks, @macrogmittrei.

I checked to see what I could do about using xdotool with Mac, and found another shell app, Cliclick (https://www.bluem.net/en/projects/cliclick/). It seems fairly simple and I can get it to click at specific points.
I have never used [shell] before and I'm not well-versed in shell/terminal use, so I'll have to see what I can do. The help documentation for [shell] is not a lot, but I suppose it assumes I know what I'm doing.
I just need to get pd to send some single lines of text to the terminal.

sounds like you have found something simpler but the tuiomouse mentioned in the mouse control thread is not limited to the pd window. and might give you another option. (here is a version of the tuio stuff. the mouse click is not set up but maybe you can figure it out.) at one point I did have a working version but its been a long time since.
TuioMouse-helps.pd
from the Readme:
TuioMouse will use the first available TuioCursor as the principal mouse pointer.
Additional TuioCursor ADD and REMOVE events are treated as mouse button PRESS and RELEASE events.

another way to go about it might just be to get the link for the sound file and use [readanysf~] but that is not what you are trying to do.
https://puredata.info/downloads/readanysf

That seems to work as well, thanks.

Is there a simple way to send commands to the osx terminal using the [shell] object? I can't get it to pass anything usable through [shell], I can't figure out if I have to use a certain prefix or not.
All the examples I've found online are to do with specifying a directory, moving or creating files - all I want to do is pass a command as if I had typed into the terminal window itself.

don't know but you can also try:
motex/system

@youaresound, I tried that, but with no result. And [system] has got no outlet for me to print out any possible error messages.

If anyone has successfully used [shell] to send commands to the osx terminal, I'd like to hear how.

@Hasselbalch Maybe....... https://stackoverflow.com/questions/5588064/how-do-i-make-a-mac-terminal-pop-up-alert-applescript
Not really an osx user..... so unsure. But that article suggests that it is similar to "echo" in Widows.

And...... https://forum.pdpatchrepo.info/topic/7335/inteface-pd-with-shell-scripts
David.

@whale-av, ah. Thank you. The [echo message] prefix seem to give a readout with [print STDOUT] in Pd's own log/terminal, but nothing happens in osx terminal. Nothing is printed in there nor is it executed.

The format I need to use for the app, cliclick is [cliclick c:x,y] which will execute a button press on a specific point. But a comma seems to divide the messages sent to [shell] as it prints out [shell: old process still running]. So if [shell] interprets a comma as a command divisor, I might not be able to use without a workaround.

I read through the second thread as well, but I couldn't really find the answer. I might be way in over my head with this.

@Hasselbalch My previous link back to this forum might get you on your way with [shell], but I see you want to click in a browser.
Another "maybe"........ https://learn.adafruit.com/raspberry-pi-open-sound-control/interacting-with-a-web-browser
You would need to send an osc message through [netsend] to a javascript.
David.