i try to use speech recognition with this python script in pure data:

import speech_recognition as sr
import socket
s = socket.socket()
host = socket.gethostname()
port = 3000
s.connect((host, port))

while (True == True):
  r = sr.Recognizer()
  with sr.Microphone() as source:
      r.adjust_for_ambient_noise(source, duration = 1)
      print("Say something!")
      audio = r.listen(source, phrase_time_limit = 5)

  try:
      message = r.recognize_google(audio) + " ;"
      s.send(message.encode('utf-8'))
      print("Google PD Message: " + message)
  except sr.UnknownValueError:
      print("Google Speech Recognition could not understand audio")
  except sr.RequestError as e:
      print("Could not request results from Google Speech Recognition service; {0}".format(e))

this is the test patch: pythonspeech.pd
it should change the canvas color if the patch recognizes the word red/ blue/ green/ yellow /white /black from the microphone input while the python script is running.

I have two questions regarding the patch:

It seems that the python script sometimes stops to work, does anybody know what is the error? I think it has something to do with the while loop, because it worked fine without, but I am not sure (and i do want the loop).

Is there something like pyext for python 3.x and Pure Data 64bit (Windows)?

At the moment the script uses Google Speech Recognition and needs an internet connection because of that, but it seems to be possible to use CMUSphinx offline instead (I do not know a lot about all that).

the initial idea of my "research" was to find the most similar subtitle of a movie compared to the spoken input and jump to the according position.
it seems to be more complicated than I naivly thought, but still theoretically possible with machine learning technologies like word2vec or doc2vec i do not think that it is possible with pure data alone (perhaps i am wrong)...
still nice that it is possible to implement basic speech recognition in pure data...

@Jona Very interesting! I was and still interested in speech recognition in pd. 10 years ago, I did this:

I was trying to sound like a native english speaker, very funny to hear myself talking this way, and not being able to say three properly.

Cheers~

@EEight that is nice. Your 10 year old approach seems almost more reliable than mine with google speech recognition now. What language or library did you use back then?

@Jona Sphinx, not sure it is still availabe or not:
http://probing.wikidot.com/sphinx3-to-pure-data
Last blog post from 2017, looks alive: https://cmusphinx.github.io/

I coded a gui to another speech recognition toolkit (mainly for japanase language) (with OSC):
http://www.workinprogress.ca/software/kiku/about/

Cheers~

this is a combination of speech to text and text to speech it is mainly copied from here: https://pythonspot.com/speech-recognition-using-google-speech-api/
it works offline with sphinx too, but then it is less reliable.

import speech_recognition as sr
from gtts import gTTS
import vlc
import socket
s = socket.socket()
host = socket.gethostname()
port = 3000
s.connect((host, port))

while (True == True):
  r = sr.Recognizer()
  with sr.Microphone() as source:
    r.adjust_for_ambient_noise(source, duration=1)
    print("Say something!")
    audio = r.listen(source,phrase_time_limit=5)
 
  try:
    pdMessage = r.recognize_google(audio, language="en-US") + " ;"
    message = r.recognize_google(audio, language="en-US")
    s.send(pdMessage.encode('utf-8'))
    print("PD Message: " + message)
    tts = gTTS(text = str(message), lang="en-us")
    tts.save("message.mp3")
    p = vlc.MediaPlayer("message.mp3")
    p.play()
  except sr.UnknownValueError:
    print("Google could not understand audio")
  except sr.RequestError as e:
    print("Google error; {0}".format(e))

What I would like to achieve: I have an CSV file with ~60000 quotes. The recognized word is send with [netreceive] to Pure Data and the patch chooses randomly one of the quotes where the recognized word appears. That does work.
My question: Is it possible to send the choosen quote with [netsend] back to python and transform it there into speech?
Somebody says a word and the computer answers with a quote where the word appears...
Does it make sense to use Pure Data for that task or better just use Python (but I do not know how to do that only with Python yet...)?
The Universal Sentence Encoder sounds really promising for "understanding" the meaning of a sentence and finding the most similar quote. But that is far too complicated for me to implement...
https://www.learnopencv.com/universal-sentence-encoder/

Do you want your computer speak this given quote? If so, on OSX its very easy to use the build in text to speech engine through [shell] and terminal. I guess it could work similar with other OSes too.
Maybe didnt hit your point, i'm only here with half the brain at the moment..

@Johnny-Mauser exactly that (even if it is not a real world project (yet) but discovering the possibilities of speech recognition and pure data). I can probably transfer it to Windows if you (or somebody else) can give me some more hints how that is done with Pure Data and OSX. That is what i found about speech recognition and OSX in the mailing list: https://www.mail-archive.com/pd-list@iem.at/msg45054.html

it does work now on windows with this tool: https://www.elifulkerson.com/projects/commandline-text-to-speech.php
and the [sytem] object from the motex library.
with this python code:

import speech_recognition as sr
import socket
s = socket.socket()
host = socket.gethostname()
port = 3000
s.connect((host, port))

while (True == True):
  r = sr.Recognizer()
  with sr.Microphone() as source:
    r.adjust_for_ambient_noise(source, duration = 1)
    print("Say something!")
    audio = r.listen(source,phrase_time_limit = 5)
 
  try:
    pdMessage = r.recognize_google(audio, language = "en-US") + " ;"
    message = r.recognize_google(audio, language = "en-US")
    s.send(pdMessage.encode('utf-8'))
    print("PD Message: " + message)
  except sr.UnknownValueError:
    print("Google could not understand audio")
  except sr.RequestError as e:
    print("Google error; {0}".format(e))

and this patch: speechtoquote.pd
here is the csv file that I used: https://github.com/akhiltak/inspirational-quotes/blob/master/Quotes.csv

I recognized one problem: while the text to speech command prompt is running via [sytem], the pure data patch is "frozen". is there a way to solve it (then I can play for example a female and a male voice together)?
I also tried the [sys_gui] method: https://forum.pdpatchrepo.info/topic/10168/is-it-possible-to-execute-an-exe-from-within-puredata/10
But somehow I can not execute an .exe file with it (message to [sys_gui] : exec "path to .exe")...

@Jona You should be able to open an executable sending a message into [sys_gui]......
[exec "C:/Program Files (x86)/pd/bin/pd.com"(
Note the quotes that take care of spaces in the path (just like a batch file).

If you then want it to open a file then you need to give the path to that file as well........
[exec C:/path....../whatever.exe C:/path......./file.extension(

If the file is within the Pd folder then you can use a Pd global variable for the relative path.........
$::sys_libdir ..... being the C:/path_to_pd
[exec wish85.exe $::sys_libdir/lib/tk8.5/demos/widget(
Note that wish.exe is in the Pd folder so no path is required.

The last one could be useful if you are sharing a patch.
You are correct though, Pd is frozen (hands user input to the new executable.

You can open multiple copies of Pd using a batch file......
...... BUT it needs a file to open.
Eg........

start "C:\Program Files (x86)\pd\bin\pd.com" start C:\Users\"David"\Desktop\PDMusic\minx\minx_run.pd
start "C:\Program Files (x86)\pd\bin\pd.com" start C:\Users\"David"\Desktop\PDMusic\minx\minx_run.pd
exit

Note..... slashes the other way for a batch file.
And then use one of them frozen for [sys_gui]
David.

@Jona Hello again...... I forgot to explain that your frozen Pd is still working, and you can communicate with it through ports (netsend and receive) so you could use a second instance of Pd to do the GUI work.
That was my thinking behind using a batch file to open more than one instance of Pd.
David.

@whale-av thanks a lot for your explanations, i think i understand the netsend object better now. and it works with multiple pd instances. it seems that each pd instance needs its own port?
pdtexttospeechwindows.zip
this ist the gui instance:

and this are two of the netreceive instances:

@Jona Good to see it working!
Yes, generally netsends and netreceives have matched ports (one port only).
For comms with verification of receipt (tcp) it is vital.
But for udp you can "broadcast"...... much like streaming internet radio.

So you could have all your [netreceive] objects as [netreceive 3001 1] and broadcast to your router the [netsend]
So, on my network the primary router address is 192.168.1.1
That is the address that will serve up its web-page. Often referred to as the "gateway".
Its broadcast address will be 192.168.1.255
255 is reserved specifically for this purpose so......
[connect 192.168.1.255 3001( will set [netsend 3001 1] to broadcast to any/all copies of [netreceive] listening on port 3001 on any computer on the local network.
If that is what you want?
David.

@whale-av thanks, everything works fine. I was just wondering if every PD instance needs its own port. obviously not if it receives udp, thanks for your explanations.