Hello everyone!
This might be more of a Python than a Pd question, but I thought perhaps some one would have a suggestion:
I am trying to stream data from a web server into Pd using netreceive and Python.

I have written some python code to get data from the server and I can see this working because it prints the data out in Python. However, I want to send this to Pd and I am not sure how to do this.

I have been experimenting with sockets to do this and
have followed examples found in previous threads
and now have no problem sending static values and even text between Python and Pd. Pd acknowledges the connection and receives and prints the results.

However, I cannot seem to figure out how to replace a static variable, or "hello watermellons" or whatever with a dynamic value to send to Pd.

Any ideas? I can't use a normal GET message (like in purest_json library or [httpget] because I cannot find the full http address for the server (I have the server address and port and then use a series of names to connect to the data I want, so it isnt a usual http addy unfortunately....

thanks in advance for any ideas...
Jorge

@Coalman Never used Python, but would imagine it needs a loop (with a sleep inside?) that samples the incoming data and sends it on to Pd.
David.

@Coalman said:

hello watermellons

I'm confused; you get values from the server and assign them to a variable, right? So since you're getting the values in python correctly, what's wrong with sending the contents of that variable to pd using socket.sendall or something?

@David thank you! yes this is what I am trying to do.

@seb-harmonik-ar I am using socket.sendall but my problem is actually how to assign variables that are filled dynamically (and then refilled with the next update from the server.

Basically, I have:

1. a connection to a server
2. a way of sending a static value from python to Pd (with sockets)

But I can't figure out how to get the values I am receiving from the server into the sending part of the python script (in place of a static variable). I could post the code if that helps? I didn't want to turn this into the python forum, but since Pd is really going to be driving the project and there are some great coders lurking on here, I thought it was worth a try....

@Coalman I'm not the most knowledgable about networking or python tbh, but I think the thing I'm missing is how the data is structured such that you can't put it into a variable and send that over the socket easily when it is received. Why can't you send the contents of a non-static variable in the same place you send your static variables?
you may as well post the code if it isn't too long

@seb-harmonik-ar thanks for helping think this through, I am at a very low level with python myself, but it is useful for this kind of stuff.

I don't think the problem is sending the variable over, because, as you say you could send a variable over in a simple way. Its more of how to assign dynamically updated values (from the server) to a variable.

I will try to show you what I mean:
here I define a function:

import socket

def handle_data(trace):
    #print('Received the following trace:')
    print(trace)
    print()````

#(I am leaving out the code that sets up a client and queries data from a server because it is a bit #long, 
#but I am using a package called obspy which is a package for doing this with seismographic data)

#then I do this:

s = socket.socket()
host = 'localhost'
port = #put an open port here

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

s.connect((host, port))

b = client.run()

s.sendall(b + ";")

# problem is to get the data that prints from the client.run() sent over.  
#That is what I have been trying here with "b".  
#The way I do it for a static value is this 
#(posting this because it might actually help someone else doing this stuff)

s.connect((host, port))
mess = "yello"
Msg = mess + " ;"
s.send(Msg.encode('utf-8'))

# the ";" is because Pd likes messages in this 'FUDI' format 
#which I think needs the semicolon like normal message boxes in Pd 
#(and also if I am not mistaken there should be tab-separations between 
#parts of. a message so that they can be separated as lists are in Pd)

I notice the bottom of the code box is cut off in my browser, so if that doesn't make sense for you, try to click the box and scroll up to see the rest of the code.

@Coalman did you try sendall with Msg.encode as well?
another thing to check is to make sure that b is the right type (str I think?)

@seb-harmonik-ar I think you are right about the variable type, a string would be better since the data type returned from the server is a mixture of alpha-numerics, spaces, and periods

I have tried these things and despite trying several combinations of ways of writing the variable and encoding it I still don't seem to be able to send across what prints from the code above. I basically want to send to Pd the same data I am able to print from the ```
print(trace)
above, of course isn't necessary to print it first, and I am successfully sending strings across without printing it first
the odd thing is that I can send an arbitrary string across fine and receive it in Pd
(like yelkdsjf 032742n sdkjfh \ sldfjsldfkjlkdsjf 8493 sends fine using the code above (in place of "yello")
but the data I am getting from the server (a line of which looks like "
IU.SFJD.00.BHZ | 2021-08-19T12:45:49.019538Z - 2021-08-19T12:46:05.269538Z | 20.0 Hz, 326 samples
doesn't send.
in both cases I am adding the ";" at the end as Pd requires for messages...so it must be something about how I am dealing with the data I am receiving from the server because the actual data doesn't seem any weirder or to have any other prohibited characters than my arbitrary test data and it sends ok.

so it must be that I am writing it to a variable wrong somehow, or
maybe utf-8
isn't the best encoding for something like that
will continue to dig....

@Coalman Maybe define the send as ascii or convert to a string of the decimal values of the ascii characters in python before sending? ......... https://www.codegrepper.com/code-examples/python/ascii+to+decimal+python
[list tosymbol] will convert the decimal back to characters in Pd........ and it might then be easier to diagnose any problems.

Also there is a dedicated binary in your Pd distribution called pdsend (it is in the bin folder along with the Pd binary). It can be called in Python (probably needs to be copied to a folder where Python will find it).
Then use [netreceive] in Pd to receive what is sent through pdsend.
There is a code snippet here........ https://guitarextended.wordpress.com/2012/11/03/make-python-and-pure-data-communicate-on-the-raspberry-pi/ ...... unfortunately a screenshot so it will need to be typed out again.

BUT the comma in the message "IU.SFJD.00.BHZ | 2021-08-19T12:45:49.019538Z - 2021-08-19T12:46:05.269538Z | 20.0 Hz, 326 samples" will split the message when it arrives in Pd.
AND more importantly.... I am pretty sure that a pipe "|" in Python passes information (before the pipe) to an operator (after the pipe) and that could be why the message fails....... trying to pass "IU.SFJD.00.BHZ" to a non-existent operator "2021-08-19T12"
David.

@Coalman You say you want to send trace but the code tries to send b.

What i would do is (not particularly familiar with python):

• Get the data from the server
• Save the data into a variable
• Print the variable to see if the data is actually there
• Send the variable to Pd using the method that worked with the static data

Edit: In this examples the messages sent by sendall are mostly encoded to utf-8 or serialized or converted to bytes: https://www.programcreek.com/python/example/1928/socket.sendall

Hi,
you can take a look in my project. Even if the python code is running into FreeCAD, it implements a double direction client/server communication.
See https://github.com/FlachyJoe/FCPDWorkbench/blob/main/pdserver.py

@whale-av , @ingox , @FFW
Thank you so much for these detailed suggestions and examples. I have been submerged in other stuff this morning, but I will look through all of these and report back with the solution for the next person trying to do this.
I think with all this to look at I should be able to sort one, these examples look great.

best,
Jorge

First off thank you all for your help and suggestions!
I managed to find a solution with a little help from my friends.

So, my main problem turned out to be that I needed to create my socket BEFORE defining my function. Then only after both of these are done I could set up the string to send to Pd (because Python does this sort of weird recursivey thing).

Also as some of you suggested my string was named badly, first I was using a the wrong variable data type ("b") then I tried to make a string using "str" as a name which confuses Python as "str" is the string data type. So for anyone else dealing with this be sure to call your string something like "data" or "lsdfdjkhgs" but not "str"!

Anyway, it works now and I can send data over to [netrecieve]
incidentally @whale-av 's suggestion to use Pd's built in messaging scripts located in the application folder (pdsend & pdreceive) brought up a lot of interesting possibilities that I will be exploring later and for anybody wanting to send messages between Pd and other programs this looks like a cool way to do it.

anyway, I will share the code here in hopes it will help someone else:

import socket
from obspy.clients.seedlink.easyseedlink import create_client

# first get all the socket declaration out of the way
s = socket.socket()
host = 'localhost'
port =  #put (your) open port here, this is also part of Pd's [netreceive] argument
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host, port))

# then we define our callback which happens when we receive data
# out socket is already set up so we can reference it here in the callback
def handle_data(trace):
    # trace is an obspy.trace class so we must convert to string
    # I have to mess around with this "trace" because it is part of the class my client code is using
   data = str(trace)

   # we can then neatly append a ; with a "f-string" to make it work with Pd's FUDI protocol (Pd messages end in ";")
   data = f'{data};'

   # or
   # data = str(trace) + ';'
   # if you like
   
   # and send it over our socket
   s.sendall(data.encode('utf-8'))

#then AFTER all this my code connecting to the server (that returns the data as a string "trace" above, but the entire part for communicating with Pd is up there, and surprisingly simple despite the 'strange loops'!