Hello community!

My main question is: how to deal with the i/o-errors, pd chronically suffers from?
I get them on many occasions:

• resizing a table
• sending and processing (routing, splitting into packages) messages/lists of mediocre length
• sending/receiving messages from many objects
• updating gui elements

It's a pain in the ass! processing long messages is a quite basic feature for realtime audio.
Of course the I/O-Error comes with an audio click, which is inacceptable in a live situation.
(I am using pd with jack. There is no xrun in jack, by the way)
Fun fact: pd also prints the I/O-Error message if audio processing is switched of.

My impression is, the problem lies in pd's basic design:
Messages and audio processing is being handled in the same single main loop.
So if message processing takes a little longer, audio gets out of sync.
(Of course this can happen easily on message bursts, e.g. if you want to synchroniously control many objects.)
Can anyone confirm this?

What are the common practices to deal with this flaw on the USER SIDE?
Somewhere I read that I should keep audio processing and gui/message processing in seperate patches/pd instances.
However, this is only possible to some degree.
However, how could this be achieved? How can I force the gui patch to use a seperate thread? do I have to run it in a seperate pd instance?

From the PROGRAMMERS ANGLE: are there any efforts to improve pd's design? (e.g. calculate messages and audio processing concurrently, e.g. in sperate threads). Since this flaw has annoyed me since I have started to use pd, and since I have some experience with programming, I would like to offer my help to fix it!
Has this problem been solved in any of pds "forks" yet?
I love pd, but it still has profound issues, in my humble opinion.

Messages and audio processing is being handled in the same single main loop.

AFAIK: Yes.

Has this problem been solved in any of pds "forks" yet?

AFAIK: No.

From the PROGRAMMERS ANGLE: are there any efforts to improve pd's design?

FYI: I'm currently working on making my fork multithreaded to attempt to solve that...

I love pd, but it still has profound issues, in my humble opinion.

+1

@EsGeh You should not be having these problems. At all. Except "resizing a table".
Even that should not cause a problem if it is hidden in a sub-patch and you are not reading from it as you resize.
I can run a 64ch in 64ch out mixer controlled by osc messaging from 64 tablets in Pd, at 3ms latency without a glitch (Extended on Windows7)

Are you running Pd at the same samplerate as your soundcard?
Are you trying for too small a buffer to improve latency?
Have you given Pd root priority (chmod 4755)?

Could you post some more detail..... os, soundcard, Pd version?
Chapter 2.5 ........ https://puredata.info/docs/manuals/pd/x2.htm/?searchterm=i/o error
explains what Pd gets up to, and why it is best to hide unused gui's, and there is more help on that site.

You can run more than one instance of Pd and communicate between them through ports, and that will sprout 2 instances of wish, which spreads the gui load as well. Whether your OS will run them all in the same processor I don't know. Recent OS's should automatically spread them about.

I have had problems trying to control a gui through more than 2 levels of gop.
If you wish to post a problem patch then we can see if it works well on our systems, which might narrow it down.
David.

@whale-av: thats interesting.

Details about my system and configuration:

OS.: arch
pd-version: Pd-0.49.0 ("") compiled 17:22:56 Sep 26 2018
pd config:

audioapi: 5
noaudioin: False
audioindev1: 0 2
audioindevname1: JACK
noaudioout: False
audiooutdev1: 0 2
audiooutdevname1: JACK
audiobuf: 11
rate: 41000
callback: 0
blocksize: 256
midiapi: 0
nomidiin: True
nomidiout: True
path1: /home/samuel/.sgPD
npath: 1
standardpath: 1
verbose: 0
loadlib1: sgCLib/sgScriptLib
loadlib2: sgCLib/sgInputC
loadlib3: zexy
nloadlib: 3
defeatrt: 0
flags: 
zoom: 1
loading: no

sound-server: jackd
jack config:

> jack_control dp
--- get driver parameters (type:isset:default:value)
            device: ALSA device name (str:set:hw:0:hw:0)
           capture: Provide capture ports.  Optionally set device (str:set:none:hw:0)
          playback: Provide playback ports.  Optionally set device (str:set:none:hw:0)
              rate: Sample rate (uint:set:48000:44100)
            period: Frames per period (uint:set:1024:256)
          nperiods: Number of periods of playback latency (uint:set:2:2)
             hwmon: Hardware monitoring, if available (bool:notset:False:False)
           hwmeter: Hardware metering, if available (bool:set:False:False)
            duplex: Provide both capture and playback ports (bool:set:True:True)
          softmode: Soft-mode, no xrun handling (bool:set:False:False)
           monitor: Provide monitor ports for the output (bool:set:False:False)
            dither: Dithering mode (char:set:n:n)
        inchannels: Number of capture channels (defaults to hardware max) (uint:notset:0:0)
       outchannels: Number of playback channels (defaults to hardware max) (uint:notset:0:0)
            shorts: Try 16-bit samples before 32-bit (bool:set:False:False)
     input-latency: Extra input latency (frames) (uint:notset:0:0)
    output-latency: Extra output latency (frames) (uint:notset:0:0)
       midi-driver: ALSA MIDI driver (str:notset:none:none)

root-priority: Hmm. I guess no. I'll try your suggestion, and see if it helps!

Configuration is pretty standard, I guess.

My impression is that certain events in my patch cause some messages being sent to multiple destinations, which causes pd to get out of sync.
It's slightly difficult to isolate the problem(s) into a small example, since I am using my own library of abstractions for parsing/generating recursively structured messages and automatically/dynamically connecting objects. I use this e.g. to let one object track properties of another one. It's nothing too fancy, all just native pd.
The amount and length of messages is still very moderate. It's disappointing to see pd struggle already.
(e.g. a "sample recorder" would record audio and update the "sample" object, by sending a "length" of the recording in ms, which in turn also distributes this message to ~3 "sample player" objects).
I will try to make a small example, that shows the issues I am facing.

Thank you for your help so far!
Any suggestions are welcome

Have you given Pd root priority (chmod 4755)?

why should I do that? this would make pd run as root always right?
pd already runs with realtime priority, as far as I can see:

$ pd -rt &
Jack: JackClient::SetupDriverSync driver sem in flush mode
Jack: JackLinuxFutex::Connect name = jack_sem.1000_default_pure_data
Jack: Clock source : system clock via clock_gettime
Jack: JackLibClient::Open name = pure_data refnum = 4
Jack: JackClient::PortRegister ref = 4 name = pure_data:input0 type = 32 bit float mono audio port_index = 7
Jack: JackClient::PortRegister ref = 4 name = pure_data:input1 type = 32 bit float mono audio port_index = 8
Jack: JackClient::PortRegister ref = 4 name = pure_data:output0 type = 32 bit float mono audio port_index = 9
Jack: JackClient::PortRegister ref = 4 name = pure_data:output1 type = 32 bit float mono audio port_index = 10
Jack: JackClient::Activate
Jack: JackPosixThread::StartImp : create non RT thread
Jack: JackPosixThread::ThreadHandler : start
Jack: JackClient::kBufferSizeCallback buffer_size = 256
Jack: JackClient::Init : period = 5804 computation = 100 constraint = 5804
Jack: JackPosixThread::AcquireRealTimeImp priority = 5
Jack: JackClient::ClientNotify ref = 4 name = pure_data notify = 2
Jack: JackClient::kActivateClient name = pure_data ref = 4 
Jack: JackClient::Connect src = system:capture_1 dst = pure_data:input0
Jack: JackClient::ClientNotify ref = 4 name = pure_data notify = 18
Jack: JackClient::ClientNotify ref = 4 name = pure_data notify = 18
Jack: JackClient::Connect src = system:capture_2 dst = pure_data:input1
Jack: JackClient::Connect src = pure_data:output0 dst = system:playback_1
Jack: JackClient::Connect src = pure_data:output1 dst = system:playback_2
Jack: JackClient::ClientNotify ref = 4 name = pure_data notify = 18
Jack: JackClient::ClientNotify ref = 4 name = pure_data notify = 18

$ ps al | grep pd
F   UID   PID  PPID PRI  NI    VSZ   RSS WCHAN  STAT TTY        TIME COMMAND
0  1000  1212   746  -7   - 259980 183892 -     SLl  pts/0      0:19 pd -rt
0  1000  1214  1212  20   0  58984 24812 -      Sl   pts/0      0:02 wish /usr/lib/pd/tcl//pd-gui.tcl 5401
0  1000  1216  1212  -9   -   2308   796 -      S    pts/0      0:00 /usr/lib/pd/bin/pd-watchdog
0  1000  1311   746  20   0  15064  3088 -      R+   pts/0      0:00 ps al
...

@EsGeh I have seen it recommended by Miller to get audio working correctly on a RPI...... just applied to /bin/pd
You can always change it back if it doesn't help.
4755 user can read write and execute, group and others cannot write.
I am no Linux expert, but you have maybe provided enough info for @EEight to spot something.
David.

I have come to the conclusion, that sending many messages to several destinations can easily cause audio clicks.
(Same for resizing, normalizing and clearing tables, and updating many GUI elements.)
It seems crucial to optimize patches to keep the the number and length of messages minimal at all costs.
Maybe it helps to implement more sophisticated message parsing/processing in C and include as an "external" - I am going to try that.

Chapter 2.5 ........ https://puredata.info/docs/manuals/pd/x2.htm/?searchterm=i/o error
explains what Pd gets up to, and why it is best to hide unused gui's, and there is more help on that site.

The article is interesting indeed! Seems like pd makes strong guarantees about the semantics of message passing. There seems to be no easy way of solving the problem of message bursts causing audio clicks while maintaining nice predictable semantics...