@Pspiroz

My LibPd4UnityTools project was my first attempt at integrating Pure Data with Unity and I haven't updated it in a while. I would suggest using uPD~ as it is much more complete and stable. uPD~ offers the same functionality that was present in the LibPd4UnityTools with much more (efficiency, containers, step sequencer, mixer, etc.). There is one major difference though: LibPd4Unity still used Unity's audio engine by sending the output of Unity's audio sources to Pure Data, while uPD~ loads, plays and spatializes audio all within Pure Data. To note that the later is much more flexible, efficient and Pure Data friendly.

If you read my uPD~ quick start guide on Github, you'll be able to get up and running pretty easily (https://github.com/Magicolo/uPD). If there's an error that's preventing you from using uPD~, report it here and I'll try to fix it in a timely manner.

M.

@s.elliot.perez

uPD~ was made with musicians and programmers in mind as I am myself both of these. Designers that are used to the Pure Data workflow will find it quite easy to build sound or music patches that can be easily triggered within Unity (very low scripting knowledge required). On top of that, I've built a few very useful tools to simplify even more the musician's life. This includes a Wwise-like container logic integration and a step sequencer that uses special 'Instrument' patches.

It is to note that I haven't been developing uPD~ for a few months mostly because of compatibility issues (Android, iOS, and Web) that I have trouble solving.

Anyway, you should definetly try it out and give me your feedback. I know there's not a lot of documentation/tutorial for now but you can get started by following my Quick Tutorial on GitHub and by exploring the methods of the PureData class.

M.

Yes it is possible. I have developed a project that does exactly that, so for procedural gaming audio fans, I encourage you to go and get it at https://github.com/Magicolo/uPD.

There is a list of features listed in the README file a list of things I don't know how to accomplish such as Android/Web compatibility, so anyone who has insight on these would be very helpful. Also, if you find any bugs or have any suggestions post them here; I am very actively developing the tool.

Eh voilà. I've uploaded the first version of the PDPlayer at https://github.com/Magicolo/LibPd4UnityTools

It has only been tested on Windows, but should work fine on Mac (I hope).
If there isn't enough information on how to use the tool, I will add some.
Any comments or suggestions to make the tool better will be appreciated.
I will keep updating it as I use it.

@EEight I will, within the next month.

Update

I've come up with a different approach to reduce CPU usage and for 32 independent audio sources sent to Pure Data, the Total Audio CPU stays around 11% (which is way better than my previous 23%). I'm not sure if this result is viable for mobile applications; maybe with some dynamic [switch~] objects and by limiting the maximum number of simultaneous independent voices to something lower than 32...

Setup details:
-I copy each audio block data from each of the 32 Audio Sources to a big float[].
-I write this float[] to a Pure Data array.
-I read sections of the array and send the signal to the appropriate receiver.

Finally I'm not using the [unpack~] and [pack~] objects because I haven't found a good way of separating the left and right audio channels and the LibPD.WriteArray() method seems to be pretty efficient. But it has been very useful to be able to test them and they may come handy for more complex setups.

@EEight

It works! Thanks alot!
I've also experimented with the LibPD.WriteArray() method and surprisingly it seems to have the best results in my current setup.

Here are my preliminary test results on my computer (the data comes from the Unity Profiler):
-32 Audio Sources playing without requiring access to the audio buffer = ~3% Total Audio CPU
-32 Audio Sources playing with requiring access to the audio buffer (but nothing done with it) = ~6% Total Audio CPU
-32 Audio Sources playing and sent to Pure Data with the [unpack~] setup = ~13.5% Total Audio CPU
-32 Audio Sources playing and sent to Pure Data using the LibPD.WriteArray() setup = ~9% Total Audio CPU

[unpack~] setup details:
-I separate the left and right channels and sum up all the data from the 32 Audio Sources in two float[] (so I don't need to write to 64 different arrays and use the LibPD.SendList() method 64 times per audio block)
-I send the two float[] to two different [receive] each connected to a [unpack~] (to keep the left and right channels separate)
-I think that the loss in performance in comparison to the LibPD.WriteArray() setup comes from the fact that I'm using the LibPD.SendList() twice per audio block (the performance cost of the [unpack~] objects are negligible). A way around this would imply to separate the left and right channels within Pure Data but I haven't yet found a good way to do this because the channels are intertwined and iterating 48k times per second on a list can get rapidly costly in Pure Data..

LibPD.WriteArray() setup details:
-I sum up all the data from the 32 Audio Sources that will be sent to Pure Data into one float[] (so I don't need to write to 32 different arrays and use the LibPD.WriteArray() method 32 times per audio block)
-Then I write both audio channels to the same Pure Data array and read them with two [tabread~] with a small phase shift so one reads only even indices and the other odd indices. This way of separating the left and right channels is pretty efficient.

Conclusion:
The less I communicate between Unity and Pure Data, the better. In this test, the 32 Audio Source are summed up together, so in order to have individual control on each of them, I would have to send them to 32 different Pure Data arrays and use LibPD.WriteArray() method 32 times per audio block. To reduce this CPU cost, I am currently writing a tool that will gather audio blocks based on a module system (sounds that are part of the same module will be summed together while modules will be independent from one another).

If you have any ideas on how to reduce the CPU cost of the sending process, please let me know.

@EEight
I've had some issues when first setting up LibPd4Unity. There seems to be a bug at some point when the print events from Pure Data are sent to Unity. I just commented everything that had to do with print in LibPDNativeMethods.cs and LibPDNativeMessaging.cs.

Otherwise, I get this error when trying to load a patch using your .dll: IOException: unable to open patch.
I think that the libpdcsharp.dll file included in the LibPd4Unity package has been customized to work with Unity. When I compare LibPd's LibPDNativeMethods.cs and LibPd4Unity's LibPDNativeMethods.cs, there are slight differences in DLLImports and the Process method (line 396).
So I'm guessing LibPd depends on
[DllImport("libpdcsharp", EntryPoint="libpd_safe_init")]
at line 88 to work with Unity.

If you know of a workaround, it would be fantastic. I'll continue to do some research on my side.

@EEight
I'm on Windows 8 64bit. If at the same time you could compile the [pack~] object, I could experiment with sending audio back to Unity, which might be usefull for creating buses and such. I don't want to take too much of your time, so if it's too much work, [unpack~] is the priority.

Thank you very much for your help.

Hello @EEight and thanks for the reply.

Unfortunately, I am quite new to the compiling process. I will make a bit of research to try to understand how to compile externals within LibPd though any guidance would be appreciated.

I've uploaded my current solution here: https://github.com/Magicolo/LibPd4UnityTools. I've included an example as to how to successfully send the audio to Pure Data.

Instructions:
-In a Unity project, include the package LibPd4Unity (https://github.com/patricksebastien/libpd4unity)
-Replace LibPDNativeMessaging.cs with the one in my repository (I've only made the LibPd.SendList() method generic)
-Put LibPdFilterRead.cs on the Audio Listener and give it a Pure Data patch path.
-Add a new Game Object with an Audio Source
-Put the Audio Clip of your choice in the Audio Source
-On that Game Object, add AudioSendToLibPdExample.cs
-In your Pure Data patch (the one you referenced on the LibPdFilterRead.cs script) create the object [ru~ Test] (I've included the object in my repository and Test is the name I have chosen for the example)
-Connect {ru~ Test] to the [dac~]
-Play the Audio Source

The sound should go from the Audio Source, through Pure Data, and come back in Unity via LibPdFilterRead.cs AND the sound should be positioned in stereo space correctly (because the output of the Audio Source is already positioned). The whole point of this hack is to have a send per Audio Source instead of just sending everything mixed up in the [adc~].

If the [unpack~] object were to be compatible, instead of using the unefficient [ru~ Test] object, I would simply connect a [r Test] to a [unpack~] and send this to the [dac~].

**Problem: **when using LibPD4Unity, it seems that you can only send the mixed audio signal from Unity to Pure Data. This is because all the sending is done by LibPdFilterRead.cs in it's OnAudioFilterRead method and it seems that you can only have one instance of that script that it recommended to be placed on the AudioListener. This means that inside Pure Data, there is no way of differentiating the sources from one another.

Reasons to want to send the audio from Unity to Pure Data instead of just reading sound files inside Pure Data:
-To use the Unity's 2D/3D sound positioning system.
-LibPD has many incompatibilities with extended objects or Unity, so it is much less of a hassle if Unity can deal with reading all the different sound files with all their extensions,
-Unity compresses the audio files bases on build settings and can be platform specific.
-Using other Unity audio processing modules (the mixer is coming soon).
-Other reasons I'm not thinking about right now.

**Partial Solution: **I've been doing some research on the subject and I have almost found the solution. All that is needed is a script which implements the OnAudioFilterRead function and to put that script on an object with an AudioSource. Then you would use LibPd.SendList(nameOfReceiver, data) to send the list of float representing the current audio block. Then in Pure Data, you would set up a [receive nameOfReceiver] and use one of zexy's object, [unpack~] which converts a list of float into a signal Et voilà!
But, there is a catch... the objects [unpack~] and [pack~] (that does the opposite) are not compatible with libPd and/or Unity.
I have hacked a way to do it anyway by writing the list of float to an array and then reading the array with a [tabread~] but the process much too CPU intensive for what it does.

**Question: **Does anyone know a way to make [pack~] and [unpack~] objects compatible with LibPd4Unity and/or can think of an alternative solution to this problem? Pure Data has a huge potential in games, but it needs that flexibility to flourish.

Any help will be appreciated.

I'm looking for a way to send messages (symbols, lists, etc.) through a signal inlet so my data objects and their signal version can be the most similar in behavior as possible.

In reality it would combine what the [inlet~] already does with what the [route~] does (seperates signals from messages). Idealy, I would like to access this functionnality without have to write it in C. Does an object like this already exists?

Thx.