Now that it is possible to generate shader visuals with sound information in Pure Data (it was before with Gem, right?),
I was wondering how it can work the other way around. To generate sound or at least sound information with shaders and send it to Pure Data. There are some very interesting examples of how to use the GPU for sound generation. This one is Shadertoy again:
https://stackoverflow.com/questions/34859701/how-do-shadertoys-audio-shaders-work https://www.reddit.com/r/musicprogramming/comments/2cbd2s/shadertoy_has_added_glslsynthesized_audio/
This seems also interesting: https://www.fsynth.com/
Anyway, I read out one horizontal pixel line per frame and use the rgb information for generating MIDI notes.
I also use the mScale abstraction from @ingox for scaling the notes.
The player generates also MIDI notes from images, it is only silent if the whole scanned line is very dark or very bright .
My first approach was to generate audio from the pixel information like in the examples above, but the sound was always distorted (but I am sure it could work... ).
I am also interested in other methods to generate sound from visuals and visuals from sound.

GLSL_Video_Effects_V01_MusicPlayer.zip

Here is an example from the MusicPlayer:

Here is the patch that I have been working on for the past few months. OverBlast is a sampler sequencer that will be used as the UI for a DIY midi workstation with a Raspberry pi. And now with a lot of help from people in this forum I have been able to play midi sequences from midi hardware which was a huge step in its development process. Boy let me tell you learning puredata has been an experience that taught me to learn from the headaches and appreciate the highs. lol

In its current state I have been At this point there are a few known bugs. listed below.

Known Bugs

• Sample Editor - Truncate btn dose not work correctly. saves whatever is in soundfiler buffer should clip start and end positions.
• Sample Editor - Loop playback toggle dose not do anything at the moment. sample will automaticly loop.
• Sample Editor - When start and end positions are adjusted the patch will crash the next time the sample is played or looped
• Sample Editor - Loop playback toggle dose not do anything at the moment. sample will automaticly loop.
• Sample Editor - Time stretch dose not to any thing. pitch slider will only changes the pitch. Time stretch should take in account the pitch and bmp to make it a true time strecth.
• Sample Editor - Loop playback toggle dose not do anything at the moment. sample will automaticly loop.
• Sample Editor - Export as Wave dose not do anything at the moment.
• Sequencer dose not work.. WIP contains only ui design
• Song builder does not work. WIP contains only ui design

With that said if you wish to play around with this patch it will more then likely crash while using the sample editor if the start and end positions are adjusted as there is still work to be done with it. The only thing that works in the sample editor is the sample recorder. You are able to record sample and import them to the sample editor to be edited but you cant actually make edits yet.

So why release the patch in this state? Well i could use some feedback and peoples thoughts on the workflow. And simply if there is anyone out there that is interesting in lending a hand in its development. What can you do with it?

You can load samples to the mixer and play sequences from your midi hardware. Record samples from a mic to the sample editor and play them (Dont adjust the start and end positions) it may crash.

Again the song builder and sequencer do not work they are simply ui designs. Be warned if you are going to dive int he sample editor is very messy as it was when I was doing the majority of my exploration/noobing into puredata. I have been slowly cleaning it up.

I should mention you can learn midi notes then save them as a preset the pad controller is fully functional.

Sadly I cant make a video my laptop locks up when I try to use obs or xsplit.

The main patch is OverBlast.pd inside of the zip

OverBlast-SuperAlpha.zip

Apologies for cross-postings

Fifth Annual Web Audio Conference - 2nd Call for Submissions

https://www.ntnu.edu/wac2019

The fifth Web Audio Conference (WAC) will be held 4-6 December, 2019 at the Norwegian University of Science and Technology (NTNU) in Trondheim, Norway. WAC is an international conference dedicated to web audio technologies and applications. The conference addresses academic research, artistic research, development, design, evaluation and standards concerned with emerging audio-related web technologies such as Web Audio API, Web RTC, WebSockets and Javascript. The conference welcomes web developers, music technologists, computer musicians, application designers, industry engineers, R&D scientists, academic researchers, artists, students and people interested in the fields of web development, music technology, computer music, audio applications and web standards. The previous Web Audio Conferences were held in 2015 at IRCAM and Mozilla in Paris, in 2016 at Georgia Tech in Atlanta, in 2017 at the Centre for Digital Music, Queen Mary University of London in London, and in 2018 at TU Berlin in Berlin.

The internet has become much more than a simple storage and delivery network for audio files, as modern web browsers on desktop and mobile devices bring new user experiences and interaction opportunities. New and emerging web technologies and standards now allow applications to create and manipulate sound in real-time at near-native speeds, enabling the creation of a new generation of web-based applications that mimic the capabilities of desktop software while leveraging unique opportunities afforded by the web in areas such as social collaboration, user experience, cloud computing, and portability. The Web Audio Conference focuses on innovative work by artists, researchers, students, and engineers in industry and academia, highlighting new standards, tools, APIs, and practices as well as innovative web audio applications for musical performance, education, research, collaboration, and production, with an emphasis on bringing more diversity into audio.

Keynote Speakers

We are pleased to announce our two keynote speakers: Rebekah Wilson (independent researcher, technologist, composer, co-founder and technology director for Chicago’s Source Elements) and Norbert Schnell (professor of Music Design at the Digital Media Faculty at the Furtwangen University).
More info available at: https://www.ntnu.edu/wac2019/keynotes

Theme and Topics

The theme for the fifth edition of the Web Audio Conference is Diversity in Web Audio. We particularly encourage submissions focusing on inclusive computing, cultural computing, postcolonial computing, and collaborative and participatory interfaces across the web in the context of generation, production, distribution, consumption and delivery of audio material that especially promote diversity and inclusion.

Further areas of interest include:

• Web Audio API, Web MIDI, Web RTC and other existing or emerging web standards for audio and music.
• Development tools, practices, and strategies of web audio applications.
• Innovative audio-based web applications.
• Web-based music composition, production, delivery, and experience.
• Client-side audio engines and audio processing/rendering (real-time or non real-time).
• Cloud/HPC for music production and live performances.
• Audio data and metadata formats and network delivery.
• Server-side audio processing and client access.
• Frameworks for audio synthesis, processing, and transformation.
• Web-based audio visualization and/or sonification.
• Multimedia integration.
• Web-based live coding and collaborative environments for audio and music generation.
• Web standards and use of standards within audio-based web projects.
• Hardware and tangible interfaces and human-computer interaction in web applications.
• Codecs and standards for remote audio transmission.
• Any other innovative work related to web audio that does not fall into the above categories.

Submission Tracks

We welcome submissions in the following tracks: papers, talks, posters, demos, performances, and artworks. All submissions will be single-blind peer reviewed. The conference proceedings, which will include both papers (for papers and posters) and extended abstracts (for talks, demos, performances, and artworks), will be published open-access online with Creative Commons attribution, and with an ISSN number. A selection of the best papers, as determined by a specialized jury, will be offered the opportunity to publish an extended version at the Journal of Audio Engineering Society.

Papers: Submit a 4-6 page paper to be given as an oral presentation.

Talks: Submit a 1-2 page extended abstract to be given as an oral presentation.

Posters: Submit a 2-4 page paper to be presented at a poster session.

Demos: Submit a work to be presented at a hands-on demo session. Demo submissions should consist of a 1-2 page extended abstract including diagrams or images, and a complete list of technical requirements (including anything expected to be provided by the conference organizers).

Performances: Submit a performance making creative use of web-based audio applications. Performances can include elements such as audience device participation and collaboration, web-based interfaces, Web MIDI, WebSockets, and/or other imaginative approaches to web technology. Submissions must include a title, a 1-2 page description of the performance, links to audio/video/image documentation of the work, a complete list of technical requirements (including anything expected to be provided by conference organizers), and names and one-paragraph biographies of all performers.

Artworks: Submit a sonic web artwork or interactive application which makes significant use of web audio standards such as Web Audio API or Web MIDI in conjunction with other technologies such as HTML5 graphics, WebGL, and Virtual Reality frameworks. Works must be suitable for presentation on a computer kiosk with headphones. They will be featured at the conference venue throughout the conference and on the conference web site. Submissions must include a title, 1-2 page description of the work, a link to access the work, and names and one-paragraph biographies of the authors.

Tutorials: If you are interested in running a tutorial session at the conference, please contact the organizers directly.

Important Dates

March 26, 2019: Open call for submissions starts.

June 16, 2019: Submissions deadline.

September 2, 2019: Notification of acceptances and rejections.

September 15, 2019: Early-bird registration deadline.

October 6, 2019: Camera ready submission and presenter registration deadline.

December 4-6, 2019: The conference.

At least one author of each accepted submission must register for and attend the conference in order to present their work. A limited number of diversity tickets will be available.

Templates and Submission System

Templates and information about the submission system are available on the official conference website: https://www.ntnu.edu/wac2019

Best wishes,
The WAC 2019 Committee

@Cuinjune the conway implementation does work now (with and without borders). to achieve that i changed the 1-dimensional table into a 2-dimensional table. because of that i could not figure out yet how to make "shift" work again (except "shift up", which already works), but i will find a solution. the second issue is (again) saving the grid, because pd seems to crash when i return a 2-dimensional table (even without setting the pd [array]).
here is a minimal example of my issue: return_2dim_table.pd
i also wonder if it makes more sense to save a 2-dimensional lua table into a [text] object (which seems kind of an equivalent) or an [array] object. if i save into [array] i have to flatten the lua table before saving and "de"flatten before i load it back into the lua table. or get and set the array/table line by line. basically my main question is: what is the best way to save and recall a 2-dimensional lua table in pure data? everything else does work (and i think better than before).
i also put the matrix transformation functions (shift, invert and so on...) into single ofelia define objects (again), because i think it is better for keeping track of the code.
Ofelia2_GridXMatrix.pd
another issue that i found is that sending a bang into ofelia.bang does not work sometimes while a directly connected bang does work (the "Invert" bang in ofelia2_gridxmatrix.pd for example). which is not really a problem because there are easy workarounds, just wanted to mention it.
i really appreciate all the possibilities of the ofelia library, and i think i just scratched the surface....
edit: everything besides saving works again.

Seem like
[del 1000]

does not always work. sometimes it works sometimes it don't when it does not work I get a instant bang. when it does work I get a 1 second delay before the bang. When it does not work I have to delete the object and recreate it. Then it works but as soon as I add another object or restart pure data it stop working.

@jeffgrips I have it running with vanilla in Windows7.... but it was hard...... and it's a long story....
I cannot see any reason that it would not work for widows10.
I have not found any other easier better solution.
(((SEE below though for the fact that the -lib needs to be declared...!
That could be your only problem? But I doubt it..!!! )))

I used the 64-bit GEM from here (0.94-test3.zip)..... https://github.com/avilleret/Gem/releases/

I unzipped it and put it in a folder (which I called "dot") on my desktop.

Because I am using many different versions of Pure Data I left it there, and I start Pd with a shortcut that also points to the GEM folder and loads it as a library (It has to be loaded at startup as a library...... and that could be why you don't have it working.... but probably not...).
This is the contents of the line "target" in the shortcut.......

"C:\Users\David\Desktop\Pd Vanilla\pd-0.47-1.msw\pd\bin\pd.exe" -lib C:\Users\David\Desktop\dot\0.94-test3/Gem -font-size 10 -font-face "Consolas"

Be careful with forward / backward slashes.

I am pretty sure that I tipped the contents of the "QT" folder out into the "0.94-test3 folder" where gem.dll is (from within the "QT" folder inside "0.94-test3").
If you find some .dlls are missing then post back with a list, and I can upload them....... but you might not have the correct C++ redistributables from Microsoft.

But if you want to start Pd normally by double-clicking a patch then you will need to declare the library in Pd.. edit....preferences.... startup......
If you have GEM in the extra folder you should only need to put "-lib Gem" where you see that I have put "-noautopatch".......

The gem abstractions could not be found though (the "parts of Gem... .like [pix_image] pix_video] [gemwin] etc. worked fine) I copied the contents of the Gem folder from "extended" into a folder on the desktop called "GemAbstr" and set a path to it in Pd preferences.
I removed gem.dll from that folder (no conflicts....!).
The abstractions now create, but the help files for embedded objects cannot be found (different "search" for embedded objects) so I have to open my "GemAbstr" folder and open them manually.

Everything works. All the patches from https://puredata.info/downloads/extended-view-toolkit create and run correctly. The camera on my computer even works (I had never had it working before).
And it works with Spout.

If you go this route then I wish you good luck.......

@zerino has confirmed that it can work with widows10........ https://forum.pdpatchrepo.info/topic/11450/pix-and-geometric-objects-not-being-loaded-in-pd-vanilla/7
........and resolved the "help" file problem.....

For the time being you are lucky that you are using Windows.
This solution is not available for other systems.
David.

HID(FlightStick)+GEM-2d(Cartesian)to3d(Polar)-Visualizer

HID(FlightStick)+GEM-2d(Cartesian)to3d(Polar)-Visualizer.pd

Background:

Currently I am doing a lot of work (using Anduril+elvencloak+and 2 player+recorder abstractions, links to the musical pieces are at the bottom (too many to share here)) with "self-driven" remixes. (The plan is one day it will be me as I play driving those abs.)

Meaning:
I play a piece; send it thru Anduril (with randomly selected effects (14) (generally speaking correctly gain-balanced) and randomly selected fade-in/out times); and then turn elvencloak on or off (in which case I then manually manipulate the parameters).

Since the 14 effects have all been chosen because they have only 3+gain parameters (4 more are in the process of being added (body-resonance (diy2), filter (StampAlbum), noisegate (StampAlbum), & my beat-looper, they could theoritically all be controlled via one 3-d controller.

I considered (pursued) using a handheld's gyroscope/acceleremeter to manipulate the effects, then thought of how I might use a joy/flight-stick to do it if I converted the 2 xy coordinates to polar (theta and ro). (So use two values (coordinates) to create 3 values.)

So...

The patch is a visualization tool to help me learn and hopefully master the use of a 3-d footpedal (by cutting off the stick and inserting a plane of wood, which I;ve talked about elsewhere on the forum).

To use it/see what it is doing you need:

GEM + a joystick;
once you plug in your joystick select it from the hradio and click the green box to turn on HID;
then click the Create button to open the gem window.

As you move the joystick, the subsphere will move around the larger sphere and the three quasi-sliders (from left-to-right: x, y, and z) will move up and down to represent the 2d-to-3d conversion.

To be honest, I still can't digest how it moves. But as is the case with many of my patches I made it to see how it might work.

Note:

Hope you enjoy it. And as the values are all scaled to -4 to 4, it is ready to be used if you want to create a 3d footpedal for your self.

The inv toggle flips the "polarity" of the functions, as I got them from Wikipedia (in the calculations subpatch) and it said the ones they had listed were Not the traditional physics ones. The physics ones are toggle=1 and toggle=0 are the Wikipedia ones (really just relating to which values get plotted to x or y).

Links (in the order in which i posted them) to recent work (Anduril/elvencloak remixes)

p.s. for further testing I am going to be doing remixes of a Hopi (Indian) song, whales, and Gyoto Monk chants. So you might (if you like them check up on the site).

Hopefully, those will be done with me "driving" the app(s) using the footpedal so I can better learn how I might use it in a live/improv environment.

Peace, love, and joyous (or not music thru us all.

Sincerely,
Scott

Hello there,

Could maybe someone explain the order of operations of my [sel] objects on both left and right. 1473000944963-1-working-2-not-working.pd Eventually, I want them to do the same thing, but 3 examples work very differently. (1) Works perfectly (2.1) and (2.2.) do not work in the same way as (1) does, although I am only omitting a few things, and without them I think it should still work in the same way.

I am new to PD, so I really want to understand everything that's going on, and I think that if I don't it will harm my future patching.

@Pierre-Guillot
"make install" does not seem to install anything so I added object files manualy
It compiles but the linker cant seem to find pd stuffs although it is linked with pd...

cc -I"D:/WORK/DSP/pd-0.46-7/src" -Wno-unused-parameter -DPD -DVERSION='"0.0"'  -O6 -funroll-loops -fomit-frame-pointer  -Wall -W -g -o "src/adsr.o" -c "src/adsr.c"
cc   -rdynamic -shared -L"D:/WORK/DSP/pd-0.46-7/src" -L"D:/WORK/DSP/pd-0.46-7/bin" -o "src/adsr.dll" "src/adsr.o"  libCicmWrapper_la-ebox.o libCicmWrapper_la-egraphics.o libCicmWrapper_la-eclass.o libCicmWrapper_la-eobj.o libCicmWrapper_la-ecommon.o libCicmWrapper_la-epopup.o  -lpd  
libCicmWrapper_la-ebox.o: In function `ebox_dosave':
/cygdrive/d/WORK/DSP/CicmWrapper/Sources/ebox.c:819: undefined reference to `s__X'
libCicmWrapper_la-ebox.o: In function `ebox_properties':
/cygdrive/d/WORK/DSP/CicmWrapper/Sources/ebox.c:1098: undefined reference to `s_symbol'
/cygdrive/d/WORK/DSP/CicmWrapper/Sources/ebox.c:1113: undefined reference to `s_symbol'
libCicmWrapper_la-eclass.o: In function `eclass_attr_getter':
/cygdrive/d/WORK/DSP/CicmWrapper/Sources/eclass.c:639: undefined reference to `s_float'
/cygdrive/d/WORK/DSP/CicmWrapper/Sources/eclass.c:653: undefined reference to `s_symbol'
libCicmWrapper_la-eclass.o: In function `eclass_attr_setter':
/cygdrive/d/WORK/DSP/CicmWrapper/Sources/eclass.c:753: undefined reference to `s_float'
/cygdrive/d/WORK/DSP/CicmWrapper/Sources/eclass.c:775: undefined reference to `s_symbol'
libCicmWrapper_la-eclass.o: In function `eclass_addmethod':
/cygdrive/d/WORK/DSP/CicmWrapper/Sources/eclass.c:240: undefined reference to `s_float'
/cygdrive/d/WORK/DSP/CicmWrapper/Sources/eclass.c:252: undefined reference to `s_symbol'

@Pierre-Guillot

so I think the linker should be set to -lCicmWrapper but I got conflicts and undefined
can you help me out ?

cc   -rdynamic -shared -L"D:/WORK/DSP/pd-0.46-7/src" -L"D:/WORK/DSP/pd-0.46-7/bin" -o "src/adsr.dll" "src/adsr.o"  -L"D:/WORK\DSP/PD/CicmWrapper-camo-dev/Sources/.libs" -lCicmWrapper  -lc -lpd  
D:/WORK/DSP/pd-0.46-7/bin/pd.lib(pd.dll):(.text+0x0): multiple definition of `error'
/usr/lib/libc.a(t-d000927.o):fake:(.text+0x0): first defined here
D:/WORK\DSP/PD/CicmWrapper-camo-dev/Sources/.libs/libCicmWrapper.a(libCicmWrapper_la-ecommon.o): In function `epd_add_folder':
/cygdrive/d/WORK/DSP/PD/CicmWrapper-camo-dev/Sources/ecommon.c:578: undefined reference to `sys_searchpath'
/cygdrive/d/WORK/DSP/PD/CicmWrapper-camo-dev/Sources/ecommon.c:603: undefined reference to `sys_staticpath'
/cygdrive/d/WORK/DSP/PD/CicmWrapper-camo-dev/Sources/ecommon.c:603: undefined reference to `namelist_append_files'
/cygdrive/d/WORK/DSP/PD/CicmWrapper-camo-dev/Sources/ecommon.c:609: undefined reference to `sys_staticpath'
/cygdrive/d/WORK/DSP/PD/CicmWrapper-camo-dev/Sources/ecommon.c:609: undefined reference to `namelist_append_files'```