Deconvolution for RT spectral "stamping" of violin
Do you mention that for pure educational purpose, or will this impact my implementation? So far I was thinking that [bsaylor/partconv~] does that kind of stuff for me...
I made a finer freq resolution version of my patch, but the highest framewidth I could use without altering the audio was 8192. If I simply make a FFT and then resynthetize from that with IFFT I get a "chopping" effect, bound to windowing I guess.
Keeping going,
Nau
Nau, I think that zeropadding the input fft frames would be the way to go, in your case like in every similar case of spectral manipulation. From experiments, I have learned that windowing does a lot to reduce wrapping tails, because those tails are already reduced by the window, but of course only zeropadding can really exclude them. If you zeropad (making the frame twice as long), phase angles in the spectrum will not exceed |pi/2| which makes it safe to manipulate phases without wrapping tails in the output. It is often hard or impossible to see subtle tail wraps in an IR plot, but they will alter the result to an extent.
Katja
Thank you again, Katjav.
But the procedure explained in the thesis already makes a weighted kind of averaging in order to take into account that wrapping issue. Am I right?
Here is a citation of the document (p.96):
Once we have an estimation of the body impulse response for each individual frame, we proceed to averaging them in order to obtain an accurate estimation. The reason for averaging is obvious: the input signal is not exciting all frequency components in all recorded frames. An important issue for computing the average is that, since we are working in a bin-by-bin estimation, we must take into account the effects of the window applied to the signals. This fact led us to weight each bin estimation by the energy contribution of the corresponding bin in the input signal spectrum, so that frames for which the energy is low for the bin under estimation (more likely to be affected by window sidelobes) give less influence to the average.
For "energy contribution" I used (magnitude^2).
Anyway, those techniques are both interesting and this zeropadding technique is definitely one of those I need to learn, so I'll start with implementing it in a dumb analysis/resynthesis test. I think I'm gonna cross http://puredata.hurleur.com/sujet-3992-zero-pad-fft-block and http://puredata.hurleur.com/sujet-6682-convolution-reverb-rjdj topics together to sort it out. I'm still wondering if Hann remains the good window to do such a zeropadding, or should I go for something steeper.
I'll go for a 32768 frame size, as experimented in the thesis. I've beeing doing 4 times overlapping so far.
Nau
Yop,
I made a simple analysis/resynthesis patch that runs N=32768 long frames without any click. So the behaviour explained before should rise from the other bigger patch I were playing with. I'll figure it out later.
Now I'd like to convert this simple patch to implement zeropadding of the input signal. I'm not sure I will be able to make a live processing because creating all those zeroes "live" seems a bit tricky to me. I'll go for a static computing scheme first.
I found this explanation useful : http://www.bitweenie.com/listings/fft-zero-padding/
If I want to keep the number of samples N of incoming signal but interpolate it I should go for 2N or 4N-long zeropadded frames, right ? Am I wrong if I think that an overlapping factor of [block~] different from 1 wouldn't work without further complications ?
I'll try that tomorrow.
Nau
Nau, the article you link to has important remarks about zeropadding, but it does not talk about the reasons to do zeropadding. A useful animation is on wikipedia, showing how the energy of two convolved signals is spread over a wider period in time: http://en.wikipedia.org/wiki/Convolution.
Zeropadding in Pd was briefly discussed here: http://puredata.hurleur.com/sujet-6682-convolution-reverb-rjdj. Combined with windowing, it is basically a matter of using a zeropadded window and do extra overlap (to compensate for the zero's).
In convolution 2N zeropadding is sufficient but for filter inversion it depends on how extreme you want the inverse to be. Theoretically the inverse of certain filters may have infinite length, there is no simple rule.
Katja
Thnaks, that's pretty much what I had in mind, and i reassures me.
@nau said:
Yop,
I made a simple analysis/resynthesis patch that runs N=32768 long frames without any click. So the behaviour explained before should rise from the other bigger patch I were playing with. I'll figure it out later.
Nau
I tried to go further in this simple zeropadded analysis/resynthesis patch. I don't know where I messed up when I wrote the upcited words, but the amplitude modulation is still there in my latest patch. Simply logical.
It seems now to perform a correct 4 times zeropadded work, using four zeropadded hann windows, or square windows at will. Beyond the work on zeropadding, I understood that when you sample analysis objects you have to take care of the synchronization between "simple" bang messages and blocks. Hence the use of a "latched" [bang~]. If you don't do this you bang tables to be written to before the next block starts and you get a (unknown length) bunch of zeros (very small values) in front of your table...
This level of control will allow me, I hope, to efficiently compare the values obtained with or without zeropadding. And they should match every 4 values of the zeropadded one...
I use a 3ghz single core and it lags at the end of every block, and it's not only the audio, the written tables are stained too. But I guessed the computer was to blame and tried it on a more powerful computer and it worked without any problem.
Thank you,
Nau
@nau said:
Yop,
I made a simple analysis/resynthesis patch that runs N=32768 long frames without any click. So the behaviour explained before should rise from the other bigger patch I were playing with. I'll figure it out later.
Nau
I tried to go further in this simple zeropadded analysis/resynthesis patch. I don't know where I messed up when I wrote the upcited words, but the amplitude modulation is still there in my latest patch. Simply logical.
It seems now to perform a correct 4 times zeropadded work, using four zeropadded hann windows, or square windows at will. Beyond the work on zeropadding, I understood that when you sample analysis objects you have to take care of the synchronization between "simple" bang messages and blocks. Hence the use of a "latched" [bang~]. If you don't do this you bang tables to be written to before the next block starts and you get a (unknown length) bunch of zeros (very small values) in front of your table...
This level of control will allow me, I hope, to efficiently compare the values obtained with or without zeropadding. And they should match every 4 values of the zeropadded one...
I use a 3ghz single core and it lags at the end of every block, and it's not only the audio, the written tables are stained too. But I guessed the computer was to blame and tried it on a more powerful computer and it worked without any problem.
Thank you,
Nau
Yop,
here is a patch presenting a situation very similar to the one presented in the previously cited tuto http://www.bitweenie.com/listings/fft-zero-padding/ , but in realtime.
Could someone validate this way of doing zeropadding ?
I'm a bit confused by one point : in the zeropadded subpatch I dont see many "windowing lobes" in the fft when I add the results of the four [fft~] objects. These "lobes" are shown in the bitweenie example, though, as it shows the result of a single zeropadded frame. My patch does exhibit such lobes if I delete the six objects located in the grey frame (3/4 of the data is discarded).
So my patch seems to work as described in the bitweenee article when I discard 3/4 of my signal, but when I combine four temporally separated zeropadded versions of my signal I get a smooth spectrum (no lobes anymore) out of the peaks'zone and get four peaks in the middle. Is it normal ?
I can't persuade myself this patch works as it should, nevertheless everything's seems to be fine. For example I get more points in between the ones got with 1024-long frames, but didn't change the waveform frequency resolution, so it's logical still to get 4 peaks instead of the two theorical ones (spectral leakage). But the gaps between those peaks are kinda improvement, from the graphical point of view at least.
Everything seems to be fine, but I'm still dubious. I know that spectral tweaking can be tricky, and I did this without the use of delay lines, that was my first attack angle as it has been advised to me in this thread n(edit : more precisely in this thread : http://puredata.hurleur.com/sujet-6682-convolution-reverb-rjdj , mentionned earlier in this very thread). Maybe should I put delay lines in another corner if this patch ? I'm unsure of everything.
The 8192-long frames with no zeropadding (last object, bottom pos in the vradio) seem to "wobble" at the theorical 5.38hz frequency (44100/8192), but it could not be a problem in my deconvolution patch is it takes care of this effect in its scheme.
What do you think about all this ?
Thanks,
Nau
http://www.pdpatchrepo.info/hurleur/zeropadded_ar_article.pd
I was fiddling with zeropadding recently, in an attempt to make the perfectly phase locked phase vocoder in Pd. Took me a while to realize why I had 'holes' in my output. Well it was the zeropadding of course.
If you zeropad an IR, process in frequency domain and bring it back to time domain, you can easily discard samples, like the non-causal part of an IR. But in a real time patch, you really have to transform the zeropadded signal in an upsampled fft block. For example if the non-zero signal part is 1024 samples, transform it with a 2048 point fft. But this is no trivial thing to do in Pd. A signal upsampled by the [block~] object appears diluted with zero's, that's not what you want. A solution could be to reposition the non-zero samples using [tabread~] / [tabwrite~] with an index table designed for the purpose. And after ifft, dilute again before downsampling.
In the end I did not bother to implement it. The perfectly phase locked phase vocoder seems to be a spoof. It rotates the zeropadded signal segment within the fft frame, with boundaries and all. Nothing we can't do in time domain.
Attached is a patch showing plots for constant phase shifts, linear phase shifts and random phase shifts with and without zeropadding. I found it revealing to see how constant phase shifts make the phases change within the frame, while linear phase shifts implement a time delay, not only for the input frequencies but for the frame boundaries too. Could of know that before, right. But I'm the type who doesn't understand math concepts until they're illustrated with plots.
Katja
Thank you Katjav,
i'll check it soon.
However here the 32768-samples-long frames with no overlap deconvolution patch is working for two days !
I made a shitty test tune, sorry for the earache.
More will come soon.
I added a bit of freeverb, constant params through the test.
Piezo is a piezo mic in the bridge.
DPA is DPA4099 at ~9 cm.
Guesse what is simu !
http://bitshare.com/?f=npc8ju9k
Thanks again,
Nau
Hey nau, the simu sounds great, is it really the inverse-filtered piezo recording?
Thank you Katjav !
Yes, it's Ben Saylor's partitionned convolution of piezo.wav with the IR I finally managed to calculate, using the Perez procedure and your minphase IR procedure discussed before, using magnitude only to reconstruct it.
Seems that the other comments on this first test are unanimous (in real life too, I mean), it's encouraging.
I cleaned and tested further, and it seems rather consistent as 4 different glissandi played with two different bows spit out rather similar frequency responses, IRs and psychoacoustical feelings.
Still more commenting and bettering to do on this patch, but I'll try to post it tonight or tomorrow night.
Nau
So you can now go on stage with your piezo-amplified violin and play like it were captured by an expensive condenser mic? And all done in Pd. This is terrific, congratulations!
I wonder how long your inverse filter kernel is, and if the energy is really concentrated near index zero. If all went right and you did not try to overcompensate, it should be possible to cut down the filter length to some 2048 points (with smooth fade out), thus making the inverse filtering a reasonably efficient procedure. If you set [bsaylor/partconv~] partition size to Pd's default block size, you'll add no latency at all with your inverse filter.
Katja
@katjav said:
So you can now go on stage with your piezo-amplified violin and play like it were captured by an expensive condenser mic? And all done in Pd. This is terrific, congratulations!
Thank YOU, Katjav. Indeed, even crossmixing the direct and the simulated audio becomes possible, if wanted.
@katjav said:
I wonder how long your inverse filter kernel is, and if the energy is really concentrated near index zero. If all went right and you did not try to overcompensate, it should be possible to cut down the filter length to some 2048 points (with smooth fade out), thus making the inverse filtering a reasonably efficient procedure. If you set [bsaylor/partconv~] partition size to Pd's default block size, you'll add no latency at all with your inverse filter.
Katja
Everything is fine, I do confirm! (but I let you check by yourself 
) Of course I could have made a programmatic error somewhere, but I checked it carefully and...it sounds good.
Procedure :
**(you can skip this if you only want to hear how it sounds)
first run "ir_construction" patch. Select an input (measurement, "custom stereo" wav file), retrieve minphase kernel and save it into a file (I already made it for you).**
Then run "convolviolin" patch, select a demo tune and an IR, play and switch between channels to hear the differences between piezo pickup, condenser simulation, and actual condenser microphone audio.
link : http://bitshare.com/?f=piu22sll
Next tests : improving mic placement, concatenate the G string glissando with an A string glissando to see how response changes (and maybe a glissando on a C string in place of the G string), kinda cookbook approach of the filter (it's still a bit "boxy" isn'it ?.
Of course this procedure seems compatible with many instruments or devices, but we'll have to find an efficient excitation scheme for the measurements. Anyway with these patches it's just a matter of trying it...
Thank you again,
Nau
I've downloaded and listened your patch, remarkably there's no substantial frequency response difference between the long filters and the reduced ones. And between the original mic and simu, maybe the most important difference is a sound level inequality. The simulator filtering is really convincing. You got a great tool there, Nau!
If you set [bsaylor/partconv~] to partition size 64 instead of 512, the filter will not add to the system latency (512 samples add ~11 ms at default samplerate). By the way [freeverb~] has the nasty habit of producing subnormals after the input has ended, and cpu load will rise spectacularly. Feed it a subliminal level of anything to avoid subnormals.
Katja
@katjav said:
I've downloaded and listened your patch, remarkably there's no substantial frequency response difference between the long filters and the reduced ones. And between the original mic and simu, maybe the most important difference is a sound level inequality. The simulator filtering is really convincing. You got a great tool there, Nau!
Yes, it didn't work for a while, and suddenly turned to a working tool, I think when I realized that [bang~] had to be used somehow in order to "sync" sampled data. And that's (again) thanks to you, as I came to understand this trying to tweak a zeropadding patch
@katjav said:
If you set [bsaylor/partconv~] to partition size 64 instead of 512, the filter will not add to the system latency (512 samples add ~11 ms at default samplerate). By the way [freeverb~] has the nasty habit of producing subnormals after the input has ended, and cpu load will rise spectacularly. Feed it a subliminal level of anything to avoid subnormals.
Katja
I already use a small partition size here, but only left this high value in the public patch in order to be shure (oops, lapsus) that it would suit anybody's config. Forgot this isn't actually bound to soundcard settings but to PD settings, rather.
Thank you for remembering me that freeverb~ issue, I had noticed it, but had not investigated yet.
I'm curious to make a pizzicato test.
Nau
Hi there,
I made tests in the last evenings, covering these points :
I read other articles and "they" seem to say (as Katjav wrote earlier) that least squares deconvolution would be really interesting, but couldn't read the original articles (by Clarkson, among others) so don't know much about implementation. Anyway it seems more like something to be done within Matlab or Octave rather than within Pd.
I wished I had found an already usable .m program suited to my needs but I hadn't.
I will order a 3/4 viola C string to play it in place of the G one, I wonder if the low-end will be reinforced (if needed) and hope that excitation will still be honest in the high-end.
Anyway I'm aware of the fact that not only dsp procedures could be improved; I'd like to give a shot to a better audio levels management, both in the analysis/inverse filtering computing and in perception tests. As Katjav suggested differences in the perceived audio level do alter spectral perception. I'm gonna start to follow this topic : http://puredata.hurleur.com/sujet-4703-correct-ear .
Would be nice to make something like JAPA (http://kokkinizita.linuxaudio.org/linuxaudio/ , other interesting applications on this page !), in order to compare levels from that point of view.
I'm also dreaming of a pd-stompbox.
@Roger said:
I am currently deciding whether to run the PD on Beagle Board Cubiboard or Wanda board all of which can host a full linux system.
[...]
Anyway the Open Music Labs Stomp shield kit looks like a nice short cut.
http://wiki.openmusiclabs.com/wiki/StompShield
this also interfaces with the computer in their demo , the MP3 files on the site do the stomp shield more justice.
Arduino is too weak to get decent audio resuts. I looked at Cubieboard and Wanda board specs, they are very good but there's (a lot of) work to get Pd working on it I guess (and Pd vanilla won't be able to perform partitioned convolution). It seems that it is not 100% sure it will be possible to get Pd working on these platforms.
But what about CCRMA Satellite (https://ccrma.stanford.edu/~eberdahl/satellite/)? It could work "out of the box". Pd-extended already installed, and tests have been made with an external soundcard with good results. Thrilling.
Worth mentioning : the Logidy EPSi (http://www.logidy.com/?pid=1)seemed to be a winner as a connvolution pedal, but in the manual it is stated that "In the current version of the firmware, the first 128 samples of the impulse response are ignored. If the impulse response includes a direct signal component at the beginning, it will be ignored. This is an advantage for reverb sounds, but needs to be worked around for direct convolution sounds like guitar speaker simulation, something the EPSi was not specifically designed for."
Thanks,
Nau
@nau said:
I read other articles and "they" seem to say (as Katjav wrote earlier) that least squares deconvolution would be really interesting,
erh... I don't know what that is?
I'm also dreaming of a pd-stompbox.
Cubieboard is interesting because it has an onboard soundcard and is more powerful than RaspberryPi. My friend has one. When I tried to compile my own externals for it, I got a problem with the float type and Pd crashes on them. Apparently I had set some flag wrong in the makefile. But of course it should be possible to selectively compile externals for Cubieboard, with a bit more perseverance. [bsaylor/partconv~] needs fftw so it is not the easiest external to build I guess. You could first try and build it on a regular (Intel) computer with Linux to see if you get it done at all.
Katja
@katjav said:
@nau said:
I read other articles and "they" seem to say (as Katjav wrote earlier) that least squares deconvolution would be really interesting,
erh... I don't know what that is?
Sorry, actually it was the "inverse Kirbeby".
@katjav said:
@nau said:
I'm also dreaming of a pd-stompbox.
Cubieboard is interesting because it has an onboard soundcard and is more powerful than RaspberryPi. My friend has one. When I tried to compile my own externals for it, I got a problem with the float type and Pd crashes on them. Apparently I had set some flag wrong in the makefile. But of course it should be possible to selectively compile externals for Cubieboard, with a bit more perseverance. [bsaylor/partconv~] needs fftw so it is not the easiest external to build I guess. You could first try and build it on a regular (Intel) computer with Linux to see if you get it done at all.
Katja
Satellite cost like 200euro, like 3 times a Cubieboard, but how many times greater would the dev time (pd compilation) be? That's a question.
Nau
Oops! Looks like something went wrong!
