Posted: Wed Feb 03, 2010 5:10 pm Post subject:
Spectral Vocoder Subject description: Realtime spectral analysis and additive resynthesis. G2 goes frequency-domain. :)
Hi,
I've been reading up on DFT and FFT and pondered how much of it could be applied to the G2. It was obvious pretty quickly that a complete frequency-domain representation of a given audio signal and consecutive processing was totally out of the question. However, by severely limiting the amount of frequency bins, simple stuff could be achieved.
My idea was that a monophonic, harmonic and highly periodic signal can be represented by the sum of it's partials and their magnitudes. So, let's say, if I want to graft the spectral information of a source signal onto this monophonic pitch, I only need to extract from the source signal the magnitudes of the sinusoids that directly correspond to the frequencies of these partials. This limits the required frequency bins down to a very small amount -which the G2 can handle.
The result of that idea is this patch, which I call a spectral vocoder. In some ways it is similar to the "classic" vocoder, in other ways not at all. The classic vocoder is based on analysis and resynthesis bandpass filter banks, which arbitrarily filter and process the source and carrier signals without knowing much about them. But in this case I know that the result shall be a pitched monophonic signal, so therefore I know a) exactly which information I want from the source signal (the magnitudes of the sinusoids that correspond to the partials of my monophonic pitched signal), and b)how to best represent this signal: by the sum and magnitudes of it's partials. In other words: by additive synthesis.
Since the standard computer DSP programming approach to measuring the magnitudes of sinusoids in a source signal is impossible to realise on the G2, I opted for an old "analog" technique, which is to heterodyne the source signal with the sine and cosine of the frequency to be measured, and taking the square root of the quadratures of the results. (This actually is nothing else but transforming rectangular coordinates into polar coordinates, with the polar radius being the magnitude of the sinusoid). Quadratures are easy on the G2, and thanks to a building block by Mike Estlick, square roots are possible too.
Patched in the most economical way possible, the process described above uses close to 25% of a DSP. And since I always wanted to build a patch that completely maxes out an expanded G2 , here finally was the opportunity. But instead of tediously filling up all VA and FX slots with these analysis stages one by one, I used the G2s polyphony feature, each voice conveniently acting as a macro for a single analysis/resynthesis band. These bands are controlled by the FX area via interslot busses. So I get 30 bands in total from an expanded G2, which is as good as it gets. Of course you can use less bands by turning down the voice count (or using an unexpanded G2 ), but the resulting sound will be increasingly dull.
So, at last, a single monophonic voice that maxes out your expanded G2!
The controls:
INPUT: Select channel for the source audio
INPUT COMP: Compress the source audio to taste
Resp1 and Resp2: Response characteristics of the analysis stages. Controls the sensitivity, speed and immunity to inharmonic frequency content.
FORMANT: Shift formants up and down (simply by offsetting the analysis and resynthesis frequency bands)
HI THRU Freq/Level: Just like classic vocoders, let's you add some high frequency content of the source signal, especially good for sibilants and "air".
KEY PLAY: Attack and Release times of the keyboard envelope.
PRE-ANALYSIS EQ: Tweak the source audio before it goes to the analysis stage.
POST-RESYNTHESIS EQ: Tweak the resynthesized result.
...and there was DSP power left for a delay module (yay! ).
There's a short demo mp3 of this patch too, with me talking into the microphone and playing some keys. G2 goes Antares.
This is probably as close as you can get the G2 into frequency-domain territory. (But never say never )
cheers,
tim
SpectraVocoderTK.pch2
Description:
Monophonic vocoding by realtime spectral analysis and additive resynthesis. Monophonic monster patch that completely maxes out an expanded G2.
Glad you like it. It was quite an adventure. In the beginning I had a pitch-tracker as well to track the pitch of the incoming signal to extract the partials thereof and port those to the pitch of choice in the additive resynthesis stage. This resulted in the formants being shifted as well, producing the "mickey mouse" effect. However, we all know that the pitch-tracker is so-so, so it caused occasional warbles in the spectral distribution, and anyway -who want's mickey mouse? .
It was so obvious to extract the sinusoid magnitudes from the input signal directly (without pitch-detection) that I didn't think of it -hence porting to the desired pitch only those magnitudes that correspond to it's partials, and therefore retaining formants. Easier this way, no pitch-track warbles, and no mickey-mouse.
You can hear the high-frequency partials ring a little bit, which makes it sound more artificial than it should. This is due to the square root function being based on feedback, which is noticeable especially in the higher partials which have high frequency and low amplitude. There's no way around this unfortunately. It would sound better otherwise.
cheers,t
EDIT: If you want some fun, try twiddling the partial quantizer module, which defines the partial spacing. You can tweak it eg. to only odd harmonics, sounds pretty weird.
Joined: Apr 13, 2009 Posts: 11 Location: Germany, near Munich
Posted: Fri Feb 05, 2010 5:47 am Post subject:
...that's embarrasing! - how do you make things like this? - all the time?? - again and again???...i feel pretty depressed now {being unable to do things like this myself}...
Joined: Jan 23, 2006 Posts: 526 Location: Sweden
Audio files: 36
Posted: Fri Feb 05, 2010 12:34 pm Post subject:
tim wrote:
(...) It was so obvious to extract the sinusoid magnitudes from the input signal directly (without pitch-detection) that I didn't think of it -hence porting to the desired pitch only those magnitudes that correspond to it's partials, and therefore retaining formants. (...)
This is due to the square root function being based on feedback, which is noticeable especially in the higher partials which have high frequency and low amplitude. (...)
obvious? uh, hm, what was that middle part again?
thanks for sharing, tim. keep up the excellent work!
Joined: Oct 13, 2007 Posts: 6221 Location: near Austin, Tx, USA
Audio files: 267
Posted: Tue Feb 09, 2010 6:20 pm Post subject:
The main part that I don't understand is how does the patch determine what the fundamental frequency of the voice is? I don't know the G2, so looking at the patch won't help me there. Anyone understand that part of it?
Les _________________ "Let's make noise for peace." - Kijjaz
Oh, all these "shocked" emoticons... what did I do.
inventor wrote:
The main part that I don't understand is how does the patch determine what the fundamental frequency of the voice is?
It doesn't. That's the whole point. It extracts from the source signal only the sinusoid magnitudes that correspond to the harmonic series of the monophonic destination pitch. This is then synthesized from these magnitude measurements via additive synthesis.
It's a bit expensive for only one resulting voice (to put it mildly ) from an expanded G2, so almost posted it in the "experimental" subsection. It's just that when I read up on DFT, I thought "Come on, the G2 must be able to do this, if only a very little bit." It worked out. Glad you like it.
Joined: Oct 11, 2004 Posts: 204 Location: London
Audio files: 1
G2 patch files: 1
Posted: Sat Feb 13, 2010 10:00 am Post subject:
this one is so much fun, thanks Tim!
i made my engine boot with this one for now...
it was just what the doctor ordered, as i added an xlr input recently with a knob, so i can vocode with just a mic and the engine. and a keyboard, erh, you lucky G2 key owners.
Thank you Carlo. Seeing/hearing ones patches used for artistic expression is the ultimate compliment.
cheers,
t
Tim,
I gave credit to you, on my blog entry, for the Spectral Vocoder. It's easy to use a patch like yours "for artistic expression" when they are so beautifully built
Thanks to you _________________ homepage - blog - forum - youtube
Quote:
Don't die with your music still in you - Wayne Dyer
Joined: Oct 13, 2007 Posts: 6221 Location: near Austin, Tx, USA
Audio files: 267
Posted: Wed Feb 17, 2010 2:44 am Post subject:
I played Carlo's song on my radio show and mentioned the g2 patch of tim's after the song. You guys are collaborating so well it makes for great radio!
Les _________________ "Let's make noise for peace." - Kijjaz
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum You cannot attach files in this forum You can download files in this forum
Forum index » Clavia Nord Modular » G2 Patches - Completed » Audio In
FAQ
Calendar
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
Chat Room
, here finally was the opportunity. 
)
I use on of my G2's primarily for vocal processing, so I'll throw this at my singer and see what he thinks.
-hence porting to the desired pitch only those magnitudes that correspond to it's partials, and therefore retaining formants. Easier this way, no pitch-track warbles, and no mickey-mouse.
...
