granular synthesis methods for ChucK

Inventor · Posted: Wed Mar 26, 2008 1:43 am Post subject:

Nicely done, Frostburn, I enjoyed the sound sample and the code looks very well written as is your usual style. Now along the same lines, I am wondering if we can use this DigitalTape class to create songs such as this one by Amy X Neuburg:

http://amyxneuburg.com/sound/tattoo.seg.mp3

It has a vocal track that you can discern, and underlying that is noisy stuff that sounds somewhat like grains of the vocal track. I think it's really samples of other singing, but we could approximate that with grains.

I'm thinking that a ChucK program could record the vocals and then construct that underlying grainy chaos from the recording. The program could construct many layers of granularity, or really a continuum of granularity from fine to coarse simultaneously. We would then have the ability to automatically generate this kind of song from any vocal track.

That could be a very effective music creation tool. Just sing the lyrics and a complete enjoyable song is automatically generated by ChucK.

In addition to vocals, it might work well using a single instrument as the source, such as a sequence from Guitar Lab. Create one good musical track of any single source and out pops a finished song.

I may give this a try if you or kijjaz don't feel so inclined, what do you think?

Frostburn · Posted: Wed Mar 26, 2008 12:11 pm Post subject:

My few experimentations with DigitalTape suggest that it is some kind of a super filter. It can do flanger, reverb, time dilation, scrathing... Well pretty much anything and even some more.

I've added cubic interpolation and better loop control for DigitalTape but there are some nasty bugs related to the new system that I've yet to catch.
I'll be posting when it's ready.

kijjaz · Posted: Wed Mar 26, 2008 12:28 pm Post subject:

This turns out to be very general as i expected.
Soon we'll have many read and write heads flying around,
creating some monster machine hahhaha -_-"

I wanna try one take with the idea also.
i'll try to make something that sounds different in character.

Frostburn · Posted: Wed Mar 26, 2008 2:24 pm Post subject:

Ok, known bugs crushed.
I'll post the source now and start writing example files and documentation tomorrow.

digitaltape.ck

Description:

DigitalTape
Now with cubic interpolation and looping with fractional start and end marks.

Download (listen)

Filename:

digitaltape.ck

Filesize:

12.26 KB

Downloaded:

608 Time(s)

Inventor · Posted: Wed Mar 26, 2008 3:14 pm Post subject:

Very cool, guys. Now I want to write a signal control interface for it and plug it into Synth Lab. I'll wait and see what you do with it first, though.

quintopia · Posted: Wed Mar 26, 2008 8:08 pm Post subject: my patch

I finally "finished" a patch for a piece I call Entropy. It's a press-and-play kind of thing, and I hope it helps stimulates some ideas for ways to use granular synthesis.

There is still one bug that I never cracked since I just edited the output to remove its effects. Namely, sometimes, when certain envelopes key on near the end, there are some clicks and pops from clipping artifacts.

Be sure to turn your speakers or headphones down before listening to it live.

granularprojcbetter.ck

Description:

Download (listen)

Filename:

granularprojcbetter.ck

Filesize:

11.74 KB

Downloaded:

648 Time(s)

Inventor · Posted: Wed Mar 26, 2008 8:34 pm Post subject:

How interesting. I'd like to hear it but it won't run on my miniAudicle or on the command line version for some reason. It looks like an interesting song, but it won't run. Can you post an mp3?

Inventor · Posted: Wed Mar 26, 2008 8:56 pm Post subject:

I was just looking over the DigitalTape class, and I think there is a good way to hook it up into Synth Lab. I have already established floating point input and output from between the analog signals and thet methods of a class. It's just a $ int cast to add integer capability. Plus I have already established logic levels for receiving bits from analog signals. I can just add an input node slider or two for each function call and an output slider for each return value. Hmm... it would be a good idea to have this in Synth_Lab, so I think I will do it.

quintopia · Posted: Wed Mar 26, 2008 10:43 pm Post subject:

mp3 of above patch sample results posted

after further review, i think there was one other bug i wasn't noticing for one reason or another. anyway, the results sound fine.

entropyfinalbetter.mp3

Description:

Download (listen)

Filename:

entropyfinalbetter.mp3

Filesize:

4.72 MB

Downloaded:

556 Time(s)

Inventor · Posted: Thu Mar 27, 2008 8:22 am Post subject:

Well, iti was OK. I liked the music at the end best. It's a good example of granular synthesis as I understand it to be, where you have shredded up the sound at the beginning into smithereens. That is kind of cool in a way. The only problem I had was the tweedling sound that exists somewhat at the beginning and especially in the middle such high pitched sound hurts my ears, but that's my limitation not yours. My ears hurt when women shout for example. Might be better with lower frequencies I guess.

Frostburn · Posted: Fri Mar 28, 2008 4:53 am Post subject:

I discovered a glitch in the looping and had to revise it.
Interpolating array writing in to the reverse direction with fractional looping wasn't as easy as I had thought. :P

Just a few more bug hunts to make sure and it should be ready for releasing.

EDIT: Source posted
some documentation can be found in the comments

up next: writing examples

but first:

Frostburn · Posted: Sun Mar 30, 2008 8:17 am Post subject:

I added anti aliasing cosine interpolation to the reader but I'm still pondering on how to implement it on the writer side...

digitaltape.mp3

Description:

Letting a write head and a read head roam freely on a piece of Digital Tape.

Download (listen)

Filename:

digitaltape.mp3

Filesize:

208.88 KB

Downloaded:

808 Time(s)

digitaltape.ck

Description:

Digital Tape version 0.2 beta

Download (listen)

Filename:

digitaltape.ck

Filesize:

33.58 KB

Downloaded:

348 Time(s)

Inventor · Posted: Sun Mar 30, 2008 9:08 am Post subject:

Wow that mp3 was awesome, It seems like a very professional effect. I tried to get the example to work by removing the /* and the */ plus putting a digitaltape.wav file in the samples directory but it only made silence. Not sure why that was, but I'll mess with it to try to make it work. It was running though, because it slowed my computer down to the point where the mouse was jumping when I moved it.

Frostburn · Posted: Sun Mar 30, 2008 9:57 am Post subject:

kijjaz · Posted: Sun Mar 30, 2008 11:17 pm Post subject:

wow.. the sound quality of the above example is great. it's clear and warm. i like it.
i'm still roaming around northern part of Thailand and still don't have time to play with your digital tape. -_-"
it's so exciting...

Inventor · Posted: Sun Mar 30, 2008 11:56 pm Post subject:

I hope you're having fun roaming about Thailand, kijjaz!

Inventor · Posted: Mon Mar 31, 2008 12:21 am Post subject:

As for DigitalTape, I appreciate your efforts to make it a high quality audio tool, but it's just plain unusable on my old Mac. I let it run for five minutes each on miniAudicle and command line ChucK with --silent, and no sign of completion. It did produce a 26-second output file, but it was inaudible. DigitalTape will just have to wait until I get a faster machine, which is not soon.

Inventor · Posted: Mon Mar 31, 2008 1:31 am Post subject:

Oh, and um, I don't intend the above post to be taken the wrong way. We're all pushing the limits of ChucK in various ways including CPU usage as we explore what we can do with this awesome tool. LiSa and SndBuff will have to do if you haven't got the oomph to run DigitalTape, I guess is the story.

Frostburn · Posted: Mon Mar 31, 2008 5:25 am Post subject:

Yeah, I totally understand.
Working with Digital Tape as it is requires patience and a good cooling system. I cannot do anything about that. You can try running everything with 0 => myDigitalTapeWriter.interp => myDigitalTapeReader.interp; but it's still too slow for realtime on most machines. Doing advanced stuff in ChucK that requires sample precission and a lot of operations will always be out of the reach of live gigs and impatience. (unless ChucK gets super optimized all of a sudden)
Maybe one day I'll port it to ChucK on the source level and then it'll become usable. No promises though. :)

kijjaz · Posted: Mon Mar 31, 2008 1:11 pm Post subject:

Inventor: Or the example needed to be run at an exact sample rate..? 44,100hz?

Inventor · Posted: Mon Mar 31, 2008 2:17 pm Post subject:

Frostburn · Posted: Mon Mar 31, 2008 2:19 pm Post subject:

ChucK has a default sample rate of 48,000 when running JACK. That's what I used.

kijjaz · Posted: Mon Mar 31, 2008 2:58 pm Post subject:

This is my first take of what i said to be a digital tape with a character.
The design turns to be very much like Frostburn's.
I still haven't implemented interpolation to the read head yet,
but tried a new method i come up with for writing.

the example code tries to record a simple sine tone in the begining
and then use one write head and one read head to scatter the sound of the begining part of the tape around randomly with random write speed.

it's that thing i want to do with this granular idea in ChucK.

for now, it sounds quite digital, but surely has a character.
the writing part still has problems. I'll correct the mathematical model.

it still has the ArrayOutOfBound error, i'll fix that very soon!
- - -

edited: Ahhh!! the algorithms are all so wrong. dont' take it seriously hahah
(but it's fun to listen to)

kijjaz-DigitalTape01-0.1.ck

Description:

kijjaz's digital tape verion 0.1 alpha

Download (listen)

Filename:

kijjaz-DigitalTape01-0.1.ck

Filesize:

4.52 KB

Downloaded:

292 Time(s)

Frostburn · Posted: Mon Mar 31, 2008 3:34 pm Post subject:

Oh ok. That example sounded like cutting tape in to pieces and splicing them back together after a good shake. Except it was more crazy and digital sounding. I liked it! Smile

However to me digital means high quality and tape brought up the idea of a looped medium to work on. The sense of unviolatible topology in me wouldn't have dared to cut such a pretty picture in to separated individual grains. There must be continuity and preferably smoothness (differential continuity).

But now that I think about it. Tape fits better in with the grainy splicy idea. While turntable would fit my approach.
I think I'll change the name in to DigitalTurntable so that you can use Digital Tape.

Of course you can break the sound buffer in to grain fragments with my classes but my intention is to create an abstract turntable with read and write heads that is not bound by the limits of time. (Only by the limits of processing power and available data). And the main focus is on quality instead of lightness so it's better/only suited for audio rendering than live performance.

Inventor · Posted: Mon Mar 31, 2008 3:40 pm Post subject:

Woah, take me to your leader! That sounds like a professional quality recording done for an electromusic track. It made a similar impression as Frostburn's mp3 - well done! It evan ran on my klunky old machine, wow.