electro-music.com   Dedicated to experimental electro-acoustic
and electronic music
 
    Front Page  |  Radio
 |  Media  |  Forum  |  Wiki  |  Links
Forum with support of Syndicator RSS
 FAQFAQ   CalendarCalendar   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   LinksLinks
 RegisterRegister   ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in  Chat RoomChat Room 
 Forum index » DIY Hardware and Software » ChucK programming language
Guitar Note Detection
Post new topic   Reply to topic Moderators: Kassen
Page 1 of 1 [18 Posts]
View unread posts
View new posts in the last week
Mark the topic unread :: View previous topic :: View next topic
Author Message
Inventor
Stream Operator


Joined: Oct 13, 2007
Posts: 6221
Location: near Austin, Tx, USA
Audio files: 267

PostPosted: Sat Apr 19, 2008 2:10 am    Post subject: Guitar Note Detection
Subject description: Algorithms for extracting the notes from the signal...
Reply with quote  Mark this post and the followings unread

Yesterday I spent a long time working on the attached guitar tracker software. It is supposed to monitor the audio signal from a guitar and pick out the notes. In its present form it doesn't work all that well in my testing because my guitar samples are kind of mixed up, but I think the algorithm is sound.

I'm working with 30 second guitar samples from my Guitar Lab software, and it does pick out existing frequencies from the sound source. The only problem is that it often picks up a harmonic instead of the fundamental. This is partly due to the fact that the guitar models often have more energy in the harmonics than in the fundamentals.

So I wrote the algorithm to first get all the peaks in the FFT spectrum, then walk through the fundamentals and stomp on the harmonic multiples of them. It often works but often doesn't. I'm hoping that once I get a real guitar, the "clean" signal from it will have more energy in the fundamentals.

Another approach I need to look into is envelope tracking. One of the guitar models, though rich in harmonics, has a definite envelope shape at the beginning of each note. I could rectify and filter to extract that information, then use it in conjunction with the FFT to detect notes.

I thought others might have some suggestions, so I'm posting about it. How would you do this?


Guitar_Tracker2.ck
 Description:
A crude guitar tracker. Detects prominent fundamentals and harmonics.

Download
 Filename:  Guitar_Tracker2.ck
 Filesize:  7.09 KB
 Downloaded:  308 Time(s)


_________________
"Let's make noise for peace." - Kijjaz
Back to top
View user's profile Send private message Send e-mail
Kassen
Janitor
Janitor


Joined: Jul 06, 2004
Posts: 7678
Location: The Hague, NL
G2 patch files: 3

PostPosted: Sat Apr 19, 2008 6:18 am    Post subject: Re: Guitar Note Detection
Subject description: Algorithms for extracting the notes from the signal...
Reply with quote  Mark this post and the followings unread

Inventor wrote:

I thought others might have some suggestions, so I'm posting about it. How would you do this?


I have one... but it's a very different approach... and meant to be set loose on files, not in realtime.

What I do is "ride" the RMS, if the RMS goes from increasing to decreasing that must be a peak, I reason. At that point I store the last time it went from decreasing to increasing as that would be the start of the attack of this note or hit.

This is ok but it results in too many notes being detected as it will also detect volume modulations.

So; I only store the time of a "valley" in the wave (when it starts going up again) if the amplitude has decreased to -say- 75% of the last peak or less. That would still have false-positives when the volume fluctuates at a low amplitude but this is much less of a issue since my goal is to detect the n loudest hits in a drum-loop so low-volume noise won't get through anyway.

All of the hits/ notes detected in that way end up in a array to index them and after I did that I remove ones that are shorter then -say- 50::ms.

This tends to give me a set of indexes of a drum-loop that's quite workable for automatic breakbeat cutting but that's a different problem then the one you are trying to solve so the solution is different as well. It's inspired by the settings of Soundforge's "auto-region" tool, I'm not sure they use a similar method but the parameters are quite similar. It's not a realtime tool since it will store the start of the envelope's attack after (and depending on) the peak so there's already latency there and I'm more concerned with dividing the loop up in musically sensible sections then I'm with actually detecting each and every hit.

May still be of some interest?

_________________
Kassen
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Frostburn



Joined: Dec 12, 2007
Posts: 255
Location: Finland
Audio files: 9

PostPosted: Sat Apr 19, 2008 7:39 am    Post subject: Reply with quote  Mark this post and the followings unread

I did have a look in to this but no luck yet. (I've only managed to get the strongest partial out of a signal, not the fundamental)

There are so many cool effects that depend on good fundamental tracking that this surely would be a great tool.

I'm busy with school and all but once I'm done with the exams I should have a whole summer to chuck.

So yeah, I'm still alive but maybe still not chucking for a while Razz

_________________
To boldly go where no man has bothered to go before.
Back to top
View user's profile Send private message
Inventor
Stream Operator


Joined: Oct 13, 2007
Posts: 6221
Location: near Austin, Tx, USA
Audio files: 267

PostPosted: Sat Apr 19, 2008 7:56 am    Post subject: Reply with quote  Mark this post and the followings unread

Yes, I like the idea of "riding the RMS". I'm a little unsure, however, of how to code it. I remember from school that RMS is a calculation for periodic signals, but here the signals are not periodic. Is it done with a sliding window, as is the case with an FFT? I did something similar like this:

Signal in => FullRect fr => LPF lpf => HPF hpf => Gain div => blackhole;
lpf => div;
div.op (4);

Where the LPF is sort-of like an envelope (but not really) of the signal, and hpf is supposed to be the slope of that envelope. Then hpf / lpf is a normalized representation of that envelope and the software looks at div.last () for a threshold transition.

It didn't work, but then again I didn't try to tweak it either. Most of the reason it didn't work is that the signal envelope was nearly flat. I opened up the wav file in audacity to see the waveform and there were no positive transitions in envelope at note beginnings. On another file there were, but not significant enough to detect easily.

However, the above is not as good as an RMS calculation, because RMS will integrate all of the energy including harmonics. If each note consists of a sudden increase in harmonics, RMS will catch it I think. My only issue is how to do RMS with a sliding window calculation. Maybe RMS would just have to be approximated somehow? Something like this perhaps:

Signal in => Gain mult => Gain buff => LPF int => blackhole;
buff => mult;
mult.op (3);

Then poll int.last () and use sqrt() on it, compare to previous values. Could be something like that, eh?

_________________
"Let's make noise for peace." - Kijjaz
Back to top
View user's profile Send private message Send e-mail
Inventor
Stream Operator


Joined: Oct 13, 2007
Posts: 6221
Location: near Austin, Tx, USA
Audio files: 267

PostPosted: Sat Apr 19, 2008 7:59 am    Post subject: Reply with quote  Mark this post and the followings unread

Frostburn wrote:
I'm busy with school and all but once I'm done with the exams I should have a whole summer to chuck.

So yeah, I'm still alive but maybe still not chucking for a while Razz


Good to hear from you, Frostburn! I was just thinking last night: "I wonder where Frostburn and Kijjaz have been?". Even if you don't have time to code, do try to post once in a while to let us know you didn't drop off of the face of the earth! Plus if you happen to comment on any of the posts, that's a bonus. Cheers!

_________________
"Let's make noise for peace." - Kijjaz
Back to top
View user's profile Send private message Send e-mail
Kassen
Janitor
Janitor


Joined: Jul 06, 2004
Posts: 7678
Location: The Hague, NL
G2 patch files: 3

PostPosted: Sat Apr 19, 2008 8:18 am    Post subject: Reply with quote  Mark this post and the followings unread

Inventor wrote:
Yes, I like the idea of "riding the RMS". I'm a little unsure, however, of how to code it.


Oh, those nice Princeton / Stanford people already did that for us, there's simply a RMS Uana.

I think, BTW, that Perry's envelope follower based on squaring the signal, then pulling it through the filter is also a RMS implementation but I may be misunderstanding stuff there again.

_________________
Kassen
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Inventor
Stream Operator


Joined: Oct 13, 2007
Posts: 6221
Location: near Austin, Tx, USA
Audio files: 267

PostPosted: Sat Apr 19, 2008 8:23 am    Post subject: Reply with quote  Mark this post and the followings unread

Kassen wrote:
Inventor wrote:
Yes, I like the idea of "riding the RMS". I'm a little unsure, however, of how to code it.


Oh, those nice Princeton / Stanford people already did that for us, there's simply a RMS Uana.


Well, lookey here, a nice little RMS Uana all set up for me to code! Go Princeton! If I ever have kids, they're going to Princeton, haha. I'll have to code that up once this coffee gets me going this morning.

_________________
"Let's make noise for peace." - Kijjaz
Back to top
View user's profile Send private message Send e-mail
Kassen
Janitor
Janitor


Joined: Jul 06, 2004
Posts: 7678
Location: The Hague, NL
G2 patch files: 3

PostPosted: Sat Apr 19, 2008 8:26 am    Post subject: Reply with quote  Mark this post and the followings unread

Inventor wrote:

Good to hear from you, Frostburn! I was just thinking last night: "I wonder where Frostburn and Kijjaz have been?". Even if you don't have time to code, do try to post once in a while to let us know you didn't drop off of the face of the earth! Plus if you happen to comment on any of the posts, that's a bonus. Cheers!


/\
||

_________________
Kassen
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Kassen
Janitor
Janitor


Joined: Jul 06, 2004
Posts: 7678
Location: The Hague, NL
G2 patch files: 3

PostPosted: Sat Apr 19, 2008 8:28 am    Post subject: Reply with quote  Mark this post and the followings unread

Inventor wrote:

Well, lookey here, a nice little RMS Uana all set up for me to code! Go Princeton! If I ever have kids, they're going to Princeton, haha.


Don't forget Ge went to Stanford. I think Spencer also moved West.

_________________
Kassen
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Inventor
Stream Operator


Joined: Oct 13, 2007
Posts: 6221
Location: near Austin, Tx, USA
Audio files: 267

PostPosted: Sun Apr 20, 2008 3:09 am    Post subject: Reply with quote  Mark this post and the followings unread

Kassen wrote:
Don't forget Ge went to Stanford. I think Spencer also moved West.


OK, Princeton OR Stanford would be fine!

I put your RMS tracking feature in, Kassen, and it improved the situation quite a bit, plus i added a low pass filter change that attenuates harmonics.

I am fortunate in that I know I'm tracking a guitar and the guitar's frequency range is really quite limited. The strings can pluck basically 80 Hz to 800 Hz, with fingering notes up to 1500 Hz (all fundamentals). So really we're only looking at about one order of magnitude in frequency that we have to consider.

Plus, I think when I get a real guitar the "clean" signal from it should be much more detectable than what I'm working with now. We'll see...


Guitar_Tracker3.ck
 Description:
Tracks guitar fundamentals a somewhat better than the previous one, has Kassen's RMS feature.

Download
 Filename:  Guitar_Tracker3.ck
 Filesize:  7.73 KB
 Downloaded:  238 Time(s)


_________________
"Let's make noise for peace." - Kijjaz
Back to top
View user's profile Send private message Send e-mail
Inventor
Stream Operator


Joined: Oct 13, 2007
Posts: 6221
Location: near Austin, Tx, USA
Audio files: 267

PostPosted: Sun Apr 20, 2008 4:04 am    Post subject: Reply with quote  Mark this post and the followings unread

Wow, I tweaked the FFT size on that previous file and it started working very well. Here is an example of the output:

Code:
17 91.516113 0.000636 2.798322 41.815130
15 80.749512 0.000358 2.672298 39.648263
19 102.282715 0.000275 2.558877 43.740706
17 91.516113 0.000237 4.456310 41.815130
61 328.381348 0.000326 4.164487 63.934424
18 96.899414 0.000696 3.901847 42.804676
18 96.899414 0.000499 3.585915 42.804676
75 403.747559 0.000428 3.329862 67.511400
37 199.182129 0.000444 3.094754 55.279016
29 156.115723 0.000135 2.939087 51.061348


The columns are FFT bin number, frequency, amplitude of fundamental, average notes per second, and MIDI number. They are all pretty much correctly detected notes. The average is 3 notes per second, which is how fast the recording goes so that's good. And the MIDI numbers are about right. I think that because of the low pass filter, it might not be as good at detecting higher notes like 1kHz, but that's OK for now.

_________________
"Let's make noise for peace." - Kijjaz
Back to top
View user's profile Send private message Send e-mail
Kassen
Janitor
Janitor


Joined: Jul 06, 2004
Posts: 7678
Location: The Hague, NL
G2 patch files: 3

PostPosted: Sun Apr 20, 2008 8:08 am    Post subject: Reply with quote  Mark this post and the followings unread

Inventor wrote:

I put your RMS tracking feature in, Kassen, and it improved the situation quite a bit, plus i added a low pass filter change that attenuates harmonics.


Considering the guitar sound where notes start with a burst on noise before settling on the pitch I'd LPF before the pitch detection but not before the RMS one. This is because the burst of noise is quite loud and so easy (and useful!) to detect volume-wise while it doesn't convey much pitch information.

While perhaps more expensive as you'll need to Uanae chains I think that might significantly improve the reaction to plucks.

_________________
Kassen
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Inventor
Stream Operator


Joined: Oct 13, 2007
Posts: 6221
Location: near Austin, Tx, USA
Audio files: 267

PostPosted: Sun Apr 20, 2008 8:24 am    Post subject: Reply with quote  Mark this post and the followings unread

Kassen wrote:
Considering the guitar sound where notes start with a burst on noise before settling on the pitch I'd LPF before the pitch detection but not before the RMS one. This is because the burst of noise is quite loud and so easy (and useful!) to detect volume-wise while it doesn't convey much pitch information.

While perhaps more expensive as you'll need to Uanae chains I think that might significantly improve the reaction to plucks.


Yes, it requires two FFT's, but it does seem to work better, good call Kassen.

_________________
"Let's make noise for peace." - Kijjaz
Back to top
View user's profile Send private message Send e-mail
Kassen
Janitor
Janitor


Joined: Jul 06, 2004
Posts: 7678
Location: The Hague, NL
G2 patch files: 3

PostPosted: Sun Apr 20, 2008 8:40 am    Post subject: Reply with quote  Mark this post and the followings unread

It's really quite remarkable how far you can get with this very simple way of looking at it all. You will need to get the response rate up for this if you want to do it in realtime so I think this is a good idea. Fortunately guitar picking is a very brief and very sharp event so to avoid false-positives from for example handling the body of the guitar itself you could even consider a HPF in the chain that aims to detect plucks.
_________________
Kassen
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Inventor
Stream Operator


Joined: Oct 13, 2007
Posts: 6221
Location: near Austin, Tx, USA
Audio files: 267

PostPosted: Sun Apr 20, 2008 9:20 pm    Post subject: Reply with quote  Mark this post and the followings unread

Haha, I just added some HID interfacing for the guitar tracker software. Now it has a pitch shifter, which you can turn on by pressing "p", feedback "f", and reverb "r". Once enabled, the mouse then controls the amount of feedback and reverb with the horizontal and vertical motion, respectively. I'm pleased with the smooth operation of both, though when feedback saturates it shuts off rather than clips - have to do something intereresting here. But overall it's quite good. Oh, press "c" for clean guitar to turn off all three features at once. Soon this will evolve into the stage control software that I've been talking about so much.


Guitar_Tracker5.ck
 Description:
Tracks guitar notes and has kbd/mouse controlled effects.

Download
 Filename:  Guitar_Tracker5.ck
 Filesize:  13.09 KB
 Downloaded:  208 Time(s)


_________________
"Let's make noise for peace." - Kijjaz
Back to top
View user's profile Send private message Send e-mail
Inventor
Stream Operator


Joined: Oct 13, 2007
Posts: 6221
Location: near Austin, Tx, USA
Audio files: 267

PostPosted: Tue Apr 22, 2008 10:22 am    Post subject: Reply with quote  Mark this post and the followings unread

Attached is my latest attempt at a guitar tracker. As with the earlier ones, put a short guitar wav file in Guitar_Tracker.wav, then ChucK the program and press "f" to enable feedback. (note: to prevent unwanted keystrokes in the source code or elsewhere, click on your desktop after starting the program and before pressing "f"). Then move the mouse in X to vary feedback gain from -1 to 1, and in Y to vary feedback delay from 0.1 ms to 100 ms.

The dynamic mouse-control is quite interesting, you can loop the cursor around in circles to create interesting sounds that are not really guitar-related. I found that adding delay to the feedback path prevents the problem of sound going silent due to excessive clipping, I guess because the delayed sound will not superimpose on top of the original sound as was the case before with no delay. At any rate, ChucK's ability to smoothly track the mouse motion and to vary the feedback without audio glitches is promising and leads me to believe that the eventual HID interface will be similarly satisfying.

Anyway, just keeping you posted on the latest progress. Enjoy.


Guitar_Tracker7.ck
 Description:
The latest guitar tracker with 0.1 ms to 100 ms delay in feedback loop under mouse control.

Download
 Filename:  Guitar_Tracker7.ck
 Filesize:  23.66 KB
 Downloaded:  238 Time(s)


_________________
"Let's make noise for peace." - Kijjaz
Back to top
View user's profile Send private message Send e-mail
nescivi



Joined: Mar 23, 2005
Posts: 94
Location: Montreal

PostPosted: Thu Apr 24, 2008 8:19 pm    Post subject: Re: Guitar Note Detection
Subject description: Algorithms for extracting the notes from the signal...
Reply with quote  Mark this post and the followings unread

Inventor wrote:
I'm hoping that once I get a real guitar, the "clean" signal from it will have more energy in the fundamentals.


Well, I'm guessing that you won't, unless you pull the string in the middle and let it go from there. But since normally you play the guitar exciting the strings at the body of the guitar, you'll be inciting more of the higher harmonics than the fundamental one.
I'm betting that that is part of how we distinguish the guitar sound (apart from the body resonance and all).

but you should try and search for guitar pitch estimation algorithms, since I am sure there are a lot around, as this is an area which has high commercial interest as well. DAFX conferences might be a place to start looking.
Back to top
View user's profile Send private message Visit poster's website
Inventor
Stream Operator


Joined: Oct 13, 2007
Posts: 6221
Location: near Austin, Tx, USA
Audio files: 267

PostPosted: Thu Apr 24, 2008 9:40 pm    Post subject: Reply with quote  Mark this post and the followings unread

Ah, an interesting observation nescivi. I will go do some Googling on the topic now!

You might find the video in the first post of this thread to be interesting, if you haven't seen it yet:

http://www.electro-music.com/forum/topic-25122.html

_________________
"Let's make noise for peace." - Kijjaz
Back to top
View user's profile Send private message Send e-mail
Display posts from previous:   
Post new topic   Reply to topic Moderators: Kassen
Page 1 of 1 [18 Posts]
View unread posts
View new posts in the last week
Mark the topic unread :: View previous topic :: View next topic
 Forum index » DIY Hardware and Software » ChucK programming language
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Forum with support of Syndicator RSS
Powered by phpBB © 2001, 2005 phpBB Group
Copyright © 2003 through 2009 by electro-music.com - Conditions Of Use