Trading and audio/synthesis

Inventor · Posted: Fri Dec 11, 2009 7:27 am Post subject:

OK, I have been playing around with this adaptive filter most of the night now and it seems to be showing the beginnings of some hopes of working. In the plot shown below, 500 of 512 points are plotted (only 500 due to an insane limitation of Appleworks 6). The last 20 points plotted are predictive as are the 12 not shown at the end.

As you can see, the algorithm tracks the input closely and when I shut off the input signal to see how it predicts, it settles on the zero axis. This is in agreement with the input data that you cannot see. I do not yet trust the algorithm, however, because it seems to exhibit one of a few behavoirs listed following.

1. It will settle on zero.
2. It will oscillate around the correct solution.
3. It will show a periodic signal which is incorrect.
4. It will not work at all with some input parameter combinations.

I have found by attempting to fit three different segments of the data that I gathered and formatted (which is a bit of effort) that a carefully specified set of input parameters will properly reproduce the signal and predict it for only a very short while. These input parameters do not vary much depending on which of the three sub-data sets that I attempt to model, so that's a good thing.

I should also note that this method requires evenly spaced data samples and the input data is not evenly spaced. It is marked with timestamps that I cannot process without writing a Perl script or something, so what I did was just assume the data was evenly spaced and deal with it that way. This is not kosher and some additional algorithm such as a curve fit or simple interpolation is required to pre-process the data for use by the adaptive filter.

After I read your comments I will take the next step. Advice please?

Les

AdaptiveFilter1.jpg

Description:

Example of Adaptive Filter 1 Program

Filesize:

99.1 KB

Viewed:

385 Time(s)

This image has been reduced to fit the page. Click on it to enlarge.

mosc · Posted: Fri Dec 11, 2009 9:39 am Post subject:

Looks like you have been working hard at this, Les. Some comments:

Don't worry about the timestamps of the data. Traders just call each data point a bar. Bars can be monthly, weekly, daily, hourly, all the way down to 1 minute and even less. Trading algorithms generally use any time scale. Weekends and holidays are usually ignored in most cases.

The objective of these algorithms is to generate some sort of trading signal - buy and sell queues. In the case of your algorithm, the time scale seems a little bit fast. You should add, if possible, some sort of slow down to this.

Shifting the result of the algorithm forward in time is done, but not without risks. The lag appears to be eliminated, but it is illusionary.

There is usually some number of samples that must be processed before the algorithm starts to work. If, for example, you are using a 512 FFT, then you should process at least 512 data samples before putting out the first result. If you are using a 50 bar moving average, then it will take 50 bars to get going, so to speak.

blue hell · Posted: Fri Dec 11, 2009 10:07 am Post subject:

Inventor · Posted: Fri Dec 11, 2009 12:28 pm Post subject:

To address all four points:

1. There is a smoothing parameter. I will do more smoothing to get a clearer buy/sell decision. I added the "ringing" effect to act as a buy/sell signal. In the above example the dollar goes down then stabilizes, a sell signal.

2. I shifted the output data by a fixed amount equal to one of the input parameters. I understand why the data is shifted by exactly this amount of lag so there should not be a problem. This is why I feel the shifting has minimal risk, but of course it requires much testing.

3. The startup delay is precisely as you stated, Howard.

4. Jan, I felt that this method would not work in real time also when I first tried it, however after playing around with it for hours I thought about it and yes, it works in real time. This is because the computer can run through the calculation almost instantaneously and because all data to the current trade (with some availability delay) is known. It's just a calculation, not a violation of causality. Or did I explain that correctly?

Next up, I have realized that I can run an optimization algorithm to continuously guess at best-fit values of three of the input parameters, and set the fourth one to a known good value. This will automatically tune the algorithm for the given input data.

Further, I can look at the prediction values and determine a buy or sell decision automatically and update it in real time on the screen. I can also produce a numeric strength value to the decision.

By optimizing and deciding, the trader can just run the program continuously and when clear decisions print out then make the trading actions appropriately. I'll keep you posted.

Les

blue hell · Posted: Fri Dec 11, 2009 2:24 pm Post subject:

bantri · Posted: Sat Dec 12, 2009 3:00 pm Post subject:

bantri · Posted: Sat Dec 12, 2009 3:09 pm Post subject:

In short... the most important fact in that book is:

Cycles can be predicted as soon as you spot 1/4 of the full cycle... and that fact gives superior predictability when compared with moving averages.

mosc · Posted: Sat Dec 12, 2009 9:36 pm Post subject:

I have Ehler's book, Rocket Science for Traders. Oddly, or maybe not, it is a great book about Digital Signal Processing. He uses the Hilbert Transform a lot.

I read somewhere that you can use Hilbert Transforms instead of Dome Filters to build frequency shifters (the electronic music tool). I wish I could find more information about that somewhere.

bantri · Posted: Sun Dec 13, 2009 3:11 am Post subject:

His new book seems interesting....

http://emini-watch.com/category/john-ehlers/

Inventor · Posted: Sat Jul 03, 2010 2:46 am Post subject:

I got motivated by my brother to work on the trading stuff and wrote a ChucK program to do it. The plot below shows the price of Apple stock from 05 to 07 in blue and the program's model of it in green. The program predicts a gain of $24 and the actual gain is $47. I figure in this game, a clear buy indicator is a good thing even if it is half-off. Or is that half-baked? I dunno...

Les

eChucKTrader1.jpg

Description:

eChucK Trader Plot

Filesize:

87.85 KB

Viewed:

355 Time(s)

This image has been reduced to fit the page. Click on it to enlarge.

bachus · Posted: Sun Jul 04, 2010 6:44 pm Post subject:

Here is a respected theorist with a formula. Shocked

mosc · Posted: Tue Jul 06, 2010 12:26 pm Post subject:

bachus · Posted: Wed Jul 07, 2010 5:39 am Post subject:

It is my entirely uninformed opinion that markets and society are too irrational and too filled with the unpredictable to be modeled with present knowledge. So I don't take such theories seriously, though I too wouldn't be the least surprised if we were entering a true depression.

nobody · Joined: Mar 09, 2008 Posts: 1687 Location: Not here

"Markets" just show off the stupidity of society. We've got people going hungry and homeless in "developed" societies because of "markets". So it's all garbage, and the sooner we get rid of it, the better off everyone will be.

bachus · Posted: Wed Jul 07, 2010 6:16 am Post subject:

Antimon · Posted: Wed Jul 07, 2010 6:59 am Post subject:

mosc · Posted: Fri Jul 09, 2010 1:30 pm Post subject:

Inventor · Posted: Fri Jul 09, 2010 6:50 pm Post subject:

I'm just setting up the program and giving it an initial set of test data to chew on for now. To get a prediction I set the algorithm to continue after the data is done, as you say. Then I look at all the wiggling plots and see if the trend is up or down. This simple Apple 05 to 07 test case is an easy one to predict though, up up up up.

Next I'm going to add file i/o so I can get an online data service and pump their data into the program. But first we are going to select an online brokerage firm and invest in a few stocks that we like. Any recommendations?

Les

mosc · Posted: Sat Jul 10, 2010 11:59 am Post subject:

Without a doubt - Tradestation.

You can sync up your data with your algorithms. The language they use for the DSP is called Easy Language. It's easy, and integrated directly into the data feeds. You shouldn't use ChucK for trading, IMHO, anymore than using Fortran for music. The Easy Language is a standard for this sort of thing and there are millions of free pieces of code you can get from the net, and you can exchange code with others. Also, Easy Language has built in back testing, a parameter optimizer (very useful), and it can make trades for you automatically. It can query you account and determine which trades are open. Trading cost is very reasonable. It is rated highest regularly by trading magazines.

Inventor · Posted: Sat Jul 10, 2010 12:11 pm Post subject:

Yes, I really really want to go to a tradestation account when i can afford to buy a pc powerful enough for it. Right now we're only playing with a small amount of money and my bro won't go for the PC. But later when we've got more of a bankroll a PC will be a smaller part of the pie. Until then I will have to piece something together myself and ChucK, strangely, has what's needed I think. I'm just working without a budget for now, so I have to do a custom job.

Les

mosc · Posted: Sat Jul 10, 2010 12:21 pm Post subject:

You really do not need a powerful PC for Tradestation. I run on a very modest one running XP. You can run it on your Mac, BTW. People have had great luck using Parallels and Boot Camp. I'd go with Parallels myself.

Inventor · Posted: Sat Jul 10, 2010 3:11 pm Post subject:

Awesome news Howard! That's perfect, but why not Boot Camp?

Les

mosc · Posted: Sun Jul 11, 2010 1:31 pm Post subject:

Well, you can run Tradestation, or any other windows app in a window while running OSX. Bootcamp is a dual boot solution, which is cool too, of course. I've seen a video on the web where someone is using Parallels with Tradestation. I've been trying to find that but haven't had any luck yet.

Inventor · Posted: Sun Jul 11, 2010 2:40 pm Post subject:

OK that is a good reason to use Parallels. Also the link you gave me in chat demonstrates it on video very nicely - and in text. Now my only question is that my Mac mini has only 2Gigs of RAM, so do I need to get an XP disk or can I get a Windows 7 disk?

Les