mindmoneymarkets

Will mean reversion bounce back?

Excuse the pun with the post title (Unintentional honestly!)

I have to admit I'm absolutely stumped by the things I'm investigating at the moment.

I would normally give up and move on, but I believe these things have implications for robust adaptive system development.

It concerns the Agg M indicator which I've blogged about a few times.

Aggregate M Indicator

As a quick recap, the Agg M works on the principle that markets are mean reverting in the short term but trend following in the longer term. The Agg M blends these two principles, with the longer term trend helping to smooth the more profitable but volatile short term mean reversion.

I wanted to have a look at making the Agg M adaptive, mainly because it broke down in May.

This is based on the bog standard settings of opening trades at the next close and holding for 1 day. Buy above 0.5 and sell below 0.5. I tested the FTSE 100 and the SPX, results were roughly similar.

'Broke down' is a bit excessive, but especially on the SPX, there were indications in May that the Mean Reversion part of the system was not firing. May saw a huge upsurge in volatility, a market condition that is 'supposed' to good for Mean Reversion, yet the Agg M had its worst month for a long time.

Month	FTSE	Agg M
Jan-10	-4.71%	-4.71%
Feb-10	2.63%	1.95%
Mar-10	7.25%	7.47%
Apr-10	-0.88%	-0.88%
May-10	-7.38%	-7.38%
Jun-10	-1.77%	3.59%

An Adaptive Agg M

I got thinking, well if the Mean Reversion part is falling down, that could be easy to manage and isolate. The indicator is built on the premise that markets are mean reverting in the short term, but what if they stop becoming mean reverting and do something else instead (or the type of mean reversion changes etc?).

So how do we track if the system is starting to become unprofitable?

My intimidate thought was to try the Expected Value levels on a rolling basis to see if it provides an means of switching the entire system on/ off or at least the mean reversion part.

To cut a long (and very boring) story short, I tried many different combination and rolling EV lengths, but applying the filter based on system performance tended to decrease overall performance, not increase it.

huh?

The chart below shows the problem visually. The problem with the Agg M, even with the recent under performance is that it is never consistently unprofitable. If anything, there is a rhythm whereby it bounces from weak performance to strong performance and back again. The data is of the FTSE 100 from 1997 to 2010.

A score of 0.5 or above means the system has been profitable for the last 50 periods.

I ran a rolling 50 period correlation comparing today's Expected Value Score with the Expected Value score from 50 periods ago.

The correlation strength waxes and wanes, but the clear bias is for the correlation to be negative. In other words a strong performance 50 periods ago is more likely to mean a weak performance today. Conversely, a weak performance 50 periods ago is more likely to mean a strong performance today.  

This makes using an Expected Value filter very hard because each time the system looked like it was about to break down, it was actually more likely to go on and make bigger profits. 

Correlation does not equal causation, but there appears to have been a relationship there.

Can we use this?

I ran a filter as follows:

Trade the indicator as normal except when today's rolling Expected Value is 50% greater than 50 periods ago. In addition, today's Expected Value also has to be greater than yesterdays. Logically with a negative correlation, this should improve performance because when the EV is 50% higher than the past, the more likely outcome is for the system to under perform, so we cut these trades out.

 

Month	FTSE	Agg M	AggMA
Jan-10	-4.71%	-4.71%	-4.71%
Feb-10	2.63%	1.95%	1.95%
Mar-10	7.25%	7.47%	7.47%
Apr-10	-0.88%	-0.88%	2.10%
May-10	-7.38%	-7.38%	-6.23%
Jun-10	-1.77%	3.59%	3.59%
Jul-10	0.00%	0.00%	0.00%

Month	FTSE	Agg M	AggMA
2001	-14.94%	24.42%	26.49%
2002	-25.78%	16.15%	23.15%
2003	15.53%	-0.70%	6.62%
2005	16.06%	13.66%	14.59%
2006	10.94%	5.80%	6.40%
2007	5.24%	-6.50%	-4.36%
2008	-31.76%	4.95%	24.08%
2009	23.36%	23.55%	33.35%
2010	-4.85%	0.05%	4.17%

AggMA = The Adaptive Agg M.

Slightly better performance in 2010, but a better total performance.

Thoughts

While interesting and potentially useful, the findings to not answer my original question on how to tell when Mean Reversion is falling apart. Neither does it provide a good way to finally say that a system has bit the dust. It seems a good way of working out when a method has over performed in the short term, but not as a switch on or switch off, which is what I was looking for.

Other thoughts:

• Indicators/ methods, just like markets have their own rhythms. Perhaps a completely unprofitable method could be made profitable by exploring its rhythm.
• Perhaps measuring the number of consecutive days a method goes with an EV under 0.5 might help as an off switch.
• Building adaptive systems is bloody difficult!

Any thoughts/ suggestions/ comments appreciated.

AggZ Indicator

David Vardi posted yet another interesting idea last week called the Agg Z. Similar to the Agg M, the Agg Z aims to be mean reverting in the short term and trend following in the longer term.

You're best having a read of David's original post for the full description. I believe excel versions of his indicators are on their way, but until then, curiosity go the better of me, so here's an excel version of the indicator.

Download AZIAuto

The returns are decent, but according to my data have come off the boil this year. It also seems that it would benefit from a stop loss of sorts. 

I tried various settings and unfortunately, unlike the Agg M indicator, couldn't get much value out of it at extreme values. It seems buy above 0 and sell below 0 works best. The stop here is simply 1 SPY point. No doubt more intelligent stops could be used.

I did note that the long side of things works out a little better. 

Long only:

Short only:

It produces more profits than the Agg M, but with more volatility.

Best short term indicators

There have been some great posts on short term mean reversion indicators. A recent one here from Market Sci looked at various 2 period indicators, while Trading The Odds looked at a few more under slightly different settings.

In a past post I wondered if you could improve the AMI by using a different MR component. The first task is therefore to findout which MR is best. The guys mentioned above have already done the lions share of the work so hat tip there.

While doing this, I wondered, can you apply a relative strength tracking filter to various systems in a sub category and switch between them depending on progress. You can see the below that different systems responded to the market at different times, switching between them in some way could improve returns - could

Anyhoo, I only got part way through my list, but here's the findings so far for

DV2(a) - Buy/ Sell at close of next day
DV2(b) - Buy sell at close of that day
Ultimate Oscillator

First, here's the results if buying >0.5 and selling <0.5

 

	spy	DV2	DV22	UU
CAGR	3.24%	12.70%	14.88%	9.42%
Max DD	56.65%	12.89%	22.08%	27.46%
Return	51.32%	373.37%	506.65%	222.24%

First, here's the results if buying >0.8 and selling <0.2

 

	spy	DV(a)	DV2(b)	UU
CAGR	3.24%	9.18%	12.74%	11.03%
Max DD	56.65%	13.86%	12.70%	15.36%
Return	51.32%	213.07%	375.45%	289.64%

You can hopefully see how the different indicators got there groove on at different times. Even the different DV2 options performed better at different times.

Can you perform a rotation model on systems/ indicators just like you can with momentum stocks? Who knows, I'll try to find out.... Have a good weekend.

Agg M under the hood

I've been musing about the Aggregate M Indicator introduced by the fantastic David Vardi. I've done some work on this here including an auto spreadsheet you can download.

There's been some great posts across the blogosphere comparing the various short term mean reversion indicators. I like the AMI indicator because it combines trending and Mean Reversion inputs.

The AMI uses a 252 period Percent Rank for the trend and a 10 period negative percent rank for the mean reversion. I wondered, can the indicator be tweaked by using different sub components e.g. a moving average instead of the 252 PR for the trend and a RS 2 for the Mean Reversion.

Before answering this, I wanted to see how the two components of the AMI were performing. To do this I split the indicator trading just the MR part and then just the trend part.

First, here's the combined results on the S&P500 (SPY). No commission etc.

Here's AMI using >0.5 as the buy trigger and <0.5 as the short trigger.

 

Here's AMI using >0.7 as the buy trigger and <0.2 as the short trigger.

Mean Reversion component only

What if we try to isolate the different components of the AMI.

Let's see how the Mean Reversion component performs on its own:

AMI MR component ONLY using >0.5 as the buy trigger and <0.5 as the short trigger.

 

As a note, my 'stop' here is a notional one at 1 point just for reference. This to me implies that the Mean Reversion component on its own can experience to volatile days, which kind of makes sense. The MR component appears to bring the most potential, but also the most volatility or at least exposure to volatility. The 1 point stop implies that profitable days are likely to go against you before they turn around.

AMI MR component ONLY using >0.7 as the buy trigger and <0.2 as the short trigger.

 

Again it appears that the MR component is producing the bulk of the combined profits, but again exposing you to greater volatility.

Trend component only

Now lets look at the trend following component on its own. 

AMI Trend component ONLY using >0.5 as the buy trigger and <0.5 as the short trigger.

 

The trend component isn't exactly smooth, but interesting to see the opposite effect with regard to the stop and volatility. Here the stop helps things, implying that moves that go against you generally continued to do so throughout the day past the 1 point stop.

AMI Trend component ONLY using >0.7 as the buy trigger and <0.2 as the short trigger.

 

Again, the stop helps thing somewhat with the bulk of returns coming in the credit crunch era.

Conclusions

The trend component appears to be reducing the volatility from the MR component. Interesting to see they're both playing their part in different ways.

In another post I'll see what happens if we substitute in different components. More soon.

The challenge...

I must admit I've been flagging this last week and the trading system challenge took a back seat to the day job. However, aside from watching the Giant Slalom tomorrow it's relatively clear waters ahead for the next few weeks.

I really enjoy taking tips and inspiration from Dr Brett Steenbarger's blog. I can't remember the link, but it was something along these lines that caught my eye. In a previous post, the good doctor shared research that showed that great achievers showed tremendous energy (well duh, you might say), but it stuck a chord with me as I was flagging last week.

Energy/ focus/ commitment of purpose all intertwine to give us that drive to excel. It's available all times, yet tantalizingly out for reach or in hiding a lot of the time. Confidence plays and big role, creating a virtuous circle with all the other good stuff above.

It is certainly easier to read about than do, but I find that at least reflecting on this gives me a boost.

So with this in mind, I've set myself the following challenge:

Complete research on the following ideas by March 20th

1.       Split the Agg M strategies. A) which are the best trend following indicators? B) Which are the best MR indicators?

2.       Can these be combined better than the current Agg M model. E.g. is there a better MR component than the -percent rank used.

3.       Bollinger band/ vix based system.

4.       Pairs based vix system as outlined here. I wonder if this is trade able using futures or spread betting or whether the fees would destroy and edge.

5.       Update PE/10

6.       Evening US forex system.

7.       UK B/O system. In a recent edition of the magazine I edit, I noted that forex markets tend to be trend following (breakout) biased during the UK morning session (07.00 to 11.00) and mean reverting during the late US session (18.00 to 21.00). I want to see this bias can translate into a robust system.

Agg M Scale in & out (properly this time)

This can't be seen as a clean test of Agg M as I already know the results of the strategy. I used it mainly to build up the reporting in my spreadsheets, but have found it very complex to be able to display seemingly simple things like that most of the shelf packages do for you.

As I was putting some of my performance metrics together, I noticed that the long trades accounted for the bulk of the profits. No surprise there necessarily as markets spend more time going up than they do down (maybe until recently!)

I got my phrasing wrong in a previous post, it should have been staggering long and shorts not scaling in. This bit of research does both - scaling in and staggering entries depending on the score of the Agg M.

To get into the practice, I did an out of sample test of the FTSE 100 from 1994 to 2007 and found the system performed well if you gave it a long side bias (i.e. long when above 0.4, not 0.5). I added in the results from my previous research which went long at >0.7 and short at <0.2 so you entered 1 position at stage 1 and another position/ contract at stage 2. This gave you 2X leverage when above 0.7/ below 0.2 and 1X leverage when above 0.4 and below 0.39.

I then followed up with the test using the data on the FTSE to date.

As I mentioned, this isn't a clean investigation because I already know the results roughly from the recent data. The spreadsheet below is all over the place, but allows you to have two different staggered long/ short entries and to weight these differently (e.g. to weight long trades more if you want)

W = Weight.

NB 2008 data above is only up to the 10th of Jan 2008.

 
Unfortunately I can't say I've started my proper testing yet as I can't get my performance metrics to work easily in excel and there is probably a lot of subconscious hindsight bias in this. Hope this is of interest though.

Notice how the system went messy at time when the market went messy. I wonder if there could be a good filter to apply involving range breakouts or some sort of dochian channel. The system seems to get the trend right when there is one, but struggle when there's no direction. 

To be honest, after all this, I liked the look of the simple 2X leverage if >0.7 or <0.2 rather than scaling in.

Spreadsheet here:

Download Agg M Scale3

Spreadsheet snags and how to fix them.

Thanks for the comments regarding the Agg M stuff.

There are some snags in the spreadsheet. I'm certainly no expert in coding or VBA - For the auto download function I adjusted another spreadsheet I found at http://www.gummy-stuff.org/ - Lot of amazing stuff there. I'm sure someone who actually knows what they are doing with VBA will be able to sort them better than I can. Only really intended to use these myself so didn't anticipate other problems. All good though, the more eyes on this stuff the better.

Snags & fixes

1. 'Today' isn't downloading - The Macro only pulls data from the yahoo finance page for the previous day when its added to this page for example: http://uk.finance.yahoo.com/q/hp?s=^FTSE. Hopefully not too much of an issue though because the signals don't go long 'tonight's' close but tomorrow's close. So by the time the data because available tomorrow you should be able to have a signal for that day's close. 

2. Sometimes line 8 is blank which messes things up. I've put some instructions on the modified spreadsheet below which gets around this.

3. Some pairs don't have enough data on yahoo finance and screw up the data. QQQQ is an example of this. The only way around this is a manual adjustment.

4. Column headings - added these to the adjusted spreadsheets below.

5. Manual version - Below you'll also find a different version of the Agg M (not leveraged, entry at 0.5). This time the data runs so that the most recent date is at the bottom. You can manually entry data or copy and paste from yahoo historical data. The signals should automatically update. Feel free to play around.

Downloads:

Revised auto spreadsheet. Download Agg M Scale

Spreadsheet for manual entry. Download PRSmanual

Agg M Scale in & out

I was playing around with scaling in and out of the Agg M indicator the other day and found some very consistent results. The returns were very smooth when buying if the Agg M was above 0.7 and sold if below 0.2. It does reduce the number of trades so I wondered what it would look like if you applied 2X leverage.

I'd only really consider doing this with a system that has smooth returns as this appears to. 

Works just as well on the FTSE 100 and SPY. The 0.7 and 0.2 scaling is skewed to the long side because markets spend longer trending higher than they do trending lower.

Works very well indeed. 

Spreadsheet here: Download Agg M Scale

Full credit once again must go to DV.

AGG M System/ Indicator

I've been lapping up the excellent content over at CSS Analytics in recent weeks and one system in particular caught my eye.

The aggregate M indicator appears to be based around the work that David has done on the DV indicator. You can read the full post on it here.

I'm getting ahead of myself here with this analysis, but I'm prepping an article so I thought I'd kill two birds with one stone.

I managed to figure out the formulas for the system mentioned and it indeed looks pretty good.

Here's results for the S&P 500 (SPY) and for comparison, Tesco.

It's interesting that it really comes into its own around 2008 which is when Mean Reversion strategies really kicked into gear.

 

 

More on the state of mean reversion strategies here .

I think it need some sort of filter, but its not a bad starter for 10.