How to quantify trading system edge?

http://elitetrader.com/vb/showthread.php?s=&threadid=17082&perpage=6&pagenumber=1

acrary   Registered: Apr 2002 Posts: 700

05-05-03 04:54 PM Here's the forerunner to my edge measurement method (which I've described many times). 1). Take all of the long trades and add up the total profits plus commissions and slippage for a year. 2). Pull out the same number of trades with the same holding period randomly from the year. Add the total profits made on the random trades. 3). Do the random test 1000 times. 4). Sort the random tests by total profits from lowest to highest. 5). Locate where your total profits were in relation to the 1000 tests. This is the ranking. The closer to the middle e.g. 500, the less the edge. Ideally the test will be above 700 or upper 30%. 6). Do the same test using short side trades. 7). Do the same test over multiple years. You'll get a statistic that shows how well the system is doing independent of whether the overall market was up or down. Here's a example for the daytrading continuation system that I posted in another thread. I didn't finish the system, so it only has trades for the long side. First here's the summary report from 1997 - present:

acrary   Registered: Apr 2002 Posts: 700

05-05-03 04:58 PM Next the annual summary report. This is for the SP market, so to find the number of points divide the $ profit by the multiplier (250). 2002 +17,125/250 = 68.5 pts. 2001 +34,125/250 = 136.5 pts. 2000 +35,225/250 = 140.9 pts. etc. Also note the number of trades per-year. 2002 24 2001 24 2000 19 etc.

acrary   Registered: Apr 2002 Posts: 700

05-05-03 05:02 PM Next process the numbers in a simple basic program. The results show the long trades ranked: 2002 90.5 or beat 905 of the random tests. 2001 98.1 2000 95.3 1999 70.3 1998 78.1 1997 90.1 From this little test, I would say this system has a tradeable edge. Attachment: ran.gif This has been downloaded 256 time(s).

acrary   Registered: Apr 2002 Posts: 700

05-05-03 05:03 PM Here's the basic code in case anyone wants to convert it to their platform of choice. Attachment: ran.txt This has been downloaded 345 time(s).

acrary   Registered: Apr 2002 Posts: 700

05-05-03 05:13 PM A friend of mine (with a phd in stats), told me I should've done a chi square goodness of fit test with a significance level of .01. The only problem I had was that it only told me if I had a edge...not the degree. Given that I now use 5,000 samples in the test and combined it with some Monte Carlo testing, I find it to have real value. This can be combined with other tests to measure the mean and std. dev. to provide a confidence level, but since the number of samples is very small...it will probably only go out to the 95% confidence level that the mean is positive. Anyway, that's my .02