you might want to check for bias in the positions. You are simply using a random length window to sample the data. If the data has positive bias (which it should for your type of data), and you are taking both long and short slices of the same data population, the long bias will cause the net expectation to be negative since you are taking short trades in roughly equal proportion. What happens if you run the same experiment, with the same underlying data with long side only?
You might check to see if the problem is associated with your input data or in the program by using synthetic input data - say a table of values, each price $ 10. The price values are all the same. If your program still shows a loss then there is something in your code that calculates the loss.
That's an interesting thought. The reason why I chose to threw randomly short or long orders at the market is that usually the price action itself is biased, at least over a reasonable period of time. So, if my run contains only long trades and the market trends up during that time, I would expect slightly better results than vice versa. Having said that, I will pick up the idea someone posted earlier and perform two parallel runs, one with longs and one with shorts. I am going to do that over the weekend with market replay data and we'll see where that goes.
Right, I should have also clarified... the longer the length of the sample window, the more likely the bias will be up. If you are somehow sampling from a smaller time frame it should reflect the bias of the prevailing trend in that window. If you have access to statistical tools (R is free for example), you can simply run a gamut of tools to analyze your data; which might be more informative than running the empirical backtest trials.
I said earlier it is about 250 trades in one continuous run. I think it is very difficult (impossible?) to prove if a significantly higher number of trades - like the 10,000 you suggested - would bring out significantly different results. By the way - I made about a total of 8 runs (about 2,000 trades), but with different parameters, like different futures, different trading times and different maximum times of trade durations. They were all negative, with no recognizable pattern as to which parameters would give better results. But, as you said, the total number of trades might be too small to draw conclusions from that.
Always change the seed value each time a random number is needed, or always change the seed value each time the program is run (once a day)? The latter is preferred. Either way, check your sequence and verify that it is indeed random. I agree with intradaybill, you'd need many days of trading to verify that this is indeed profitable. Test during bull market, bear market, and volatile/nontrending market. From March 9, 2009 - April 26, 2010 (bull market) see if you beat the indexes.
If the exercise is NOT ABOUT MAKING MONEY, then post only the BID PRICE for the entry and the exit, so you can eliminate the spread from the results. I think the exercise is a good place to start a strategy. The next thing you need to do is slowly start adding one new rule at a time (maybe one a week) to see if you can eliminate some of the losing trades, but keep most of the profitable trades. I have done this method of strategy development and it works. For instance, I discovered that most of the Long trades that I entered while the DOW index was negative, turned out to be losing trades. So I added a rule that required the Current Days Dow Move > 10, and so on... I assume that is how everyone does it. Ed
I like your idea of eliminating the bid/ask spread. Now the outcome has to be close to zero, provided the sample is big enough. We'll see.
It won't be a wash because if the market is predominently down during the test period you'll get stopped out too many times.
Finally, I did what edbar suggested: Instead of actually trading the randomly generated entry and exit signals, I registered the bid prices and calculated a theoretical profit/loss based on the bid prices of each entry/exit pair. After a few hundred "trades", the results were close to zero, regardless if the market tended up or down or sideways. In my opinion, this proved what several respondents already suggested: That the reason for the consistent losses with random trades is indeed the bid/ask spread.