Bootstrapping provides a means of estimating the Sampling Distribution. Statistical inference has three distributions: Data distribution in the Population (e.g. distribution of all conceivable daily returns that would be earned by the rule's signal “over the immediate practical future”) Data distribution in the Sample of size N (e.g. distribution of daily returns in the backtested sample) The Sampling Distribution, i.e. the Data distribution of the Sample Statistic, the attribute of the sample that is of interest (e.g. average daily return, or average monthly return, or Sharpe Ratio, or Profit Factor, etc.). So, for example, if you take 1000 trades and measure the average monthly return, and then take another 1000 trades and measure the average monthly return again, you likely get a different value for average monthly return. That is, the measured value of average monthly return has its own distribution, distinct from the Population and Sample distributions. Bootstrapping is one method of estimating the Sampling Distribution (of say the average monthly return) by resampling with replacement from an original sample, and under the assumption that the rule being tested has no predictive value (e.g. so that the generated Sampling Distribution of average monthly returns will be centred on zero). http://www.amazon.com/Computer-Intensive-Methods-Testing-Hypotheses-Introduction/dp/0471611360 Once the Sampling Distribution has been generated, hypothesis testing can be done along the lines of: H0: Rule generates an average monthly return <= 0 H1: Rule generates an average monthly return > 0 The observed average monthly return of X% per month corresponds to a p-value of Y if we use the Sampling Distribution generated by Bootstrapping. So we accept/reject H0 at the Z% level, etc.
Abattia, thank! Excellent! One question: what do you mean by “over the immediate practical future”. How long is that for swing trading for example?
“Immediate practical future” is a quote from David Aronson (“Evidence-based technical analysis”). “[The] immediate practical future refers to all possible random realizations of market behaviour over a finite future. It is as if there were an infinite number of parallel universes, where all universes are an exact duplicate except for the random component of the market's behavior.”
What is the difference between infinite time in one universe and finite time in infinite universes? I see none.
“... It would be unreasonable to assume that the dynamics of financial markets are stationary, and so it would be unreasonable to expect that the profitability of a rule will endure in perpetuity. For this reason, the population with respect to TA rules cannot refer to returns occuring over an infinite future....[However..] unless one is willing to assume some persistence of predictive power, all forms of TA are pointless... Therefore, the immediate practical future refers to all possible random realizations of market behavior over a finite future.”
That statement does not make any sense to me. If you throw a coin infinite times it is like throwing the same coin once in infinite universes. I think Aronson confuses stationary with the frequency definition of probability. It does not make any difference whether you will test a rule in infinite time or in infinite universes when you are looking at averages. If you throw a fair coin in infinite time you will get prob of heads = 0.5 and if you throw a coin in infinite universes you will again get prob = 0.5. There is another problem here that parallel universes are about ensembles only and cannot describe non-ergodic processes. They cannot provide a measure of drawdown and probability of ruin because there is no time average.
Hi, thanks for the response. In the case of a coin flip, I believe you are correct that there is no difference between the two views of how the Population might be defined. But the coin flip distribution is stationary; population mean and variance are unchanged over time. In the case of markets though, the usual observation is that they do change ... ALTHOUGH SOMETIMES (OVER SMALL TIME HORIZONS) IT IS NOT TOO MUCH OF A MISTAKE TO MAKE THE ASSUMPTION THAT THEY DO NOT. I believe that is the only point that the author is trying to make; i.e. using the concept of infinite time would lay him open to the criticsm that he had forgotten about nonstationary processes. I think that is all he is saying.
I understand but the concept of infinite universes is also strange or more than that. So to avoid criticism about infinite time he resorts to infinite universes? Sounds peculiar. I think (a) Either he wants to say something we do not understand] (b) or he does not understand what he says Otherwise, what is the practical significance of infinite universes for traders?
Say you want to measue the average height H of a person living in Manhattan (population 1.6+ million?). You can measure the height of 10 (or 1000) people at random, and calculate the average height h of this group of 10. Is h the same as H? No. It is a good estimate of H, but it is not the same as H. If you take another 10 people (or 1000 people) and repeat the measurements and calculations of h (the average) a large number of times, you get more estimates. The more 10-person measurements and calculations of h that you make, the more of a picture you build up of the distribution of h, and so the better you can estimate H. But you can only ever know for sure what H is by measuring the heights of all 1.6+ million people. If a backtest of 1000 trades results in an average trade of d $/trade, that is analagous to measuring h in order to estimate H above. So we measure d $/trade in order to estimate the value of D $/trade that describes the whole Population. But what is this Population in the backtest case? So that the measurement is useful for us as systematic traders, Aronson says the Population is the distribution of all conceivable daily returns that would be earned by the rule's signal “over the immediate practical future”. That is, it is not just all the returns that are actually going to occur in the “immediate practical future”. It is “all conceivable” returns, since the market exhibits a mix of both systematic and random price changes., which in turn each have their own distributions. To describe the realization of all this Aronson says “it is as if there were an infinite number of parallel universes, where all universes are an exact duplicate except for the random component of the market's behavior.” So he is suggesting that this is a way of conceptualizing the concept of Population that is required. But doubtless there are alternative ways. I don't think he is saying that there are not. How would you do it?
Thanks Abattia but I do not think your example related well to trading and to the fact that what we are interested is time-averages, not mean values. This is the reason I am saying that I fail to understand Aronson. I will try to make this clear by noting that what we are interested is the way series of trades affect returns. The mean value obtained from parallel infinite universes is not very relevant to this. You see there is a difference between market returns in the immediate future in parallel universes and the time-average of those returns in the particular universe of interest. Traders fail because the time-average turns negative although I believe the mean from infinite universes is positive. Example: heads win =+1 and tails loses -1 Mean is zero but losses over 4 tosses can be -4.. The infinite universes give the mean and you think this is an even game but the time-average can make you broke if all you have is 4 units to play with.