Backtesting and Strategy Validation - How Much is Enough?

Discussion in 'Trading' started by bhause, Aug 13, 2011.

  1. Btw, it wasn't just my eyes that were looking at all the data.

    There were many more. They invested in the idea as well. It wasn't like I just sold some random people a get rich quick scheme. This research looked very promising.

    In the end you come to find out that you should REALLY take into consideration ( 1.5 * tick size ) of slippage to all stop orders. That is IMO a pretty solid starting point. So that would be for example to the equivalent of losing .375 on each ES trade with a stop order.

    Don't forget you probably won't ever get filled on your number for limit orders either in a live environment as opposed to a backtest. You always get filled on the backtest. This is usually of course due to not recreating the entire order book, which is time and money consuming to say the least.

    These are just 2 problems out of the unlimited ones you will encounter. Each can be devastating on their own. To be sure I can promise you they will ruin you over a long enough timeline.
     
    #31     Aug 13, 2011
  2. Buy1Sell2

    Buy1Sell2

    Proper backtesting along with prudent money management in the market absolutely works. Those that say it can't are either failing in analysis or misusing money. simple as that.
     
    #32     Aug 13, 2011
  3. Define proper backtesting.

    Then after you define it tell me what software will let you perform your "proper backtesting."

    It most likely will not be anything you will be able to utilize. Proprietary software ( a small percentage of it btw ) will be the only way.

    The only way you will be able to do this is with a lot of money invested into the project or try to land a job somewhere that already has.

    If you go the first way there is a 95% you will end up like me. If you go the other way good luck finding that job. Let me know when you do. They are few and far between.

    And also if you are only paid for wins and a high watermark ( no salary ) good luck sticking with it through more than two quarters of a year with no income.
     
    #33     Aug 13, 2011
  4. Buy1Sell2

    Buy1Sell2

    Software is not necessary for backtesting. --Other than software which allows different time frame charts to be displayed along with OHLC and a few scant indicators.-- One uses backtesting to learn recognition of previous market action and how it likely affects the charts going forth. In addition, one must develop a stop strategy based upon market support and resistance and learn to never have a risk of more than 2% of TLNW. This can all be done by backtesting previous market action. For, if we don't study what has happened, how do we know how to react in the future.---Never borrow money to trade.
     
    #34     Aug 13, 2011
  5. you > me
     
    #35     Aug 13, 2011
  6. ahahahaahahaaaaahhehehehehohoho

    i wanna take morrrrrre from them....morrrre...please give me morrree of them

    :D :D
     
    #36     Aug 13, 2011
  7. kut2k2

    kut2k2

    You'd be amazed. Supercomputers are no better than their programmers, and those PhD programmers mostly aren't the geniuses you're assuming they are. Most can't think outside the box, in fact they get paid to NOT think outside the box (random walk theory and other garbage). Most are great with post-collection data analysis, they've used it all through school. Few are adept with simultaneous data collection and analysis, which is the heart of successful trading. Anytime I see wavelets and Fourier anything mentioned in connection with trading, I laugh. Those are exactly the wrong tools for trading, but great for optimizing what happened in the past. :eek:
     
    #37     Aug 14, 2011
  8. I`m amazed with Jack''Fourier''Hershey optimizing the future
     
    #38     Aug 14, 2011
  9. So what was wrong with all of your designed algos,Kid?

    Can you summarize it please in a few lines?
     
    #39     Aug 14, 2011
  10. was you response to "I know of 8 major backtesting flaws".

    Yes Sir. I am not going to reveal them now bacause it would take a lot of time. I can start by a historical review. There was a paper posted here once with specific backtesting examples. I mean too specific. It showed how two major popular backtesters skipped valid signals by using trivial entry generation. Do you know how many got screwed because of that? How many thought they found a good system but when the missing signals were added it was actually terrible?

    I can also mention to you as a bonus the issue of data. Depending on the adjustments made to the series in relation to system code, the results may be false.

    Use google

    Kid is correct. You have to do it yourself.
     
    #40     Aug 14, 2011