if you weren't successsful at trading before and u wisely stopped,visit www.traderfeed.blogspot.com and read his posts on psychology of trading,i could tell u where the mrket is going 3-4 times a day on most days and if you haven't gottten the knack down yet ,you wouldn't make much off the info,conversely i have a trader friend who i've known for 20 years and he'll call me from the golf course,get the numbers, put on a trade, keep golfing and make money on the trade. The point is ,even with a system it would only be useful if you were a good trader.
I think 1500 trades is fine, or even 300. But 3 weeks of data isn't that much. Trends change, for example that 3 weeks could be down and you start trading and then the trend changes and goes up and the system doesn't work any more. I wonder why they limit it to 3 weeks? In the video they use 2 weeks and it was very quick, just a few seconds. I imagine they limit it to 3 weeks and offer something larger to bigger clients or keep it for themselves.
Ron, This is very interesting indeed, but only problematic for big players on no so liquid markets. This, as slippage issues, can be improved in tests by multiplying costs transaction by 3.(simple trick, works good!) (to answer the initial post: 300 trades aren't enough at all. A couple of thousands is the lower limit. Overfitting is a tough issue: you need some to stick to your market, but too much of it will make your strategy unstable. This really is the tricky part) thx. Alex
Backtesting seems like having a girlfriend and then getting married. The results are not gonna be the same after the honeymoon. You can bet the farm on that one. Systems traders and those that seek to automate the trading game seem to have a common goal and that goal is to eliminate their emotions from the game. Another goal seems to be a false and very weak attempt to enforce discipline. Like ammo said, UNTIL a trader can learn how to trade in the first place there is nothing to automate. Being an engineer as the original poster said was the job in the past says a lot about why he lost. Trading is not like other jobs, thats why 90% lose. Trading is about "UNLEARNING" everything you know and "UNDOING" your natural instincts of "FIGHT or FLIGHT" Ok, the ending of the Le Mans race is about another hour. Great race, fantastic. 24 hours , NO WUSSIE machines need apply. Just started raining again, thats a game changer for last hour. Remember this...........if you do not know how to trade, a system is silly. The only backtesting that makes any sense at all is to correct your mistakes as you learn. We all make mistakes. deal with them.
The price data you are using in your backtests, if not accurate, could mislead you to think you have something profitable, when in fact it only appears to be profitable because your price data is not accurate.
I think this would be a problem if the price data is less accurate against you. It could easily be more accurate in your favor. I've noticed data discrepancies before when comparing charts and fills so it's always an issue.
This is easy to say, but inquiring minds would like to know why? So far we have: #1- invalid / erroneous data #2 - difficulty getting filled #3 - market conditions change For a hypothetical example, let's say we identify a trigger that will produce a big gain. At the time of the trigger, the price is stable and also stable at the time of exit (no big runup). That would remove #2. Let's say this system is tested for 5 years (thousands of trades). Is that enough to rule out #3? For #1, I don't think we can do anything about it other than to choose testing platforms with reliable data. Any variances should go against as much as in favor..
For now, I have come to the conclusion after looking at just the same questions you have the back testing can give you a idea if you may have something, that's all. The problems with back testing as I see it are 1. No fills and slippage. 2. Have such a large sample size that doesn't reflect the current volatility. 3. As related to 2 can you really withstand the max draw down the system shows? Most can't in real life I believe. 4. Probabilities are critical to successful trading, but unlike blackjack let's say game theory doesn't IMO work as well. There is nothing close to a fixed sample size in trading, it's what you make it, which is different from some card games where you can count. Pros 1. Potentially allows you to remove some of the emotion of trading and can create a more disciplined approach. IMO if you are going to back test do it manually make sure it's true to the current volatility and realize it's just a starting point nothing more. Hope this helps. Look forward to seeing some productive feedback from others since this is a important subject. Post edit: Thinking more about volatility and some of the posts. Let say those 1500 trades occurred over the last few weeks. Well the VIX went from 18 ish to 23 recently. How is back test going to successfully handle such a big swing. I don't think it can.
All of the data in the OM is for 15 trading days. So all you have to do is cut and paste the results into a spreadsheet to keep tabs on the system for longer. I have strategies that have run well for over 4 months. Here is an example of one I trade. I am not going to show any details of course but this one is very very powerful system I have running in full auto. There is another piece of the puzzle which people that don't have it can't see. When you buy this they turn on a thing called the trade report which shows all the details of all the trades. I export that to a tool I use called stocktickr. Here is what stuff like that looks like if you want to get into the nitty gritty. But basically with a spreadsheet you can easily keep tabs on the system for as long as you like. https://www.stocktickr.com/report/oddsmaker/?resultbd20850a
Anyone saying you can test against 2 weeks of data, pick the top performer, and expect that behavior to continue is being dishonest or has never attempted it. (Alright there is a small chance you MIGHT discover a "magic system" this way, but don't count on it.) Getting backtest results to continue performance is almost always MUCH harder than that. But of course the software companies don't want you to know that because it would affect their software sales. If peolple realize mechanical trading might be just as hard or harder than manual trading they might not try. I can take a simple sma crossover system, test it on all symbols on the nasdaq, from sma 9 to 50, for past few months, and get many symbols that produce unbelievable profits, 1000s of percent return per year. But I can guarantee you those symbols will not continue to produce that profit, or will produce a loss. The same is true for most systems. Optimizing against a longer time frame such as 2 years should in theiry produce more reliable results that continue, however the results will be lower. Why lower? By increasing time period you are putting more constraints and requirements on the system. Without getting into a detailed explanation, this simply makes the top performing optimization more "conservative" and thus produces lower profit. I once optimized a ES system for a client against past 3 months, found result that were amazing. Informed client that behavior could not necessarily be depending on to continue long. Tested against 2 years, found top performer, but it was much less - .5 points average a day. cunparis I am actually going to make a YouTube video about this soon I'll send you a pm so you can watch it.