Dont know the correct term for this but by info preserved i mean if at in sample period avg growth angle is some figure and if out of sample has half or less the in sample growth then it lost lost 50-60% info or more. Not sure why anyone would assume an average value for one timeseries have to correspond to another average value for another timeseries. Since price is non-stationary and always changing, this is what you would expect. Also, the information is in the raw source-data itself, not in your aggregates, so no information is really lost? It's just that the data is changing and we as traders have to adapt to changes in markets. Is there some articles / link that references this concept? Since price is non-stationary, it means it's hard to know what periods to use or what to use as base reference. MTF may help, but doesn't eliminate the problem entirely, and also creates a new arbitrary relationship between measurements. Yes and by having more trades shorter term id hope to reduce drawdown, risk etc. This is also an assumption, but in trading you need big winners to pay for small losers, so there are limits to reducing drawdown and losses, especially since the movements are hard to predict correctly. Do your system account for how to have big winners or many medium and stable winners? Cost is not big problem with forex but with stocks it can be .Curreltly on eur/usd i get only 1-3 trades per month and with long testing periods (15years) drawdown periods can reach over 2-3 month even on in sample data. Yes, cost is killer for stock-trading unless one gamble or start out-of-box thinking. I agree this kind of long-term is difficult to lock money in and wait. Which is why I started on a new system. So more trades will be the result of that, but not my aim. Have bridges to terminal programs implemented more than year ago but have not traded live for long yet. If using bridges can test validity of strategy tester vs metatrader , ninjatrader, but never used mt4 for learning parameters because it single threaded and slow, custom implementation is running much faster. Thats the way i found the problem of algo already using last unifinished bar also. So far solution has been to avoid last bar data as it takes away lot of problem. But it also reduce test results and creates lag If your system is any good, it won't depend on a couple of bars (1-30), but at least 600-6000 bars. The less bars, the less data, means higher uncertainty in calculations. So skipping last bar can be fixed later, and shouldn't be a showstopper.
For measuring it i have code that divides the results generated from the in sample and out of sample periods to 2 , next gets 2 avg values for both periods and draws avg lines , the angle of differences between in sample and out of sample period is what gets to % of info preserved or commonly found by algo between in and out of sample data. Maybe there is some other term for this or it uncommon method, but i find it as useful figure to get algos adoption info without generating and opening balance chart. It is nothing more than ideas and assumptions. No classification algo but can generate sortable table to manually see where is biggest loss , profit or order size ,enty, exit etc common info. It could account if enable limiting min and max % loss per trade if thats what you mean. I have previously tested with generated last bar that continues the avg direction of previous bars but i ended up just disabling it. For momentum algos if working with little data it changed alot.
For measuring it i have code that divides the results generated from the in sample and out of sample periods to 2 , next gets 2 avg values for both periods and draws avg lines , the angle of differences between in sample and out of sample period is what gets to % of info preserved or commonly found by algo between in and out of sample data. Maybe there is some other term for this or it uncommon method, but i find it as useful figure to get algos adoption info without generating and opening balance chart. I don't really know what is common or not, and it might make sense to you. However, I'd think what period is in-sample and out-of-sample is arbitrary, but I guess you could use this as a predictor or confirmation if they match? It is nothing more than ideas and assumptions. No classification algo but can generate sortable table to manually see where is biggest loss , profit or order size ,enty, exit etc common info. It could account if enable limiting min and max % loss per trade if thats what you mean. Good with ideas and I think it's the only viable way and everyone need to find their own way. I have previously tested with generated last bar that continues the avg direction of previous bars but i ended up just disabling it. For momentum algos if working with little data it changed alot. Yes, this makes sense then compared to running against very few bars.
Only using it as figure that describes data on chart , if use it with ML ago it could create feedback loop and render all "out of sample" results useless in future. It is same as if you have printed paper with balance data , then visually seperate in sample and out of sample periods and draw 2 linear avg lines over both periods.Out of sample will probably be 1/2 or less.
Only using it as figure that describes data on chart , if use it with ML ago it could create feedback loop and render all "out of sample" results useless in future. If used in wetware, isn't this creating feedback loop also? It is same as if you have printed paper with balance data , then visually seperate in sample and out of sample periods and draw 2 linear avg lines over both periods.Out of sample will probably be 1/2 or less. This has less with in-sample and out-of-sample, and more with how data has changed in time I would wager. It's not like the market knows when you're sampling in/out. However, what if you try to detect the opposite?
Yes , to give watware live feedback about optimization progress. What do you mean by detecting opposite, Can you describe the idea more?
Just a guess because I don't know your specifics, but you say out of sample will be 1/2 or less, but I'm not sure why. If you use a higher period for "out of sample", the average will converge to less extreme values by mathematics alone. So, in what situation and/or period will the average become higher than "in sample"? I think higher price deltas may be more interesting than lower price deltas. Just play around with it. It's all sort of a puzzle anyways
It good point to have less settings, but to me it seems just luck on algo if it work decent with both past & future data , maybe there is some clever optimization methods also that help but idk any that make unlucky algo optimize well for future using past data. What ways you using instead of ML?
I'm curious about ML, but for my current usage it seems too complicated. However, it might be interesting to use in the future, when I'm ready for it. Am more into creating a backtester and seeing how simple rules works on past data. This may sound trivial, but you can also test on data you haven't made the algo from, ie. other time periods, other instruments and markets. If something works generally, there's a bigger chance it's not just luck, maybe So if interested, maybe search for backtesting on these forums? It's been mentioned, but not very easy to find info on it since it's a personal path of discovery that won't make much sense to anyone else.