Exactly! I know at least two such vendors who do that. You have to be very careful of programs that do not let you hide the OOS but instead claim to internally do the split to IS and OOS. From what I understand in the Price Action Lab software the in-sample is exclusively used for search and the OOS can be any file the user chooses even on a USB stick. I rarely use OOS tests because I do not think they are realizable and I prefer not wasting data but above all the fact is that machine designed strategies is the present and the future and whoever ignores them is like ignoring the telephone when it was invented and sticking to telegraph.
Through the same logic, I suppose canned algorithms can be used in sports betting, making billionaires out of ordinary people every day. Really?
Precisely because of this risk of wasting data, what I prefer to do is the N-fold cross-validation. Split your data set into N groups, then use N-1 for training, and the remaining 1 for testing, and repeat N-times choosing a different group each time. This will not only give you an OOS estimate of performance, but it will also give you error bars on your OOS estimates. A lot of people somehow object to this procedure because of some irrational worry about mixing of test / train sets which is not happening at all. You are just imagining N parallel universes where you did the single OOS test once. You are saying, what are the chances that I got a typical or atypical result from my single OOS test.
This is a good point. I think selling canned dreams is a scam. But in the case of machine generated strategies you still have to do a lot of work to get a system that works. The work is the edge. I guess some programs that advertise generation of a system with no work needed from the part of the trader are scams. No such program can exist and if it is sold it is either trivial or a scam. Trivial things are like putting a NN together with a few indicators or an evolutionary algorithm with some patterns from a TA book.
I'm not sure where you got the 3-bar thing. PAL finds patterns with up to 6 bars. Most had 4 or 5 bars when I checked. As far as the 4%, this was just an example. Here is the result with 2% target and stop-loss and win rate > 70%:
Here is the out of sample test. Please note that the program did not have access to the out of sample while it was doing the search: The combined profit factor was > 3.12. In this case there are two short signals. What is interesting to me is that I get very high profit factor for some patterns, higher than in the in sample. This is an indication that they have predictive power. I think this program works and I see that from other results I have run. I will be busy for a while. I am not interested in a fight. I just showed you these results because you thought 4% was too much. I think 2% is a reasonable target. Many patterns exit after 2 bars. Some exit after 1 bar. The losers exit after 3 to 4 bars on the average.
I have to admit that this test overcomes both problems of the previous test. An exit after 2 to 3 bars is most likely within the prediction horizon of a pattern, and would also not be much affected by any trend bias. So this suggests that isolated patterns have indeed predictive power. Still, it does not explain why the decision tree tests found no predictive value of isolated price patterns. Maybe I used too short time frames. I'll also be busy in the next weeks, but must eventually look into that in more detail.
You tried a method for a few days and then you say it doesn't work??? You have to work on a method sometimes for years before you can make money on it.
Why asking for people to trust you? This is not an interrogation...Everyone here has his own opinion like everyone here has (at least) one computer... Risk aversion has nothing to do with this. If a program generates complete systems that work then nobody should sell it because with little money, risk averse trading and compounding you can become rich in 5 years without disclosing the edge. If the program needs a lot of work done by the user to get a complete system then you introduce curve-fitting, selection and data-snooping bias, subjectivity, emotions, and a host of other problems. In that case I agree it makes sense to sell it but the chances of it working for most people are slim to none. It must be priced accordingly then. The price of the product you mentioned is not justified for the amount of work one must put and the possibilities. There are much cheaper programs that maybe do a better job.
If you apply this logic to other industries, it sounds ridiculous. Why would anyone teach me to drive, if they can do driving themselves and make more money (less competition). Why would anyone sell graphics program if they could make money using it?