Just how reliable is backtesting?

Backtesting is futile. You are now learning that lesson. Forward testing is all that is useful in the plan development process.

 

Backtesting is forward testing for those doing it right.

 

"He who can't dance says the music is bad."

 

Yep, if you've only got 1Min OHLC data, then only build and back test systems which work on the Close of the Bar that way ( open of a new bar maybe ? ), the back test will be identical to a live test.

Simples!

 

You're completely ignoring the topic of curve-fitting of parameters. You only described how to do a very basic back-test properly.

 

Ahhh but all you can do is Curve fit, run 2 data sets, 1 6months and before, 1 last 6months, test on the older 1, then confirm still viable on newer 1.

Just got to hope your curve fit, still has some profit making time left, before you have to re curve fit.

 

Well, curve-fitting implies to me that the strategy doesn't work on any out of sample data.

 

Exactly!
If you start to curve fit you will always run behind the facts. In best case you will find what was the best setup in past. But you need a setup for the future.

 

What you folks are describing with the "curve fitting" is exactly why you have a separate training set and test set. It is not statistically correct to validate your learned parameters (the aforementioned "curve") against the same set you learned them from.

 

Exactly! Re-curve fit daily, if trading daily.