Trading without backtesting first is like jumping off a plane at 10.000 feet and saying "Gee, I wonder, is that a parachute or a backpack on my back? Oh well, I will find out soon, if it works it works..."
I haven't read the book yet (just ordered it today) so I can't comment yet on what the traders in the book were referring to. It is very difficult to backtest a setup that a discretionary trader is looking for. For a systematic trader you need to back test but I don't believe you do need it for a discretionary trader. Instead, a discretionary trader might start with trading 100 shares of a setup and as he feels that its working slowly increase share size. If it doesn't work he moves on. The testing is done in the market...nothing can be more realistic than that.
The probleme with arb trading is that it does not last long for the retail trader, sooner or later "Big Money" will notice what you are doing and will ALWAYS find a way to stop you from eating their lunch. Unless you are one of them, of course.
Of course you also need to backtest a discretionary system! Just open your charting software, go back 10 or 15 years back, and start testing from let's say January 2, 2000, daily chart after daily chart, or whatever your time frame is, making sure you hide the data you are trying to predict. Repeat this process after each buy/sell signal then see if you could have made a profit with no benefit of the hindsight, fictional trade after fictional trade, up to April 2014.
Drownfurw: Thank you for the elaborating the story I myself mentioned so imprecise, I just really don't understand all its technicalities. Though a crazys story. The trader himself had people from his office to manually multiple the trade setup, getting the most money out of it as long as the setup worked. (hed had some carpel tunnel syndrome) Because of that he couldn't keep it as his own secret... (not because of the carpel syndrome, LOL) Though probably some other persons/companies also figured the setup out simultaneously with.
Thats exactly what the traders in the book mostly thought about it. I guess this approach favor my own trading style. And then probably a one easy reason for me to like the book. And then we got subjective about the matter, all the suddenly... Thanks for sharing my interest about the book...