Optimization vs Robustness

Hi
Can you please help me define the relationship between these two concepts. Is robust system not in need of optimization?
Thank you,
Dima

 

A lot has been written about this, can keep you busy for a long time. In my experience, when small incremental changes of a parameter value have a big effect on the results a system is not robust. Also when a system tests profitably on only a small range of parameter values it is not robust.

 

hausse has nailed it for you...

let's say you optimize and find that at period parameter of 20 is optimal. You then test period = 19 and it loses money. System is not robust and is in fact bullshit.

Robustness across not just all parameters but also time frames, time periods and instruments is essential to avoid the fantasy land of fooled-by-randomness.

 

Great answers.

To give an example, suppose you run several simulations of moving average combinations, and you plot some performance metric (like expected profit) vs. simulated combinations;
if it performs well for one fine tuned combination (parameter, say 50/100 cross), but much lower on surrounding combos, then it is not robust. If you tried to run it in practice, it is unlikely that it would perform as ideal as the best parameter you simulated.

Ideally, you would want a flat response over as many parameter simulations as possible. The idea is to make it robust or resilient towards different input conditions, since you are only testing a small universe of the possible data your system will encounter.

Generally, optimizing is finding the best possible parameter (say you found 50/100 was best in example above).
You however, also want it to be robust under different conditions, so a trade off has to be made between optimization over a finite set of input data and robustness of performance
over a much larger universe of unseen data.

 

a robust model is not sensitive to the assumptions you are making about the market/environment you are trying to model.

Ask yourself the question: "If all my assumptions are wrong, hows the model going to hold up?"

If your model has many parameters, which need to be tuned through optimisation, then generally it wont be robust. This is especially true for environments with high levels of uncertainty ( eg financial markets), but not necessarily true for physical systems with well defined & consistent relationships( ie not the markets)

 

thanks..that is very concise definition of robustness....but what about optimization....or is the optimization REQUIRED to see if a system is ROBUST?

 

time-frames and instruments....instruments from the same type of market - eg currencies or all types of markets (eg bonds, coca, eurusd)?

 

dtrader98,
you say surrounding combinations...is there any way to define this more precisely? percentage of the total number of combinations? is there any book on this? Thomas Strigman's comes to mind...

 

what you mean by flat response?

 

hi and thanks....what is the maximum acceptable number of parameters in a system...as i understand it - the less the better?