What kind of R^2 (or other metrics) do you see for your ML algos?

Discussion in 'Automated Trading' started by nooby_mcnoob, Mar 31, 2019.

  1. Asking for a friend.

    Bonus: does the R^2 transfer well into win rate?
  2. Sharpe ratio is the most important metric in my experience.
    fordewind likes this.
  3. R1234


    Not ML but i aim for greater than 0.9. The more reversion systems the higher the r^2
  4. Thanks, how does the r^2 transfer into win rate for you?

    This is before I'd even think about Sharpe ratio. Mostly looking at eliminating my manual decision making.
  5. Adjusted R^2 generally between 0.03 and 0.06.

    Translates fairly well to win rate basis 1:1 RR.
  6. R1234


    Not super high as one might assume given that r^2. It varies by system with a range of 50% to 65%.
    nooby_mcnoob likes this.
  7. That seems super low. What definition are you using?
    stevenk85 likes this.
  8. On the model Y ~ 1 + Yhat, or Y = beta1 + beta2*Yhat; Y and Yhat normalized to zero mean and unit sigma, R^2 is just beta2^2.

    Fpr 1:1 RR and my expected time-to-next-forecast, an R^2 of 0.05 is quite high. That translates to an average win rate at 1:1 of about 54%, and if, say, the 10% of forecasts nearest zero are filtered out, the win rate is higher still. This is pretty provable analytically given that the joint distribution Y,Yhat is elliptical (in actuality it breaks down in the tails, you'll never get to, e.g. 90% win rate on a 5% R^2, except by luck, no matter how much you expand the no-trade zone around zero)
    destriero, djames, sle and 1 other person like this.
  9. sle


    Interesting. Do you actually filter out the low strength forecasts or use scale the trades based on the signal (obviously, clipping at some reasonable level)?
  10. Both. Filter out the signals near zero and a constrained scaling on the signals that get past the filter. The scaling may not be obvious as I'm also trying to maintain something close to an equal risk contribution trade basket, so the signals are also effectively scaled by expected relative vol and correlation (done step-wise, first scaled by signal strength then by ERC).
    #10     Apr 1, 2019
    jtrader33 likes this.