Thanks, how does the r^2 transfer into win rate for you? This is before I'd even think about Sharpe ratio. Mostly looking at eliminating my manual decision making.

On the model Y ~ 1 + Yhat, or Y = beta1 + beta2*Yhat; Y and Yhat normalized to zero mean and unit sigma, R^2 is just beta2^2. Fpr 1:1 RR and my expected time-to-next-forecast, an R^2 of 0.05 is quite high. That translates to an average win rate at 1:1 of about 54%, and if, say, the 10% of forecasts nearest zero are filtered out, the win rate is higher still. This is pretty provable analytically given that the joint distribution Y,Yhat is elliptical (in actuality it breaks down in the tails, you'll never get to, e.g. 90% win rate on a 5% R^2, except by luck, no matter how much you expand the no-trade zone around zero)

Interesting. Do you actually filter out the low strength forecasts or use scale the trades based on the signal (obviously, clipping at some reasonable level)?

Both. Filter out the signals near zero and a constrained scaling on the signals that get past the filter. The scaling may not be obvious as I'm also trying to maintain something close to an equal risk contribution trade basket, so the signals are also effectively scaled by expected relative vol and correlation (done step-wise, first scaled by signal strength then by ERC).