While not specifically regarding Kelly: I think you get the best, i.e. most accurate or "honest", PF and general system test if you test without compounding. I always test two base scenarios all the time: fixed position size (e.g. 100 shares, 1 contact), and volatility scaled position size. I don't like compounding as it very much can be dependent on start date of your simulation.