There is no viable strategy without the concept behind it. What is your concept? And you eat nobody's lunch since you buy from one trader, when you open position, and sell to another one, when you close it.
If a market is strong enough to set an ATH, then it's a strong market and for a breakout trader, that can be an entry signal. There are any number of ways to set stops and manage the BO trade. As a pullback trader, I would wait for the pb post-breakout. Sometimes, I won't get an entry and that's OK as I get plenty of signals. Often, my entry will be higher than the breakout entry, and that's also OK.
I use R and not to sure how I would test this. It would take hundreds of lines of code using quantmod and performance analytics. Any other alternatives you recommend?
I see you mention "markets" however do you find this is also true for companies? The main reason to support buying breakouts is because of human psycology and the asymetric flow of information. However it would seem that buying a ATH would be coming late to the party.
I think it is the illiquid stocks that will perform the best with this strategy. For the reason of slower information flow.
'the party" is a relative variable. A trader wants a piece of a move, some larger than others but getting all of a move is a rare thing. Breaking an ATH can well signal the start of a new "party".