exactly and this was my ques #1 in my first post. So, I guess the right way to calibrate this observer effect would be to do a backtest and look at your fill rate. and compare it with your real fill rate - maybe that will give some idea of how much by your presence you are affecting the order book. Its a game really but other participants play this game every day. What I wonder is how large a warchest one needs to calibrate different aspects of this process and understand one's chances of survival in this game.
The approach I would be making assumptions about how your interaction will affect the order book, then model it.
I would highly advise you not to throw $80K away doing this, the HFTs will eat you for lunch. Read Dark Pools
Gmst, You should put your effort towards something else. Trying to make the bid/ask spreads is a very crowded trade and requires decent capital outlays. Here are some of the problems. Nobody is going to let you write direct to the exchange with a 100k account and no history with them. So you are left using an execution provider with an API. Like TT. Now you have to collocate with wherever that execution provider's gateways are. Which could be in Aurora or possibly not. You are behind 1) people who have built 'nirvana' TT boxes, 2) anybody directly connected 3) everybody directly connected who is trying to be really fast. Basically behind anybody who is actually possibly squeezing out ticks in these low volume markets. And they have low fee structures. The simpler the strategy, the more competitive everything gets. If I were starting over again, I'd do stocks. Not trying to arb anything just catching statistical reversions or running PCA on overlooked, thin etfs or closed end funds. Or kospi. There are lots of relationships that are more difficult to recognize/program that don't require extreme speed or even automation. If you had 100% confidence in your ability to roll everything out yourself and understood everything the exchange is going to require and you had a great strategy that you've seen work at a prior shop and you knew exactly how to reproduce it then 500k might get you by as a minimum. Depending on the exchange and product.
Principal component analysis. Logical statement, but the ramifications appear to be lost on many here. Folks should not be spending much time on strategies that they will not be able to implement successfully due to lack of capital, relationships, knowledge, or the business savvy to acquire the above. Perhaps a better approach would be to start with the basics of what the market is. Everyone desiring or needing to transact needs liquidity in order to do so. Different players, objectives, reasons for trading, risk profile - but all need to get their orders filled at an acceptable average price. Therefore competition for limited liquidity. This competition, especially in periods of lower liquidity, creates consistent opportunities for those who apply a sufficiently accurate and consistent analysis. Profile the various categories of participant, understand how, why, and where they are likely to trade, and identify opportunities to participate in this order flow. Various ways to display the time and sales / market depth information the market gives - identify what information you need to make a call on how others will trade next with high probability, display this info in a format you can use, and test - you have the basics of a strategy. How good you are at this will dictate whether you can compete further away from the market (latency), and without the benefit of low costs. I don't have a clue on the high level maths that garachen uses, nor the technical savvy to be fully automated. This doesn't stop me being able to exploit short term order flow imbalances which accommodate my size and risk profile. Understand where you fit in the food chain, what resources you can access, and design a strategy based on this. None of this is easy, and to get to the level where you can compete successfully when paying retail costs will likely be beyond many or most. The level and quality of work required will ensure that there is very little competition in this type of manual trading. Probably the more sensible approach for the independent trader to follow, if the skills are present or can be developed.
Statistical reversions, PCAs on thin etfs - are you talking in terms of intraday movement? Garachen, many thanks for chipping in with advice. I have about 80% confidence in my ability to roll out everything and understand exchange requirements. I have not seen such a strategy work anywhere before, so no prior experience. And my warchest is considerably lower maybe around 80k at most. So, thanks for providing some thoughtful benchmarks. Appreciate it.