Hi everyone, I have been learning how to do automated trading for a few months. Still doing paper trading for testing. I will be doing scalping, day trading, closing position at the end of the day. My current focus is Emini S&P 500 future, may add other future later. Here is my setup: I am using Interactive Broker (IB), I am using R for backtesting specific/customised strategy, otherwise I am using Amibroker for backtesting to quickly check if an algorithm is profitable. My current data feed is IB history and order book, collecting this using IB Java API running 24x7 on my Ubuntu box (will change to IB Controler running on AWS later) I am using Java/Scala to derive signals from time series data, based on my IB datafeed above. In summary, Purpose: scalping Emini future Datafeed: IB Data Collection/Data Transformation: Java/Scala Backtesting: R, Amibroker Strategy Discovery (automatically discover profitable strategy) : R, Python (using machine learning) My background: I work as a developer and a machine learning researcher. What is your setup? Any better data feed? my budget is about $100/month. Any recommendation to improve my setup? I am really new in this automated trading field. My personal blog is http://Trading.Computer Thank you. Trading.Computer
I think you are duplicating code that is already available in open source. Look at www.quantopian.com, it uses IB for trades, has 100s of example systems, trades minute data, will have futures data within months, open source downloadable, and on and on. The also have a lecture series on developing algos, and a contest. If you algo is really good they plan to raise money for it in 2016. Good luck,
Yes, it will be part of my setup work. I have been using the API to collect data-feed from IB, now working on transforming raw time series data in higher level features. After having all the features ready, then doing automatic strategy discovery (in about 2 months) Not sure yet, I will post the backtest result later. Thank you, I will check it, if they are opensource, then I can borrow the code, integrate them into my setup. If I can, I will always reuse and integrate, rather than reinvent the wheel. E.g. like IBController, someone has written it, I will just use it. Similar to R wrapper of IB, called IBroker.
It looks very complicated (multiple languages is asking for trouble) My setup is IB for execution and data, and my own python package for both implementation and back testing. I spend 10 bucks a month on data. More information on my blog qoppac.blogspot.co.uk GAT
I agree with globalarbtrader, several technologies makes it more complicated. I'm guessing that you want to automate the following parts: - Entry signal trigger - Exit signal trigger - Money and risk management - Order management - House keeping stuff (data validation and clean-up, data backup, etc.) Which will give you a 100% automated system. For a start you want at least have the first 3 points implemented. Then you can implement the rest later once, if of course you want to continue (read you are making money).
Globalarbtrader, thanks, I have read your blog, very informative. Rb7, agree with your thoughts, updated my design below. About multiple languages, agree, it is getting complicated. I thinking to write all automated modules on Java/Scala and the interactive/manual modules can be on any tools (e.g.Amibroker) or any languages such as (R/Python/Java) The automated modules: Modules for data collection, near real time in-memory data processing integrated with execution will be on Java/Scala using IB API (entry/exit signal, order management, money & risk management) Modules for data transformation and feature construction on time series data (L1 market data and L2 order book) will be also in Java/Scala. (including all house keeping) The interactive modules: Modules for human guided 'trading-strategy' discovery will be using R, it is a semi-automatic search for trading-strategies with some human defined constraints/parameters to limit the search space. Modules for interactive strategy exploration, I will use anything, such as Matlab, Amibroker, Machine Learning Libraries on {R, Python, Java} on single machine, e.g. Linux Ubuntu i7 5930K/64GB RAM or Hadoop/Spark on multi AWS nodes if required. That is, the automated part in Java/Scala on IB, so using single technology. The interactive exploration part is on any languages/tools on IB+additional datafeed if required. I am currently working on data transformation modules on Java/Scala. It will take about one month to finish the data transformation part, working 4 hours per night after finishing my day job. Any thoughts? Thanks, Trading.Computer
Maybe you've lost before the game has even started? If that is true, wouldn't you like to know as soon and as cheap as possible? Invest your 4 hours/day in something more productive. Trading is not a growth industry. You're unlikely to compete in that product with a short term automated strategy and certainly not with an infrastructure budget of $100/month. To give you some perspective, there is a guy here with clearing costs around a tenth of what you'll pay at IB, IOM seats, and who writes direct to the exchange. He said he had no automated strategies running on the S&P futures. It seems you've made the common error of starting with what you know about (programming), instead of learning about the environment you want to enter. There probably isn't much point going into why this is a poor choice of contract / trade frequency and also data feed, as 1) the information is already available on a search 2) if you need help getting past the easy bits, you aren't going to crack the hard bits Perhaps look at other ways into the industry if you have a keen interest...it attracts a lot of bright people who like solving problems...but also those who lack the honest self-evaluation necessary to recognise whether they have a realistic chance of competing effectively...like moths to a flame and it is sad to see when their efforts could be so much better employed elsewhere. Then I cannot fathom why you've missed his excellent points about transaction costs and have opted for a high frequency / daytrading strategy. Will just re-state my point that if you cannot make the best of what is already available for free, the simple stuff, you haven't a prayer when you run up against something difficult. I've recently learned that I have a compulsion to try to rescue. To save others from the adverse consequences of their decisions. I've acted this out on ET trying to discourage newbies who obviously weren't going to make it, as well as in my personal life. I've earned a little more awareness of this tendency but I've had a little bit of a slip up today. Cheers.
Hey TC, Your approach seems fine. Better than most. I use IB as my broker and IQFeed for my datafeed. You only need to update to IQFeed if you need it's features. I think AmiBroker is a good choice. You can do a lot for not much money. I have recently started fooling around with NinjaTrader since you can program it in C# (MultiCharts.net would be fine also). Programming in AmiBroker sucks (just my opinion) but its power otherwise is great. I use Java for automating intra-day systems, so obviously I think that's a good choice also. R and Python are very popular for research though I haven't used them. Using the tools you have chosen will keep your costs low while you find systems that you like. Don't let anyone tell you that you're doing it wrong. The fact that you are already a programmer, understand machine learning, statistics pre-processing data, and looking at AmiBroker, R, and Python for research, puts you way above most people to start. I know people are always saying "keep it simple", but it's not, in my experience. Systems can be very complex. Your mileage may vary. I assume you have a fair amount of trading experience. If not, get a subscription to S&C mag (you can download all of the past issues), start reading to get ideas of methods you think are interesting. Spend time observing the behavior of your ideas and paper trade manually. This observation time takes a different mind set than programming but is necessary IMO. Have fun and don't lose money.
Yes, that is one of the intentions of this thread That is my day job I think the honest self-evaluation in this field is clear, whether my algo is profitable or not I will get the my customised automated trading ready, then I will find a niche spot, certainly not in nano seconds, not even mili seconds, probably 30 seconds, or more...., if not S&P futures, may be something else. All of these will be just parameters in my automated framework Which features you need from IQFeed? The devil is in the details, be able to program by ourselves give us an edge Thanks, I will subscribe to S&C I will back test, paper trading test, forward test with small amount of money, then increase if the algos can scale up. Thanks to both, Ghost_of_Blotto & dholliday Trading.Computer