I am trying to build an infrastructure for detecting relative value opportunities in options. The idea is to make the infrastructure "employer-independent", even though I will most certainly use it while working here. The final goal is to be able to open a separate shop, should I decide to do so, with the least amount of downtime. I will also prefice right away that I am not a "true" IT person - even though I've majored in Math/CS some years ago, I have been trading options for the last 18 years and it certainly did not make me any smarter. Thus, I decided to go with a web service-style approach. The data stream from exchanges, the vol surface fitter and the volatility database would all reside on a private server (I take it putting a server in some reasonable colocation service is dirt cheap these days if you don't need light-speed latency). Also, on the same server would reside my volatility RV code (which I would write in R) and whatever results would be displayed in the form of simple HTML pages or downloadable CSV files. Obviously, a lot of choices have to be made and questions have to be answered. That's where I hope the collective intelligence here would help me: (a) What operating system I go with? Linux sounds like a natural choice but it will somewhat restrict my software choices. It does, however, seem to be most cost effective and scalable. (b) Which database should I use? So far I am leaning toward MySQL - free, scalable, good administration UI. Oracle is the other choice. (c) The volatility fitter/updater - should it be developed in C++, Java or some .NET platform? I can see positives to all of these, but most importantly C++ is the easiest to interconnect with R, has the least number of performance issues (and since we are fitting vol surfaces for several thousands of stocks a few times a day, performance is important). (d) The development process is probably the biggest question, maybe I should have put it in the first place. My choices here are fairly limited: 1. I can probably take a stab at this myself with some bits and pieces outsourced to contractors. It's the cheapest way, but it will take a long time to get anywhere considering that I am working 12-hour days as is. 2. I can hire a pure IT person to work on the non-financial components. The issue, however, would be that pretty much everything will be financial in one or another form. Also, support of the system will be reliant on that person. Also, there are regulatory issues with me hiring someone. 3. I can (probably) find a student that could work on this project in his/her spare time, pay something or other to support him/her and, once the whole fund idea goes live, make him a partner. Any other issues/problems you guys see? Any thoughts or suggestions?
a) What operating system I go with? Linux sounds like a natural choice but it will somewhat restrict my software choices. It does, however, seem to be most cost effective and scalable. >> I think linux would be very well suited to this type of project - at least for the server side aspects. (b) Which database should I use? So far I am leaning toward MySQL - free, scalable, good administration UI. Oracle is the other choice. >>MySql makes sense - has plenty of integration tools for using it as part of a distributed system. i.e. has things like off the shelf spreadsheet connectors. The free version of Oracle only allows the use of one processor/core last time i looked. (c) The volatility fitter/updater - should it be developed in C++, Java or some .NET platform? I can see positives to all of these, but most importantly C++ is the easiest to interconnect with R, has the least number of performance issues (and since we are fitting vol surfaces for several thousands of stocks a few times a day, performance is important). >> Go open source on the analytics, so java or C++. C# is a bit new and not suitable for linux. (d) The development process is probably the biggest question, maybe I should have put it in the first place. My choices here are fairly limited: 1. I can probably take a stab at this myself with some bits and pieces outsourced to contractors. It's the cheapest way, but it will take a long time to get anywhere considering that I am working 12-hour days as is. >> It will hoover up your time and distract you from trading. You have to know 'how to do it' and 'do it '+ trade... Unless you take a sabbatical from trading. 2. I can hire a pure IT person to work on the non-financial components. The issue, however, would be that pretty much everything will be financial in one or another form. Also, support of the system will be reliant on that person. Also, there are regulatory issues with me hiring someone. >>The pure IT person will still need to have a pretty good grasp of what the thrust of it is - remember, you've been doing this for 18 years so it will piss you off when it takes 6 months of explaining stuff to point where you're singing off the same songsheet. Plus, you're doing options (tricky) AND relative value of options which could be mind boggling to an outsider. 3. I can (probably) find a student that could work on this project in his/her spare time, pay something or other to support him/her and, once the whole fund idea goes live, make him a partner. >>don't know.
you dont need to worry about most of this. Use the cloud... http://www.heroku.com/ https://www.picloud.com/ https://mongolab.com/ https://www.alwaysdata.com/ http://cloudnumbers.com/ :eek:
SQL server developer edition is like 40 dollars and is fully functional. Buy that as pretty much any vendor you use will support it.
What is the advantage of using SQL server over MySQL, for example? If anything, I would be stuck with MS windows that is not an ideal server platform. PS. If I were to go with anything commercial and needed scaling, I'd go with KDB for my HFish data and Oracle for my vol data. For the stuff I am doing, MySQL should be plenty, however.
If you are looking to be employer independent then I figure you will have to set up for the eventuality that you will be sourcing data from some third party vendor. Pretty much all the options data vendor supports SQLServer for their table schema as well for automatic scripts for getting daily updates and data corrections. If you use anything other than SQLServer then you will be responsible for all this yourself and will be adding unnecessary IT work. Also, these databases aren't that big: for EOD data, they generally are less than 1TB and can easily run on a Windows box.
That's a valid point. On the other hand, I have never seen anyone use vendors data directly out of the box, usually people fit some sort of parametric models into it etc. At the very least, how would I know if the vendor uses the same IV calculation as I do (e.g. same way of looking at time, at forwards etc)? Yes, I agree - even with the variety of supporting data tables, I can't see it take too much space. It's the high frequency data that really needs a lot of Ts.
You don't and that's annoying. But in any case, it's easier to manipulate the data in the same ecosystem.
True. My general experience has been that you want to get raw option prices from a vendor and do you own post-processing. I, obviously, might be wrong since every place I've seen was either a large desk or a large fund.
When i was at a large bank i had the same opinion. Who wouldn't calculate their own historic ivols? But on your own, getting historic dividends, borrow and rates is annoying. It's unlikely that you will have a substantially better implied vol model than industry standard black schole models. At least that's my impression, but I could be wrong. I don't know how sophisticated vol funds run their businesses. If your calculations are consistent (using the same model throughout) then I guess it's okay and a lot of data providers will give your their pricing models along with their data.