Automated Trading for a Living

Discussion in 'Automated Trading' started by Allistah, Apr 30, 2013.

  1. gmst

    gmst

    Thanks net for the detailed answer. Above is excellent performance :)
     
    #51     May 4, 2013
  2. Tell me - I am QUITE happy. We do start having issues though when the grid is running and I do queries for analysis. No way to do that at the same time - the discs overload.

    Well, the plan is to get a separate new server for this anyway soon (as in: June) and then we get a 72 disc capable case and start (a) putting in some SSD that our RAID controller will use as transparent cache automatically and (b) I have enough slots that I can move the core sim data over to another set of discs in a separate filegroup.

    The later will mean that when OLAP overloads the disc subsystem it will NOT influence - at all - the transational part ;)
     
    #52     May 4, 2013
  3. gmst

    gmst

    Net slightly tangential question but what is your academic + work experience background? Ninja/MC is completely retail - then how did you move so fast from ninja days to everything custom-designed. The way you are doing things is like big institutions will do. Its pretty impressive even if you are working on this with 2 partners. Who put in the money to get all this infrastructure in place? Just wondering. Answer only if its not too personal :)
     
    #53     May 4, 2013
  4. 20 years IT consulting, architecture, specialized on large systems / large databases. The money - you have any idea how well people on the top are earning? ;) Especially when they choose their residence well - in the beginning I was paying a lower than average wage to my people and gave them a chance to work on something big (which is rare here) and it came out costing me 2 days worth of my work per month. Was lucky to strike a nice long term risk management project at the same time - which kept me employed nonstop for 15 months. I do IT for quite some time - and had a little luck in my life - so the financing was never a big deal for me. PLUS - hardware REALLY came down.

    At the end you have to see it as a business - and I did start investing into it. Making sure I stay - as an IT service operation - always in the plus ;) Worked like a charm. I can deduct all costs from taxes and seriously, it really got me a job or two that were well paid ;)
     
    #54     May 4, 2013
  5. gmst

    gmst

    Thanks well thought out very informative. I especially like the part - "and it came out costing me 2 days worth of my work per month".

    Discussion in this thread and the other thread has been motivating for me to say the least. I can program in vba/C/MATLAB and work in excel/basics of oracle. Other than that, my computing skills are seriously below-par. Never studied computer science in university. To compete successfully and succeed in today's high-tech "systematic" trading world, to be able to design systems "properly", I am thinking getting an M.S. in computer science from a top place will be useful. I just might apply for an MS but I guess deadline for this fall might be already over.
     
    #55     May 4, 2013
  6. ofthomas

    ofthomas

    no, he is not... in concept yes, but in actuality he is far from it... you should not compare based on what you assume actual banks do...

    he has re-invented a lot of things and if you understood grids you would see the stress points that will be ahead... the nodes checking for work vs. being scheduled the work and lacking automatic job restart... as you scale it becomes an issue... at least he is in the right path with booting off iscsi (am sure he will figure out eventually being able to provision nodes and rebuild nodes in less than 5 minutes :) .... an institution will just go out and create a CBA for symphony and deploy the grid foundation based on that... not create it from scratch...

    dont take me wrong, he has invested a lot of time and effort and created something to suit his needs but scaling will prove interesting as the workload increases...
     
    #56     May 4, 2013
  7. ofthomas

    ofthomas

    you should be looking at entry level arrays that will enable you to present to ZFS ... keep in mind not all your data at rest is active all the time, so placing everything on SSD is not advantageous, best to tier it... imo, which is restricted to what you are saying here without knowing actual vendors and physical setup...
     
    #57     May 4, 2013
  8. vicirek

    vicirek

    50 core Xeon Phi is co-processor using PCIe bus with own Linux OS using old OpenMP model plus some Intel in house developed software packages.

    It looks exciting on the surface but I suspect it will be a flop because implementing it will be no easier than GPGPU code. It is based on x8086 processor which is essentially sequential single threaded unit with artificial multithreading injected by time splicing, catching and instruction prediction and it is no match to hardware based multithreading offered by GPU.
     
    #58     May 4, 2013
  9. gmst

    gmst

    Interesting take. Well, I don't understand grids. As I said I can program in C/MATLAB/VB/VBA and am an advanced excel user and understand basics of oracle. Thats all I know about computer science. I don't understand linux, I have never created a binary file based data storage system. In my trading, I am 90% systematic, trade intra-day mid-frequency. The way I see forward for me is to first expand my portfolio to include more markets and simultaneously expand my research infrastructure for pursuing systematic trading across futures/equities space across geographies.

    How does the idea of taking a year off and going back to university to pursue an M.S. in computer science sounds for a person like me? Post this MS, I have no intention of joining a tech role, rather I would like to use this tech knowledge to design my own stuff and implement my infrastructure for systematic trading. Idea is to have enough skills/capital to develop into a systematic trading firm in 5-10 years time. Does this make sense?

    This is a general question - anyone feel free to answer.
     
    #59     May 4, 2013
  10. Allistah

    Allistah

    Has anyone used Tradestation's optimizer in addition to their walk forward optimizer before? Sounds like it is a whole lot better than having nothing.
     
    #60     May 4, 2013