Database organization

Discussion in 'App Development' started by cjbuckley4, Oct 18, 2014.

  1. It is not a stretch given that virtually not a single quant desk at any hedge fund or ibank uses sql to handle time series data. There is a reason they use KDB and other columnar specialized data bases. Take my advice or leave it, but tucking gigabytes or terabytes of time series based data into a rdbms/sql is just the wrong way to approach this. I explained why so you have the free choice of course to do anything that pleases you. Good luck with your development.

     
    #111     Feb 8, 2015
  2. Butterfly

    Butterfly

    RDBMS are perfectly capable of handling time series, not sure why volpunter, in his infinite "wisdom", think only "KDB and other columnar specialized data bases" can handle those. The storage engine of such databases doesn't exclude them from handling SQL requests and create relations between tables. Actually, some columnar based BigData DB engine will do exactly that in SQL, see MonetDB for example.

    Again, volpunter confusing concepts of storage engine with the full functionality of a RDBMS.
     
    #112     Feb 8, 2015
  3. Butterfly

    Butterfly

    I fail to see how quants at hedge funds and Investment Banks could manage all those data by themselves with low level tech when in reality they would focus more on strategies, since that's why they are being paid for.

    Also, I don't see the IT departments at those huge Investment Banks letting "individuals" plugin on their workstation and company servers, software that has no been approved or not gone through some kind of formal process for integration by the company IT department. Recipe for disasters if they let individuals "play" with their software and create IT incidents because of their incompetence.

    There is no way that KDB and other proprietary binary files would make the cut for approval by the Enterprise Architects for inclusion in the company IT resources.

    Those binary files full of useless market data remains what they are, toys for geeks in their basement.
     
    #113     Feb 8, 2015