Martin I don't agree with 85% of what you are saying about the use of HDF5 in relation to it's use in financial time-series databasing, but I don't have time to go into it now. I am responding so that I remember the challenge that I made and so that you can hold me to respond to that challenge later. nitro
My suggestion is reiserfs. Although any modern journalling filesystem will have broadly similar properties. Some would say that a filesystem isn't really a database; but hdf5 isn't really a database either. The difference lies in their respective shortcomings. A filesystem falls short of being a "real" database because it doesn't manage structure at the record level. hdf5 falls short of being a "real" database because it can't handle concurrency. Otherwise they are really quite similar. Different applications will be able to cope differently with these compromises. One thing is certain though, filesystems are faster than hdf5. After all, hdf5 is layered on top of a filesystem. Anyway, it's always interesting to discuss this. As long as I've shared my own concerns with HDF5 I feel like I've done my part. Martin
I just realized that the K programming language is a derivative of APL and a cousin to the J programming language. Now I see why this is so desirable... Are you saying they distribute free versions of k+ and kdb+ for personal use? nitro
Has anyone looked into Vertica? A column major database from Michael Stonebraker's latest academic spinoff. www.vertica.com I signed up for their early access program but haven't heard boo. It looks a little too heavily read-optimized for my needs but still, pretty darn cool. nitro still looking forward to your comments on the 85% you disagreed with me. Martin
http://www.proxy80.com/cgi-bin/nph-...urceforge.net/forum/forum.php?forum_id=584739 MonetDB: Fast, but not much more, if any, than MySQL 5.0 Still, benchmarks for retrieving sequential, time-series data may show it to be a winner. Still awaiting the ODBC documentation....none to be found.
This might be useful to some. Simple Persistence for Java is an open source object-relational persistence library that uses a custom query language and built-in database support to simplify object persistence in Java applications. In this article, software architect Sami Salkosuo introduces the library and walks you through its zero-admin, zero-config approach to object persistence. http://www-128.ibm.com/developerworks/java/library/j-sp4j/?ca=dgr-lnxw07Java
I'm using OCaml to build my (commercial) algorithmic trading tookit. Apart from performance on par with C, OCaml has built-in marshalling which can be used for persistence. The approach to tick storage for OCaml is to just dump the tick arrays to disk and organize them in directories. Also check out http://cristal.inria.fr/~starynke/persil/ which uses marshalling with various data stores.