Kt, If your goal is only to store quotes from a packet, then sheesh, just about anything that takes a packet and stores it to do disk qualifies as a "database." The real meat is in the quering of realtime data with complex queries using a straightforward language on time-series in realtime. Again, I urge you to read the streamDB papers because they go into this very question. Like I said above, you can do it all in C++ & HDF5 and coding functions that do what you want in C++. But unless you have a language that you have built for doing complex queries, it is half a database imo (let alone ACID, fault tolerant, transactional, etc etc.) HDF5 doesn't have a Query Language on streams AFAIK. BTW, take a look at db40. Very interesting idea of using objects instead of relations (Object Database), but using classes to do queries. And it's free!!. This may be the best compromise between a stream DB and hand coding all these db/query language in C++: http://www.db40.com nitro
I will take the time to read the documents. Thank you. Call me crazy, but I like the exercise of coding up my own function-aggregate interfaces and utilizing pre-formed interfaces sparingly. Must be due to my (faulted?) meticulous nature for granularity, originally coming from my architectural/building/engineering thinking process. As I gain experience, I find that more and more, I'm appreciating higher level interfaces (or languages) in my code... when needed or efficiently acceptable. -kt
how could you even use mysql for realtime tick capture and analysis. using sql would be near impossible when compared to using simple vectors.
Not analysis - just capture and storage. Read time series into memory for analysis - into vectors or arrays or whatever.
For those interested: C# Prototype Release ==================== We are pleased to announce the prototype release of a C# API for HDF5. It consists of C# wrappers for the HDF5 C library for the .NET environment. In this first prototype, wrappers are implemented for many of the basic C functions, and a detailed example is included. A future prototype will provide a comprehensive set of C# wrappers for the HDF5 C library. This release is available, at: ftp://ftp.ncsa.uiuc.edu/HDF/pub/outgoing/hdf5/Csharp-prototype/HDF5dotNet.zip Once you have downloaded and uncompressed the file, you can access the documentation from the following location: .\HDF5DotNet\HDF5DotNet\doc\output\HDF5DotNet.chm (double-click on the file) The list of currently implemented wrappers along with general instructions for using the wrappers is provided in the release notes at: ftp://ftp.ncsa.uiuc.edu/HDF/pub/outgoing/hdf5/Csharp-prototype/PrereleaseNotes.txt
Another intersting appproach that has many advantages. The question is how efficient is it: http://www.hibernate.org/ This may be the best compromise to them all.. nitro
That is the key question. I only took a brief look at the site, but if the underlying data store is relational then it seems the primary advantage of this product is ease of coding (i.e. it handles the object to relational mapping for you), but would still have the performance disadvantages of writing ticks directly to a RDB. I did follow your link to db4o and I quite like the sounds of it. They make a compelling argument against object-to-relational tools such as this. Thanks for posting the links.
hibernate caches objects. specifically java objects. are you proposing to store your ticks or aggregation of ticks in java class and use hibernate? I think the hdf5 implementation is the best for no money and kx if you want to spend money.