Decent article here that discusses the issues: http://www.anandtech.com/storage/showdoc.aspx?i=3531 The ageing problem is mostly with writes and is highly dependent on the controller chip and firmware used in the drive. The forthcoming ATA TRIM command already supported by some new drives will also mitigate the problem. Linux kernel filesystem/block device support for TRIM will be out very soon. Windows 7 will support it sometime in the future.
I believe for your needs SQLIte is probably the right solution. If you r needs grow then you will need something more powerful. The simplest solution depending from the task is the best one - SQLite is very capable and if your tasks are small and don't require a lot of data probably it can fit your needs. I mentioned there is a tendency among the vendors to rewrite all they products to .NET which is not always right. Too much overhead, resources and it is slow, I don't see any other benefit other than development and maintenance is faster under .NET - that is indisputable. But we need performance. You have to cut all what is unneeded to get the speed. SQLite is a good example of this - Dr.Hipp is not willing to add anything more which is good sign it would not get bloated. It is not right for what we do but I like the approach. By the way - it is used from Bloomberg. The problem we had finding a solution using conventional approach is we have a lot of data and extensible algorithms to test the best strategy - it needs lot of experimentation and the time to get your answer right or wrong is the critical factor. That is why we decided to develop our own solution. Not easy decision because it takes time and investment, but it will compensate in the long run - it is patent pending and we can offer it to the trading public at some point - a very fast screener and back-testing platform. That is what I think is lacking on the security analysis market today and most needed.
There is a different access pattern with SSD and the optimal way to write/read is still not researched well. When they stop the deterioration problem we will wait for 1 year to look for any other hidden surprises. they will get cheaper too.
Storing just the level one OPRA feed and associated greeks goes through ~3Gb a day, uncompressed. Falcon SSD $900 here http://www.guru3d.com/article/gskill-falcon-128gb-ssd-review/' That is 8x more expensive than the same size high end SCSI drive, with, AFAIK, no support for RAID (for redundancy).
A quick note - Vertica is more oriented toward large warehouses like inventory analysis, etc. We need more time series oriented DB.
Anyone with experience using hybrid drives? I haven't but it seems like a possible solution. Not sure if they can be used in a RAID array though...
Mr Start's comment is roughly correct. Vertica is often used by financial-markets firms in conjunction with tools from Streambase, which is a quasi-affiliated company. Both firms [and C-Store, as mentioned elsewhere on the thread] are associated with Mike Stonebraker, a rockstar computer scientist. Both firms offer excellent products, but [ to answer Mr Nitro's question] we came to believe the heavy cost load of buying both Vertica and Streambase wasn't appropriate relative to our scope and requirements, and we moved toward open source solutions instead, at least for now. But YMMV.