4GB is more than enuf, you need fast processors for that kind of work. a "cray" doesn't explain much, it's just a brand of super computer, it doesn't say anything about the processors involved, interconnects, memory etc. FYI, a quad core Intel 2.93Ghz can do over 150 Gigaflops/second. more than enuf for backtesting stock data.
mobo (this one or a tyan thunder) http://www.newegg.com/product/product.asp?item=N82E16813131140 two dual-core optys (discretion here as to how nuts you go on price): http://www.newegg.com/product/product.asp?item=N82E16819103552 10k HDDs for apps: http://www.newegg.com/product/product.asp?item=N82E16822136012 Seagates for storage: http://www.newegg.com/product/product.asp?item=N82E16822148140 Registered memory: http://www.newegg.com/product/product.asp?item=N82E16820134339 PSU to make it go vroom-vroom: http://www.newegg.com/product/product.asp?item=N82E16817703005 2 of these for 8 displays: http://www.newegg.com/Product/Product.aspx?Item=N82E16814133165 and squoosh it all into a CM Stacker: http://www.newegg.com/product/product.asp?item=N82E16811119121 Anywhere from $3-5k depending on how nuts you go. Wait two years. Rinse & repeat.
As in every computing issue that deals with speed, you need to 2 things that need to be considered: 1. Well programmed software. 2. Strong computing power. I can run the same test done in Tradestation, > 1/100 of the time on C#. If you are having problems, it's time you learn to program or re-evaluate your strategy so that it works on a regular computer. Another alternative is to cluster.
Why SPSS? Depending on what you're doing, that might be the reason why it takes you so long to process your data. Why not SAS(http://www.sas.com/) or even better Matlab(http://www.mathworks.com/)? Both support multithreading better and with a quad core you can easily process numbers like crazy. In matlab you can even write processes that interact with the Tradestation API in case you want to automate it. I personally use WinRATS(http://www.estima.com/) since it's designed specifically for time series stuff. But I wouldn't recommend it for backtesting since it's more academic than practical. And if you really really want the fastest performance possible.. write your own C code running on separate computer running Linux(free). Using something like Ta-Lib(http://ta-lib.org/ free) or R(http://www.r-project.org/ free) will save you a lot of time. Then process with the datafeed and then trade with Tradestation on your trading computer. Regardless, the limitation doesn't seem to be your system but your software. SPSS sucks at time series stuff, but rocks at cross sectional data and point-click interface. Their coding is very simple too which is always a good thing.
Do the sums and think about processing speed in dealing with time series. You neither want to be recalculating them on each iteration of an optimizer nor reading them off disk.
Yeah, clustering is the way to go The only problem is that your software has to be written for the cluster. Try to google for Beowulf or go directly to beowulf.org. It's a free cluster software. In the telecom industry I am working on a cluster with 32 nodes (midi configuration). The problem is that you have to write special code for it and use a compiler that was specially build for it to make use of all the nodes. The advantage is good performance and 'on-the-fly' node replacement. If you have a couple of 100k for hardware and clusterware + another couple of 100k for 'hand-written' software be my guest.