 |
rosy2
Registered: Aug 2006
Posts: 1399 |
07-27-11 05:03 PM
you said you have a csv file of data. Isnt that file already sorted by date? so you dont need to worry about sorting. you just get the data put it in an object and add that to a list. This is trivial. You probably spent more time typeing on this forum thread than it would take to code it. if you need a sort then put the objects in a tree
|
| |
|
Edit/Delete • Quote • Complain |
dcvtss
Registered: Mar 2006
Posts: 426 |
07-27-11 05:19 PM
Quote from keyser1:
have you thought of using a database? sql server express or ms access?
'find me the lowest price within this date range' is easy to do in sql.
Its also relatively easy to do in c# .net using linq -- a bunch of technologies to learn, but speed of development will be much faster in .net; execution speed will be slower than c++, but that can be solved by having a faster machine.
The answer to your question really depends on
1. How fast do queries have to be
2. How much data is there
3. How often are you adding data
4. How often are you querying data
What he said...I've set up a MySQL DB for EOD OHLC data from excel in about 25 minutes, including installing the MySQL software, defining the tables, and importing the csv data. Use a free front end like HeidiSQL or phpmyadmin and it does the imports for you. Best thing is you can use excel to fromat the date string in MySQL's format before you import and then once it's in there as a native timestamp it's easy to query as keyser said.
|
| |
|
Edit/Delete • Quote • Complain |
Maverick1
Registered: Jun 2002
Posts: 521 |
07-27-11 05:29 PM
Quote from rosy2:
you said you have a csv file of data. Isnt that file already sorted by date? so you dont need to worry about sorting. you just get the data put it in an object and add that to a list. This is trivial. You probably spent more time typeing on this forum thread than it would take to code it. if you need a sort then put the objects in a tree
Yes the file is sorted by date. Here's the issue though: Each date has ~390 minutes of price data associated with it. For example:
09/11/2009 9.30 o, h, l, c
09/11/2009 9.31 o, h, l, c
and so on to
09/11/2009 16.14 o, h, l, c
That completes one day, and then starts the next which could be 9/14/2009 due to weekend, say.
What I want to do is simple analysis for now: for ex, find the time at which the low was made for each date in my data file. That requires that I be able to iterate over each date in sequence and because of weekends and holidays, that sequence is not a simple as +1 each time.
From the responses so far, it looks like best bet is to convert the date string into a long using strptime()?
|
| |
|
Edit/Delete • Quote • Complain |
jedwards
Registered: Jul 2009
Posts: 312 |
07-27-11 06:24 PM
Quote from Maverick1:
Yes the file is sorted by date. Here's the issue though: Each date has ~390 minutes of price data associated with it. For example:
09/11/2009 9.30 o, h, l, c
09/11/2009 9.31 o, h, l, c
and so on to
09/11/2009 16.14 o, h, l, c
That completes one day, and then starts the next which could be 9/14/2009 due to weekend, say.
What I want to do is simple analysis for now: for ex, find the time at which the low was made for each date in my data file. That requires that I be able to iterate over each date in sequence and because of weekends and holidays, that sequence is not a simple as +1 each time.
From the responses so far, it looks like best bet is to convert the date string into a long using strptime()?
If this is all you want to do then just stick it into a map where the key is the timestamp. Using minute data you can reasonably store years worth of data in memory on a current system.
strptime and mktime will work, although I myself used ICU libraries (ICU4C) because I was dealing with various time zones trading data from Hong Kong, Germany, and the US. It's more complex but doing things like date addition is a bit easier.
If you start getting into tick data, then you might need to go with something more complex because some data feeds are adding millisecond components to the timestamp.
|
| |
|
Edit/Delete • Quote • Complain |
Mr_You
Registered: Dec 2010
Posts: 265 |
08-01-11 01:58 AM
FYI, PostgreSQL is also an excellent "enterprise class" quality database that is opensource and free. Easy to use and install on Windows and other OSs.
|
| |
|
Edit/Delete • Quote • Complain |
| Receive
an email whenever a new post is added to this thread by subscribing
to it. |
|
|
|
|
