 |
NetTecture
Registered: Mar 2009
Posts: 1010 |
07-27-12 05:39 AM
Quote from amazingIndustry:
getting rid of all double, float variable types may not apply to all trading strategies. You could go lower and lower level and arrive at coding assembler but that would get into a very deep discussion about the pros and cons. Not everything has to run on integer values, the speedup sometimes warrants that, yes, but often times it is negligible.
Actually it does wonderfully apply to trading.
Start with bars - open, high, low, close are TICKS - not arbitrary numbers. So, store and aggregate them i nticks.
Then go fro mthere like another 6 digits (hard factor) and use integer from now on. That is a fixed factor from the orriginal bar to the expanded resolution one. But instaed of 6 digits (like 1.000.000) use a power of 2 number, so going down to the original tick resolution is a simple shift.
You may not be able to avoid ALL divisions, but most of them.
Obviously that doesn ot help if some overly smart developers decided to use floats all over their framework.
It is EXTREMELY expensive to dean with any price comparison after any mathematical operation in floats because not only do you ahve to do the operation, but also have to.... round it to the proper resolution, which means you basically are wasting a LOOOOT of cycles.
Even something like x * TickSize in float is NOT guaranteed to be equal to A * TickSize + B * TickSize - the two floats may be CLOSE, but not IDENTICAL.
So that in ints, and they are not identical, but then I compare:
((A * TickSize) >>x) + ((B * TickSize)>>X).
Funny thing is that >>X is a VERY cheap operation these days thanks to barrel shift registers. Most likely 1 tick.
The alternative is:
Math.Round(A*TickSize, x) + Math.Round(B*TickSize,x)
And you do NOT want to know the cost of the Math.Round operation. It is EXPENSIVE.
That is NOT relevant when you do real time processing, but when you do extensive backtests + optimization it may cut down processing times by a factor of 10 and more, depending how heavy the operation is ;)
When you use modern Opterons you get the additional benefit that the 2 cores of a bulldozer module... have ONE SHARED FPU UNIT - but two separate Integer units ;)
|
| |
|
Edit/Delete • Quote • Complain |
amazingIndustry
Registered: Aug 2009
Posts: 570 |
07-27-12 06:09 AM
With all due respect but I think you did not read my post carefully. I never claimed that it would not apply to trading, quite the contrary, I stated clearly that it has it applications and can speed up things. I claimed that the previous poster apparently is dealing with much more trivial issues in his code that could hand him at least an order of magnitude speedup if he implemented more efficient code. One aspect is what you pointed him to but its something that can be very complicated and time consuming to implement. Please consider that most financial and math/quant libraries DO PERUSE double or float variable types, you would have to re-write ALL such libraries or write your own from scratch if you wanted to go all integer. This is in most cases not feasible nor advised. We do not need to argue about your points because yes they are correct and I think most everyone will agree with your suggestions but you need to balance that with the benefit you derive from that vs the time it takes to implement and what specific libraries the project is accessing. Blanket statements are rarely the solution. It really depends what computations the code within the strategy in question is performing.
There are very important differences even whether you divide a float by 2 or multiply it by 0.5 in terms of precision and computational requirements. Yes they all matter but my claim is: First tackle the issues that can be adjusted and changed in a matter of minutes rather than delving right into issues that require hours if not days to carefully plan, think through and then implement.
Quote from NetTecture:
Actually it does wonderfully apply to trading.
Start with bars - open, high, low, close are TICKS - not arbitrary numbers. So, store and aggregate them i nticks.
Then go fro mthere like another 6 digits (hard factor) and use integer from now on. That is a fixed factor from the orriginal bar to the expanded resolution one. But instaed of 6 digits (like 1.000.000) use a power of 2 number, so going down to the original tick resolution is a simple shift.
You may not be able to avoid ALL divisions, but most of them.
Obviously that doesn ot help if some overly smart developers decided to use floats all over their framework.
It is EXTREMELY expensive to dean with any price comparison after any mathematical operation in floats because not only do you ahve to do the operation, but also have to.... round it to the proper resolution, which means you basically are wasting a LOOOOT of cycles.
Even something like x * TickSize in float is NOT guaranteed to be equal to A * TickSize + B * TickSize - the two floats may be CLOSE, but not IDENTICAL.
So that in ints, and they are not identical, but then I compare:
((A * TickSize) >>x) + ((B * TickSize)>>X).
Funny thing is that >>X is a VERY cheap operation these days thanks to barrel shift registers. Most likely 1 tick.
The alternative is:
Math.Round(A*TickSize, x) + Math.Round(B*TickSize,x)
And you do NOT want to know the cost of the Math.Round operation. It is EXPENSIVE.
That is NOT relevant when you do real time processing, but when you do extensive backtests + optimization it may cut down processing times by a factor of 10 and more, depending how heavy the operation is ;)
When you use modern Opterons you get the additional benefit that the 2 cores of a bulldozer module... have ONE SHARED FPU UNIT - but two separate Integer units ;)
|
| |
|
Edit/Delete • Quote • Complain |
NetTecture
Registered: Mar 2009
Posts: 1010 |
07-27-12 06:24 AM
I agree. And that is where there is ONE thing only to solve it efficiently - a profiler. Plus possibly some simple mathematical optimizations. For example I know one library where a moving average is calculated by - adding the elements in it, then dividing by number of elements.
Evers time.
instead of adding up, remember that for next run, then remove the element falling out and add the one going in.
For a longer moving average that is significant (50 additions instead of 1 addition, 1 substraction).
But a profiler will show you where the problem is straight in. The one from Visual Studio (mind you, not the lower ties) is VERY good in that and ALSO can properly deal helping debugging issues with TPM.
Now we just need Visual Studio 2012 released sooonish... their profilers are a LOT better than the one from 2010.
|
| |
|
Edit/Delete • Quote • Complain |
amazingIndustry
Registered: Aug 2009
Posts: 570 |
07-27-12 06:27 AM
I run VS11 beta and the profiler is awesome especially the one profiling concurrent operations. I love it.
P.S.: By the way, if the compiler and profiler did their job, then the entire operation of floats is pipelined. So if you have a loop, for example, the entire loop will only take x cycles longer to run on the FPU rather than each loop iteration. The difference then between running on ints vs floats becomes negligible. (Even running a decent loop of 1000 elements) on floats vs ints and assuming about 5 cycle overhead on each iteration will only cost you an extra 10 or so microseconds, depending on how your CPU is clocked. This 5 cycle overhead on the whole loop will be so small that you would not even be able to measure it on Windows machines.
Quote from NetTecture:
I agree. And that is where there is ONE thing only to solve it efficiently - a profiler. Plus possibly some simple mathematical optimizations. For example I know one library where a moving average is calculated by - adding the elements in it, then dividing by number of elements.
Evers time.
instead of adding up, remember that for next run, then remove the element falling out and add the one going in.
For a longer moving average that is significant (50 additions instead of 1 addition, 1 substraction).
But a profiler will show you where the problem is straight in. The one from Visual Studio (mind you, not the lower ties) is VERY good in that and ALSO can properly deal helping debugging issues with TPM.
Now we just need Visual Studio 2012 released sooonish... their profilers are a LOT better than the one from 2010.
|
| |
|
Edit/Delete • Quote • Complain |
bidask201
Registered: May 2012
Posts: 19 |
08-02-12 04:20 AM
Take a look at vectorwise for a sql solution focused on optimizing for modern processors. Still only say 3x the speed of other dbs on tpc-h. Be ready to hand over huge sums of money though
|
| |
|
Edit/Delete • Quote • Complain |
| Receive
an email whenever a new post is added to this thread by subscribing
to it. |
|
|
|
|
