Yah, tickData is insane on the prices.... It's hard to describe without going into too much detail. I assume what I am doing is too good to be true (I have a few algorithms that appear to give good results), but if it is not, I want to first test if 100% then apply it in real life and then capitalize. Like I said, my algorithms are purely and solely numerical analysis. moving alrogithm was probably a very bad term. My algorithm is like a mutating algorithm in that it tests its predictions on a scale of hours and 'moves' up to a day, 2days, week , year, etc.....(hence the poorly named moving)
Things to keep in mind: 1. Note that the overall market has been up over 15% during that period. 2. Roughly 80% of the movement of any individual stock is a result of market movement. Thus, if you have been doing only long trades, you have been "pushed" by the market. 3. You may be confusing brains with a bull market. 4. Run your system from 5/1/2010 until present. What do you see? 5. Also include commissions and spreads. 6. Have you looked at when your trades trigger? I have a suspicion that many may have triggered on the same dates. If so, that would be difficult to trade. You'll want to check that out. Good luck.
Thank you for advice: 1. I have done analysis for 2006-2010 and always am about %70 percent and in some cases go extended periods of time with 90% 2. These are essentially day trades. In my simulations, I analyze stock data at the closing bell and then buy at closing. The stock is then sold the next day no matte what. 3. Possibly but hopefully not probably 4. 45/60 = 75% | Breakdown: 43 went above 3% next day, 2 went above 10% and the other 15 did not even go above 0%. 5. Commissions included...what do you mean by spreads (I have very very little knowledge of actual stock trading and terms...im a number cruncher)
The Predictors: "One of the fundamental truths about the markets is that the dynamics are nonstationary," Norman explains. "We see no evidence for the existence of an attractor with stable statistical properties. This is what characterizes chaos -- having an attractor with stable statistical properties-- so what we are seeing is not chaos. It is something else. Call it an 'even-stranger-than-strange attractor,' which may not really be an attractor at all. The market might enter an epoch where some structure coalesces and sits there in a statistically stationary pattern, but then invariably it disappears. You have clouds of structure that coalesce and evaporate, coalesce and evaporate. Prediction Company's job is to find those pieces of structure that have the strongest signal and persist the longest. We want to know when the structure is beginning to emerge or dissolve because, once it begins to dissolve, we want to stop betting on it." So, your moving algorithm is what one needs!
First, see my note above regarding dates. Many equities strategies trigger on just a few days of the year. Spreads: The difference at any point in time between the bid and ask. The data you are using shows only the actual traded price. But you will not get that price in reality. For example, a closing trade of the day may have been logged by Yahoo at at 21.20. But you would have paid a price slightly higher. Same as exiting the trade. In selling, you would have paid slightly lower than the actual trade logged in the data. With a logged price of 21.20, the bid (higher) could have been 21.25. The ask (lower) could have been 21.15. Stocks run with different spreads. Some are very tight. Others are wide. Generally, higher the volume the tighter the spreads. Lower the volume, the higher the spreads. If any of your stocks are low volume (I'd say less than 300K shares per day), you can get some fairly wide spreads. Finally...not to discourage you...relying on opening or closing daily prices for a strategy is not terribly reliable. Spreads can be very wide at the open and close...and you will seldom get the price that is listed in the final logs of any data source. For example, let's say you're trading 1000 shares. A couple tenths on each side of the spread can easily knock $100 off your expected gain. You may have something...but these are all things to look into before you quit your day job.
That is why i think my approach is very different and that is my aim. My situation is that I need more data to analyze. The data alone will be in the terabyte range, but on top of that; to analyze the results and save the results will take terabytes more along with a tremendous amount of processing power... Im sure what i'm doing has been thought of before, but probably net ever implemented due the requirements of development (programming and mathematical algorithm development), infrastructure (hardware to store and process), and actual data
I have my algorithm tend toward higher volume stocks so I don't think spread will be too much of an issue, but I will modify my algs to take into account bid/ask. Yes, this is my goal to test against a more substantial and inclusive data set. Thx so much for your advice.
Just an idea: to collect intraday data i've found it pretty easy to import real time price (or volume etc) using DDE into Excel. Many broker software packages support this (thinkorswim for example). Then you can just write a simple macro that copies the price data into columns. I snapshot prices every 5 seconds for some of my macros. You end up with not exactly tick data, but its much more detailed than something like closing minute bars. You can then write some pretty powerful market scanning tools from that data set. http://readtheprospectus.wordpress.com/tag/dde/
How do you know your win rate when you say in the first place that you have not tested it for 0.5%? What do you mean the algorithm is right 7% of the time if you have not carefully applied stops and targets? May I suppose that you put a very small target like .5% and a very large stop like 10% and you got a 70% win rate? If that was the case, this is not enough to be profitable. You would need very high win rates above 90% to make money that way. You also said you have minute data but then you said if you had ninute data you would test the system. You could test for conservative exists by checking for stop before target. This does not require tick data.
I didn't say I know the win rate. I said I can accurately pick above 70% of the time a stock at end of day today which will at some point in time the next day gain 0.5% The algorithm is correct in picking a stock 70+ percent of the time. Yes that is exactly what i'm saying, I need the minute by minute data to determine whether or not this is feasible. The feasibility test is simply, "does the stock hit my sale for profit point of 0.5% before hit hits the sale to stop losses point of x%." Once I have the minute by minute or tick data, I can test the feasibility if this is even realistic in general. After this, I can run optimization algorithms to determine the appropriate profit point (if it is lower than 0.5%) and appropriate stop lost point.