Twitter Data Set and AI/Machine Learning to look for implied direction in SPY

Discussion in 'Strategy Development' started by Spectre2007, Sep 1, 2018.

  1. It was nice talking to ##### over the phone. I think we have zeroed in on a project that fulfills course requirements(AI/Machine Learning) and time constraints.

    Description:
    Look at past historical daily (EOD/End of Day) data on SPY, the historical data is available in CSV format from yahoo I believe. The two components looking at in the historical data, are Daily Range (High - Low), and change for the day of SPY.

    Daily Range Variable = DRV
    Change Variable = CV

    Twitter Universe State
    Twitter Universe State A = Bullish or Upward implications for price of SPY
    Twitter Universe State B = Bearish or Downward implications for price of SPY
    Twitter Universe State C = Neutral or no meaningful implication

    The supposition is that the twitter universe changes as the stock market changes, the proxy we are using for the stock market is the SPY or SP 500.

    A high range or DRV with a large negative CV implies a downward bias.
    A high range or DRV with a large positive CV implies a upward bias.
    A high range or DRV with a small positive CV implies a upward bias.
    A high range or DRV with a small negative CV implies a upward bias.
    A low range or DRV with a small positive CV implies a upward bias.
    A low range or DRV with a small negative CV implies a upward bias.


    I'm told free TWTR universe data is only available for the past 7 days. We will need to look at the past 7 days of data. And correlate it to DRV and CV. The classification of 'high' range. A running average for the past 7 days need to be made. Anything that exceeds past 7 days will be considered 'high'. The same with CV. What we are looking for in the end is to watch for real time changes in the twitter universe that may correlate with impending price movements in SPY, we can front run and place orders to take advantage of twitter universe implications.
     
    Last edited: Sep 1, 2018
  2. If you roughly look at this candle chart, the conditional states are correct.
     
    • spy.PNG
      spy.PNG
      File size:
      103.6 KB
      Views:
      27
  3. fan27

    fan27

    How can you effectively backtest this strategy if you can only access the previous 7 days of TWTR data?
     
  4. Need to collect forward data, wont be able to backtest it. Trying to find if anything in the TWTR universe is present before large dislocations present themselves.