HOME FORUMS BROKERS SOFTWARE BOOKS CONTACT US
Elite Trader Your Account  •  Become a Member  •  Help  •  Search    
    Forums ›› Technically Speaking ›› Programming ›› database selection......  


Post A Reply
    Page 1 of 2:   1  2  
cdcaveman
 

Registered: Aug 2011
Posts: 3514

 

07-30-12 02:13 AM

Ok... I've been doing some research trying to figure out the best solution for my situation...
Currently i'm trading with Interactive brokers.. I trade mainly options.. I have bought Hoadleys excel fiancial add in and started messing around with it.. I've got some experience with programming but less and less as you go down the abstraction level. Meaning i know web application languages... I've taking a class in visual basic and didn't complete it.. And i'm not scared to dig in and learn new things... That being said
I've been thinking about scraping bid ask last and volume for options of stocks that i follow and insert them into a db.. Does anyone have experience with this?

    Edit/Delete Quote Complain
cdcaveman
 

Registered: Aug 2011
Posts: 3514

 

07-30-12 02:37 AM

Just to add to what i'm trying to do .. I'm going to use web applications as well. . So i was figuring on mysql. . I found drivers for mysql to excel so i c an still use my spread sheets and yet still serve the data out to my web apps

    Edit/Delete Quote Complain
Synesthesis
 

Registered: Aug 2011
Posts: 7

 

07-30-12 09:53 PM

Seems to me like a good solution if you can get it working with minimum effort that way.

You might take a look into other database types, but it might not be worth the hassle. To me the relational aspect of sql doesn't make much sense when working with tick data. It also makes databases both more complicated and slower to use than some alternatives.

You can look into key-value stores. In essence they are simple and speedy, e.g. Redis.

CouchDB is document store (which features you might not use), might be preferable over Mysql because it has a JSON interface. But that all depends on what web interfaces you use.

    Edit/Delete Quote Complain
cdcaveman
 

Registered: Aug 2011
Posts: 3514

 

07-31-12 01:50 AM

relational tables just aim to reduce duplicates... IE if you have a binary imput like put or call.. you can turn that into a boolean value of 0 or 1 and have the definition in another table that references it.. once you start getting into huge amounts of data this stuff becomes critical.. i'm no database administrator.. but relational tables do start to make sense quickly when data starts to accumulate.. but realistically one would have to ask.. how much can some of this data can be compressed using relational tables? the quote of database fields below is from live vol website.. alot of these fields make sense to store.. but as you can see not alot of them could be compressed by relational tables.. so the need for it in this case is negligible.

Field List:
Option Quotes

Time, Root, Expiration, Strike, OptionType, Open, High, Low, Close, TradeVolume, BidSize, BestBid, AskSize, BestAsk, Underlying Bid, Underlying Ask, {Regional Exchange: BidSize, Bid, AskSize Ask} x[# of exchanges]
Option Calculations

Time, Root, Expiration, Strike, OptionType, Open, High, Low,Close, TradeVolume, BidSize, BestBid, AskSize, BestAsk, Underlying Bid, Underlying Ask, Implied Underlying Price, Active Underlying Price, Implied Volatility, Delta, Gamma, Theta, Vega, Rho
Option Trades

Time, SequenceNumber, Root, Expiration, Strike, OptionType, Exchange ID, TradeSize, TradePrice, TradeConditionID, CanceledTradeConditionID,BestBid, BestAsk, UnderlyingBid,UnderlyingAsk, {Regional Exchange: BidSize, Bid, AskSize Ask} x[# of exchanges]


PS one cool feature i've thought would be interesting to replicate of the live vol software is
Trades and Quotes (TAQ)

Livevol® also offers the complete recorded history of equity and options tick data including an API to simulate real-time playback. Ask the Livevol® team for additional information.

    Edit/Delete Quote Complain
2rosy
 

Registered: May 2012
Posts: 336

 

07-31-12 03:50 AM


Quote from cdcaveman:

Ok... I've been doing some research trying to figure out the best solution for my situation...
Currently i'm trading with Interactive brokers.. I trade mainly options.. I have bought Hoadleys excel fiancial add in and started messing around with it.. I've got some experience with programming but less and less as you go down the abstraction level. Meaning i know web application languages... I've taking a class in visual basic and didn't complete it.. And i'm not scared to dig in and learn new things... That being said
I've been thinking about scraping bid ask last and volume for options of stocks that i follow and insert them into a db.. Does anyone have experience with this?



simplest is to use sqlite (no admin headache).

    Edit/Delete Quote Complain
cdcaveman
 

Registered: Aug 2011
Posts: 3514

 

07-31-12 03:55 AM


Quote from 2rosy:

simplest is to use sqlite (no admin headache).



ok .. there is some good to Mysql because any web application will already come prebuilt with a mysql db.. meaning i wouldn't have to add a datasource and connection to the application to query the inserted options data... its just automatically available.. what do you mean by admin headache? i wouldn't be hosting the db myself.. it would be on a hosting account.

    Edit/Delete Quote Complain
    Page 1 of 2:   1  2  
Post A Reply


Receive an email whenever a new post is added to this thread by subscribing to it.
 
Rate This Thread:

Forum Jump:
 

 

   Conduct Rules  -  Privacy Policy  -  Day Trader -  Day Trader Forum -  Best Trading Software -  Sitemap Copyright © 2013, Elite Trader. All rights reserved.    
 
WHILE YOU'RE HERE, TAKE A MINUTE TO VISIT SOME OF OUR SPONSORS:
Advantage Futures
Futures Brokerage & Clearing
AMP Global Clearing
Futures and FX Trading
Bright Trading
Professional Equities Trading
CTS
Futures Trading Software
DaytradingBias.com
Professional Trading Analytics
ECHOtrade
Professional Trading Firm
eSignal
Trading Software Provider
FXCM
Forex Trading Services
Global Futures
Futures, Options & FX Trading
Interactive Brokers
Pro Gateway to World Markets
JC Trading Group
Direct Access Trading
MB Trading
Direct Access Trading
MultiCharts
Trading Software Provider
NinjaTrader
Trading Software Provider
OANDA
Currency Trading
optionshouse
Option Trading & Education
Rithmic
Futures Trade Execution Platform
SpeedTrader
Direct Access Trading
SpreadProfessor
Spread Trading Instruction
thinkorswim by TD Ameritrade
Direct Access TradingAdvertisement
TradersStudio
System Building & Backtesting
Trading Technologies
Trading Software Provider
Trend Following
Trading Systems Provider