One thing to mention, the stupidest firms (although the richest LOL!) are the ones that avoid all this complexity and just "mod it out". Meaning, they just try to do [statistical] arbitrage and market making. You don't need to be particularly smart to do that. Just have enough money to throw at the problem with computer scientists and relatively simple ideas. Retail traders are forced onto the much much MUCH harder place of predictive trading. Imo, the only way a retail trader can survive is to use options to mitigate their risk [mostly from the get-go] and let the position play out.
Here is a question every retail trader should be asking. Is there a way to tell, using only technical analysis (some chart indicator), when technical analysis is very likely work in say the next n minutes/hours/days/weeks? it is a recursive analysis, and is similar to Soros "Reflexive" theories. That way, assuming the indicator exists, retail traders can sit on hands when their predictive powers are low, and act hard when predictive powers are high. They still mitigate risk using option [spreads]. So hit hard when odds in favor, say if you normally trade 1000 times a year say buying 1 lot of SPX/NDX/DJX options, maybe now you only trade 10 times a year, but with size of say 10 or 20 SPX/NDX/DJX whatever option spreads on each trade.
I've been doing something similar for past few months...namely leg spreads across a wide strath of strike prices...there is directional risk at the outset, but only for a short duration...the "modern markets" are becoming more binary...one way flows lower/higher/lower, etc...not to mention the "speed" of the markets is another obstacle to a purely directional approach...three days of the week the market might slow to a crawl and then on the fourth day can cover an entire day's range in 15 mins...the vertical nature (binary) of these moves puts smaller directional traders at a disadvantage if they aren't early to the move since some backing and filling will invariably trigger stops.
You quoted my post, but you don't state if you believe my thesis? And if you do, what indicator do you use to tell you this time is the right time to enter market because T/A is working? Or is it less scientific?
Many thanks to the contributors and especially the OP ' Nitro" in this thread . The knowledge and intelligence exhibited here outstanding . Ive been travelling this path for many years and i appreciate the resources presented in this thread . Days / weeks of reading and watching to do , i am sure i will add more knowledge here than the rest of ET combined .... Kudos to all , huge appreciation to all for sharing
ah good point...I believe that TA can work at times as I also believe that there are cycles and time/price symmetry BUT it can be distorted by certain "machinations"...Regardless, I find that if I'm too invested in a certain outcome developing it will affect my trading...So I keep it simple and just piecemeal leg into the spreads (both sides of the market), make some minimal adjustments and then let them do their thing...I had this idea going back to last fall, when I missed most of that upside rally...even though at times I had played the bounces very well, it was the "unhedged" element of the positions that caused a great deal of the problems... One indicator: combination of a linreg channel with a specific oscillator (but more importantly a price level)...the difference is I can adjust the r:r more favorably as opposed to taking outright positions.
The people with bucks use something like this to stream their data on to storage so they don't have to worry about losing data https://aws.amazon.com/kinesis/firehose/ Put you data into S3 through firehose. If you can't afford Amazon S3 (or other AWS), you can install Eucalyptus which is what much of AWS is based on. Redshift is not Cassandra, which is my choice for massive datastore. Not sure why I wouldn't just use Spark for my analytics. From the web: Cassandra and Redshift can not really be compared to one another. They are two completely different tools used to solve two very distinct problems. Let me start off by outlining some of the key differences between the two. Cassandra Pro's Multi-DC replication. Cassandra comes out-of-the-box ready to replicate your data to any datacenter anywhere in the world in near-real-time. Fast real-time ingestion of data. Writes in Cassandra are done in an append-only fashion to a commit log. This means that the write speed is bound primarily by IO throughput of the disks and network. Fast Reads. The SSTable format of Cassandra allows the process to easily pinpoint which file, and wherein, the data you are querying resides. Extensibility. Cassandra is an open source project part of the Apache foundation. As a result, many different technologies can, and have, been layered on. Examples include github.com SOLR, Hadoop/Hive, Spark, and many others. Con's No ad-hoc queries. You must know the partition key for the data you want. There is no querying by any column in any way. No analytics queries. There is no analytics functions such as SUM, AVG, MIN, MAX etc. Newer versions of Cassandra will soon have these at the partition level, but for data analysis this is of limited use. No Indexes (sort of). The secondary indexes in Cassandra are of limited use and do not scale well in the cluster. They are primarily designed to overcome the first bullet point in the con's section. Generally they are best when there is a low cardinality of values and a small number of servers in the cluster. Redshift Pro's Columnar Storage. Redshift stores data in a columnar format. This allows operations done on a single column to be extremely fast. Operations like MIN, MAX, SUM, AVG can compute over billions of rows in seconds. Sorted table format. The tables in Redshift are sorted according to the CREATE TABLE statement. Having data sorted allows for very dense compression and fast retrieval of information. SQL-92 Compliant. Redshift is built off of the Postgres database project. This means that most Postgres tools and drivers will work out of the box. This allows for the integration complex BI solutions with little technical overhead. Easy Administration. Redshift remains true to the Amazon form of making things dead simple to administer. Within a few minutes you can have a 100 node cluster running that is fully monitored, has backups and point-in-time recovery all at the click of a few buttons. It is also simple to scale up or down as required, and has a very big suite of instrumentation for every part of the cluster including all queries ever run. Con's DML operations are very costly and slow. Because of the sorted columnar format, doing singular DML operations like INSERT, UPDATE and DELETE are very expensive. This means your data needs to be loaded in large batch loads, typically via an ETL/ELT process. One region, one availability zone. Part of the speed of Redshift is the fact that all nodes are in a single EC2 Region and Availability Zone. This means that for multi-DC setups and full AZ fault tolerance, one or more additional clusters need to be administered and kept in-sync. Complex query optimization. Most queries you run in Redshift are going to be very fast. But when processing billions of rows of data, every last optimization can make a big impact. Understanding of how to optimize queries for Redshift is something you only gain after months or years of experience and trial/failure. Luckily the Redshift control panel gives you a very in-depth look at the query plan for all queries executed. So now comes the question I ask anyone who is deciding which database to use. What is your use-case? Need to stream data in and store it for later retrieval and display? Do you want multi-datacenter replication and availability? Cassandra may be a good choice. Need to do complex analysis of large amounts of data in a, relatively, quick time? Do you have a team of analysts that need easy-access to the data to ask any question they can come up with? Redshift may be a good choice. Need a combination of the both? Then I might suggest storing your data in Cassandra then periodically ETLing the data out of Cassandra and into Redshift for analysis. This is a common practice in a lot of the time-series analytics circles and one that I, personally, have employed and found successful.
I am seriously looking into using Googles container engine https://cloud.google.com/container-engine/