A large crowd of people crossing a pedestrian crossing in a thronging city, are they measured as random? The answer is yes and no.
@kevinkdog, I was hoping you would comment on this. I am no expert but it seems to me lots of people looking for data to give them strategies, while Imho one should come up with strategies and then have them validated/invalidated by data/backtesting. I understand there is no right or wrong, whatever works for you.
Chicken and egg. No strategy without data, and no data without strategy. QLAI, I appreciate your position, just try to expand your concept of a 'strategy' outside the box of what you have been taught. Then you will see that data / layering / and manipulation thereof can 'reveal' the opposite side.. the strategy. What is a strategy but a way to exploit a pattern in the data? Why try to reveal the patterns only using a preconceived pattern or concept without first looking at the data. One 'could argue' that due to this logic that strategies are merely a derivative of data pattern, thus making data a higher ranking when it comes to formulating strategy.
I can't trade automated strategy unless I understand it's edge. Backtest results are not enough. This is just my psychological make up. As Van Tharp said, you don't trade the market, you trade your beliefs about the market. I can't trade something I don't understand. But I can appreciate when people say - grab money first and ask questions later.
You make valid points. The video is enjoyable and something to listen all the way through when I have a bit more time available. I'm not a good one to relate to in terms of 'edge' for I don't subscribe to the notion. I understand the truth of it as it applies to other traders whom find great utility with the concept. It's just not for me. Through my efforts in MADA, (Monitor, Analyze, Decide and Act, a concept first espoused by Jack Hershey here on ET) I've observed directly that market dynamics when decoupled from time operate in a sequence of events. The dynamic is fractal in nature and the same pattern that occur on larger timescales also can be observed on the fastest. There is a goldilocks zone intraday with the 5m that requires a steady focus and concentration with a relaxed pace. It's a different paradigm and has a minority view. The essence is that Volume leads price. Volume is the independent variable with price as the dependent variable. The paradigm exists within four interlocking systems, similar yet distinct in their application (equities, futures, sectors, etc.) and ability to extract the market's full offer. It operates on boolean math, not statistics nor any sort of backtesting. It's a forward testing system right from the get-go by developing what Jack would call a fully differentiated mind. The process begins with de-constructing the market's basic granularity and through use of drills and purposeful learning, one can develop a deep and abiding understanding of the market and the ability to extract the market full offer at any given time. It's conceptually challenging in that is relies on deduction not induction to learn. It attracts it's greater than fair share of trolls, distractors and disparagers. They on the most part exist with a 'crabs in a bucket mentality.' The only caveat is that it's open source, with the intent of extracting capital from wall st to fix main st problems and that the implicit agreement is to pay it forward and freely teach it to any whom agree to the ground rules and willing to do the work. It's probably one of the hardest nuts I've ever had to crack and certainly not for everyone. With all that said, if I remember correctly, 'Zen and the Art of Motorcycle Maintenance' laid the personal groundwork decades ago for me to come to the understanding that whether there is or is not a 'pattern' out there, it makes no difference in the meaningfulness we can create in our own lives.
I don't even back-test on my fully automated 'strategy' which is really just a collection industry philosophies, general rules of thumb, and well documented technical rules (collections of formulas). Markets evolve and this is why the 'typical' type of strategies I see (essentially just some basic rules 'tied' to indicators and account/trade management) will always fail over time. I have not yet once seen a completely reliable (accounting for all live trading variables) backtesting platform, and the further one delves into trying to create one on complex systems like ML - you see it is a futile effort. As previously discussed, how can you 'backtest' an AI made up of a neural net just like your brain is essentially. Must I really save many point in time models and then attempt to replay data back to ensure all point in time variables are the same? that would be an epic task X10 the effort of getting the models up and running in the first place. So if trading on beliefs....or developing strategies etc from gut feeling - then go live and let it guide your biofeedback you to the right questions to ask and then answers and then code. Run with it, you might be surprised at the results. It all starts with a creation 'loop'. As a developer i'm sure you appreciate the relationship between being able to clearly articulate a concept, process, action, observation etc in english and the subsequent ability to fully convey this in its entirety into code. Whilst kind of a moving target and can present steep learning curves - I prefer this approach because it feels genuine to me, like I truly creating and not copying... looking at the data through a predefined lens preventing me from seeing the big picture. Indeed this is why I see data as the king of the hierarchy really. Whilst I am a complete fan of @fan27 's work - I sometimes feel like maybe we are all approaching things from the wrong angle and making things harder than they are. Its like randomly generating 'strategies' until something 'sticks', and then when it does (if you care) trying to work out 'why' without any context to all the ones that did not. Things changed for me when I learn't the fact that you can essentially place trades randomly - 1 on each instrument - with 2:1 stop loss - have enough money to cover your trades - make 100 of them to provide enough statistical offset and how can you possibly lose?!. Yes its real... some guy pro trade on youtube explains all the 'casino odds' maths. throw in some account/trade management, some trailing stops and you are in business. Then hopefully it is not so hard to get an edge over 50% to really get things moving. It's a mindblowing concept really, but maths generally dont lie so I have explored and refined this concept now trading 47 instruments in auto. So maybe we also make things more difficult than they necessarily need to be also.
The first is basically the same way I already do iterative live development for my ATS. If this is true, that's a little spooky from a matrix / there is no time perspective isn't it.