I agree with your statement in regards to the math that is actually used in quant trading. If you can do basic arithmetic and log and exponents you are good to go. But that math that is behind the simplified models in lets say MCMC is a little more intense. I also agree with the statement that most retail traders lack the skill and some juts dont want to put the effort in to learn these skills. I am not saying I know the right way, but being able to see the model that has been developed in simple terms such as IF SPY is up in the last 3 5 minute bars AND RSI is less than 20 buy on the open of the next 1 minute bar is something that everyone can understand. I believe most momentum, mean reverting or technical trading models can be clearly defined and understood this way. If you are trying to create some ultra high frequency trading strategy that is taking advantage of the market microstructure then yes this is not going to work for you. But then again you are not trading retail. I would love to hear what you would suggest is the right way? I mean that sincerely.
Most of the things I do are very simple in terms of the methodology. I use a lot of linear models, a little bit of derivative math (nothing beyond vanilla Black/Scholes). The most complex thing I use for production trading is principal component analysis and there are several occasions where I've used various optimization techniques for parameter selection. Recently I started tinkering with robust regression and other stuff in machine learning library because I am bored. Even with that included, all of the required math is at 10th grade level and anyone with a college degree should be familiar with it already. This reminds me of this podcast: https://www.thetalkingmachines.com/episodes/ai-safety-and-legacy-bletchley-park. They are talking with Nick Patterson (I think the chat with him starts at 20 min mark or something), he used to be one of the principals at Rentech. I quote: "The most important type of data analysis is to do the simple things right. Here's a non-secret about the things we did at Renaissance. In my opinion the most important statistical tool was single regression. One target, one variable... We have the smartest people around, string theorists from Harvard, and they're doing single regression. Is this stupid? Should we be hiring stupider people and paying them less. The answer is no." Are you asking "what commercial software would be a good project?" or "how do I think one can get good at quant trading?". Those are very different questions.
Can you share what is the right way to do single regression? Linear non linear? How to make it robust to fitting errors? Would love to hear your thoughts... Are you asking "what commercial software would be a good project?" or "how do I think one can get good at quant trading?". Those are very different questions.
I think you are asking tongue in cheek, but I'll give you a serious answer. It really boils down to several common sense decisions that you make along the way. First one is "what you trying to discover and what are your priors?" - are you looking to predict something, get hedge ratios, validate a parametric model or see if price series are co-integrated. Second one is "what do I want to regress?" - i.e. deciding on the linear or non-linear transformation of the data, data cleaning, treatment of the outliers, application of weights etc. Finally, you want to decide on how to integrate the results into your trading process (which is usually the most difficult part).
Thank you for your answer. Rest assured that I'm serious about asking for opinions as trading is really competitive and I think it's bad to waste an experienced person's time with trivialities. So far I've only been applying linear regressions. Logistic regression could possible be applies when mapping input variables to a single binary decision.