Not sure the OP is requiring his system to be coded in NT8. And not sure either that NT8 would be the best choice of programming language based on what he wants to do. And about how complicated this could be, well, maybe it's just me. But manipulating data (time series) and applying some type of calculation, it's pretty basic programming stuff. And I think I can say that I know what I'm talking about...
Would you collaborate with a programmer if it meant sharing your system/research? If your main problem is that the existing code is slow and inefficient and your "secret sauce" is isolated as a set of parameters embedded in the code, perhaps you can change these parameters, give the new code to the programmer to improve, then put back the correct parameters after he is done.
The short answer is that the programmer would still need to put in work figuring out how to use it. For example, the call I posted above was the result of asking the right question, inputting the correct data and reading the output (although the model does generate the associated probabilities). Meaning it's not an automated system that I just turn on/off and it will generate trading signals without any user discretion. If that was the case it could be fully automated. In that respect, I think the programmer may need me even after re-coding the original program for further guidance and adding new features. FWIW, the probabilities for that specific scenario was > 90 %. Generally, when I say high probability outcomes or post predictions that's what the model produces. There are times when the model arrives at only 50/50 or 60/40 probabilities. In the ES journal I've often said during those times that I don't have any edge or read of the day ahead or the rest of the day. It's possible that this is because the model is incomplete or I haven't yet been able to figure out all the relevant questions/parameters. That's one of the things I'm still working on and where me and the programmer may work together to improve it. The programmer has extensive experience with machine learning, so that's one of the things we've discussed utilizing in order to see if we can discover new relationships or improve the predictive power/accuracy of the model.
In that case - I'd rather just have someone on UpWork do it for me for a fee, but I think that would be hard. I've had a few good experiences on UpWork where I hired programmers to do some improvements on the code, but invariably I received the same feedback. The application is poorly documented and the code is rather complex. So, most programmers aren't really interested in messing with the code at this point as they don't understand how all the parts inter-relate. I'm 100% convinced that as of now the system is working as good as possible with the current architecture. Only a complete re-write from scratch would solve the current issues. Current bottleneck is the speed of refreshing/calculating new data. Also, I added some new calculations on my own directly in Excel.
If it was only coding an indicator in NT8 I would definitely be able to handle it myself with some work, but I have a hunch it will take me a long time in order to code something similar on my own. The alternative is of course that I take time off from everything else to learn to code. And maybe I should? A rough outline of the program/model would be this: 1) Load intraday 1-minute data from a *.csv file. I use between 5-15 years typically. This is done EOD, but also intraday when the market is in session. 2) Calculate a large amount of metrics/statistics using this data. Mostly simple arithmetics and IF/ELSE loops. I wrote the pseudo-code/math for all of these myself. 3) Store this data in a database (SQL DB as of now). 4) Load this data (multiple data-sets) into a front-end for analysis. This is Excel as of now. One of the bottlenecks probably, although I think Excel should be fine enough if handled correctly. The original programmers understood C# well enough, but not Excel as the original workbooks were sh*t. 5) Data visualization is a separate side consideration that I handle elsewhere, but would of course be nice if that could be integrated in a complete package. There's a bit more to it than this, but roughly this is it. To me, this seems like a very complex task to handle for someone just learning to code. What do you think? You still think this is something I would be able to handle? I think @RedDuke said in another thread that "Hello World" is easy, but everything after that is a different matter. That's how I feel about programming.
Except for #5, which can be tricky depending on how fancy you want it, the rest is pretty straightforward.
I smile when I hear people say "Learn to code", as if it's something which anyone can learn to do in a few weeks. It's like saying "Oh, you're going to South America next month? Learn Spanish before you go". Sounds easy, but the reality is very different. I was an Oracle database designer/programmer for 25+ years, and a very efficient SQL, PL/SQL developer, so you'd think that learning to write Python scripts would be a piece of cake right? Not so - the whole methodology, and way of thinking, for relational databases (like Oracle) and object-oriented programming (Python) are totally different. I struggled with Python, and just hired someone from Upwork to write me some scripts that I wanted. On the other hand, I was able to pick up MS Access database very easily and built a database for myself. So, @Laissez Faire, don't assume that coding complex systems can be learnt in a few weeks, and don't beat yourself up if you struggle. By all means give it a go.
Waste of time! 15 years ago has nothing to do with today, keep it real. Imo, 2 years or maybe 3 years max of back data required. Why 2-3 years, because some stocks can go that long from running up into a drawdown.
Call me crazy, but I happen to think that there is only so much predictive power you can get from analyzing historical data and figuring out where price is going given that for it to get there, lots of trades need to happen that we know nothing about since its a future event. If predicting the weather, I think that can be fairly accurate because once you account for 90% of what causes weather to happen, its fairly linear as to what will happen next. Of course some unaccounted for force can enter the scene that drastically affects the forecast, but I think this is quite rare, and there isn't a weather genie that is purposefully manipulating things. But for trading, at any moment in time, no matter how bullish of bearish the market looks, it can turn on a dime because in order for price to get to where you think its going, lots of traders have to put on the trades that you think they will put on. Momentum might look strong, but its not a case of a ball that just needs to keep rolling downhill, its more like people need to keep pushing it up hill. If all of a sudden they stop pushing or take a break, the momentum stops, and we clearly don't know when they will stop pushing. So I guess what I'm saying is that there is a limit to how accurate of a forecast you can make, and squeezing out a few more percent of probability is probably a lost endeavor. I want to focus more on execution of trades since this is something that I can control, rather than the direction of the market, since I can't control this. But of course, I'm reminded of Qtrader who was posting 100% win rates for 20 trades in a row for NQ, so maybe there is a secret code contained within the data that he cracked. Anyway, I'm enjoying your thread and learning lots from what people are posting.
If you are making less than 30 percent a month and your account is small enough you dont saturate the market when you trade, I wouldnt worry about it. That said, you really should learn to do your own programming. Try reading the source until you understand it.