This is true for all learning system approaches (NN/back-prop, genetic, support vectors, etc.). Their inability to find trading methods that are known to work on the data used for training is amazing to watch.
You put up a lot of assumptions, conjecture with very little to no evidence in all you are saying, whether it be here re Genetic Program or re Apple OS, or Linux. Of course does the probability increase to make money in financial markets the more you surround yourself with people excellent in their own fields. That includes theoretical scientists as well. And I 100% agree you need practitioners as well to get things to work, BUT where I disagree with you is that programmers are your solution to get your from inception of a vague idea to a functional and profitable trading algorithm. Programmers are just one group of workhorses you need to get all the hard work you put into, implemented. What you really need, especially in the beginning is those with ACTUALLY risk-taking experience in combination with an excellent knowledge and appreciation for math and stats. At least to get a GP-project off the ground. But hey, good luck, if you feel there is merit to it then by all means dive all in. I am not gonna be part of it because for me rotating couple factors and adjusting system parameters does not make for a good strategy, I have long enough experience in this field to tell you that GP won't get you anywhere with a mindset that you evolve couple source parameters to indicators and voila, out comes an alpha generating strategy.
Exactly correct, the data-mining/snooping bias is a big issue in almost all back-testing where the historical data is not properly used. I also think it is extremely challenging to solve. Some smart people out there may know. Historical patterns do repeat again and again somehow in some way and few people really knows when and how these patterns will repeat next time.
Interesting. But it looks like a too big project if the goal is to development a bloomberg-like terminal platform. It is possible to implement certain useful core part of a trading platform. What is the feature list you have in your mind? A few more points that may also help: C++ (much more productive than C without performance concern); Scripting language (perl, bash, or python for better job/task level parallelism, especially efficient in Linux SMP arch); use LLVM for cilk use GCC for openmp (but still buggy) vectorization optimization (most hpc computers have built-in vector hardware)
The division of a team into "brains" (MBA, BS, PhD) and "workers" (programmers) is very interesting. I see it very often with mistrust between those two and deliberate compartmentalization of the access to information. It usually leads to failure and it is outdated concept but still very common. I see it very often as end product created by programmers who deliberately did not receive in depth clarification of underlying logic, did not have the opportunity to see the big picture and were working on limited set of procedures put together in final stages by somebody else. This type of environment is usually created by incompetent business or science people who think that they have special abilities and monopoly on wisdom. The results are often amusing. Modern programmer is no longer a coder but often holds degree in science or engineering and is able to leverage his background and experience with discipline and logic of programming concepts. It is better to have a team of equals than team of brains and workhorses. Everyone is pointing at data snooping as an excuse for failure of GP to create good and lasting strategies but in my opinion the issue is deeper. People are trying to use common strategies and parameters that are failing individually and then put them together in various combinations and permutations in hope that those magically refine itself in the process and create super strategy just by uncovering hidden patterns in massive data bases of time series. This is the reason I am skeptical about it because it is very superficial and generalized idea trying to resolve fine grained problem with shotgun approach.
great example (without addressing the poster directly) of a project set to fail: a) even the mere mentioning of wanting to create a "bloomberg-like" terminal platform is ridiculous. b) Why not starting in R or Matlab as I initially suggested? Or why not at least stating that C# is much more productive than C++ which is why most banks nowadays spend much more investments in porting libraries to C# for pricing purposes as well as front-end purposes than they spend on maintaining C++ projects. I would be hard-pressed to hear of a completely new project that is based on C++ within investment banks (unless we are talking high frequency trading or low-level connectivity APIs). c) And here we have the perfect example of Linux "Chaos World": What do you need to get the basics together? Scripting languages, LLVM or cilk (whatever that is), GCC (oh still buggy? Never mind me...lol), vectorization optimization (never heard of the basic principle to first get it running, then you get it running bug-free, and then you optimize?). It makes me smirk, the Linux crowd here is so obsessed with wanting to do it all the anti-MS way that they propose overengineering things EVEN BEFORE they even have mentioned a single Genetic Programming concept. Can you listen to yourself for a single moment? The task is to analyze a time series object and generate trading strategies with the help of genetic programming!!! Why do you start talking about software libraries that have not the slightest to do with GC? Why is there no discussion about + what type of strategies are to be researched? + how parameter optimizations are to be performed? + how GP even comes in play during the strategy research process + how to reduce/omit data-mining bias and other optimization fallacies? Only and only then you should start thinking on what platform you want to run the project? And while you are at it, R/Matlab should be your starting point for proof of concept. And why should it be on Linux, how does Linux solve the problem better than working on Windows. Most people run Windows not Linux. But by all means, R runs on both OSs , so I find it stupid to suggest C++ before having ever mentioned R for this project. I am opinionated, I am sometimes wrong and am happy to admit when being presented with counter facts, I am at times overly excited, at other times overly critical, but the only thing I do not bend to is nonsense and ideas that just run counter even basic logic.
there is a reason why not a single IT or technology project would succeed without project manager. In fact, most IT companies place a huge effort and investment into improving team and team leadership techniques, a multiple of the investment goes into that than, for example, improving individual programmers' skill set (which in 2014 each one on his own is expected to be responsible for, anyway). Vicirek, I replied to a lot of your posts, but could it be that you are a little thick in terms of comprehension skills? Each time you seem to either accidentally misunderstand or maybe it is intentional aside you not even fully grasping the thread title. I never even hinted at programmers belonging to "brain or worker camps", I said projects do not work out if a programmer starts thinking which language to use and on which OS to run instead of first bringing in a SPECIALIST who actually understands the underlying problem at hand and goes about drafting sketches and prototypes and presents ideas how to solve the underlying problem. Programmers are supposed to come in much later to implement the prototype. You can have 100 programmers in the room (or 100 business men if that calms your nerves) but if nobody has the slightest idea of how GP works then you are wasting time on a lost cause. This was my last reply to you, because I start to feel it is a waste of time to argue with someone who does not seem to be able to abstract but instead paints the world from his limited "linux vs windows, open-source vs Microsoft, programmers can solve all problems" angle (and that applies to anyone with specialization unable/unwilling to look outside his limited box/world).
All above sounds like old good big and fat bureaucracy intimidating programmers and free thinkers. Specialist who will reveal his prototypes and sketches to programmers for implementation - that is a good one - and what is suppose going to come out of this? Programming is fluid and dynamic process and not rigid preset organizational chart. Thank you for not replying.
me one of those people. still with a good wish. just believe the magic is hidden in the huge historical data. thanks for summarizing my approach. appreciated your post.