"Orthogonality and the DRY Principle"

Discussion in 'Strategy Development' started by harrytrader, Feb 11, 2004.

  1. http://www.artima.com/intv/dry3.html

    "Be Orthogonal
    Bill Venners: What does orthogonality mean and why is it good?

    Andy Hunt: The basic idea of orthogonality is that things that are not related conceptually should not be related in the system. [...] A change to one should not cause a change to the other. Unfortunately, we've seen systems throughout our careers where that's not the case. "

    ok it's in the field of software engineering but the principle is general and can also be applied to trading system. In fact for everything I have done in different fields I have never seen much difference and I always apply the same principles like the one described above. Nevertheless principles alone aren't enough they must be combined of course with the specific knowledge of a field that's why a pure methodologist can't succeed if he just contents with generalities but someone who has a good knowledge of a field can really empower himself with universal principles which are in fact just scientific principles disguised under other names. For example orthogonality obviously is derived from mathematics: it is well known in maths that a good representation of datas would use orthogonal axis (especially there is so called Multicomponent Analysis techniques in advanced statistics that uses this important characteristic of orthogonality).

    The beginning of the article is here

    Bill Venners: You say in your book, The Pragmatic Programmer (Addison-Wesley, 1999), that "programmers are constantly in maintenance mode." Why?
    Dave Thomas: All programming is maintenance programming, because you are rarely writing original code. If you look at the actual time you spend programming, you write a bit here and then you go back and make a change. Or you go back and fix a bug. Or you rip it out altogether and replace it with something else. But you are very quickly maintaining code even if it's a brand new project with a fresh source file. You spend most of your time in maintenance mode. So you may as well just bite the bullet and say, "I'm maintaining from day one." The disciplines that apply to maintenance should apply globally.

    The DRY Principle
    Bill Venners: What's the DRY principle?

    Dave Thomas: Don't Repeat Yourself (or DRY) is probably one of the most misunderstood parts of the book.

    Bill Venners: How is DRY misunderstood and what is the correct way to understand it?

    Dave Thomas: Most people take DRY to mean you shouldn't duplicate code. That's not its intention. The idea behind DRY is far grander than that.

    DRY says that every piece of system knowledge should have one authoritative, unambiguous representation. Every piece of knowledge in the development of something should have a single representation. A system's knowledge is far broader than just its code. It refers to database schemas, test plans, the build system, even documentation.

    Given all this knowledge, why should you find one way to represent each feature? The obvious answer is, if you have more than one way to express the same thing, at some point the two or three different representations will most likely fall out of step with each other. Even if they don't, you're guaranteeing yourself the headache of maintaining them in parallel whenever a change occurs. And change will occur. DRY is important if you want flexible and maintainable software.

    The problem is: how do you represent all these different pieces of knowledge only once? If it's just code, then you can obviously organize your code so you don't repeat things, with the help of methods and subroutines. But how do you handle things like database schemas? This is where you get into other techniques in the book, like using code generation tools, automatic build systems, and scripting languages. These let you have single, authoritative representations that then generate non-authoritative work products, like code or DDLs (data description languages).
  2. maxpi


    Orthogonality seems to mean planes at a right angle to each other in this context. He is saying that things that should not be related at all are sometimes indeed related when you go to debug your software.

  3. If there wasn't any relation you wouldn't have any bug in software and you wouldn't even have any system at all :). The problem with concepts in general is that they are so simple that one can tend to understand them too litteraly and that can indeed conduct to catastrophies. For example in quality engineering taking the 0 default concept litteraly will conduct to the opposite result: bad quality and cost because 0 default concept doesn't mean to have absolutly 0 default at any cost, it means tending towards 0 by a process that will be iterative and cost will not increase but on the contrary will decrease: that's what Deming and Shewart said about the false understanding of quality for dozen of years in America and Europe - that's why Deming has always hated this expression which was created by Philippe Crosby whom a friend of Deming I talked to as he had known both Deming and worked with Crosby at ATT said that it was because he had poor understanding of statistics and created this metaphore to help himself unfortunately it was spreaded and misunderstood all over the world. The same for orthogonality: if you take it too litteraly, for example when you normalise a database, if you apply mechanically without understanding the real purpose of normalisation it can conduct to very poor performance. In trading system where could this apply ? It is for example in searching the best combination of indicators. What you want is indicators that are orthogonal in the sense that they contribute for each part to information to your system and not give more or less the same information (basically too correlated with each other). But if you search for perfect orthogonality you will never find it and would never create something concrete. So a concept is useful for guiding like a map but you must of course take reality's constraint into account ... that's where the fun is in fact :D.
  4. Let's consider an other example where orthogonality in trading could be searched: the forum is untitled "Strategy" but it can be useful to make the distinction between strategy and tactics. According to Britanica

    "In the theory of warfare, strategy and tactics have generally been put into separate categories. The two fields have traditionally been defined in terms of different dimensions: strategy dealing with wide spaces, long periods of time, and large movements of forces, tactics dealing with the opposite. Strategy is usually understood to be the prelude to the battlefield, and tactics the action on the battlefield itself."

    By doing so we are "layering" the trading approach so that the tactics layer won't interfere too much with the strategic layer. We can consider for example that the strategic layer is the upper scale above our chosen trading scale and tactics the lower scales.
  5. Since I had engaged in automating the trading decision so as to be able to have a quality assurance about reproducibility of performance and low risk (see http://www.elitetrader.com/vb/newreply.php?s=&action=newreply&postid=429800) this is indeed a concrete problem and the technology I decided to adopt is not so astonishingly XML. XML can allow to structure the knowledge naturally into layers and to control more easily that they are "orthogonal".


  6. XML of course is not presented to the user interface, it is only in the background for data and expert system storage (the knowledge base). The user interface will be something like this (the style of the dialog can be more or less serious this is the fun version) :
    <IMG SRC=http://www.elitetrader.com/vb/attachment.php?s=&postid=430537>
    The merlin wizard gives the analysis.

    The man on the tea pot filled with gold transforms merlin's analysis into a trading plan.

    The snake represents the "poke yoke" (Luto's thread):

    The engine is not finished yet: for example the target can be refined further but it gives an idea.
  8. The advantage of XML is that it can be more intelligent than rough datas since the datas can be turned into a programming langage notably it can be used to build an expert system rules engine. Also the orthogonality can be assured by using context which in fact is a core concept of XML query langage since a command result will depend on the context it is applied upon. Operationally the context is defined as a parental node. As I have said already context is what is of huge importance: a market's pattern will probably fail if the context is badly appreciated. Market's context is difficult to appreciate because there are many variables to be taken into account and that can conflict one with another, but by using it through XML it can benefit from the concept implemented by XML.

    Many people stopped at concept levels because they don't know how to realise the concepts concretely. That's why idea is easy, realising them is more difficult. If you remember I have posted previous year this idea:

    "Building Learning into Everyday Work

    I am building tools for my learning group inspiring from this idea which is coherent within the higher framework I am using which is famous Deming's PDCA wheel (Plan Do Check Action) of CONTINUOUS progress. "

    So I have let a few months before maturing and realising the stuff but now I have a real thing prototype that works (it is not screencopy) and I'm hapy of the result. I'm not doing the job of programming myself so I had to wait for my associate or an other engineer to do the coding but as for the directives of conception it's me, it is also me that writes the rules of the expert system since only I have the knowledge to do so.

    What I intend to add is configuring the tool either to be in learning mode for people new to the method, either in expert mode for people already accustomed to the system.

  9. I'm going to add candlestick patterns also as they are very easy to implement. It's just for fun and curiosity since I don't need them.

    #10     Feb 13, 2004