The best way to detect trend market/non trend market in programming?

Discussion in 'Automated Trading' started by GloriaBrown, Jan 24, 2021.

  1. Any idea to share?
     
  2. Girija

    Girija

    Find the slope of the line connecting high and low. You need to define what you view as non trend first.
     
  3. terr

    terr

    I would say fit a linear regression line through the period in question, then check what the slope is (in % per candle period).
     
  4. The best way for you (based on my reading of your recent posts, some things you have said therein, and of course the way you have posed this statistics/programming question) is to look into integrating an existing library such as TA-LIB which has bindings for various programming languages (Python, Java, C#, Perl, etc).

    The vast majority of statistical studies will not deliver on the expectations of users who either do not get what they offer or otherwise want from them that which they are not designed to deliver - implementing such things from scratch only to be disappointed therefore would be a waste of time.

    TA-LIB has at least TWO obvious studies you might be interested in (TA-LIB function list):
    ADX - Average Directional Movement Index
    HT_TRENDLINE - Hilbert Transform - Instantaneous Trendline

    you might seek to explore these studies through your existing trading platform if they are available (ADX or ADI is almost always included in platforms that have things like EMA, MACD, RSI, etc).

    Platforms that use TA-LIB behind the scenes don't necessarily expose all of the functionality but I know JForex from Dukascopy Bank has been moving slowly bringing in more and more.
    A demo account with them will get you access to the platform - what you'll be looking for is to see if they have implemented any chart overlay studies revolving around Trends or Trendlines - if they have - more likely than not they are doing it using TA-LIB behind the scenes since over 100+ of their existing indicators already work this way.

    Came across this MT4 AutoTrendline indicator which looks quite neat:
    there are several language implementations of functions allowing you to produce a linear regression type of trendline against some points in an array
    https://www.best-metatrader-indicators.com/automatic-trendlines-indicator/



    INSIST ON IMPLEMENTING ADX FROM SCRATCH?


    Best way is to study the 'specification'... ADX source code looks very different depending on whether it's relying on pre-existing stuff like smoothing/averaging functions or if it's truly an implementation of everything needed to make it work (TA-LIB ADX function in C, ADX function in Python #1, ADX function StackOverflow suggestions in Python) - it's best to just read the specification to understand what is going on then figure out how to write (or get) the various bits.

    Quick Overview of ADX and uses:
    ** MUST READ in regards to your actual trend detection question **
    there is no "best" way just one more way for everybody to dispute the efficacy of
    https://www.investopedia.com/articles/trading/07/adx-trend-indicator.asp

    Want to implement it, here is the spec below sourced from Investopedia:
    https://www.investopedia.com/terms/w/wilders-dmi-adx.asp

    The average directional movement index (ADX) was developed in 1978 by J. Welles Wilder as an indicator of trend strength in a series of prices of a financial instrument. ADX has become a widely used indicator for technical analysts, and is provided as a standard in collections of indicators offered by various trading platforms.

    What Is Wilder's DMI (ADX) Indicator?
    Wilder’s DMI (ADX) consists of three indicators that measure a trend’s strength and direction. Three lines compose the Direction Movement Index (DMI): ADX (black line), DI+ (green line), and DI- (red line). The Average Directional Index (ADX) line shows the strength of the trend. The higher the ADX value, the stronger the trend. The color of the lines can be altered, but black, green, and red are the default in most software.

    The Plus Direction Indicator (DI+) and Minus Direction Indicator (DI-) show the current price direction. When the DI+ is above DI-, the current price momentum is up. When the DI- is above DI+, the current price momentum is down.


    Key Takeaways
    • The DMI is a collection of indicators including +DI, -DI, and ADX.
    • Both +DI and -DI measure up and down price movement, and crossovers of these lines can be used as trade signals.
    • ADX measures the strength of the trend, either up or down; a reading above 25 indicates a strong trend.
    The Formula for Wilder's DMI (ADX) is
    [​IMG]

    How to Calculate Wilder's DMI (ADX)
    The indicator has multiple components. Here's how the components are calculated.

    1. Calculate +DM, -DM, and True Range (TR) for each period. Using 14 periods is common
    2. Use +DM when Current High - Previous High > Previous Low - Current Low. Use -DM when Previous Low - Current Low > Current High - Previous High.
    3. TR is the greater of the Current High - Current Low, Current High - Previous Close, or Current Low - Previous Close.
    4. Smooth the 14-period averages of +DM, -DM, and TR. The TR formula is below. Insert the -DM and +DM values to calculate the smoothed averages.
    5. First 14TR = Sum of first 14 TR readings.
    6. Next 14TR value = First 14TR - (Prior 14TR / 14) + Current TR
    7. Divide the smoothed +DM value by the smoothed TR value to get +DI. Multiply by 100.
    8. Divide the smoothed -DM value by the smoothed TR value to get-DI. Multiply by 100.
    9. The Directional Movement Index (DX) is +DI minus -DI, divided by the sum of +DI and -DI (all absolute values). Multiply by 100.
    10. To get the ADX, continue to calculate DX values for at least 14 periods. Then, smooth the results to get ADX
    11. First ADX = sum 14 periods of DX / 14
    12. Later values, ADX = ((Prior ADX x 13) + Current DX) /14

    What Does Wilder's DMI (ADX) Tell You?
    Wilder’s DMI shows the strength of a trend, either up or down.
    According to Wilder, a trend is present when the ADX is above 25.
    DMI values range between zero and 100.

    If DI+ is above DI-, an ADX reading of 25 or higher indicates a strong uptrend. If DI- is above DI+, an ADX reading of 25 or higher indicates a strong downtrend.

    The ADX may stay above 25 even when the trend reverses. Since ADX is non-directional, this shows the reversal is as strong as the prior trend. Traders may find readings other than 25 are better suited to indicate a strong trend in certain markets.

    For example, a trader might find that an ADX reading of 20 provides an earlier indication that the price of a security is trending. Conservative traders may want to wait for readings of 30 or above before employing trend following strategies.

    ADX Value Trend Strength
    0-25: Absent or Weak Trend
    25-50: Strong Trend
    50-75: Very Strong Trend
    75-100: Extremely Strong Trend


    Trading with Wilder’s DMI
    There are a number of ways the DMI can be used to trade, in addition to the general guidelines discussed above.

    DI Crossovers
    Traders could enter a long position when the DI+ line crosses above the DI- line and set a stop-loss order under the current day’s low, or below a recent swing low. When the DI- line crosses above the DI+ line, traders could place a short position with a stop above the high of the current day, or above a recent swing high. Traders could use a trailing stop if the trade moves in their favor to help lock in profits.

    Irrespective of whether the trader takes a long or short position, the ADX should be over 25 when the crossover occurs to confirm the trend’s strength. When the ADX is below 20, traders could use trading strategies that exploit range bound or choppier conditions.

    DI Contractions and Expansions
    The DI+ and DI- line move away from each other when price volatility increases and converge toward each other when volatility decreases. Short-term traders could enter trades when the two lines move apart to take advantage of increasing volatility. Swing traders might accumulate into a position when the lines contact in anticipation of a breakout.

    Traders should use Wilder's DMI in conjunction with other technical indicators and price action to increases the probability of making profitable trades.

    Example of How to Use Wilder's DMI (ADX) Indicator
    The following chart shows Shopify Inc. (SHOP) with both trending periods and less trending periods. -DI and +DI crossover multiple times—potential trade signals—but there is not always a strong trend present (ADX above 25) when those crossovers occur.

    If the +DI is already above the -DI, when the ADX moves above 25 (or 20, 30) that could trigger a long trade. These are marked with up arrows.

    If the -DI is above the +DI, when the ADX moves above 25 that could trigger a short trade. These are marked with down arrows.

    Contraction periods are also marked when the +DI and -DI lines become squished together. These are contractions in volatility, which are often followed by periods of larger, trending movement where the lines separate again. Breakouts from these contractions (blue boxes) may present trading opportunities.

    [​IMG]
    Image by Sabrina Jiang © Investopedia 2020
    The indicator is susceptible to creating multiple false signals, meaning the price doesn't end up moving in the same direction as the crossover dictates.Therefore, the indicator is best used in conjunction with other forms of analysis, or with additional filters applied to the signals generated.

    Limitations of Using Wilder's DMI (ADX)
    The indicator is looking at past data. It may lack predictive value in forecasting future price moves. The indicator lags and will therefore tend to indicate trend changes after the price has already reversed course. This could lead to some trade signals occurring too late to be of use. This can also occur with the ADX reading. A reading of 20, or 25, or 30 doesn't mean that trend will persist. Many trends will fizzle out after reaching such a reading. The indicator can't predict a trend will continue, only that the security trended recently.

    If using the indicator for signals, there will be whipsaws. Whipsaws occur when the indicators criss-cross back and forth, resulting in multiple trade signals that produce losing trades.
     
    Last edited: Jan 24, 2021
  5. My experience with adx is bad. It is not a good design indicator and so laggy. It shows stupid signal when raise trend suddenly becomes drop trend.
     
  6. Turveyd

    Turveyd

    Don't see anything in indicators, prefering a 40Lwma currently and 12Lwma for quicker.

    Reading slope is tricky, sometimes looks strong but it's just chart scaling and it's weak and vice versa.

    Good luck, your going to need it :(
     
  7. Thx for sharing. How do you tell it is choppy/trend with these two lines?
     
  8. Turveyd

    Turveyd

    Still working on next weeks trial, so subject to change LOL

    Currently back testing......

    40Lwma
    20Lwma High and Low seperately
    Envelope 20lwma High +0.03% ( bottom line hide )
    Envelope 20lwma Low -0.03% ( Top hide )

    Chop is when 40 and 20 are sideways, then it'll stay pretty well within the +0.03% Range, not perfect mind.

    Better waiting for direction, then joining, SL outside the 0.03% range and try to let it ride.

    Looks pretty good, so far, not as many trades as I'd like mind.
     
  9. 1 minute data?
     
  10. There is a reason I only have 14 posts in here or interact minimally with other 'traders' - this industry is full of sketchy opinion delivered with so much incontestible certainty that many not willing to pursue creative and exhaustive investigative approaches end up being thrown WAY off.

    No statistical study or indicator will deliver alpha in isolation - not one of them is designed to do this.

    No statistical study or indicator applied to a single time-frame will yield a picture you can retain high confidence in - you can slice or sample a timeseries into various windows showing you something completely different for the exact same pricing series.

    Every statistician asked to render a trendline against a price series can do so - that's statistics - whether that line corresponds with the prevailing market trend in fact: supply, demand, volumes, hidden info, external factors; will require ADDITIONAL INFORMATION if you are seeking high confidence.

    My last £150k+ of manual profits on an iPhone were driven by the following indicators used in unison and studied over H4, H1, M30, M15 and M5:

    [​IMG]

    Everything in this screenshot has been derided and rejected by somebody :confused:

    and yet exhaustive research for years yielded something I can now use 'on sight' from a mobile phone to open up positions even while multi-tasking (£1200+ friday on a slow Spot Gold while writing code for my tech-startup).

    While seeking to automate my approach, code execution exposed unconscious 'hunches' I seem to apply when working manually: esp. my view on whether I feel the daily high/low has been reached on ranging volatile instruments.

    In order to model this 'hunch' and other factors baked into a price series, I ended up taking an approach that grabs price and indicator inputs across multiple timeframes, then a number of statistics studies (stuff like this) is generated against this data, then the whole lot is ran into a machine learning model.

    The model itself is based on XGBoost used on a tree of XGBoost models that are each trained on creative variants (subsets and combinations) of the original input data.

    Many models out there will input simple OHLC price data, my pre-process converts simple OHLC data from multiple timeframes into 328 items of input for any tradeable instrument.

    The final model that processes these 328 input points comprises over 12,000 conditional statements (ordered tests) against these inputs in one massive tree.

    I have never read anything about the approach I take - not in journals, not in dissertations, not in forums, not in chats - I derived my approach by just continuously experimenting creatively and refusing to give up or yield to the strong opinion of others:

    a) Indicators don't work.

    b) Algos don't work.

    c) Machine Learning doesn't work.

    sure, I can see the pattern here...
    so, by a process of simple elimination, we are left with what does actually work? :)

    BE VERY CAREFUL - THIS IS AN INDUSTRY WHERE THE VAST MAJORITY CANNOT SURVIVE - PRIMARILY DUE TO WORK ETHIC BUT ALSO THE DEADLY WEIGHT AND OVERSHADOWING SPECTRE OF UNFOUNDED PRECONCEPTION AND SPECULATION
     
    Last edited: Jan 24, 2021
    #10     Jan 24, 2021
    yc47ib and Justrade like this.