Trend Following Journal

Discussion in 'Journals' started by danw, Dec 5, 2024.

  1. danw

    danw

    It's only our own funds, nobody else's funds under management but I backtested for $1.1m from memory (have changed since). Started live trading with slightly less assigned, then increased to slightly more.

    Margin/Equity is hard to say because I combine all my strategies in the same account. When one system is not active another may be more active but this system is the biggest user of margin. I've never checked margin that was for just this system alone (my data files don't currently have it).

    I don't have a fixed margin limit, my system must take all trades to be able to come as close as possible to the backtest. If it ever became a problem, just removing Bitcoin, Ethereum, VIX and maybe one or two other contracts would probably solve it.

    With IBKR portfolio margin I don't actually need cash to margin futures either (except for Sydney futures exchange bonds). So things like Tbills, gold ETF and other investments get included. I've seen some brokers segregate things which must be hard.

    I do track my SPX delta though, whenever it's over 100 I hedge regularly with some put options so that any stock market crash won't cause a big margin hit. Margin surprises are usually related to stock market events.
     
    #41     Jan 20, 2025
    ironchef, Picaso and newbunch like this.
  2. LBF

    LBF

    Hi Dan,

    Congrats on getting your strategy up and running! I am envious. I'm curious about a few things about your discrete trend following system. How do you prioritize signals if you have a finite risk budget? Is it based on strength of signal? If so, then does that mean you make forecasts, a la pysystemtrade? Also, do you do manage trades for volatility after the are put on or is this a Mulvaneyesk gunslinger model? If not, do you have hard risk overlays for concentration? Also, do you run an instrument diversification multiplier and/or trading rules diversification multiplier? if so, does it fluctuate with the number of markets and/or systems firing?

    Also, curious if you considered running a continuous system like Rob's or pst itself? I know Rob makes the argument (along with Quantica now it seems) that continuous systems earn better risk adjusted returns over time. I was disappointed to learn that i wouldnt be able to use RT to trade a continuous trading system, so I'm back to either running pst or continuing to develop my own simpler system.

    Thanks in advance!
     
    #42     Feb 25, 2025
    dWq54ZSnBqX44AW4 and newbunch like this.
  3. danw

    danw

    Hi again, I think we chatted on the Realtest forum?

    I got a bit lazy updating this journal tbh.. and wasn't much engagement here, not many people trading like this.

    A couple of things combined together prevents me from taking on too much risk:
    1. I put a filter behind all trades, they need to be on the good half of Seasonality, COT, or Basis from cash price to be bullish/bearish. None of these are perfect, and basis is a new thing I asked CSI to add it recently (after I saw an old version in their documentation). If you choose to upgrade, hold off until they've fixed the bugs maybe a few days. I've also been looking into fundamental factors, but that would probably come down to a per-market basis and is a huge amount of work for maybe little value
    2. A moving average trend filter, similar to how Clenow's book combined his trend + breakout into a single system
    3. Requiring a breakout on top of the above 2 filters
    4. Scaling into positions with different speeds/trade lengths, so a weak signal like a short range Bollinger breakout gets a 1/3rd size position. The weaker/faster system also has closer stops and easier exit rules so that it should also scale out first
    5. A correlation check at entry vs existing positions, reducing position size based on a correlation factor (this is also a fairly new change)
    6. Position limits for very similar instruments (eg US bonds) based on historical static correlations, in case dynamic correlations end up being temporarily different. Eg I wouldn't want 3x positions on every US bond market in the same direction at once. It's unlikely but just a safety check
    Mulvaney/Fixed style, staying in open positions until either exit rule or stop loss. However since it scales in/out via multiple strategies, some would argue it is not fixed position. It isn't scaled out based on volatility or correlation though, only based on bullish/bearish signals.

    Yes as per #6 above, but it's very wide and accommodating as I prefer the other factors to prevent it ever getting to that. A few times in the backtest they triggered and testing with them disabled affects the metrics like drawdown but not in an extremely dangerous manner. I still prefer them on, for the unpredictable future.
    I tried something similar to a diversification multiplier, where I would disable the requirement for the entry filters if say I have X number of long indices, and zero short indices, like a balance filter. The backtest for this gave very poor results and actually made things worse.
    Similarly I also tried disabling the filter if the symbol passes below a maximum correlation, but again it did very poorly.
    For sizing I don't multiply the sizing, rather I subtract from correlated items instead.

    I did consider it at one stage, but because I was already deep into Realtest with stocks and really love the rapid R&D process (I can test new ideas within minutes) it was always my preference rather than having multiple infrastructure projects to manage.
    Once the CSI integration was coming I jumped at the chance, and have since stopped manual futures trading except for a few macro positions.

    The other thing I don't like with Continuous is the need to always be connected to the market/api. I like the once/daily process, and for my stocks I have some that are only once/weekly and once/monthly.

    As for which systems earn better risk adjusted returns over time, I believe Rob's assertion on that was based on the Quantica discussion around positive skew not being included and that no amount of positive skew could compensate for a reduction in Sharpe? My memory is foggy because I was busy while I was listening to TTU but I think that was it.

    There's quite a lot of debate on TTU and other places about this, usually conceding that fixed position sizes make better absolute returns (at the expense of risk). Robs recent statement is the first time I heard it the other way around, though it may have been in reference to a better risk-adjusted return which is usually code for a "smoother ride" ie sharpe focussed. The arguments for sharpe are that a better sharpe with lower return can just be leveraged more until the return matches, but if you're inexperienced at quant work like me this can be dangerous - eg option selling has a great sharpe, and leveraging that can end in disaster. We can only guess the sharpe from the backtest, so leveraging it above what we'd use with a fixed position size isn't something I'm comfortable with.

    There are other reasons to consider non-vol adjusted systems to produce better returns in catching outliers. For example segregating any open profit and only measuring vol on the closed balance, combined with the safety step of not using open profit in sizing new positions. In realtest you can do this by disabling MarkToMarket or subtracting M2M from Equity in your sizing formula. These two steps allow you to run a portion of the account at very high vol, without risking the base capital. Not as suitable for managed money, but achievable for personal accounts. This is the method Rich Brennan and a few others discuss.

    If you believe in cutting losses and letting your profits run, as one of the principles of trend following that work (see Tom Basso's random entry test that is profitable simply by cutting losses short), then to take that to its extreme one should not adjust an open position because it is cutting profits instead of letting them run. In reality we need to take profit eventually though - so you either do it partially or completely. The use of vol in that situation is a tradeoff between factors like most things some do some don't.

    These are my preferences and opinions only, I'm not as experienced as any of the quants and fund mangers out there and I have no clients or services to sell. So I'm not going to assert or prove which method is correct, but this is what I choose based on the things I've learnt from others so far.

    Yeah it is kind of a fudge to make it work, you can do a few things:
    a) Use Reduce = True
    b) Use the dynamic sizing feature (like the clenow stocks example)
    c) Use multiple strategies

    At the end of the day it is a tradeoff, a platform will never be as flexible as coding something yourself from scratch.However python style development can be very time consuming at least initially, as can the backtesting unless you have some way to do vector based multi-threaded testing the way Realtest can do.
    To build what I have now, in python, might've taken me a year to get up and running or more rather than just a couple of months.
     
    Last edited: Feb 25, 2025
    #43     Feb 25, 2025
    dWq54ZSnBqX44AW4, Baron and newbunch like this.
  4. LBF

    LBF

    Yes, we first dialogued in RT recently!

    Thanks for the detailed response regarding your system. Very helpful in understanding the implementation details of your discrete trend following system. And once again congrats for getting it running!

    So its sounds like you are setting risk per trade based on ATR with a breakout type entry, subject to the MA Clenow filter, along with seasonality, idiosyncratic sector(groupings) concentration risks (like US bonds) and correlation overlay filters. Since you are casting a wide net with many markets, in your backtesting, what is the range of futures markets whewre you have a position on at any given time? Do you have hard risk limits for overall exposure (forgive me if i missed it)? Also, what do you do (if anything) if you become very concentrated, i.e. Cocoa in March and April 2024? Is this where you consider only realized gains for capital as the base for risk sizing new positions? I guess what I am trying to understand is your overall risk limit in your trend portfolio and the implications for ranking positions (if needed) to cut to the hard limit. Inversely, the more entries in markets and model timeframes, the more diversification. In Rob's system he uses an IDM and FDM to scale up positions to account for diversification in instruments and trading rules, Because his system is continuous, IDM and FDM are applied based number of markets and trading rules with forecasts (all of them). As a side note, in the Dynamically Optimized system, its unclear to me how this works. The net is extremely wide, 250 markets or so, but many fewer markets are traded resulting from the DO. This it seems to me would change the IDM.

    Also, it seems as if it would be easier to implement a discrete system than a continuous system. I may want to try this first using a manual trading approach similar to how it sounds you started. How did you determine the number of markets? I think I read in the thread that your capital is approximately $1M. In Rob's system (non-DO), this would limit the number of markets to around 48 instruments with a careful minimum capital inclusion test for each market. How did you approach this issue? You mentioned different timeframe trading rules, so presumably that's a minimum entry of one contract per market for each trading rule. With $1M of capital you would run up against some of the same constraints (albeit discrete vs continuous) as Rob's system, no?

    At the risk of asking too big a question for a forum and probably more appropriate over coffee, if you were to offer advice to get to implementation of the discrete system, what areas should be sorted out before going live:

    market selection (referring to the question above - how did you approach this)
    models and filters (must define)
    capital (must define)
    position sizing model (ATR based/Stops based on timeframes)
    backtesting/paper trading
    operational aspects - manual trading to start - broker idiosyncrasies/contract idiosyncrasies

    Thank you for sharing your experience!
     
    #44     Feb 28, 2025
    danw likes this.
  5. traider

    traider

    How do you know the system won't bankrupt you in a true black swan event not coverd by any backtests
     
    #45     Feb 28, 2025
    danw and MarkBrown like this.
  6. danw

    danw

    Good question! I'll answer this first as @LBF question is going to take more brain power than I have first thing in the morning :)

    Firstly, bankrupt is not the right word because I run it from a Company & IBKR did not require a personal guarantee. Also I'm not comfortable enough to ever put my full net worth into a company or trading account, I'm an investor first, a trader second. But it's still enough to make it worthwhile, and enough to piss me off big time if I lost it all.

    I'd say the question is how do I know the system won't blow up my account and cause me to dissolve the Company?

    Short answer is I don't, it's not possible to know that.
    There's no way to know this with Futures trading. Some would say trading long-only below the notional value of your account, but I'd say look at Crude oil going negative.

    So this coming Asteroid strikes earth and wipes out half of the Cocoa production in the world while you're short and the markets are closed, Cocoa gaps up 10x the next day before your stop executes.
    The only way to be sure is to not trade paper contracts like futures. You could run a complex options hedging strategy that costs half of your returns, but easier to just go buy ETFs instead.

    All we can do is accept the risk of blowup and put strategies in place to manage it (even if they hurt the backtest), and study the long track record of trend followers going back 50+ years to see how they handled the previous black swans. Research how backtesting handled those even when changing the start dates, parameters, and account size to cause the order and size of trades to run through different permutations.

    If I could plan around a future event, it wouldn't be a black swan. Thankfully trend following typically handles macro shocks better than many other strategies and often thrives on it - "Crisis Alpha" as Katy Kaminski calls it.

    This question reminded me of something from the Mulvaney interview, who also has no illusions about the risk and surely warns his investors as well:

    upload_2025-3-1_11-43-42.png
     
    #46     Feb 28, 2025
    LBF likes this.
  7. danw

    danw

    Yes, the average distance of my stops and exit rules is the multiplier I apply to the ATR to calculate risk $. Then I apply a percentage of the account, depending on how much I allocate to each sector (sectors being a manual grouping).
    Lastly I subtract a sizing factor based on the highest correlated open position.

    In the trend following backtest, the maximum number of markets with a position topped out at 68 markets out of a possible universe of 143 markets. It was 130 positions, so on average 1.9 entries per market. That was on 2018-01-30, and I note February 2018 had a 30% month in the backtest which would make sense if so many signals were firing due to trends.

    Looking at the trade list I can see a large number of them are micros and minis, which could explain why it was able to open them without hitting a limit on sizing due to correlation reductions that would have pushed anything too large under 1 contract. A lot of these are Indices and Bonds, which I allocate less risk than commodities.

    Since then I've added a shorter term trend, and a mean reversion strat. With these new strategies it makes it harder to estimate the max markets because these often trade counter-trend (and that's the idea of adding them). At the broker these positions actually reduce the contract count, eg currently I have a long term long on 1xMNQ while a short term short just opened 2 days ago on 1xMNQ also. This trade makes my net position is zero, however Realtest and Orderclerk show two positions active and keep stop losses on both in both directions (which would close out to keep just one position). My old backtest was much less likely to have positions in opposite directions so I ran that to check.

    It has no hard risk limit for total overall exposure, because if the system were to take on too much exposure I would consider the entry rules to be too loose and/or the exit rules to be too strict. Hard cutoffs worry me that they risk missing trends. I would rather smoothly reduce the risk by spreading it around in smaller positions or multi-timeframes.

    But I'm open to suggestions - if you have a good way to limit total risk, without skipping any position entries I always want to improve things if I can. Assuming the tradeoffs of such methods are worth it.

    For individual positions there is a risk limit, in case volatility becomes unusually low and ATR shrinks such that it would size a position too large.

    I don't do anything to override the open profit of a position, if it's not my original funds at risk I let it ride until it exits on an exit rule or on a stop loss. Having said that, it's not going to just wait and exit all after a 20% pullback - dividing the system into 3 different speeds (4 now) means the exit will gradually take profit. The shortest timeframe can even trade counter-trend, effectively taking profit in a way.


    This need for ranking is what I don't like about systematic stock trading, in a large universe of thousands of stocks often the result is highly dependent on setup scoring because no matter what sort of entry rule there will almost always be too many positions. Playing with scoring was another overfitting risk I had in stock trading. One of the main benefits of trend following for me is to be able to take every signal.

    If somehow the global futures markets were to become more popular, and open up 1000 contracts to us then this would become important. However with much less contracts I think it's better to curate them manually or in a way that limits positions of things that are virtually the same (eg London Cocoa and NY Cocoa you maybe want to just take the first one that signals). Also I limit exposure per sector, and check the correlations at entry time.

    I haven't looked at the rules behind the IDM and FDM, as it was mostly Rob's AFTS that interested me rather than the whole system design I was picking pieces that were useful to mine.
    But I do know the net of 250 markets used by the Dynamic is not all used - Rob has a good liquidity check that excludes a lot of them. Maybe they get liquid again, and markets do change every few years.
    I found the Dynamic optimisation interesting, but I can't be as confident in a method that is so new to me. When handing over to a system, confidence is critical and building it yourself helps.

    Possibly, unless you wanted to take pysystemtrade out of the box without many changes. I would need to change it too much and I prefer Realtests workflow.

    Manual trading - that would be extremely hard/annoying I think. When I used to swing trade, even just having 3-5 positions open and updating the stops each day could introduce typos copying from tradingview etc.

    There's nothing wrong with being "semi-systematic" if it gets you away from manual trading.


    When I added new markets I also increased my account balance a bit.

    Also I don't think it's as simple as just having a count of number of markets (if you're talking about a universe), when they are all so different in character and sizing. E.g. Japanese 10yr Bonds at $1m notional is very different to Micro Nat Gas at $3k notional. I simply experimented, by adding markets slowly and reducing the risk each time. To use micro contracts with a small allocation to indices is lower risk to have many small positions than to have a few of large commodities like Cocoa and Coffee.

    If 48 instruments is a count of active positions, and not the Universe (which I think you said was ~250), then that isn't too far off and is fairly similar. The day the backtest hit 68 markets was just a single day in a peak. A few other times it moved above 48 but the average is way below that.

    There are some contracts that it won't enter a position because it is too far below the size of 1 full contract, unless there are so few other positions open that the maximum correlation is very low. But I try and minimise this, by using micros and minis, and allowing rounding up to 1 contract if possible based on some rules.


    Getting comfortable before go-live is a very personal decision about how far you need to be in completeness of your system, and how much troubleshooting you're willing to handle while live.

    My goal was to get the MVP (minimum viable product) up and running, because I knew a lot of the required knowledge would only come after going live (and I was right). Especially things like Realtest functionality, feature requests, bug fixes that Marsten helped with all came from running the thing every day. I have a tech background, so I'm quite experienced at troubleshooting and fixing issues under pressure. Similarly I have a background in the crypto world, where you get rewarded from live experimentation eg running mining, nodes, yield farming etc. To launch this (with oversight) as an MVP was better than trading manually or paper trading. I understand not everyone is like this though and some need something closer to the final product..

    Adding markets is really time consuming. I have a long checklist, for example you have to check that IB even lets you place the order. Even if you can find it in IB, the multiplier might be wrong, or they might have regulatory restrictions for your home country. Sometimes they are just placeholders and they can't calculate margin.

    I can't remember the exact number of markets I started with but it was definitely under 100. I had the major US markets, and any important global markets that I had seen in Clenow's book and had known about previously (especially adding Euro markets).

    My initial method was to make sure all the sectors/factors were covered, and focussing on high volume popular markets. Micros and Minis were important.

    I'd suggest start with two timeframes that can scale in and out with a short term and long term, just to prevent the lumpy effect on return streams from a position exiting all on the same day.

    A lot of the functions I added were from thinking outside the box - for example the problem you mentioned about multiple timeframes possibly being too many positions for large contracts. A possible solution to this you could research try is:
    1) Assign a limit to the number of positions/timeframes for each market, based on $risk or some other systematic method
    2) Design a few sets of strategies, for say 3 contract size ranges. For example:
    a) small contracts could have 4 timeframes to scale in/out
    b) medium contracts could have 3 timeframes, short, medium, long.
    c) large contract could have 2 timeframes, evenly spaced short term and long term.
    They can still run together in one portfolio for a smoother end result.

    You're welcome, that's why I started this journal ;)
     
    #47     Mar 1, 2025
    corsair and LBF like this.
  8. danw

    danw

    Quick update on trading results - February started off ok then accelerated to an ATH result around 10th Feb. Then the whipsaws started and the last 10 days of Feb have been the worst, results are moving back towards zero. This is results since inception rather than YTD, as it's only been running 4 months.

    Now there's been way too many trades to show a market by market breakdown anymore, but you can probably guess the Long Softs comes from Coffee, and partially from Cocoa (that is now fully exited).

    Cocoa gave up about half the profit, since I only caught the end of the trend.
    Coffee has pulled back quite a few $ with no exits so far, if it continues next week I think the first exit will happen, maybe even the second if it keeps going.

    upload_2025-3-1_18-40-28.png

    Having a fairly long average hold time, meant I would have to wait quite a long time before seeing profit taking as long hold times and wide stops are more reliant on outliers.

    I decided recently to try shorter term as well to try and lock in some profits along the way, it is looking promising. I've reduced risk per trade slightly to fit in the shorter term trade counts that can potentially increase the number of positions.

    Also created some reporting that uses the broker CSV reports so that I can see the exact contract trades including real roll costs, rather than using simulated rolls (realtest assumes close to close prices and uses CSI roll dates that can change).

    I need more time to spend on reporting, but already starting to have fun with some charts:

    upload_2025-3-1_19-3-35.png
     
    Last edited: Mar 1, 2025
    #48     Mar 1, 2025
    dWq54ZSnBqX44AW4 likes this.
  9. LBF

    LBF

    This is great stuff Dan. I think I understand your system a lot better.

    Rob uses an "exogenous risk overlay" (see the chapter "Tactics" (one of the last chapters in AFTS but possibly the most important).

    This overlay doesn't fundamentally alter your signal generation or sizing approach at entry but instead applies an external multiplier to the entire portfolio's position sizing when certain risk conditions are unusually elevated. Specifically, it looks at four dimensions:

    1. Portfolio Risk Multiplier: Limits positions if the total expected portfolio volatility (estimated via covariance matrix) exceeds a certain historically high threshold, like the 99th percentile of historical volatility.

    2. Jump Risk Multiplier: Accounts for tail risks by recalculating risk using the 99th percentile of each instrument’s historical volatility, capturing extraordinary jumps in volatility.

    3. Correlation Shock Multiplier: Temporarily reduces overall exposure when correlations between instruments spike unexpectedly, especially if previously uncorrelated or negatively correlated positions suddenly move together. Again looking a the covariance matrix assuming the 99th percentile historically

    4. Leverage Risk Multiplier: Ensures total portfolio leverage doesn't exceed historically observed extremes, even when individual positions seem comfortably sized based on ATR.
    The way Rob implements this is to calculate these multipliers separately and then apply the most restrictive (lowest) one to scale down overall position sizes only during rare but critical high-risk periods (approximately 1%–5% of the time). This method doesn't skip or reject signals; rather, it reduces sizing across the board until normal conditions return, ensuring you continue to participate fully in trends without compromising the integrity of your signal-based approach.

    Your existing sector and correlation checks already go a long way toward managing idiosyncratic and group-level risks. However, introducing this type of external overlay would add an additional protective layer against broader, system-level risks—particularly relevant in extreme market scenarios.

    Curious to hear your thoughts on this or if you've previously explored something similar!
     
    #49     Mar 1, 2025
    danw likes this.
  10. danw

    danw

    Tactics my favourite chapter.
    I just pulled out AFTS and realised my bookmark is on the 2nd page of the risk section, I never got around to finishing it.

    I better read it and see what I can get out of it.

    Any ideas on the best way to reduce sizing across all positions in RealTest?

    Initially the only thing I can think of would be to exit 1 strategy per market, but which strategy? Exiting the shortest term would risk closing counter-trend positions that dampen volatility.

    Possibly some other method like ranking by correlation and "Lack of signal", to exit those that are moving towards the stop loss and are also the highest correlated might be best. Like relative momentum applied to the exits, with a correlation input.
     
    #50     Mar 1, 2025