/Users/raylai/Library/Caches/org.R-project.R/R/reticulate/uv/cache/archive-v0/n27kI_pu2kgd7cyXmF2Fh/lib/python3.11/site-packages/backtesting/_plotting.py:55: UserWarning: Jupyter Notebook detected. Setting Bokeh output to notebook. This may not work in Jupyter clients without JavaScript support, such as old IDEs. Reset with `backtesting.set_bokeh_output(notebook=False)`.
  warnings.warn('Jupyter Notebook detected. '

Loading BokehJS ...

3.11.11 (main, Mar 17 2025, 21:33:08) [Clang 20.1.0 ]
/Users/raylai/Desktop/github/raylai-co-quant-blog/posts/foundation-of-trending-system-moving-averages

Premise of a Trend Following System

Trend following strategies have been the easiest to execute and it works for different markets across many years. My previous blog post has shown that most markets are trending over long period of time, at least it seems hold in longer periods, despite the shocks and corrections in between.

The rationale of a Trend Following system is to capture the “big fish” while taking small losses. If the market trends, it is likely to go far and we hope the system to capture it. Otherwise, we will be death by thousands cuts.

This also sheds the light to the first major component of the Trend Following System - Whether we can detect if the market is trending with certain confident level.

The testing universe

I will be using a range of equity and bonds ETF, Commodities Futures and Forex futures for testing.

universe = [
  "IYY", "SPY", "QQQ", "IWM",
  "FXI", "EWJ", "VNM", "INDA", "EWY", "EWT",
  "EWQ", "EWI", "EWG", "EWU",
  "ARGT", "EWW", "EWZ",
  "SHY", "TLT",
  "GC=F", "SI=F", "HG=F", "CL=F", "NG=F", "HO=F", "RB=F", "CT=F", "ZC=F", "ZS=F",
  "ZL=F", "ZW=F", "SB=F",
  "6A=F", "6B=F", "6C=F", "6E=F", "6J=F", "6S=F", "6N=F", "DX=F"
]

import yfinance as yf

tickers = yf.Tickers(universe)

A reminder on the ideal trading system

It is good to remind myself here the goal of my ideal trading system:

I will create a trading system in six months, one that trades the ETFs, returns an average of 20 percent per year, with a maximum drawdown of 5 percent. Target Information ratio would be 4.

Things I like	Things I don’t like
Trade ETFs only	Over-complicated strategies (Pairs Trading, complex options, etc)
30% annual return, 5% MDD (I am really risk averse and don’t like losing - happy to not achieve 100% per year, but need to limit drawdown)	Average down to losing positions
Trade daily timeframe - I am a part time trader and I only have 1-2 days a week to generate trading signals	Holding position for too long (Over 40 days)
Average Trading Period - Not more than 30 days (See profit take profit)
Swing Trading style
100% mechanical - no human intervention in decision making

Of course, in a very long run, markets tend to trend. However, it could also mean I would need to take heavy drawdown. I don’t think this suits my style and personality. I would rather give up a 10x growth opportunity but the maximum drawdown is over 50-60%, for a trade-off of 30% annualised growth with draw down close to 5-6%.

Also, my ideal trading timeframe is not more than 30 days. The shorter the better as you can’t lose anything when sitting out of market.

Let’s see what we can do.

Trade Identification Tools

Trend trading works when the market is trending. It doesn’t work in markets that are not trending.

Trend Following system is as simple as that. So, how to detect trends that we can ride on? That is the question.

A trend is a relative concept. It is relative to the trader’s time horizon, and it is relative to the amount of noise and price swings that are acceptable within the trending period. Ultimately, a trend exists if you can profit from the price moves using a trending strategy.¹

We will start from the 3 basic ways to identify trends:

Moving averages (Generally expecting 30% - 35% win rate, with 2.5 - 3 R/R)
1. Moving Averages turning up or down
2. Price crossing moving averages
3. Bands and Channels
Breakouts from n-day new High or n-day new Low (Generally 60% win rate for long term B/O, but may be only a few trades available)
Linear Regression slope

1. Moving Averages Trends

Entry Rule: Buy when Moving Averages turns up \(\text{MA}_t > \text{MA}_{t-1} > \text{MA}_{t-2}\)

Time of Entry: Next Open

Testing Range: From 30 days to 120 days - [30, 45, 60, 100, 150, 200]

Date Range for test: Since 2000 - 2025

Markets: Covering Equity ETFs, Commodities Futures, Currency Futures, Bonds ETFs

Exit Rule: When Entry Rule violated \(\text{MA}_t < \text{MA}_{t-1} < \text{MA}_{t-2}\)

Profit taking and Stop Loss: No rules

Position Sizing: Single Unit or Single Contract

Distribution of returns using a 45-day moving averages

Code

from strategy.MovingAverageStrategy import MovingAverageStrategy

Code

start = "2000-01-01"
end = "2025-03-30"

results = []
bt_instances = []

for ticker in universe:
  
  data = tickers.tickers[ticker].history(start=start, end=end)
  
  bt = Backtest(data,
                MovingAverageStrategy,
                cash=10_000)
                
  stats = bt.run()
  bt_instances.append(bt)
  
  results.append(stats)

Show the code

returns_trend = list(map(lambda x: x["Return [%]"], results))
trades_trend = list(map(lambda x: x._trades, results))

fig, ax = plt.subplots()

returns_df = pd.DataFrame({'Universe': universe, 'Returns': returns_trend})
returns_df.sort_values("Returns", ascending=False, inplace=True)

sns.barplot(returns_df, x="Universe", y="Returns", ax=ax)

ax.tick_params(axis='x', rotation=45, labelsize=8)
ax.set_title("Returns from 2000 to 2025 by using 45-day MA")

plt.show()

Code

ind = universe.index("ZS=F")
bt_instances[ind].plot()

/Users/raylai/Library/Caches/org.R-project.R/R/reticulate/uv/cache/archive-v0/n27kI_pu2kgd7cyXmF2Fh/lib/python3.11/site-packages/bokeh/util/serialization.py:242: UserWarning: no explicit representation of timezones available for np.datetime64
  return convert(array.astype("datetime64[us]"))

GridPlot(

id = 'p1337', …)

Code

ind = universe.index("ZC=F")
bt_instances[ind].plot()

/Users/raylai/Library/Caches/org.R-project.R/R/reticulate/uv/cache/archive-v0/n27kI_pu2kgd7cyXmF2Fh/lib/python3.11/site-packages/bokeh/util/serialization.py:242: UserWarning: no explicit representation of timezones available for np.datetime64
  return convert(array.astype("datetime64[us]"))

GridPlot(

id = 'p1702', …)

Code

ind = universe.index("ZW=F")
bt_instances[ind].plot()

/Users/raylai/Library/Caches/org.R-project.R/R/reticulate/uv/cache/archive-v0/n27kI_pu2kgd7cyXmF2Fh/lib/python3.11/site-packages/bokeh/util/serialization.py:242: UserWarning: no explicit representation of timezones available for np.datetime64
  return convert(array.astype("datetime64[us]"))

GridPlot(

id = 'p2067', …)

Show the code

trade_return_histograms = []

for trade in trades_trend:
  trade_return_histograms.append(trade["ReturnPct"].value_counts(bins=20).sort_index())

# Construct a heat map
all_hist = []
for symbol, hist in zip(universe, trade_return_histograms):
  d = (pd.DataFrame(hist)
        .reset_index(drop=True)
        .reset_index(names="Bin"))
  d["symbol"] = symbol
  
  all_hist.append(d)
  
all_hist = pd.concat(all_hist)

fig, ax = plt.subplots(figsize=(12, 15))

sns.heatmap(all_hist.pivot(index="symbol", columns="Bin", values = "count"),
            ax=ax,
            annot=True,
            linewidth=.5,
            fmt='g',
            annot_kws={"size": 8},
            cmap="Blues")
ax.tick_params(axis='y', labelsize=8)
ax.set_title("Returns distribution (20 bins) across assets")

plt.show()

Comparing trend length for different asset classes and their performance

Because different asset classes have different noise at the period, we will compare the last 10 years performance of trading these asset classes by using different length of moving averages to understand their pattern.

We will test 5, 10, 20, 40, 80, 120 day of moving averages and to evaluate their Total P/L, Profit Factor, and Number of trades.

No position sizing is considered. They are either entered 1 unit or 1 contract.

We will be using data from 2015 to 2025-03.

Code

ma_lengths = [5, 10, 20, 40, 80, 120, 200]
trend_results = []

start = "2015-01-01"
end = "2025-03-30"

for ticker in universe:
  
  ticker_result = []
  
  for ma in ma_lengths:
  
    data = tickers.tickers[ticker].history(start=start, end=end)
    
    bt = Backtest(data,
                  MovingAverageStrategy,
                  cash=10_000)
                  
    stats = bt.run(n=ma)
    
    ticker_result.append(stats)
    
  
  trend_results.append(ticker_result)

Code

noise = pd.read_csv("./data/noise-40-day-2015-2025.csv")
noise.set_index("symbol", inplace=True)
trend_all_performances = []

for i, symbol in enumerate(universe):

  n = []
  return_pcts = []
  win_rates = []
  profit_factors = []
  num_trades = []
  sharpe_ratios = []
  expectancies = []
  
  for ma_result in trend_results[i]:
    n.append(ma_result._strategy.n)
    return_pcts.append(ma_result["Return [%]"])
    win_rates.append(ma_result["Win Rate [%]"])
    profit_factors.append(ma_result["Profit Factor"])
    num_trades.append(ma_result["# Trades"])
    sharpe_ratios.append(ma_result["Sharpe Ratio"])
    expectancies.append(ma_result["Expectancy [%]"])
    
  result_df = pd.DataFrame({
    "Length": n,
    "Return %": return_pcts,
    "Win Rate": win_rates,
    "Profit Factors": profit_factors,
    "Number of Trades": num_trades,
    "Sharpe Ratio": sharpe_ratios
    })
    
  result_df["Symbol"] = symbol
  
  trend_all_performances.append(result_df)
  
trend_all_performances = pd.concat(trend_all_performances).set_index("Symbol")
trend_all_performances = pd.merge(trend_all_performances, noise, left_index=True, right_index=True)

# Sort by Noise
trend_all_performances.sort_values("efficiency_ratio", ascending=False, inplace=True)

Code

fig, ax = plt.subplots(figsize=(17, 10))

sns.barplot(trend_all_performances,
            x="Symbol",
            y="Profit Factors",
            hue = "Length",
            palette="Blues",
            ax=ax)

ax.tick_params(axis='x', rotation=45, labelsize=8)
ax.set_title("Profit Factor from 2015 to 2025 by different levels of MAs")
ax.set_xlabel("Asset (Rank by Efficiency Ratio)")
ax.axhline(y = 1, c = "red")

plt.show()

Code

fig, ax = plt.subplots(figsize=(17, 10))

sns.barplot(trend_all_performances,
            x="Symbol",
            y="Win Rate",
            hue = "Length",
            palette="Blues",
            ax=ax)

ax.tick_params(axis='x', rotation=45, labelsize=8)
ax.set_title("Win Rate from 2015 to 2025 by different levels of MAs")
ax.set_xlabel("Asset (Rank by Efficiency Ratio)")

plt.show()

Code

fig, ax = plt.subplots(figsize=(17, 10))

sns.barplot(trend_all_performances,
            x="Symbol",
            y="Number of Trades",
            hue = "Length",
            palette="Blues",
            ax=ax)

ax.tick_params(axis='x', rotation=45, labelsize=8)
ax.set_title("Number of trades from 2015 to 2025 by different levels of MAs")
ax.set_xlabel("Asset (Rank by Efficiency Ratio)")

plt.show()

Findings

Below is a comparison between the trending asset (IYY) and Noisy asset (IWM)

Symbol	IYY	IYY	IWM	IWM
Length	80	120	80	120
Return %	0.173375	0.139606	-0.936058	0.119674
Win Rate	32.075472	29.166667	28.571429	39.130435
Profit Factors	1.192266	1.223974	0.698876	1.126358
Number of Trades	53	48	77	69
Sharpe Ratio	0.118709	0.095599	-0.260316	0.033245
asset_class	Equity	Equity	Equity	Equity
efficiency_ratio	0.190255	0.190255	0.14803	0.14803

2. Crossing Moving Averages

Entry Rule: Buy when close cross over Moving average \(P_t > \text{MA}_t\)

Time of Entry: Next Open

Testing Range: From 30 days to 120 days - [30, 45, 60, 100, 150, 200]

Date Range for test: Since 2000 - 2025

Markets: Covering Equity ETFs, Commodities Futures, Currency Futures, Bonds ETFs

Exit Rule: When Entry Rule violated \(P_t < \text{MA}_t\)

Profit taking and Stop Loss: No rules

Position Sizing: Single Unit or Single Contract

Code

from strategy.MovingAverageCrossoverStrategy import MovingAverageCrossoverStrategy

Code

start = "2000-01-01"
end = "2025-03-30"

results = []

for ticker in universe:
  
  data = tickers.tickers[ticker].history(start=start, end=end)
  
  bt = Backtest(data,
                MovingAverageCrossoverStrategy,
                cash=10_000)
                
  stats = bt.run()
  
  results.append(stats)

Show the code

returns_crossover = list(map(lambda x: x["Return [%]"], results))

fig, ax = plt.subplots()

returns_crossover_df = pd.DataFrame({'Universe': universe,
                                     'Crossover Returns': returns_crossover,
                                     'MA Trend Returns': returns_trend
                                     })
returns_crossover_df = pd.melt(returns_crossover_df,
                id_vars="Universe",
                value_vars=["Crossover Returns", "MA Trend Returns"],
                var_name="Strategy",
                value_name="Returns")
returns_crossover_df.sort_values("Returns", ascending=False, inplace=True)

sns.barplot(returns_crossover_df, x="Universe", y="Returns", hue="Strategy", ax=ax)

ax.tick_params(axis='x', rotation=45, labelsize=8)
ax.set_title("Returns from 2000 to 2025 by using 45-day MA")

plt.show()

Comparing trend length for different asset classes and their performance

We will test 5, 10, 20, 40, 80, 120 day of moving averages and to evaluate their Total P/L, Profit Factor, and Number of trades.

No position sizing is considered. They are either entered 1 unit or 1 contract.

We will be using data from 2015 to 2025-03.

Code

ma_lengths = [5, 10, 20, 40, 80, 120, 200]
crossover_results = []

start = "2015-01-01"
end = "2025-03-30"

for ticker in universe:
  
  ticker_result = []
  
  for ma in ma_lengths:
  
    data = tickers.tickers[ticker].history(start=start, end=end)
    
    bt = Backtest(data,
                  MovingAverageCrossoverStrategy,
                  cash=10_000)
                  
    stats = bt.run(n=ma)
    
    ticker_result.append(stats)
  
  crossover_results.append(ticker_result)

Code

noise = pd.read_csv("./data/noise-40-day-2015-2025.csv")
noise.set_index("symbol", inplace=True)
crossover_all_performances = []

for i, symbol in enumerate(universe):

  n = []
  return_pcts = []
  win_rates = []
  profit_factors = []
  num_trades = []
  sharpe_ratios = []
  expectancies = []
  
  for ma_result in crossover_results[i]:
    n.append(ma_result._strategy.n)
    return_pcts.append(ma_result["Return [%]"])
    win_rates.append(ma_result["Win Rate [%]"])
    profit_factors.append(ma_result["Profit Factor"])
    num_trades.append(ma_result["# Trades"])
    sharpe_ratios.append(ma_result["Sharpe Ratio"])
    expectancies.append(ma_result["Expectancy [%]"])
    
  result_df = pd.DataFrame({
    "Length": n,
    "Return %": return_pcts,
    "Win Rate": win_rates,
    "Profit Factors": profit_factors,
    "Number of Trades": num_trades,
    "Sharpe Ratio": sharpe_ratios
    })
    
  result_df["Symbol"] = symbol
  
  crossover_all_performances.append(result_df)
  
crossover_all_performances = pd.concat(crossover_all_performances).set_index("Symbol")
crossover_all_performances = pd.merge(crossover_all_performances, noise, left_index=True, right_index=True)

# Sort by Noise
crossover_all_performances.sort_values("efficiency_ratio", ascending=False, inplace=True)

Code

fig, ax = plt.subplots(figsize=(17, 10))

sns.barplot(crossover_all_performances,
            x="Symbol",
            y="Profit Factors",
            hue = "Length",
            palette="Blues",
            ax=ax)

ax.tick_params(axis='x', rotation=45, labelsize=8)
ax.set_title("Profit Factor from 2015 to 2025 by different levels of MAs")
ax.set_xlabel("Asset (Rank by Efficiency Ratio)")
ax.axhline(y = 1, c = "red")

plt.show()

Code

fig, ax = plt.subplots(figsize=(17, 10))

sns.barplot(crossover_all_performances,
            x="Symbol",
            y="Win Rate",
            hue = "Length",
            palette="Blues",
            ax=ax)

ax.tick_params(axis='x', rotation=45, labelsize=8)
ax.set_title("Win Rate from 2015 to 2025 by different levels of MAs")
ax.set_xlabel("Asset (Rank by Efficiency Ratio)")

plt.show()

Code

fig, ax = plt.subplots(figsize=(17, 10))

sns.barplot(crossover_all_performances,
            x="Symbol",
            y="Number of Trades",
            hue = "Length",
            palette="Blues",
            ax=ax)

ax.tick_params(axis='x', rotation=45, labelsize=8)
ax.set_title("Number of trades from 2015 to 2025 by different levels of MAs")
ax.set_xlabel("Asset (Rank by Efficiency Ratio)")

plt.show()

Findings

By comparing the Crossover system vs the Trend system, the performance of the Crossover system is a lot worse. The trending signal is more stable and achieving higher win rate, despite the fact that the indicator is more lagged ant not responsive.

Below is a comparison between the trending asset (IYY) and Noisy asset (IWM)

Symbol	IWM				IYY
data	Crossover		Trend		Crossover		Trend
variable	Profit Factors	Return %	Profit Factors	Return %	Profit Factors	Return %	Profit Factors	Return %
Length
Loading ITables v2.2.5 from the internet... (need help?)

Conclusion

We can generalize the trend-following profile as: - Win Rate is low—often less than 30%. - The average winning trade must be significantly larger than the average losing trade; actually, given only 30% profitable trades, the ratio must be greater than 100:30 to be profitable. - The average winning trades are held much longer than losses. - There is a high frequency of losing trades; therefore, there are also long sequences of losing trades.²

While there are many ways to improve the win rate, such as adding profit taking and stop loss, however, it also means there is a trade off, as this will miss the “Big Fish”.

Comparing the 2 methods of using Moving averages, it is also found that using the trend of moving averages is more reliable than doing price crossover. The crossover method generates too many noise during choppy market, and even in trending market the price will cross the moving average multiple times, even the trend is not changing. The crossover method makes it hard to catch the “Big Fish”

Footnotes

Trading Systems and Methods, 5th Edition (Perry J. Kaufman, 2012) - Ch8 (pp.310)↩︎
Trading Systems and Methods, 5th Edition (Perry J. Kaufman, 2012) - Ch8 (pp.319)↩︎

The key elements of Trending System

Premise of a Trend Following System

The testing universe

A reminder on the ideal trading system

Trade Identification Tools

1. Moving Averages Trends

Distribution of returns using a 45-day moving averages

Comparing trend length for different asset classes and their performance

Findings

2. Crossing Moving Averages

Comparing trend length for different asset classes and their performance

Findings

Conclusion

Footnotes