Questions:

• Can we improve the overall performance by excluding signals during months when assets consistently have performed worse?
• A valid hypothesis given the variety of pronounced seasonal patterns associated with commodities, e.g. price, volatility, supply, demand etc.

Approach:

• Identify calendar months when an AiLA asset signal appears to consistently under perform.
• Block trading signals generated for that asset, if generated during the under performing month.
• Compare AiLA index performance with and without this bad month selection.

• Define a bad month quantitatively, i.e. arbitrary choice.
• Divide returns by volatility for simple identification and treatment of outliers.
• Even with 10 years of historical data a given month only corresponds to a limited sample of daily data points.
• Consistently bad performance during a given month should be of relevant size compared to the statistical error of average monthly return observed.
• Here we define bad monthly performance as,
  • Month average return/volatility < -0.1
  • Month average return/volatility < 1.5 STD from average return of best 8 other months.
• Bad month identification applied to assets in an AiLA long/short index, comprising 102 most liquid assets.

• Rejecting AiLA signals generated during bad months, obtained from data during the same time period, dramatically improve the results.
• Identifying bad months during period 2011 to 2016, applied to signals during 2017 to 2021, indicates no improvement of the results.
• Same exercise performed using signals during 2011 to 2016, and bad months identified during the same or later period, yield the same improvement pattern.
• The case of overfitting by using bad months from the same period as performance evaluation is obvious, however, illustrates large potential impact due to very low signal-to-noise nature of trading strategies.
• Emphasize importance of rigorous research process due to risk of un-intended and subtitle sources of overfitting!

• No improvement was observed associated with hypothesis of rejecting signals during months where an AiLA asset signal appears to have consistently performed worse.
• However, the exercise demonstrates the large impact overfitting can have on trading strategy performance, due to the very low signal-to-noise ratio.
• The case of overfitting presented here is intended to be obvious, however, the same kind of problem can easily occur un-intentionally in many subtle ways, e.g. from repetitive use of training/validation data in a Machine Learning cycle.
• The results therefore serves as a reminder of the importance of a rigorous research approach in order to produce reliable results.