Kevin Warsh, the newly appointed Chair of the Federal Reserve, recently outlined his views on the future of the Fed's inflation framework and why policymakers should avoid relying too heavily on any single measure, Warsh referenced Goodhart's Law, the Lucas Critique, and the need to consider a broader - though unspecified - set of inflation indicators.
This brief post will introduce several concepts key not only for understanding Warsh's words, but also the nature of the dynamics behind predictive models in quantitative finance and several tips for readers to develop robust model frameworks.
Introduction
Whether building a factor model, training a machine learning algorithm, forecasting macroeconomic variables, or designing a trading strategy, quantitative researchers implicitly rely on a fundamental assumption:
"The relationships observed in historical data will continue to hold in the future."
This assumption is rarely true.
Historical statistical relationships are conditional rather than permanent. Their predictive power depends on the behavior of participants, the incentive structure, the competitive landscape, and the prevailing economic regime. Understanding this principle is arguably one of the most important skills separating robust quantitative research from overfitted backtests.
Tackling the Issue
Concepts such as Goodhart's Law is a famous adage stating that "when a measure becomes a target, it ceases to be a good measure.". First expressed by British economist Charles Goodhart in 1975, the law highlights how human behavior shifts when a specific metric is incentivised. Instead of focusing on the actual goal, people optimize their actions exclusively to satisfy the metric, often leading to unintended and counterproductive consequences.
Beyond Goodhart's law, there are different disciplines describing how historical statistical relationships are conditional rather than permanent:
Quantitative Research Implications
These concepts fundamentally change how quantitative researchers should evaluate empirical findings. Rather than asking
"Does this model fit the historical data?"
the more important questions become:
- Why should this relationship exist?
- Is the mechanism causal or merely correlational?
- Can market participants arbitrage it away?
- What incentives could alter the relationship?
- Under which macroeconomic regimes should it hold?
- How sensitive is it to changing market structure?
- How quickly could competitors discover the same signal?
- How will I detect when it stops working?
These questions often matter more than another decimal place of in-sample performance.
Practical Principles for Robust Model Development
A robust quantitative research process should therefore incorporate the following principles.
- Prefer structural explanations over statistical correlations: Economic intuition generally survives longer than purely empirical relationships.
- Assume every model has a finite half-life: No predictive model should be expected to remain permanently valid.
- Continuously monitor model performance: Model validation is an ongoing process rather than a one-time exercise.
- Detect concept drift explicitly: Monitor prediction errors, feature distributions, factor exposures, and residual behaviour.
- Model changing regimes: Relationships may differ during recessions, inflationary periods, financial crises, monetary tightening, liquidity shocks, etc.
- Account for crowding: A strategy's popularity is itself a state variable. Crowding often precedes alpha decay.
- Use adaptive estimation: Rolling windows, Bayesian updating, online learning, state-space models, hidden Markov models, and dynamic factor models are often preferable to static parameter estimates.
- Separate signal discovery from signal durability: Finding an anomaly is only the beginning. Understanding why it should persist is the real research challenge.
Final Thoughts
The concepts presented here are not competing theories but complementary descriptions of adaptive systems.
Each focuses on a different mechanism through which predictive relationships evolve:
- Goodhart's Law explains optimization.
- Lucas Critique explains policy-induced behavioural change.
- Campbell's Law explains incentive distortion.
- Reflexivity explains endogenous feedback.
- Efficient Market Hypothesis explains competitive arbitrage.
- Adaptive Markets explains evolutionary learning.
- Concept Drift explains model degradation after deployment.
- Regime Change explains structural macroeconomic evolution.
- Non-Stationarity explains changing statistical properties.
- Alpha Decay explains the practical erosion of investment opportunities.
Collectively, they lead to a single guiding principle for quantitative researchers and data scientists:
"Markets, economies, and human systems are adaptive rather than static. Consequently, successful models are not those that fit historical data most closely, but those that are continuously monitored, economically grounded, robust to changing environments, and designed to adapt as the underlying data-generating process evolves."
In modern quantitative research, the objective is therefore not to discover immutable laws, but to develop models that remain useful in a world where the rules themselves are constantly changing.
------------------------------
Carlos Salas
Portfolio Manager & Freelance Investment Research Consultant
------------------------------