Overfitting (Curve Fitting)

Every historical dataset contains signal and noise. An optimizer can't tell them apart. Push it hard enough and it will happily tune parameters to the noise, producing a strategy that "predicts" the past perfectly and knows nothing about the future. That's overfitting, and it's the default outcome of unsupervised parameter search rather than an exotic accident.

The warning signs

Fragile parameters: RSI(13) is brilliant, RSI(14) loses money. Real effects are smooth in their parameters, while noise-fits are spiky. Too many knobs: each added filter or threshold is another degree of freedom to memorize the sample with, and five parameters on 200 trades is a memorization engine. Implausible metrics: in-sample Sharpe above roughly 2 or 3 on a retail timeframe deserves suspicion first and excitement later. And one-regime results, where all the profit came from a single stretch of history.

The defenses

Hold data back and test on it once; forward-walk analysis systematizes this. Prefer fewer parameters, and check that performance is stable across neighboring values. Demand hundreds of trades. Stress the result with Monte Carlo reshuffles. None of these prove a strategy works. They raise the cost of fooling yourself, and that is most of what quantitative validation is.

On AlphaProve

The no-code builder lets you attach a sweep array to any field — up to 25 values per parameter — which makes searching a large grid trivial and, for that reason, makes noise-fitting easy: the more combinations you try, the better some of them fit the sample by luck. Forward-walk runs are the built-in answer, scoring each combination out of sample instead of rewarding the in-sample champion. We took a strategy across that line deliberately in we overfit a strategy on purpose.