The Optimize timeline keeps two jobs visibly separate:
- Training, used to choose funds. Window-based candidate scores and the whole-model objective come from this period.
- Evaluation, reserved for comparison. A forward, disjoint evaluation is out-of-sample verification only when the whole-model proposal was selected without all-history consistency controls.
Training performance is always selection evidence. It is never renamed or presented as verification. Hit rate, information ratio, excess CAGR, and the hit-rate gate deliberately use all shared history. When any of them participates in selection, the timeline says the evaluation dates were also visible and the comparison becomes corroborating evidence, not a strict holdout. A manually assembled proposal is labeled the same way because its expert table exposes all-history benchmark evidence before Select.
Why this matters
Ranking funds over a window is a form of curve-fitting: the winners are partly genuinely better and partly lucky in that period. A later holdout helps separate the two. If proposed swaps win in training but lose in evaluation, the comparison exposed that fragility. That is useful evidence, not a broken optimizer.
The default scoring path uses window metrics and leaves the hit-rate gate off, so these three plain presets preserve a clean date split:
- Recent check: four earlier years for selection, then the latest year.
- Stronger check: four earlier years, then the latest two years.
- Through-cycle check: an older five-year selection period, then a newer five-year evaluation; it requires ten years of fund history.
Exact dates and minimum fund history remain under Advanced.
The page warns when the setup weakens the test:
- Evaluation overlaps training: the shared span is in-sample, so the comparison is explicitly labeled as overlapping and not fully out of sample.
- Evaluation precedes training: the windows are separate, but this is not a forward holdout and is never labeled out-of-sample verification.
- Short windows: risk metrics get noisy under about a year.
- Minimum history vs. training start: funds younger than the training window are scored only from inception, a subtly unfair comparison; raise the minimum age or shorten the window.
- All-history selection controls: the dates can be disjoint while the evidence is not a strict holdout; the page labels that comparison as corroboration.
- Worst 5Y without a complete rolling span: the engine needs 5×365.25 elapsed days. Even Through-cycle's exact five calendar years can fall just short; use another active criterion or custom dates slightly wider than five years.
Compare is required before Apply, but even a clean evaluation pass is evidence from one historical period, not proof.