Knowledge · Test it
Backtesting a valuation: how reliable is a hit rate?
A backtest checks how well a signal or model would have worked in the past. You always compare with a base rate, the result without the signal.
At a glance
- Our method: We find all days on which a condition held (e.g. “below SMA200 and RSI oversold”) and measure the hit rate after 5, 10 and 20 days against the base rate of all days.
- What matters is n (sample size) and the number of independent episodes. Many days from few episodes suggest a certainty that does not exist.
- We also publish weak or contradictory results.
Formula
Edge = hit rate with signal − base rate (in percentage points)
Worked examples
| Stock | Condition / signal | Result | Base rate | n | Assessment |
|---|---|---|---|---|---|
| Alphabet | below SMA200 + RSI oversold | 82 / 76 / 70% | 57 / 58 / 61% | 79 days, few episodes | anecdotal |
| Merck | above SMA200, RSI neutral | +0.6 / +1.3 / +1.9 pts | – | 9,932 days | small but robust |
| Tesla | Daily MACD bull cross | +0.6 / −0.7 / +1.5 pts | – | 51 | no edge |
| Porsche SE | Death cross daily vs. regime weekly | −2.94% (20 days) vs. +10.75% (52 weeks) | +0.46% | approx. 10 independent windows | contradictory |
| SpaceX | – | deliberately no backtest | – | history since IPO too short | – |
Figures apply to 5 / 10 / 20 days. The chart shows the edge over the base rate. Porsche SE is missing there because the measures are not comparable.
What the number does
- The most impressive figure (Alphabet, 82%) has the thinnest basis.
- The least conspicuous (Merck, +1.9 points) is the most reliable.
- That is exactly what this page should convey: a rate without sample size and base rate is worthless.
Pitfalls
- Overlapping windows: Consecutive days are not independent.
- Few episodes: 79 days can consist of a handful of market phases.
- Overfitting: If you test many signals, you will always find one that looks good.
- Regime change: The market no longer behaves as it used to.
FAQ
- What does a backtest say about the future? Only how a signal would have worked in the past. It is an indication, not a promise, and the weaker the fewer cases and episodes stand behind it.
- Why do you report weak results? Because a backtest is only credible if you also show where it adds nothing. Tesla and Porsche SE are good examples.
- How well have your fair values done so far? It is too early to say. The oldest analyses date from Sept 20, 2026; the first three-month evaluation is possible from Dec 20, 2026 at the earliest. We will add it here then.
Further reading
- Analyses: Alphabet, Merck & Co., Porsche SE, Tesla
- Next term: Fair value
Figures from our analyses, see each example for the date. Last updated: Nov 1, 2026.
This content is for information only and is not investment advice. The examples are model calculations based on assumptions and may differ from actual value.