How many trades a backtest needs to mean anything

Your backtest came back at a 58% win rate over 34 trades, and you are already sizing real money on it. That number is not a measurement. It is a rumour. Run the same rules over the next 34 trades and anything from the mid-thirties to the hi

Your backtest came back at a 58% win rate over 34 trades, and you are already sizing real money on it.

That number is not a measurement. It is a rumour. Run the same rules over the next 34 trades and anything from the mid-thirties to the high sixties would be an ordinary result, and none of it would mean the strategy had changed.

So how many trades do you need to backtest before the output counts as evidence? There is a real answer, it is not thirty, and you can calculate it from your own results in about a minute. For most retail strategies it lands in the hundreds.

A backtest is a measurement with an error bar, and until you know how wide that error bar is, you cannot tell an edge apart from a pattern in noise.

Thirty trades cannot separate a 40% win rate from a 60% one

Start with the simplest version of the problem: the win rate.

Every measured win rate carries a standard error, which is the typical amount your measured number misses the true one by, purely because you sampled a slice of reality instead of all of it. For a win rate the arithmetic is short. Take the win rate times one minus the win rate, divide by the number of trades, take the square root.

Run it on an observed 50% win rate and the picture is bleak. Over 30 trades the 95% range runs from 32% to 68%. Over 100 trades it tightens to 40% to 60%. To pin the number to within 5 percentage points you need about 384 trades.

Read that first range again. A 30-trade sample cannot tell the difference between a system that clears its costs comfortably and one that is quietly feeding your broker. Both live inside the same error bar.

This is the same reason nobody polls 30 voters and calls an election. The poll is not wrong, it is just too small to say anything, and a win rate quoted without a trade count next to it is the trading version of that poll.

The sample you need is set by your edge divided by your noise

Win rate alone was never the right thing to measure anyway. What matters is the average result per trade and how violently that result swings around.

The statistic that combines both is the t-statistic, which is the size of your edge expressed in units of its own uncertainty. In trade terms: t equals the square root of the number of trades, multiplied by your average result per trade divided by the standard deviation of your results per trade. Measure both in R, where 1R is the amount you risked on the trade, and the units cancel.

Flip it around and you get the number you actually want. Trades needed equals your target t-statistic divided by that edge-to-noise ratio, squared.

Now put real systems through it. A 45% win rate taking 2R winners against 1R losers averages +0.35R with a standard deviation of 1.49R, a ratio of 0.23, so it reaches a t-statistic of 1.96 at about 70 trades. Drop the win rate to 40% on the same 2R target and the ratio falls to 0.14, which needs about 207 trades. A 55% win rate at 1R for 1R averages +0.10R against a standard deviation of almost exactly 1R, a ratio of 0.10, which needs about 380.

The one that surprises people: a 70% win rate scalping 0.5R winners against 1R losers averages +0.05R with a ratio of 0.07, and needs roughly 726 trades before the edge separates from noise. The highest win rate on the list is the slowest to prove.

The strategies that feel most convincing after twenty trades are usually the high win rate ones, which are exactly the ones that need the largest sample. Comfort and evidence run in opposite directions here.

— Internal note on strategy validation, Tradoki desk

Every variant you tested raises the bar for the one you kept

Here is the part almost nobody adjusts for, and it is the part that decides whether the backtest survives contact with live markets.

If you test one strategy and it clears a t-statistic of 1.96, that is the standard 5% false-positive rate. If you test 20 variants and keep the best one, you gave yourself 20 chances at that 5%, so the winner needs to clear roughly 3.02 instead. Test 45 and the bar moves to about 3.26. Test 100 and it is about 3.48.

That correction is not a Tradoki opinion. Harvey, Liu and Zhu, reviewing hundreds of published equity factors, concluded that a newly discovered factor needs a t-ratio greater than 3.0 precisely because of how much testing the field had already done. If academic finance needs a t of 3 after decades of data mining, a retail backtest that swept 40 moving-average lengths does not get to use 1.96.

The uncomfortable version comes from Berkeley Lab. Their team reports that with only five years of daily data, 45 or more independent variations of a strategy make it more likely than not that the best variant selected shows a Sharpe ratio of 1.0 or better. Not a good strategy. Just the best of 45 coin flips. The underlying derivation was published in the Notices of the American Mathematical Society in 2014, under a title that tells you what the authors thought of the practice: pseudo-mathematics and financial charlatanism.

Forty-five variants used to take a week of manual work. It now takes one prompt, which is why generating strategy code with an LLM makes this problem worse rather than better unless you count what you generated.

A pure random walk will hand you a Sharpe of 1.59

The same Berkeley Lab paper built a tool that makes the point better than any argument. They generate daily prices from a random number generator, so there is no edge in the data by construction, then let a simple strategy optimise its entry day, holding period, stop loss and direction across 55,000 combinations.

The published example run produced an in-sample Sharpe ratio of 1.59, which most traders would call a career. Applied to a second random series, the same optimised rules scored minus 0.18. Across 400 runs, the in-sample Sharpe ratios clustered around 0.9 while the out-of-sample ratios clustered around zero, exactly as they should when the underlying data is noise.

Splitting the data in two does not fix a counting problem

The standard defence is the out-of-sample split: train on the first half, test on the second, ship if both look good.

It helps, and it is not the shield it is sold as. The Berkeley Lab authors point out that the hold-out method does not control for the number of trials involved in a discovery, and that with enough attempts you can find rules that look good on both halves while having no skill at all.

There is also the quiet failure mode nobody logs. You test out-of-sample, the result disappoints, you adjust two parameters and test again. That second look spent the out-of-sample data. Do it five times and you no longer have a hold-out set, you have a slower in-sample fit, which is the same overfitting trap that swallows multi-timeframe analysis.

A defensible backtest is one you can describe in five numbers

None of this means backtesting is pointless. It means reporting a win rate and a profit factor is not reporting a backtest.

Count your variants before you start. Write down how many parameter combinations you intend to try, then hold yourself to the significance bar that number implies. Twenty variants means t of 3, and that is a decision you make before the results seduce you.

Compute the t-statistic, not the win rate. Square root of trade count, times average R per trade, divided by the standard deviation of R per trade. It is one spreadsheet formula and it is the difference between a measurement and a story.

Treat the sample as a range, not a number. Every win rate you quote should come with the width of its error bar, which is the discipline most trading myths die under.

Then forward test anyway. As a rule of thumb from what we see in cohorts rather than a measured statistic, a strategy that survives a few hundred backtested trades and then 30 days of forward testing on a demo account has cleared the two failure modes that kill most systems: too small a sample, and rules that only work when you already know the answer.

If your strategy takes two trades a week, a 400-trade sample is roughly four years of history. That is not a reason to lower the bar. It is a reason to know which side of the bar you are standing on while you trade it.

● FAQ

How many trades do you need for a backtest to be statistically valid?

There is no single number, but 30 is far too few for almost any retail strategy. The count you need is the square of your target t-statistic divided by your per-trade edge-to-noise ratio, which is average result per trade divided by the standard deviation of results per trade. A strong system with a 0.23 ratio reaches a t-statistic of 1.96 at about 70 trades; a thin one at 0.10 needs about 380.

Why is 30 trades not enough to judge a strategy?

Because the error bar is wider than the thing you are measuring. A 50% win rate observed over 30 trades has a 95% range of roughly 32% to 68%, which covers both a strategy that prints money and one that quietly bleeds. Pinning a win rate to within 5 percentage points at 95% confidence takes about 384 trades.

Does testing more parameter variations make the backtest better?

It makes the winner harder to trust. Testing 20 variants and keeping the best one pushes the significance bar from a t-statistic of 1.96 to about 3.02, and 45 variants pushes it to about 3.26. Bailey and co-authors report that on five years of daily data, 45 independent variations of a strategy make it more likely than not that the best one shows a Sharpe ratio of 1.0 or better from chance alone.

Does an out-of-sample split fix backtest overfitting?

Only partly. A hold-out split does not control for how many trials went into the discovery, and the moment you go back and adjust the rules after seeing the out-of-sample result, that data has become in-sample too. The Berkeley Lab team behind the SEBO tool found the hold-out method is not very effective at preventing backtest overfitting.

What should a trader report about a backtest instead of the win rate?

Report the trade count, the average result per trade in R, the standard deviation of those results, the resulting t-statistic, and the number of variants tested before this one was chosen. Those five numbers say whether the result is a measurement or a lucky draw. A win rate on its own says almost nothing.