Evidence Based TradingOctober 27, 202614 min read

Evidence-Based Trading: How to Test Before You Trade

Evidence-based trading means testing every rule on historical data before risking capital. Learn the hypothesis, backtest, metrics and pitfalls that matter.

Evidence-based trading means you only risk money on a rule that has been written down precisely, tested on historical data, and checked for the ways a test can fool you. It replaces "this setup looks good" with "this rule produced these results, across these conditions, after these costs." This guide walks through the full process: turning an idea into testable rules, running a backtest, reading the metrics, and avoiding the errors that make most backtests look better than reality.

TL;DR

Evidence-based trading starts with rules specific enough that two people would get identical signals from the same chart. Anything that needs "feel" cannot be tested.
A backtest measures how a rule would have performed on past data. It does not prove the pattern is real, and it does not guarantee future results.
Overfitting, look-ahead bias and ignored costs are the three most common reasons a good-looking backtest fails live.
Judge a strategy on several metrics together (win rate, profit factor, maximum drawdown), and confirm it on data the rules never saw.
Prediction has limits: even rigorous data-driven methods give probabilities, not certainties.

What evidence-based trading actually means
Step 1: Turn the idea into a testable hypothesis
Step 2: Run the backtest, and know what it can say
Step 3: Read the metrics together
Step 4: Hunt for the errors that flatter your results
Step 5: Confirm on data the rules have not seen
Can data-driven trading predict the next move?
Where Backtestify fits
FAQ
Sources

What evidence-based trading actually means

The phrase is most associated with David Aronson's book Evidence-Based Technical Analysis, which applies the scientific method and statistical inference to trading signals. The core split is between objective and subjective analysis. Objective methods use clearly defined rules that produce unambiguous signals and can be simulated repeatedly on historical data; subjective methods rely on private interpretation and cannot be tested.

That distinction is practical. If you say "I buy when price pulls back to support in an uptrend," you have a description, not a rule. How far is a pullback? What counts as support? How do you define an uptrend? Until each term is a number or a condition a computer can evaluate, you cannot test it, and you cannot learn from it either, because every outcome can be explained away after the fact.

Data-driven trading is the same idea seen from the other side: the data decides whether a rule survives, not the trader's confidence or a video's thumbnail.

Step 1: Turn the idea into a testable hypothesis

Most trading ideas arrive as stories: a chart pattern, a creator's setup, a hunch after a good week. The first job is to convert the story into a hypothesis.

A useful hypothesis has three parts: a measured condition, a predicted response over a specific horizon, and a theoretical reason the pattern should exist, such as compensation for risk or a market inefficiency. The reason matters. A pattern with no plausible mechanism is far more likely to be noise.

The CFA Institute describes the general workflow: specify the hypothesis and goals, define the rules and processes, form positions according to those rules, rebalance periodically, then compute performance and risk profiles.

Replace vague words with numbers

Practitioner guides on strategy specification agree on one habit: terms like "strong trend" or "near support" get replaced with numerical definitions. Here is what that looks like.

Vague idea	Testable version
"Buy a strong uptrend"	Close above the 50-day moving average and the 50-day average above the 200-day average
"Enter near support"	Enter when price comes within 0.5% of the lowest low of the last 20 bars
"Get out if it fails"	Exit at a stop 1.5 ATR below entry, or after 10 bars without a new high
"Risk a small amount"	Risk 1% of account equity per trade

A complete specification also covers entry rules, exit rules, position sizing, regime filters, portfolio risk limits and version control. One more habit worth copying: decide in advance what result would make you reject the idea. Treat a changed universe, timing convention, cost model or holding period as a new hypothesis rather than quietly editing the old one.

This is also how you should handle setups from YouTube creators. A video rarely gives you exact parameters, so your job is to write the most faithful rule set you can, test it, and see whether the claim holds on data the creator did not pick. We built Backtestify around that workflow: you describe a strategy in plain English, and the platform runs it on real historical market data and reports the result, winners and losers alike. Our Published Strategy Library shows what that looks like in practice, including a trendline strategy from a popular creator.

Step 2: Run the backtest, and know what it can say

A backtest simulates past performance. It tells you how a rule would have profited from an observed pattern. What it cannot tell you is whether the pattern is signal or noise. López de Prado makes the same point: backtesting assesses a rule's profitability after a theory is established; it does not prove the theory.

So a backtest is a filter, not a verdict. A rule that fails on history is a poor candidate. A rule that passes is a candidate that still has to clear more hurdles.

A few practical requirements:

Enough data, across different conditions. Results depend on the market behavior during the test period, so a test that covers only a rising market says little about a falling one. Our note on how much historical data a backtest needs goes deeper.
Real prices, not hand-picked screenshots. Simulate every signal the rule generates, not the examples that look best.
The whole sequence of trades. Examine each trade, not only the final equity number.

If you are starting from zero, our step-by-step guide to backtesting a trading strategy covers the mechanics, and the methodology page explains how Backtestify simulates a strategy.

Step 3: Read the metrics together

No single number describes a strategy. Each metric answers one question and hides another.

Metric	What it tells you	What it can miss
Win rate	The share of trades that made money	Size of wins versus losses. A 70% win rate can still lose money if the losers are large
Profit factor	Gross profit divided by gross loss	Inflated by omitted costs or a very small number of losing trades
Maximum drawdown	The largest peak-to-trough decline in the test	Only the worst case seen so far. Future losses can exceed the sample
Sharpe ratio	Return relative to overall variability	Depends on data frequency and scaling, and flatters strategies with hidden tail risk

The definitions are simple. Maximum drawdown is the largest percentage decline from a peak to a trough in the period measured, which makes it the best single proxy for how painful the strategy would have been to hold. Pair it with your own tolerance. If a 35% drawdown would make you abandon the rule, the rule does not work for you, whatever its profit factor.

A quick worked example

Imagine a strategy with 100 trades. It wins 60 times for an average of $100 and loses 40 times for an average of $120. Gross profit is $6,000 and gross loss is $4,800, so the profit factor is 1.25 and the win rate is 60%. That looks decent. Now subtract commissions and slippage of $15 per trade, or $1,500 in total. Net profit falls from $1,200 to a loss of $300, and the profit factor drops below 1. The strategy that looked positive was not, once the costs were counted. (The numbers are illustrative, not a real result.)

Trading costs turn gross profit into loss

For a deeper look at one risk-adjusted measure, see our explainer on the Sharpe ratio, its benchmarks and limits.

Step 4: Hunt for the errors that flatter your results

This is where evidence-based trading earns its name. Most failures come from the test, not the market.

Overfitting and data mining

Test enough variations and one will look brilliant by chance. Backtest overfitting occurs when too many model variations are tested relative to the available data, so the model targets random patterns that fail out of sample. The statistical fix is to account for how many things you tried: significance has to be adjusted for multiple testing, and the "haircut" method reduces Sharpe ratios more for marginal strategies than for very strong ones.

Overfit curve misses unseen data points

The scale of the problem has been measured. A 2016 study of 888 algorithms found that common backtest metrics such as the Sharpe ratio had little value in predicting out-of-sample performance, and that more backtesting was associated with a bigger gap between backtest and live results.

Two habits help. Keep a log of every variation you try, because the count is part of the evidence. And prefer plain parameters: simple, round values such as 20, 50 or 200 reduce the risk of fitting a strategy too closely to the past. If you are weighing how many rule tweaks are too many, the logic is the same as in how confirmation bias in investing leads people to keep the results that agree with them and discard the rest. For FX-style risk models, the same discipline shows up in FX value-at-risk backtesting rules. Our own guide to spotting and stopping overfitting gives concrete checks.

Look-ahead bias

Look-ahead bias, using information from a later moment to decide a trade at an earlier one, can invalidate test results. A classic case: a rule that buys at the day's low, which nobody knows until the day ends. We list four quick tests to expose look-ahead bias.

Ignored costs and unrealistic fills

A test that ignores friction is optimistic by design. A backtest must model transaction costs, bid-ask spreads, market impact and execution delays, otherwise a seemingly profitable model may be impractical. Also check execution details such as how stop-loss triggers are handled, since a stop that fills at the exact price in a simulation may gap through in real trading.

Other traps

The CFA Institute lists randomness, regime changes, survivorship bias, skewness, fat tails and structural breaks as things a single historical backtest may miss. Its remedies are rolling-window or walk-forward testing, scenario analysis, simulation and sensitivity analysis. Our broader pieces on why most backtests lie and backtesting like a regulator cover these in more depth.

Step 5: Confirm on data the rules have not seen

Out-of-sample testing means building the rules on one stretch of data and checking them on another. It helps, with one catch: repeated revisions after seeing results mean the test is no longer truly out of sample. Treat the holdout like a sealed envelope. Open it once.

A workable routine:

Write the full rule set and the rejection criterion.
Test on the first portion of your data and fix obvious errors.
Freeze the rules.
Run them once on the later, unseen period, with realistic costs.
Compare the original and any improved version side by side on recent data, and keep both results.
Only then trade small, and track live results against the backtest.

For a final gate, the strategy validation checklist lists ten checks to run before you risk capital, and the swing trading backtest guide applies the same logic to a specific style.

Can data-driven trading predict the next move?

Sometimes, in a probabilistic sense, and never reliably enough to skip risk management. A review of machine-learning investment research found mixed forecasting results and an ambiguous picture of real-world success. It also noted that hit rate measures directional accuracy but not the size of price changes, so a model can be right most of the time and still lose money. The same review points to selection bias and cherry-picking of the best-performing models as reasons published results run ahead of live ones.

The sensible stance: use forecasts as one input, expressed as probability and confidence, and size positions so a wrong call is survivable.

Where Backtestify fits

We built Backtestify for traders who want the evidence before the trade. You enter a strategy in plain English, run it on real historical data, and see win rate, profit factor, maximum drawdown and every individual trade. You can then improve the rules and compare the original with the improved version on recent data, forecast with probability and confidence statistics, and export to TradingView Pine Script. We publish reports for losing strategies as well as winning ones, because a record that only shows winners is not evidence.

The Free plan lets you get started on the billing page. Pro costs $29 per month or $190 per year and adds unlimited backtesting, improvement and forecasting. You can see the plans on the Backtestify homepage, or check the Forecast tool and the trading journal for tracking live results against your tests.

FAQ

Is technical analysis a sham?

Subjective technical analysis, where signals depend on private interpretation, cannot be tested, so it cannot be shown to work. Rules that are objective and precise can be tested, and some fail while others hold up. The evidence-based answer is to test each rule rather than accept or dismiss the whole field.

What is the difference between evidence-based and data-driven trading?

In practice they overlap. Data-driven trading uses historical data to choose or tune rules. Evidence-based trading adds a hypothesis, a reason the edge should exist, and checks for overfitting, bias and costs before any money is risked.

Does a good backtest mean a strategy will make money?

No. A backtest shows hypothetical past results, and it does not show whether a pattern is signal or noise. Treat a strong result as a reason to test further, with out-of-sample data and realistic costs.

Which metric matters most?

None alone. Read win rate, profit factor and maximum drawdown together, because each hides what the others show. Maximum drawdown is often the deciding one, since it tells you whether you could stay with the strategy through its worst stretch.