Strategy Validation Checklist: 10 Checks Before You Risk Capital

A strategy is validated when it survives honest attempts to break it: clean data, an untouched out-of-sample period, realistic costs, nearby parameters, other markets, and a bad run of luck. This checklist gives you ten checks in that order, plus a sign-off gate that tells you when to reject a strategy or pause it. It works for your own idea, a setup from a video, or a strategy an AI tool wrote for you.

TL;DR

• A backtest is a hypothetical simulation made with hindsight. Treat a good equity curve as a claim to be tested, not as evidence.
• Run the checks in layers: rules and data timing first, then out-of-sample and walk-forward tests, then robustness, costs, and drawdown survival.
• Report gross and net results separately. Costs alone can cut a strategy's return by more than half.
• Log every variant you tried, including the ones you threw away. Otherwise your "winner" is just the best of many guesses.
• Passing means an edge is plausible, not certain. Live execution still needs its own checklist.

Table of Contents

• Why a validation checklist beats a single backtest
• The 10-point strategy validation checklist
• How to validate claims from videos and AI-generated strategies
• The sign-off gate: when to reject or pause
• Execution review: the part backtests skip
• A day-to-day trading checklist to pair with it
• Perspective: why we publish the losers too
• Run the checklist on your own strategy
• FAQ
• Sources

Why a validation checklist beats a single backtest

A single backtest answers one narrow question: did these rules make money on this data? It cannot tell you whether the data was clean, whether the rules peeked at the future, or whether you ran 200 variations and kept the best.

Regulators say this plainly. SEC-filed risk disclosures note that hypothetical backtested performance is built with hindsight, that alternative models can produce materially different results, and that it does not indicate future results. Read that as a design requirement. Your checklist should label every result as hypothetical and forbid yourself from treating it as a return guarantee.

The best way to think about validation is falsification. You are not trying to prove the strategy works. You are trying to kill it with the cheapest tests first, and trust only what is left. If you want the full mechanics of running the test itself, our step-by-step guide to backtesting a trading strategy covers it.

The 10-point strategy validation checklist

A 2026 validation workflow and the FIDES preprint list the minimum research checks as: executable rules, point-in-time data, no look-ahead, an untouched out-of-sample test, walk-forward testing, cost and slippage modeling, parameter sweeps, multi-asset and multi-regime tests, Monte Carlo trade-order resampling, and adjustment for the number of trials. The sections below turn that into ten checks you can tick off.

1. Write rules a stranger could execute

Entry, exit, stop, position size, timeframe, and instrument must be unambiguous. "Buy a strong breakout" is not a rule. "Buy at the next open after the daily close exceeds the prior 20-day high" is.

This is also where you catch vague claims from videos. If the creator cannot state the rule precisely, the backtest will test your interpretation, not their strategy.

2. Use point-in-time data

Use prices and fundamentals as they existed on each date. Adjusted or restated data, and datasets that only include companies that still exist today, flatter results. Record the data source, date range, bar size, and how splits and dividends were handled.

3. Rule out look-ahead bias

Look-ahead is usually an accident: a signal computed on the close but traded at that same close, or an indicator that repaints. A quick test is to delay every signal by one bar. If the edge vanishes, it was never tradable. We walk through this in 4 quick tests to expose look-ahead bias.

4. Lock an out-of-sample period and touch it once

Split your history. Develop and tune on the first part. Keep the last part sealed. Run it once, at the end. If you peek, adjust, and rerun, it is no longer out-of-sample, it is just more development data.

A strong headline Sharpe ratio cannot compensate for a contaminated holdout. Write the date range of the holdout down before you start.

5. Run walk-forward testing

Walk-forward analysis optimizes on one window, tests on the next, then rolls forward. It shows whether your process keeps working as conditions change, rather than whether one set of parameters fit one era. Our guide on walk-forward analysis for algo and retail traders covers window sizing, and this 2026 walk-forward validation guide is a useful second read on the same idea.

Look for consistency across windows, not the best window. If most out-of-sample segments are flat or negative and one is spectacular, you have a story, not an edge.

6. Model costs and slippage, and show gross versus net

Costs deserve their own decision gate. A BIS working paper on currency momentum found that a best strategy fell from nearly 10% to about 4% per year once full quoted spreads were applied. The authors also note that full quoted spreads may overstate actual effective spreads, but leaving spreads out clearly overstates profit.

So do three things:

1. Report gross and net returns side by side.
2. State your exact assumptions: commission, spread, slippage per trade.
3. Show a sensitivity range (for example, costs at 0.5x, 1x, and 2x your estimate) instead of one number.

Short-horizon strategies are the most exposed. A slow position in a liquid market can tolerate a simple per-trade cost. A strategy that trades often, or at larger size, needs spread, market impact, latency, and fill probability considered too.

7. Sweep nearby parameters

If your moving average length is 21 and the strategy only works at 21, you have found noise. Test 15 through 27. A real edge shows up as a plateau: results degrade gracefully as you move away from the chosen setting. A single spike surrounded by losses is a reject.

8. Test other assets and market regimes

Run the same rules, unchanged, on related instruments and on different periods: trending, choppy, high-volatility, low-volatility. You do not need the strategy to win everywhere, but you need to know where it loses and why. A swing strategy, for example, should be checked against sideways markets specifically, as we describe in our swing trading backtest guide.

9. Resample trade order with Monte Carlo

Your historical trade sequence is one of many possible orderings. Shuffle or resample the trades thousands of times and look at the distribution of drawdowns, not the single path you happened to get. The question is whether the worst plausible paths stay inside your real loss tolerance. See our write-up on Monte Carlo backtesting and tail risk for how we approach it.

10. Keep a trial log and adjust for it

The least obvious failure is process bias. Every parameter, asset, timeframe, and variant you tried belongs in a log, including discarded experiments. If you test 100 variants and report the best, its apparent edge is overstated. Selection-bias-aware statistics that account for the number of trials are the proper correction, and the log is what makes them possible.

How to validate claims from videos and AI-generated strategies

Online strategy content has a verification problem, and AI-generated strategies make it worse. The FIDES study had a language model generate 40 strategies on 8 liquid US ETFs and tested them on 2023 to 2024 out-of-sample data. 32 of the 40 claimed to beat buy-and-hold, but only one actually did, a 3.1% hit rate against the claims. The authors present this as a measurement protocol, not proof that all AI strategies fail. But the lesson transfers to any strategy you did not build yourself.

Reconcile three different things:

• What was said: the stated rationale and the claimed results.
• What the code does: the rules as actually implemented.
• What the results show: the outcome on data the author did not tune on.

High consistency between the first two does not establish profitability. Only the third does. When a creator shows an impressive run, write down the exact rules, code them once, and test them on a period after the video was made.

The sign-off gate: when to reject or pause

Checks without a decision rule just produce a nicer-looking spreadsheet. Set these before you look at results. Reject or pause the strategy when:

• Out-of-sample performance collapses relative to development.
• The edge depends on one isolated parameter spike.
• Costs erase the edge.
• The most adverse Monte Carlo paths exceed your loss tolerance.
• Capacity assumptions fail at the size you intend to trade.

Even a pass leaves open questions. Live latency, venue costs, and future regime changes are unresolved by any historical test, so a pass means an edge is plausible, not certain. For the framing of why you must keep the hypothetical label, the SEC-filed disclosure is worth reading in full.

Execution review: the part backtests skip

If you are moving from simulation to real orders, record more than entry and exit prices. The BIS Markets Committee describes execution review as covering pre-trade conditions, real-time fills, and post-trade transaction-cost analysis, with accurate timestamps and benchmarks suited to the order. In practice, log spread, slippage, market impact, rejected trades, fill ratio, and implementation shortfall, and compare against a benchmark such as arrival price, VWAP, or TWAP.

Benchmark choice depends on trade size, frequency, and urgency. That is why there is no universal "best" execution setting: you are always trading off market risk, market impact, and certainty of getting filled. Our guides on futures backtests with an execution-first checklist and options strategies with realistic fills show how this changes by instrument.

A day-to-day trading checklist to pair with it

Statistical validation checks the strategy. A trading checklist checks you. IG's February 2025 guidance describes three stages: pre-trade assessment, in-trade risk and exit management, and post-trade journaling and review. Use it to confirm that live trading follows the rules you validated.

A compact version:

1. Before: Is the setup exactly the one you tested? Are market conditions and scheduled news inside the conditions you validated? Is risk-reward and position size within plan?
2. During: Is the stop where the plan says? Are you managing the exit by rule, not by emotion?
3. After: Did you log the trade, the fill versus expected, and what you would change? Compare live results with the backtest's expected range.

If you trade lower timeframes, a retail trader's guide to backtesting is a helpful complement for the testing side.

Perspective: why we publish the losers too

We built Backtestify because most trading strategies people follow have never been checked against what actually happened in the market. A strategy explained confidently in a video is still just a claim. The data either supports it or it does not.

That is why we test popular strategies, including ones from well-known YouTube creators, on real historical data and compare the original rules side by side with an improved version on recent data. And it is why we publish results for both winning and losing strategies. A library of only winners would teach you nothing about how often claims fail.

Our view of validation is simple. Look at every trade, not only the summary. Read win rate, profit factor, and maximum drawdown together, because any one of them alone can mislead. And treat the equity curve as the start of an argument. You can see how we simulate in our methodology, and browse examples in the published strategy library. For a regulator's mindset on this, see how we backtest like a regulator, and for the broader problem, why most backtests lie.

— WAJDI

Run the checklist on your own strategy

You can describe a strategy in plain English in Backtestify and get a report with win rate, profit factor, and maximum drawdown, then see how an improved rule set compares on recent data. You can start on the Free plan. Pro, at $29 per month or $190 per year, adds unlimited backtesting, improvement, and forecasting. Pick it up from the Backtestify home page when you need to run many validation passes.

One limit applies however you test: no tool turns a hypothetical result into a promise. Use the output to decide what to reject, and size any real trade for the losses you can afford.

FAQ

How many checks does a strategy need to pass?

All the gate conditions, not a majority. A strategy that survives out-of-sample testing but whose edge disappears after costs is still a reject. Treat the sign-off criteria as separate vetoes rather than a score to average.

How do I validate a strategy I saw in a video?

Write the exact rules, code them once, and test them on data from after the video was published. Then run costs, parameter sweeps, and other assets. If the creator's rules are too vague to code without guessing, you cannot validate the claim as stated.

What is the most common reason backtests fail validation?

Usually it is look-ahead bias, ignored costs, or overfitting to one parameter setting. A delayed-signal test, a gross-versus-net comparison, and a parameter sweep catch most of these quickly and cheaply.

Does passing validation mean a strategy will make money?

No. Backtested results are hypothetical, made with hindsight, and not indicative of future results. Passing shows an edge is plausible. Live latency, execution costs, and changing market regimes can still change the outcome, so start small and compare live results with your tested ranges.

Sources

• SEC-filed risk disclosure on hypothetical and backtested performance
• FIDES: A Concordance Protocol for LLM-Generated Trading Strategies (arXiv, 2026)
• BIS working paper 366: Currency momentum strategies
• Bank for International Settlements
• IG: The Essential Trading Checklist (February 13, 2025)

Recommended

• Forward Testing for Traders: 5 Checks for Auditable Results
• 8 Steps to Backtest Futures Strategies for Developers: Execution-First Checklist
• 4 Quick Tests to Expose Look-Ahead Bias in Quant Backtests
• 50-Day Purge Gaps: Out-of-Sample Testing Rules for Traders