Validation report: GOLD 1h
do not trust
Failed
- Deflated Sharpe ratio: Charged for all 13 variants tried on this strategy so far, the probability the Sharpe is above zero is 0.0%.
- Haircut Sharpe and multiple testing: Charged for 13 trials, the Sharpe of 0.76 becomes 0.00 (adjusted p = 1.000); it is not significant at 5% after the multiple-testing correction (p must be below 0.05).
- Random-entry benchmark: The strategy's result beats 50% of 1000 random-timing versions with the same trades; 95% is the bar.
Cautions
- Data quality: 71 of 40,000 bars (0.178%) have a problem: 9 return spikes, 62 gaps.
- Probability of backtest overfitting (CSCV): PBO = 0.93: in 93% of 12,870 in-sample / out-of-sample splits, the variant that looked best in-sample did no better than the median variant out of sample. But the in-sample winner still made money out of sample in 98% of them: the choice between variants is noise, not the edge.
- Sub-period consistency: 4 of 7 calendar years were profitable (57%); 60% of years with no year above half the total is the bar.
- Alpha versus buy-and-hold: Beyond what holding the asset explains (beta 1.04), the strategy adds 1.7% a year with a t-stat of 1.09; above 2 is the bar.
Integrity
Is the simulator right?
| Check | Status | Value | Threshold | What it found |
|---|---|---|---|---|
| Known-answer tests | pass | 2.91e-11 | every case within 0.01% of capital; no trades returns capital exactly | The engine reproduced all 4 hand-worked answers on these bars (buy and hold, short, costs, no trades); worst error $0.000000. |
| Look-ahead: truncation test | pass | 108 | trades entered before (cut - longest hold) identical at 60/75/90% cuts; within max(5 bars, 1% of the cut) of that, a trade only the full run has is an end guard, not a leak | Cutting the data at 60% (65 trades), 75% (83 trades), 90% (108 trades) left every earlier trade unchanged: none of the decisions it could compare depended on the bars that were cut. |
| Look-ahead: one-bar delay test | pass | 0.9943 | keeps >= 25% of net profit with every fill one bar later | Filling every trade one bar later keeps 99% of the net profit ($128,580 of $129,320); the edge is not a one-bar artefact. |
| Null-data test | pass | 0.3 | no significant profit on 20 noise series: fail if profitable significantly more often than a coin (sign test p < 0.05, series with no trades are ties and dropped); warn if the median of the series that traded is positive | On 20 noise series with this market's returns in random order it made money 30% of the time (median $-1,412): no edge appears where there is none to find. |
| Independent re-computation (shadow engine) | pass | 1.46e-10 | equity curve and net profit within 0.01% of capital | An independent re-computation of fills, slippage, sizing, commission and equity agrees with the engine to $0.0000 on $100,000 of capital. |
| Data quality | warn | 0.0018 | clean passes; gaps and spikes warn (holidays and news cause them); structural corruption on more than 0.1% of bars fails | 71 of 40,000 bars (0.178%) have a problem: 9 return spikes, 62 gaps. |
Execution
Would the fills have happened?
| Check | Status | Value | Threshold | What it found |
|---|---|---|---|---|
| Bid/ask spread fills | pass | 525.81 | realistic run profitable and keeps >= 50% of net profit | Paying a 1.5 bp half-spread on every fill costs $526 (0% of the baseline profit). With every realistic cost on, the strategy makes $72,021; it keeps 56% of the baseline profit. |
| Slippage and market impact | pass | 830.98 | realistic run profitable and keeps >= 50% of net profit | Volatility-scaled slippage on every fill costs $831 (1% of the baseline profit). With every realistic cost on, the strategy makes $72,021; it keeps 56% of the baseline profit. |
| Execution latency | pass | 740.11 | realistic run profitable and keeps >= 50% of net profit | Filling 1 bar late costs $740 (1% of the baseline profit). With every realistic cost on, the strategy makes $72,021; it keeps 56% of the baseline profit. |
| Financing and carry costs | pass | 55,190.43 | realistic run profitable and keeps >= 50% of net profit | Financing the position while it is held costs $55,190 (43% of the baseline profit). With every realistic cost on, the strategy makes $72,021; it keeps 56% of the baseline profit. |
| Cost stress | pass | - | profitable at 2x costs | Profitable at every cost multiple up to 3x. |
| Break-even cost | pass | 742.50 | break-even >= 3x assumed cost (warn >= 1.5x) | The strategy breaks even at 742.5 bps round trip, 185.6x the 4.0 bps assumed; 3x is the bar. |
| Capacity | info | 4.81e+08 | info: notional where impact halves profit | Square-root market impact halves the profit at about $481,133,254 a trade (4.29e+04x today's size, $4,293,510,210 of capital). Treat it as an order of magnitude: it assumes the volume column is units traded, and FX/CFD volume is usually tick counts. |
Significance
Is it more than luck?
| Check | Status | Value | Threshold | What it found |
|---|---|---|---|---|
| Probabilistic Sharpe ratio | pass | 0.9858 | PSR >= 0.95 (warn >= 0.90) | The Sharpe of 0.76 has a 99% probability of being above zero; 95% is the bar. |
| Minimum track record length | pass | 3.8139 | data covers MinTRL | A Sharpe of 0.76 needs 3.8 years of history to be 95% sure it beats zero; this test has 6.8 years. |
| Autocorrelation-adjusted Sharpe (Lo) | info | 0.7442 | adjusted Sharpe > 0 | Correcting the Sharpe of 0.76 for autocorrelated returns gives 0.74; the correction barely changes it. |
| Deflated Sharpe ratio | fail | 5.83e-05 | >= 95% | Charged for all 13 variants tried on this strategy so far, the probability the Sharpe is above zero is 0.0%. |
| Haircut Sharpe and multiple testing | fail | 0 | haircut Sharpe > 0 and adjusted p < 0.05 | Charged for 13 trials, the Sharpe of 0.76 becomes 0.00 (adjusted p = 1.000); it is not significant at 5% after the multiple-testing correction (p must be below 0.05). |
| White's Reality Check | pass | 0.026 | p below 0.05 (warn below 0.10) | p = 0.026: if none of the 12 variants searched truly beat cash, a best variant this good would turn up by luck 2.6% of the time. |
| Hansen's SPA test | pass | 0.016 | p below 0.05 (warn below 0.10) | p = 0.016: Hansen's test, which ignores clearly bad variants, puts the chance that the best of 12 beats cash only by luck at 1.6%. |
| Romano-Wolf stepwise test | pass | 0.016 | chosen variant significant at 5% after stepwise correction | 12 of 12 variants stay significant after the stepwise correction; the chosen one is among them (adjusted p = 0.016). |
| Permutation test | pass | 0.0099 | p < 0.05 (warn < 0.10) | On 100 shuffled versions of the price history the strategy did as well as on the real one 0 times (p = 0.010); below 0.05 means the real order of prices mattered. |
| Bootstrap confidence intervals | pass | 0.0694 | Sharpe 95% lower bound > 0 (warn if median > 0) | Resampling the daily returns puts the Sharpe between 0.07 and 1.45 (95% interval); even the low end is above zero. |
| Monte Carlo trade resampling | pass | 0 | P(loss) <= 20% and ruin <= 1% | Resampling the trades 1000 times ends at a loss 0.0% of the time, with a 0.0% risk of ruin. |
| Random-entry benchmark | fail | 0.498 | percentile >= 95th (warn >= 90th) | The strategy's result beats 50% of 1000 random-timing versions with the same trades; 95% is the bar. |
Overfitting
Does it hold on data it never saw?
| Check | Status | Value | Threshold | What it found |
|---|---|---|---|---|
| Walk-forward analysis | pass | 6.387 | efficiency >= 0.5 and OOS profit > 0 | Walk-forward efficiency is 6.39 with out-of-sample profit of $41,515; the fitted settings carried over. |
| Anchored walk-forward | pass | 1.6734 | mean efficiency >= 0.5 and OOS profit > 0 | Refit on all data up to each test window, the strategy made $42,164 out of sample with efficiency 1.67 (out-of-sample profit per bar over in-sample). |
| Probability of backtest overfitting (CSCV) | warn | 0.9328 | PBO below 0.25 (warn below 0.5); fail only if the winner loses out of sample in 5%+ of splits | PBO = 0.93: in 93% of 12,870 in-sample / out-of-sample splits, the variant that looked best in-sample did no better than the median variant out of sample. But the in-sample winner still made money out of sample in 98% of them: the choice between variants is noise, not the edge. |
| Combinatorial purged cross-validation | skip | - | 75% of paths Sharpe > 0 and median > 0 | 2103 days of history is too short to cross-validate trades held up to 1612 days: each of the 6 groups (350 days) is smaller than the purge plus embargo (1634 days). |
| Trial registry | info | 13 | - | 13 distinct parameter variants have been tried on this strategy and dataset (0 before this run); every significance test charges for all of them. |
| Minimum backtest length | pass | 4.9799 | data span at least MinBTL | After 13 trials a Sharpe of 0.76 needs 5.0 years of data to beat what luck alone would produce; this test has 6.8. |
| Sealed holdout | info | 0 | opened once, and profitable | The last 20% of bars is sealed; never opened. |
| Out-of-time incubation | skip | - | profitable with Sharpe > 0 on bars after the first validation | First validation of this strategy: data up to 2024-11-21 is now frozen, and bars added after it will be scored on the next run. |
Robustness
Does it break easily?
| Check | Status | Value | Threshold | What it found |
|---|---|---|---|---|
| Parameter sensitivity | pass | 1 | at least half the grid profitable | 100% of the 12 parameter combinations are profitable. |
| Plateau score | pass | 1 | at least 70% of neighbours keep half the Sharpe (warn 40%) | 100% of the 3 settings one step from the chosen one keep at least half its Sharpe of 0.81. |
| Noise injection | pass | 1 | >= 80% profitable and median keeps >= 50% (warn >= 60% profitable) | With small random noise added to every price, 100% of 20 reruns stayed profitable and the median kept 90% of the baseline profit. |
| Timing jitter | pass | 1 | >= 80% profitable and median keeps >= 50% (warn >= 60% profitable) | With every entry and exit moved up to 2 bars at random, 100% of 20 reruns stayed profitable and the median kept 100% of the unshifted replay profit. |
| Sub-period consistency | warn | 0.5714 | >= 60% of years profitable, no year > 50% of total (fail < 40%) | 4 of 7 calendar years were profitable (57%); 60% of years with no year above half the total is the bar. |
| Regime breakdown | pass | 3 | profitable in >= 2 of 3 volatility regimes (warn 1) | Profitable in 3 of 3 volatility regimes (low, mid, high); $100,604 in up-trends and $30,099 in down-trends. |
| Outlier dependence | pass | 100,121.41 | profitable without the best 5 trades (warn: only without the best 1) | Without its 5 best trades the strategy still makes $100,121 of its $129,320, so it does not live on a few outliers. |
| Cross-asset generalisation | pass | 0.6667 | at least half the other markets profitable (warn at least one) | Unchanged, the strategy made money on 2 of 3 other 1h markets (BTCUSD, SPX500). |
| Stress-scenario replay | info | -15,803.58 | no window loses more than 25% of the capital it started with | Through 2 stress windows the worst was 2022 rate shock at -$15,804 (-10.7%) with a 22.7% drawdown inside it. |
Risk
What is the return, really?
| Check | Status | Value | Threshold | What it found |
|---|---|---|---|---|
| Tail and drawdown risk | info | 0.2346 | - | The worst drawdown was 23.5% and the longest spell below a high lasted 1212 days; on the worst 1% of days you lose 3.3% on average. |
| Alpha versus buy-and-hold | warn | 1.0882 | alpha t-stat > 2 (warn > 1) | Beyond what holding the asset explains (beta 1.04), the strategy adds 1.7% a year with a t-stat of 1.09; above 2 is the bar. |
| Turnover and exposure | info | 21.1493 | - | About 21 trades a year, held 232.6 days on average, in the market 99% of the time at 0.09x average leverage. |
Governance
Can anyone check it?
| Check | Status | Value | Threshold | What it found |
|---|---|---|---|---|
| Reproducibility manifest | info | 529e81e7f22e87cf | - | Script 94d8e4ef4ce3, 40,000 bars 529e81e7f22e, options 1132c7f9e211, seed 42, python engine, 20 side datasets: the same inputs give the same numbers. |
| Signed validation report | info | - | - | The exported report is signed when you export it: a SHA-256 digest anyone can recompute, and an HMAC with this machine's key (created on first export). |
| Independent review | info | - | - | Ask Kepler with /audit for an adversarial review of the code on screen. |
| Live versus backtest tracking | skip | - | - | Live versus backtest tracking arrives with LIVE, in a future update. |
Reproducibility
Everything that decides the numbers above. Same inputs, same result.
- Script SHA-256
- 94d8e4ef4ce373f02b826e4122a47d2bf5f27ef82e04db9e04804b5c7b213d5e
- Bars SHA-256
- 529e81e7f22e87cf1fab9e4a8b94948fff706e77a59330b4064c3e86bb6f352f
- Bars
- 40000 (1518789600 to 1732165200)
- Options SHA-256
- 1132c7f9e211554cdb3005c981d96c103a903df68c63101da95095aea124ce80
- Params SHA-256
- 31211bf2efef2394e050b8077c8bc2c50b48b13aac0202a71e32485c949abd47
- Engine
- python 0.0.1
- Versions
- numpy 2.5.3, pandas 3.0.5, python 3.12.13, qubitquantterminal 0.0.1
- Seed
- 42
- data[AAPL] SHA-256
- fe59c5f2e22d2c821fe5c847ceec89e1b6280e4f6e6a4903ccd545d1a7228d0c
- data[BRENT] SHA-256
- 6393f785a2c7329ff3f523313face3d88e84d065c91fc9d6d28aa9bd668337db
- data[BTCUSD] SHA-256
- f27ce0cb54763d9eb7721f7751f39e49fc893dc884e18e31d787453b16892115
- data[EURUSD] SHA-256
- 408780e23cda4a64ac9ee019fef45b0356316bcadb653f09514eb906fa0661e8
- data[GOLD] SHA-256
- 83c96eb66ee8b6fc6b1ded8ae702f52116445a57f20b7b40e78b39479b2947cd
- data[NAS100] SHA-256
- bc7c3e620163bcee31c3d455b83c6bbcf271e44c97ceb5b993efa0d261ae5e99
- data[SILVER] SHA-256
- ef2a395f5b046cdabc5d9ea62ec5b0e81d7f65ad9a1982bdcddd673673afba52
- data[SPX500] SHA-256
- 9fc1e1cd4c1ab9bac19ff8386692ccb05885a198c1e01dfae99b7ff1f6c2c755
- data[USDJPY] SHA-256
- 8c1e80b85dd863ea38c17eab571d0d1ade49753aae2ab62ec3f71a84349fa3a7
- data[USYIELDS] SHA-256
- 47c8a7cfb4928c35bcbd48f1ecf8a835ab579c41a2e6638de007d6f8e51b5b0b
- data[aapl] SHA-256
- fe59c5f2e22d2c821fe5c847ceec89e1b6280e4f6e6a4903ccd545d1a7228d0c
- data[brent] SHA-256
- 6393f785a2c7329ff3f523313face3d88e84d065c91fc9d6d28aa9bd668337db
- data[btcusd] SHA-256
- f27ce0cb54763d9eb7721f7751f39e49fc893dc884e18e31d787453b16892115
- data[eurusd] SHA-256
- 408780e23cda4a64ac9ee019fef45b0356316bcadb653f09514eb906fa0661e8
- data[gold] SHA-256
- 83c96eb66ee8b6fc6b1ded8ae702f52116445a57f20b7b40e78b39479b2947cd
- data[nas100] SHA-256
- bc7c3e620163bcee31c3d455b83c6bbcf271e44c97ceb5b993efa0d261ae5e99
- data[silver] SHA-256
- ef2a395f5b046cdabc5d9ea62ec5b0e81d7f65ad9a1982bdcddd673673afba52
- data[spx500] SHA-256
- 9fc1e1cd4c1ab9bac19ff8386692ccb05885a198c1e01dfae99b7ff1f6c2c755
- data[usdjpy] SHA-256
- 8c1e80b85dd863ea38c17eab571d0d1ade49753aae2ab62ec3f71a84349fa3a7
- data[usyields] SHA-256
- 47c8a7cfb4928c35bcbd48f1ecf8a835ab579c41a2e6638de007d6f8e51b5b0b
Integrity of this report
- Digest (SHA-256)
- 076ce691f671303e6b702533e7d6fd65c4d556cc7ec320ad1f083963ad39e4b8
- Signature (HMAC-SHA256)
- d3c16769d5bbe8881d109c362579af5a8c4d61b3f3cf2ffa8088a395ab2c5241
- Key fingerprint
- 7fb0192a5416a338
The digest is SHA-256 over this report's canonical JSON (sorted keys, no whitespace, UTF-8), every field except digest and signature. Anyone can recompute it: if it matches a digest published when the report was made, the report has not been edited since. The signature is HMAC-SHA256 with a secret key kept on the machine that ran the validation; it proves integrity to the holder of that key only. It is not a public-key signature and cannot prove who made the report to anyone else.