What Is Realistic Options Backtesting? A Practical Guide for Serious Traders

Repository reference: cutebacktests

Abstract

Realistic options backtesting starts from an unpleasant fact: most attractive option backtests are too flattering. They use information the strategy would not have had at the decision time, they pick contracts too cleanly, or they aggregate risk in a way that makes a multi-symbol portfolio look smoother than it really was. In this repository, the strongest evidence for that claim is not philosophical. It is the March 8 audit recorded in Backtesting Framework Issue Summary, where five concrete framework defects were patched and 91 targeted regression tests passed afterward.

That audit matters because each defect changed a different layer of inference. One bug could silently reuse the wrong strike when the relevant underlying-price bucket changed. Another let stop_touch style ORB entries use same-bar information and then fill intrabar. Two more issues overstated portfolio quality by flattening per-symbol returns and misaligning top-level PBO and DSR diagnostics. If you want realistic options backtesting, that is where the work starts. It starts with causal information sets and honest portfolio math.

Question

The practical question is not "how do I make a backtest look more conservative?" The real question is: what assumptions have to hold before an options backtest deserves to influence capital allocation?

In this repo, the answer became clearer only after the framework stopped flattering the strategies. A realistic options backtest must get at least four things right. It must use a causal signal timestamp. It must select contracts using the information that existed at that timestamp. It must estimate fills with a model that reflects tradable quotes rather than fantasy prints. It must aggregate portfolio risk at the realized daily level, not by flattening unrelated per-symbol rows into a smoother series.

Method: What Realistic Options Backtesting Requires

I would define realistic options backtesting as a four-layer discipline.

First, the signal layer has to be causal. The March audit changed stop_touch so that the touch bar creates the signal, but the entry happens on the next bar open. That sounds like a small implementation detail until you realize how many momentum and event-drive systems can be flattered by same-bar entry logic. A backtest that lets the strategy react to the completed bar and still fill inside that same bar is not conservative. It is using information from the future.

Second, the contract-selection layer has to be time-correct. In the audit, the contract selection cache had ignored the relevant underlying price bucket used for moneyness ranking. That meant different entries on the same day could silently reuse the wrong strike. For an options backtest, that is not a bookkeeping nuisance. It changes the instrument under test.

Third, the execution layer has to stay close to the tradeable surface. That means quote-aware logic, spread-aware filtering, and a refusal to assume that the mid is always available in size. The broader repo keeps returning to this principle because many strategy ideas survive at the stock-signal layer and die when the option-expression layer is priced honestly. The ORB audit documented this clearly in ORB Framework Audit, where broad ORB lanes weakened sharply once realism and parity standards were enforced.

Fourth, the portfolio layer has to measure risk from actual combined daily PnL. The audit removed combined Sharpe and Sortino calculations that had been built from flattened per-symbol daily return lists. That change matters because multi-symbol option research is especially vulnerable to accidental smoothing. If two symbols trade on the same day, the portfolio estimator should see one realized day, not two independent fragments pretending to be separate daily observations.

Evidence / Results

The March 8 audit patched five issues that changed the scientific meaning of the repo:

Two exact behavior changes from the audit are worth remembering. stop_touch entries are now "signal on bar t, enter on bar t+1 open." Combined robustness diagnostics are now split properly into dashboard and selection scenarios, with selection_pbo and selection_dsr measured on combined folds rather than on the wrong granularity. That is what realistic options backtesting looks like in code. It is specific, measurable, and usually less flattering than the previous version.

The downstream effect showed up in strategy research almost immediately. The repo's own high-level summary in Toward The One Piece Of Sharpe is blunt: once the framework became more honest, most broad ideas weakened, several died completely, and only a small set of lanes remained credible. That is the outcome a serious researcher should expect. Better measurement does not usually make the opportunity set larger.

What Worked

What worked was the willingness to treat simulation quality as research quality. The audit patched behavior and then added regression coverage around cached contract universes, next-bar stop_touch semantics, prior-bar overnight MR semantics, combined-day risk aggregation, and combined-fold PBO and DSR usage. The result was not a prettier backtest. The result was a narrower but more believable search space.

That narrower search space still produced a real survivor. The current lead paper bot, c66_opening_compression_option_native_short_balance_dte35_v1, survived the harsher process with base out-of-sample return 19.18%, stress-medium 16.70%, stress-harsh 15.56%, and 76 out-of-sample trades in all three cases, as summarized in baseline_summary.json. This is important because it shows the purpose of realism. Realism is not there to kill everything. It is there to distinguish the few lanes that survive honest pressure from the many that only looked good before the audit.

What Failed

The obvious negative result is that a large share of prior excitement had to be discounted. Broad ORB is the cleanest example. After the realism fixes and audit, the repo did not conclude that ORB was dead in every form. It concluded something more specific and more useful: a narrow ORB pocket still had life, but broad ORB search mostly did not. The surviving pocket was directional ORB with 5-7DTE, a 5 minute opening range, and range-stop geometry. The broad 0DTE, 1DTE, and 2-3DTE lanes did not survive as general claims.

That distinction matters because unrealistic backtests fail in two ways. Sometimes they overstate one strategy. Other times they hide the fact that most of a family is weak while a narrow descendant still has merit. Once the framework became stricter, the repo could tell those two stories apart. That is much more valuable than a generic claim that "ORB works" or "ORB does not work."

There is also one unresolved lesson from the audit itself. The repo intentionally did not patch the default fill-model mismatch between orb_confluence and orb_conviction in that same change set. That was treated as a product-level default decision rather than as an immediate defect. This is scientifically healthy. A realistic backtest discipline should separate proven implementation flaws from open design choices. The first category invalidates evidence if left unresolved. The second category changes interpretation and therefore needs explicit documentation.

Takeaway

Realistic options backtesting is not a stylistic preference. It is the minimum standard for deciding whether an options strategy deserves more attention. In practice that means causal entry semantics, time-correct contract selection, quote-aware execution assumptions, and portfolio metrics computed from real combined daily PnL.

The broader lesson from this repository is simple. When the simulator got more honest, the opportunity set got smaller. That was progress, not disappointment. If you want the next layer down, Historical Options Backtesting: Data, Fills, and Slippage That Actually Matter focuses on the data stack itself, and Backtest vs Paper Trading: Why Good Trading Results Break in Live Markets shows what still changes when research leaves the lab. Join the research log to get the next backtest and failure report.