Introduction

Algorithmic-trading research spends most of its compute not on a single backtest but on search: the same strategy is run thousands of times with slightly different parameters, and only a handful of scalar objectives is read back. This regime has a defining property — the engine runs many times, the analysis happens once — and it inverts the cost model of a single backtest. Any fixed cost paid to set up and step a backtest (build an event loop, instantiate a broker, allocate an object per bar) is paid on every combination in the sweep. A per-run cost of a few seconds, invisible in one honest backtest, becomes the entire bill across ten thousand runs.

Backtest engines fall into a small number of paradigms, and the paradigm sets the ceiling on sweep throughput. An event-driven engine walks the series bar by bar, materializes each bar as an object, fires a user callback (next/on_bar), and routes any resulting order through a broker that validates, fills, and books it into a ledger. This is the architecture of the popular open-source backtesters, and it is trusted precisely because it mirrors how live trading works. A vectorized or compiled engine, by contrast, expresses the whole strategy as array operations or a JIT-compiled kernel, with no per-bar Python at all. The claim under test — the framework tax — is that the per-bar machinery of the event-driven paradigm, useful for one high-fidelity validation run, is pure overhead during search, and that it can make a mature-looking event-driven engine slower than a throwaway Python loop.

This paper is a controlled measurement of that tax. We take one strategy, one seeded dataset, and one parameter sweep, and we run them through four engines that differ only in per-bar execution machinery — everything upstream (the indicator math) and the strategy semantics are shared verbatim. We then measure, from one seeded run, three things: that all four engines produce provably identical trades (the equivalence gate); the wall-time each pays to run the whole sweep; and the resulting overhead ratios, whose headline is the inversion in which the event-driven framework loses to the naive loop while the compiled kernel ties the vectorized engine.

Scope and honesty constraints.

Three constraints bound the claims. First, we do not install or time any third-party backtesting framework (backtrader, bt, backtesting.py, PyAlgoTrade, zipline, nautilus_trader, vectorbt, …). All four engines are our own; the event-driven one is a minimal but faithful distillation of the per-bar object/callback/broker round-trip, not any named library, and we fabricate no numbers for libraries we did not run. The companion blog post's eight-engine table is therefore not reproduced here; what reproduces is the underlying phenomenon. Second, timing is a CPU-relative benchmark on the machine that produced results/results.json: absolute combos/s is hardware-specific, and only the ordering of the engines and the ratios between them are claimed to travel. Third, all numbers derive from one seeded run of one public harness (scripts/run_all.py), with the engines in scripts/engines.py and the shared indicator/strategy code in scripts/wma.py and scripts/strategy.py.

The equivalence gate: identical trades make the timing gap pure

The trap in any engine comparison is that a “fast” engine may simply be doing less. If one engine books 40{,}779 trades and another books 40, the second is not faster — it is wrong, and the comparison is meaningless. A speed number is only interpretable if every engine is provably doing the same work.

We enforce this with an equivalence gate, and it is the integrity mechanism of the entire study. Before any timing, the harness runs all four engines over every parameter combination and asserts that they emit a bit-identical trade list: the exact same number of trades, the exact same (entry, exit) bar index for every trade, entry/exit prices equal to an absolute tolerance of 10^{-9}, and total net PnL equal to 10^{-9}. The vectorized engine is the reference; any disagreement aborts the run. Table 1 reports the outcome: over 80 (fast, slow) combinations, all four engines produce 40{,}779 trades each, with a maximum PnL delta of 0 and a maximum price delta of 0 against the reference. The trade lists are not close — they are identical to the bit.

Exact parity is what makes the study clean. Because every engine produces the same trades, the difference in wall time cannot be attributed to one engine computing a different, cheaper strategy. The timing gap is therefore pure overhead: the cost of the machinery each engine wraps around the identical underlying computation. This is a strictly stronger guarantee than the \pm 1-trade tolerance a cross-library comparison must accept (different libraries disagree on bookkeeping conventions — whether the final open position is force-closed, whether the initial entry counts as a trade); because all four engines here implement one identical execution convention, parity is exact, not approximate. The test suite additionally verifies that the gate fails when an engine is made to silently drop a single trade, confirming that it actually protects against a “fast-because-wrong” engine rather than passing vacuously.

The four engines and what differs between them

The workload is deliberately plain, so that the only thing that differs between engines is the engine itself. The strategy is a Hull-moving-average cross: go long while a fast \mathrm{HMA} of the close is above a slow \mathrm{HMA3} and flat otherwise, with one-bar-delayed execution at the close (the position held over bar t is the signal computed at bar t{-}1). The HMA and HMA3 indicators (compositions of linearly weighted moving averages; Section 8) are precomputed once per combination in float64, outside every engine, so what is timed is only each engine's signal\totrade work and not the moving-average arithmetic. A round-trip fee of 0.09\% is charged per closed trade, identically by every engine, so it can never break parity. The sweep is 80 (fast, slow) combinations — fast in \{6,10,14,18,22,26,30,34\}, slow in \{40,60,80,100,120,140,160,180,190,200\} — over a single seeded geometric-random-walk close series (n=30{,}000 bars, initial price 30{,}000, per-bar log-return standard deviation 5\times10^{-4}, zero drift, numpy.random.default_rng(0)). The four engines are:

• vector_numpy — vectorized NumPy. The position series and the trade list are whole-array operations plus a canonical trade extractor; there is no Python bar loop. This is the reference the gate compares against.

• numba_jit — the identical scalar per-bar kernel as the loop, JIT-compiled with Numba (imported if available, never faked). Same algorithm as python_loop, but the interpreter is removed from the inner loop.

• python_loop — a hand-written pure-Python for loop over bars with scalar bookkeeping: track position, remember the entry bar, append a trade on each flat-cross. No framework, no broker, no bar object — but the interpreter pays for every bar. This is the “naive pandas loop” control, the honest floor the event-driven engine ought to beat.

• event_driven — a minimal but honest event-driven framework. Per bar it materializes a Bar object, fires a Strategy.on_bar callback (a method call with its own frame), has the strategy read broker/position state and submit an Order through a Broker, and has the broker validate the order, fill it at the bar's close, mutate position state, and append a Fill to a transaction ledger. This is the per-bar object/callback/broker round-trip that real event-driven engines carry, distilled to its essence — and nothing gratuitous.

The critical design point is that the engines share everything except per-bar execution machinery. They consume the same precomputed indicator arrays, apply the same one-bar-delayed convention, extract trades to the same convention, and charge the same fee. The numba_jit and python_loop engines run the same algorithm; the only difference is compiled versus interpreted. The event_driven engine runs that same trade logic wrapped in the object/callback/broker layer. So each pairwise wall-time difference isolates exactly one variable: interpreter versus compiler (loop vs. numba), array formulation versus per-bar loop (vector vs. loop), and bare loop versus framework machinery (loop vs. event-driven).

Results: the tax, and the event>loop inversion

Timing is the best-of-k (k=3) wall time to run the whole 80-combo sweep per engine, after one untimed warmup (which also triggers Numba compilation off the clock). Table 2 reports the sweep time and throughput, and Table 3 the overhead ratios that are the point of the paper.

The paradigm gap.

The two engines that delete the per-bar Python — vectorized NumPy and the Numba-compiled kernel — run the sweep in \approx 0.03 s; the two that keep a Python step per bar — the naive loop and the event-driven framework — take 0.384 s and 0.660 s. The event-driven framework is 22.6\times slower than the vectorized engine (Table 3); the naive loop is 13.2\times slower. That order of magnitude is the framework tax in its plainest form: 80 combos \times 30{,}000 bars is 2.4 million bar-iterations per sweep, and every one of them, in the per-bar engines, is a fistful of Python-level operations that the array and compiled formulations never execute.

The inversion (the headline).

The instructive number is not at the extremes but in the middle of the table: the event-driven framework is 1.72\times slower than the naive Python loop. The loop is not a clever engine — it is a bare for loop with scalar bookkeeping, the throwaway control — and yet the event-driven framework, with its bar objects, callback frames, and broker round-trips, loses to it. The reason is blunt: on the identical trades, the event-driven engine does strictly more per bar than the loop. It allocates a Bar object, dispatches a method call into on_bar, reads broker state through attribute lookups, and (on a state change) constructs and routes an Order, appends a Fill to the ledger, and mutates the broker's position. The loop skips all of it and pays only the interpreter's per-bar cost. The feature-complete, object-per-event machinery that makes an event-driven engine trustworthy for one realistic run is, across a sweep, a per-bar tax that sinks it below the naive baseline.

Paradigm, not language.

The Numba engine settles which variable is responsible. It runs the same scalar algorithm as python_loop, differing only in that it is JIT-compiled rather than interpreted, and it is 13.7\times faster than the loop — enough to close the entire gap to the vectorized engine, which it effectively ties at 0.96\times the vectorized engine's time (Table 3). Put the other way, the event-driven framework is 23.5\times slower than the compiled kernel. Deleting the interpreter from the inner loop, with the algorithm held fixed, recovers the whole order-of-magnitude difference. The tax is therefore levied on the paradigm — per-bar Python execution, whether bare (the loop) or dressed in framework machinery (event-driven) — and not on any property of the strategy, the indicator, or the language as such. The remedy is to remove the per-bar interpreter, by array formulation or by compilation; the two ways of doing so land within four percent of each other.

Discussion

The tax is a property of the paradigm, not the library.

We built the event-driven engine ourselves and kept it minimal: one bar object, one callback, one broker round-trip, one ledger append per bar. It still lost to the naive loop by 1.72\times. A real framework carries more per bar — analyzers, observers, multi-asset portfolios, richer order lifecycles — so the inversion we measure is a lower bound on the tax a full-featured event-driven backtester pays. The mechanism does not depend on any library being poorly written; it is intrinsic to executing a per-bar callback-plus-broker loop in Python across a sweep. This is also why the fix is not “optimize the framework” but “change the paradigm for search”: the Numba A/B shows that removing the per-bar interpreter, with the algorithm untouched, recovers the whole order of magnitude.

One run versus ten thousand.

The per-bar machinery of an event-driven engine exists to buy fidelity: a realistic broker, an order lifecycle, fills that respect the market. For a single validation run — one strategy about to be deployed — that fidelity is the product and its runtime cost is negligible. The tax is only punishing when the machinery is used for the wrong phase: a large search, where thousands of configurations are ranked and 99\% are discarded, and where fidelity per point barely matters. The practical rule that follows is to search on a vectorized or compiled engine and validate the survivors on the event-driven one, with an equivalence gate (exact here, \pm 1-trade across libraries) enforcing that the two engines measure the same strategy. This study is the measurement that motivates that split; the decision rule itself is engineering guidance, not a claim of this paper.

The equivalence gate as method.

The general lesson beyond backtesting is that a performance comparison between two implementations of “the same” computation is only meaningful if “the same” is enforced, not assumed. A discrete, exact invariant — here the trade list, bit-identical across engines — converts a speed benchmark from a plausibility argument into a proof that the timing gap is pure overhead. Without it, the event>loop inversion could always be dismissed as the loop cutting a corner; with it, the inversion is unambiguous.

Limitations

• CPU-relative timing. All wall-clock numbers are best-of-k times on one machine. Absolute combos/s is hardware-, build-, and load-specific and is not claimed to reproduce; only the ordering of the engines and the ratios between them are claimed to travel, and they are large enough (an order of magnitude to the vectorized engine, 1.72\times for the inversion) that no reasonable hardware difference flips them. Unlike the parity block, the timing block is not bit-reproducible across runs by construction.

• No third-party frameworks. We install and time no named backtesting library. The event-driven engine is our own minimal distillation of the per-bar object/callback/broker pattern. The companion blog post's eight-engine table (with specific libraries and an MLX GPU row) is not reproduced; we reproduce the phenomenon those numbers illustrate, and we invent no figure for any library we did not run.

• One synthetic DGP and one strategy family. Numbers are specific to a single seeded geometric-random-walk series (n=30{,}000, close \approx 30{,}000, per-bar volatility 5\times10^{-4}) and one HMA-cross strategy over 80 (fast, slow) combinations. The magnitude of the tax scales with the number of bar-iterations and with how much per-bar machinery the framework carries; the ordering — per-bar Python engines far behind array/compiled ones, event-driven behind the bare loop — is the invariant.

• Minimal, not adversarial, event engine. Our event-driven engine is deliberately lean, so it understates the tax of a full-featured framework. A heavier per-bar object graph would widen the inversion, not close it; the direction of the result is robust, its exact size is a floor.

Conclusion

Parameter search multiplies whatever an engine pays per bar across an entire sweep, so the paradigm of the engine decides its search throughput. On one identical, parity-locked workload — 40{,}779 trades, bit-identical across four engines, over 80 HMA-cross combinations of a seeded 30{,}000-bar series — a minimal event-driven framework is 22.6\times slower than a vectorized NumPy engine and, the headline inversion, 1.72\times slower than even the naive Python loop it ought to beat: its per-bar bar objects, callback frames, and broker round-trips are strictly more work than a bare interpreted loop. The tax is paid to the paradigm and not the language, as the Numba engine proves by running the loop's exact algorithm compiled and going 13.7\times faster — enough to tie the vectorized engine at 0.96\times its time. The equivalence gate is what makes the conclusion clean: because every engine emits the identical trade list, the timing gap is pure per-bar overhead and nothing else. If a backtest library is your parameter-search bottleneck, the fix is probably not a bigger machine but a different paradigm for the search phase — and, as measured here, even the pandas loop you were too embarrassed to keep would beat the framework.

Reproducibility.

All numbers derive from one seeded run. scripts/run_all.py regenerates results/results.json from seed 0 (Python 3.12.3, NumPy 2.4.6) with Numba present; the four engines are in scripts/engines.py and the shared HMA/HMA3 and trade-extraction code in scripts/wma.py and scripts/strategy.py. The parity block (identical trades, the integrity mechanism) is byte-deterministic across runs; the timing block is a CPU-relative benchmark and is not bit-reproducible by construction, so the manuscript quotes timings from the committed run and pins them within rounding. The synthetic series is a geometric random walk from numpy.random.default_rng(0) (initial price 30{,}000, per-bar log-return standard deviation 5\times10^{-4}, zero drift, n=30{,}000). tests/ contains deterministic invariant tests for the equivalence gate and the robust timing orderings, and scripts/check_paper_numbers.py verifies every headline numeric literal in this manuscript against results/results.json.

Formulas

The HMA/HMA3 and WMA definitions, the overhead-ratio definitions, and their mapping to results/results.json fields are collected in paper/FORMULAS.md.

References