% The Framework Tax: per-bar event-driven overhead at exact trade-count parity.
% Controlled, seeded, CPU-relative performance study. Every number in this
% manuscript comes from results/results.json (seed 0), produced by
% scripts/run_all.py; the engines are in scripts/engines.py.
% Primary arXiv category: cs.PF (Performance).
% Cross-list: cs.SE (Software Engineering); further cross q-fin.CP (Computational Finance).
\documentclass[11pt]{article}
\usepackage[margin=1in]{geometry}
\usepackage{amsmath,amssymb}
\usepackage{booktabs}
\usepackage[numbers,round]{natbib}
\usepackage{microtype}
\usepackage[hidelinks]{hyperref}
\setlength{\emergencystretch}{2em}

\newcommand{\eng}[1]{\texttt{#1}}

\title{The Framework Tax:\\
Per-Bar Event-Driven Overhead at Exact Trade-Count Parity,\\
and an Equivalence Gate That Makes the Timing Gap Pure}
\author{Eugen Soloviov}
\date{July 2026}

\begin{document}
\maketitle

\begin{abstract}
Parameter search runs a backtest engine thousands of times but reads one scalar
out the other end, so whatever fixed per-bar cost the engine pays is multiplied
across the entire sweep. Event-driven backtesters --- the architecture that
materializes a bar object, fires a strategy callback, and routes an order through
a broker on every bar --- pay that cost in full. We isolate it in a controlled,
seeded study. Four engines compute the \emph{identical} Hull-moving-average-cross
strategy over a synthetic price series ($n=30{,}000$ bars, close $\approx
30{,}000$, seed $0$): a vectorized NumPy formulation with no Python bar loop, the
same scalar kernel JIT-compiled with Numba, a naive pure-Python per-bar loop, and
a minimal but honest event-driven framework (bar object $+$ \eng{on\_bar}
callback $+$ broker order routing $+$ ledger). The integrity mechanism is an
\emph{equivalence gate}: all four engines must emit a bit-identical trade list ---
$40{,}779$ trades in total across $80$ (fast,\,slow) combinations, the same
(entry,\,exit) bar for every trade, PnL equal to $10^{-9}$ --- so any wall-time
difference is \emph{pure per-bar overhead}, never one engine quietly doing less
work. On this parity-locked workload the event-driven framework is $22.6\times$
slower than the vectorized engine and, the headline inversion, $1.72\times$
slower than even the naive Python loop it ought to resemble: its bar objects,
callback frames, and broker round-trips are strictly more per-bar work than a
bare interpreted loop. Numba, running the \emph{same} algorithm as the loop but
compiled, is $13.7\times$ faster than the loop and effectively ties the
vectorized engine ($0.96\times$ its time), showing the tax is paid to the
\emph{paradigm} (per-bar Python) and not to the language. We do \emph{not} install
or benchmark the third-party frameworks a companion blog post named; all four
engines are our own, timing is CPU-relative (the ordering and ratios travel,
absolute combos/s does not), and the blog's eight-engine table is not reproduced.
This study accompanies a \href{https://marketmaker.cc}{marketmaker.cc} blog post.
\end{abstract}

\section{Introduction}
\label{sec:intro}

Algorithmic-trading research spends most of its compute not on a single backtest
but on \emph{search}: the same strategy is run thousands of times with slightly
different parameters, and only a handful of scalar objectives is read back. This
regime has a defining property --- the engine runs many times, the analysis
happens once --- and it inverts the cost model of a single backtest. Any fixed
cost paid to set up and step a backtest (build an event loop, instantiate a
broker, allocate an object per bar) is paid on \emph{every} combination in the
sweep. A per-run cost of a few seconds, invisible in one honest backtest, becomes
the entire bill across ten thousand runs.

Backtest engines fall into a small number of paradigms, and the paradigm sets the
ceiling on sweep throughput. An \emph{event-driven} engine walks the series bar by
bar, materializes each bar as an object, fires a user callback
(\eng{next}/\eng{on\_bar}), and routes any resulting order through a broker that
validates, fills, and books it into a ledger. This is the architecture of the
popular open-source backtesters, and it is trusted precisely because it mirrors
how live trading works. A \emph{vectorized} or \emph{compiled} engine, by
contrast, expresses the whole strategy as array operations or a JIT-compiled
kernel, with no per-bar Python at all. The claim under test --- the
\emph{framework tax} --- is that the per-bar machinery of the event-driven
paradigm, useful for one high-fidelity validation run, is pure overhead during
search, and that it can make a mature-looking event-driven engine slower than a
throwaway Python loop.

This paper is a controlled measurement of that tax. We take one strategy, one
seeded dataset, and one parameter sweep, and we run them through four engines
that differ \emph{only} in per-bar execution machinery --- everything upstream
(the indicator math) and the strategy semantics are shared verbatim. We then
measure, from one seeded run, three things: that all four engines produce
provably identical trades (the equivalence gate); the wall-time each pays to run
the whole sweep; and the resulting overhead ratios, whose headline is the
inversion in which the event-driven framework loses to the naive loop while the
compiled kernel ties the vectorized engine.

\paragraph{Scope and honesty constraints.} Three constraints bound the claims.
First, we do \emph{not} install or time any third-party backtesting framework
(\eng{backtrader}, \eng{bt}, \eng{backtesting.py}, \eng{PyAlgoTrade},
\eng{zipline}, \eng{nautilus\_trader}, \eng{vectorbt}, \dots). All four engines
are our own; the event-driven one is a minimal but faithful distillation of the
per-bar object/callback/broker round-trip, not any named library, and we
fabricate no numbers for libraries we did not run. The companion blog post's
eight-engine table is therefore \emph{not} reproduced here; what reproduces is
the underlying phenomenon. Second, timing is a \emph{CPU-relative} benchmark on
the machine that produced \texttt{results/results.json}: absolute combos/s is
hardware-specific, and only the ordering of the engines and the ratios between
them are claimed to travel. Third, all numbers derive from one seeded run of one
public harness (\texttt{scripts/run\_all.py}), with the engines in
\texttt{scripts/engines.py} and the shared indicator/strategy code in
\texttt{scripts/wma.py} and \texttt{scripts/strategy.py}.

\section{The equivalence gate: identical trades make the timing gap pure}
\label{sec:gate}

The trap in any engine comparison is that a ``fast'' engine may simply be doing
\emph{less}. If one engine books $40{,}779$ trades and another books $40$, the
second is not faster --- it is wrong, and the comparison is meaningless. A speed
number is only interpretable if every engine is provably doing the same work.

We enforce this with an \emph{equivalence gate}, and it is the integrity
mechanism of the entire study. Before any timing, the harness runs all four
engines over every parameter combination and asserts that they emit a
\emph{bit-identical} trade list: the exact same number of trades, the exact same
(entry,\,exit) bar index for every trade, entry/exit prices equal to an absolute
tolerance of $10^{-9}$, and total net PnL equal to $10^{-9}$. The vectorized
engine is the reference; any disagreement aborts the run. Table~\ref{tab:parity}
reports the outcome: over $80$ (fast,\,slow) combinations, all four engines
produce $40{,}779$ trades each, with a maximum PnL delta of $0$ and a maximum
price delta of $0$ against the reference. The trade lists are not close --- they
are identical to the bit.

Exact parity is what makes the study clean. Because every engine produces the
same trades, the difference in wall time \emph{cannot} be attributed to one
engine computing a different, cheaper strategy. The timing gap is therefore
\emph{pure overhead}: the cost of the machinery each engine wraps around the
identical underlying computation. This is a strictly stronger guarantee than the
$\pm 1$-trade tolerance a cross-library comparison must accept (different
libraries disagree on bookkeeping conventions --- whether the final open position
is force-closed, whether the initial entry counts as a trade); because all four
engines here implement one identical execution convention, parity is exact, not
approximate. The test suite additionally verifies that the gate \emph{fails} when
an engine is made to silently drop a single trade, confirming that it actually
protects against a ``fast-because-wrong'' engine rather than passing vacuously.

\begin{table}[t]
\centering
\caption{The equivalence gate (seed $0$, $n=30{,}000$, $80$ combos). All four
engines emit a bit-identical trade list: the same total trade count, the same
(entry,\,exit) bars, and PnL/price equal to the reference within $10^{-9}$. A
representative single combo, (fast,\,slow)$=(22,40)$, books $788$ trades.
Because the trades are identical, any wall-time difference is pure per-bar
overhead.}
\label{tab:parity}
\begin{tabular}{lrrr}
\toprule
Engine & Total trades & Max PnL delta & Max price delta \\
\midrule
\eng{vector\_numpy} (reference) & $40{,}779$ & --- & --- \\
\eng{numba\_jit}                & $40{,}779$ & $0$ & $0$ \\
\eng{python\_loop}              & $40{,}779$ & $0$ & $0$ \\
\eng{event\_driven}             & $40{,}779$ & $0$ & $0$ \\
\bottomrule
\end{tabular}
\end{table}

\section{The four engines and what differs between them}
\label{sec:engines}

The workload is deliberately plain, so that the \emph{only} thing that differs
between engines is the engine itself. The strategy is a Hull-moving-average
cross: go long while a fast $\mathrm{HMA}$ of the close is above a slow
$\mathrm{HMA3}$ and flat otherwise, with one-bar-delayed execution at the close
(the position held over bar $t$ is the signal computed at bar $t{-}1$). The HMA
and HMA3 indicators (compositions of linearly weighted moving averages;
Section~\ref{sec:formulas-pointer}) are precomputed once per combination in
\eng{float64}, \emph{outside} every engine, so what is timed is only each
engine's signal$\to$trade work and not the moving-average arithmetic. A
round-trip fee of $0.09\%$ is charged per closed trade, identically by every
engine, so it can never break parity. The sweep is $80$ (fast,\,slow)
combinations --- fast in $\{6,10,14,18,22,26,30,34\}$, slow in
$\{40,60,80,100,120,140,160,180,190,200\}$ --- over a single seeded
geometric-random-walk close series ($n=30{,}000$ bars, initial price $30{,}000$,
per-bar log-return standard deviation $5\times10^{-4}$, zero drift,
\eng{numpy.random.default\_rng(0)}). The four engines are:

\begin{itemize}
\item \textbf{\eng{vector\_numpy}} --- vectorized NumPy. The position series and
  the trade list are whole-array operations plus a canonical trade extractor;
  there is no Python bar loop. This is the reference the gate compares against.
\item \textbf{\eng{numba\_jit}} --- the identical scalar per-bar kernel as the
  loop, JIT-compiled with Numba (imported if available, never faked). Same
  algorithm as \eng{python\_loop}, but the interpreter is removed from the inner
  loop.
\item \textbf{\eng{python\_loop}} --- a hand-written pure-Python \eng{for} loop
  over bars with scalar bookkeeping: track position, remember the entry bar,
  append a trade on each flat-cross. No framework, no broker, no bar object ---
  but the interpreter pays for every bar. This is the ``naive pandas loop''
  control, the honest floor the event-driven engine ought to beat.
\item \textbf{\eng{event\_driven}} --- a minimal but honest event-driven
  framework. Per bar it materializes a \eng{Bar} object, fires a
  \eng{Strategy.on\_bar} callback (a method call with its own frame), has the
  strategy read broker/position state and submit an \eng{Order} through a
  \eng{Broker}, and has the broker validate the order, fill it at the bar's
  close, mutate position state, and append a \eng{Fill} to a transaction ledger.
  This is the per-bar object/callback/broker round-trip that real event-driven
  engines carry, distilled to its essence --- and nothing gratuitous.
\end{itemize}

The critical design point is that the engines share \emph{everything except
per-bar execution machinery}. They consume the same precomputed indicator
arrays, apply the same one-bar-delayed convention, extract trades to the same
convention, and charge the same fee. The \eng{numba\_jit} and \eng{python\_loop}
engines run the \emph{same algorithm}; the only difference is compiled versus
interpreted. The \eng{event\_driven} engine runs that same trade logic wrapped in
the object/callback/broker layer. So each pairwise wall-time difference isolates
exactly one variable: interpreter versus compiler (loop vs.\ numba), array
formulation versus per-bar loop (vector vs.\ loop), and bare loop versus framework
machinery (loop vs.\ event-driven).

\section{Results: the tax, and the event$>$loop inversion}
\label{sec:results}

Timing is the best-of-$k$ ($k=3$) wall time to run the whole $80$-combo sweep per
engine, after one untimed warmup (which also triggers Numba compilation off the
clock). Table~\ref{tab:timings} reports the sweep time and throughput, and
Table~\ref{tab:overhead} the overhead ratios that are the point of the paper.

\begin{table}[t]
\centering
\caption{Per-engine sweep wall time and throughput (best-of-$3$, seed $0$,
$80$ combos, $n=30{,}000$). CPU-relative: absolute combos/s is
hardware-specific; the \emph{ordering} is the phenomenon. The two engines with
no per-bar Python (vectorized, Numba) are an order of magnitude faster than the
two with a per-bar Python step (loop, event-driven), and the event-driven engine
is the single slowest.}
\label{tab:timings}
\begin{tabular}{lrr}
\toprule
Engine & Sweep time (s) & Throughput (combos/s) \\
\midrule
\eng{numba\_jit}    & $0.028$ & $2843.3$ \\
\eng{vector\_numpy} & $0.029$ & $2741.9$ \\
\eng{python\_loop}  & $0.384$ & $208.2$  \\
\eng{event\_driven} & $0.660$ & $121.2$  \\
\bottomrule
\end{tabular}
\end{table}

\paragraph{The paradigm gap.} The two engines that delete the per-bar Python ---
vectorized NumPy and the Numba-compiled kernel --- run the sweep in $\approx
0.03$\,s; the two that keep a Python step per bar --- the naive loop and the
event-driven framework --- take $0.384$\,s and $0.660$\,s. The event-driven
framework is $22.6\times$ slower than the vectorized engine
(Table~\ref{tab:overhead}); the naive loop is $13.2\times$ slower. That order of
magnitude is the framework tax in its plainest form: $80$ combos $\times$
$30{,}000$ bars is $2.4$ million bar-iterations per sweep, and every one of them,
in the per-bar engines, is a fistful of Python-level operations that the array
and compiled formulations never execute.

\paragraph{The inversion (the headline).} The instructive number is not at the
extremes but in the middle of the table: the event-driven framework is
$1.72\times$ slower than the naive Python loop. The loop is not a clever engine
--- it is a bare \eng{for} loop with scalar bookkeeping, the throwaway control ---
and yet the event-driven framework, with its bar objects, callback frames, and
broker round-trips, loses to it. The reason is blunt: on the identical trades, the
event-driven engine does \emph{strictly more per bar} than the loop. It allocates
a \eng{Bar} object, dispatches a method call into \eng{on\_bar}, reads broker
state through attribute lookups, and (on a state change) constructs and routes an
\eng{Order}, appends a \eng{Fill} to the ledger, and mutates the broker's
position. The loop skips all of it and pays only the interpreter's per-bar cost.
The feature-complete, object-per-event machinery that makes an event-driven
engine trustworthy for one realistic run is, across a sweep, a per-bar tax that
sinks it below the naive baseline.

\paragraph{Paradigm, not language.} The Numba engine settles which variable is
responsible. It runs the \emph{same} scalar algorithm as \eng{python\_loop},
differing only in that it is JIT-compiled rather than interpreted, and it is
$13.7\times$ faster than the loop --- enough to close the entire gap to the
vectorized engine, which it effectively ties at $0.96\times$ the vectorized
engine's time (Table~\ref{tab:overhead}). Put the other way, the event-driven
framework is $23.5\times$ slower than the compiled kernel. Deleting the
interpreter from the inner loop, with the algorithm held fixed, recovers the
whole order-of-magnitude difference. The tax is therefore levied on the
\emph{paradigm} --- per-bar Python execution, whether bare (the loop) or dressed
in framework machinery (event-driven) --- and not on any property of the
strategy, the indicator, or the language as such. The remedy is to remove the
per-bar interpreter, by array formulation or by compilation; the two ways of
doing so land within four percent of each other.

\begin{table}[t]
\centering
\caption{Overhead ratios (seed $0$), each isolating one variable. The
event-driven framework is $22.6\times$ the vectorized engine and $1.72\times$
the naive loop (the inversion). The Numba kernel --- the loop's algorithm,
compiled --- is $13.7\times$ the loop and effectively ties the vectorized engine.
Ratios are what travel across hardware; absolute times do not.}
\label{tab:overhead}
\begin{tabular}{lrl}
\toprule
Ratio & Value & Isolates \\
\midrule
event-driven / vectorized & $22.6\times$ & full per-bar framework tax \\
event-driven / naive loop & $1.72\times$ & framework machinery vs.\ bare loop (inversion) \\
naive loop / vectorized   & $13.2\times$ & per-bar interpreter vs.\ array formulation \\
event-driven / numba      & $23.5\times$ & framework tax vs.\ compiled kernel \\
numba speedup / loop      & $13.7\times$ & compiler vs.\ interpreter (same algorithm) \\
numba / vectorized (time) & $0.96\times$ & compiled kernel ties array formulation \\
\bottomrule
\end{tabular}
\end{table}

\section{Discussion}
\label{sec:discussion}

\paragraph{The tax is a property of the paradigm, not the library.} We built the
event-driven engine ourselves and kept it minimal: one bar object, one callback,
one broker round-trip, one ledger append per bar. It still lost to the naive loop
by $1.72\times$. A real framework carries more per bar --- analyzers, observers,
multi-asset portfolios, richer order lifecycles --- so the inversion we measure is
a \emph{lower bound} on the tax a full-featured event-driven backtester pays. The
mechanism does not depend on any library being poorly written; it is intrinsic to
executing a per-bar callback-plus-broker loop in Python across a sweep. This is
also why the fix is not ``optimize the framework'' but ``change the paradigm for
search'': the Numba A/B shows that removing the per-bar interpreter, with the
algorithm untouched, recovers the whole order of magnitude.

\paragraph{One run versus ten thousand.} The per-bar machinery of an
event-driven engine exists to buy fidelity: a realistic broker, an order
lifecycle, fills that respect the market. For a single validation run --- one
strategy about to be deployed --- that fidelity is the product and its runtime
cost is negligible. The tax is only punishing when the machinery is used for the
wrong phase: a large \emph{search}, where thousands of configurations are ranked
and $99\%$ are discarded, and where fidelity per point barely matters. The
practical rule that follows is to search on a vectorized or compiled engine and
validate the survivors on the event-driven one, with an equivalence gate (exact
here, $\pm 1$-trade across libraries) enforcing that the two engines measure the
same strategy. This study is the measurement that motivates that split; the
decision rule itself is engineering guidance, not a claim of this paper.

\paragraph{The equivalence gate as method.} The general lesson beyond backtesting
is that a performance comparison between two implementations of ``the same''
computation is only meaningful if ``the same'' is \emph{enforced}, not assumed. A
discrete, exact invariant --- here the trade list, bit-identical across engines
--- converts a speed benchmark from a plausibility argument into a proof that the
timing gap is pure overhead. Without it, the event$>$loop inversion could always
be dismissed as the loop cutting a corner; with it, the inversion is
unambiguous.

\section{Limitations}
\label{sec:limitations}

\begin{itemize}
\item \textbf{CPU-relative timing.} All wall-clock numbers are best-of-$k$ times
  on one machine. Absolute combos/s is hardware-, build-, and load-specific and
  is \emph{not} claimed to reproduce; only the ordering of the engines and the
  ratios between them are claimed to travel, and they are large enough (an order
  of magnitude to the vectorized engine, $1.72\times$ for the inversion) that no
  reasonable hardware difference flips them. Unlike the parity block, the timing
  block is not bit-reproducible across runs by construction.
\item \textbf{No third-party frameworks.} We install and time no named
  backtesting library. The event-driven engine is our own minimal distillation of
  the per-bar object/callback/broker pattern. The companion blog post's
  eight-engine table (with specific libraries and an MLX GPU row) is
  \emph{not} reproduced; we reproduce the phenomenon those numbers illustrate,
  and we invent no figure for any library we did not run.
\item \textbf{One synthetic DGP and one strategy family.} Numbers are specific to
  a single seeded geometric-random-walk series ($n=30{,}000$, close $\approx
  30{,}000$, per-bar volatility $5\times10^{-4}$) and one HMA-cross strategy over
  $80$ (fast,\,slow) combinations. The \emph{magnitude} of the tax scales with the
  number of bar-iterations and with how much per-bar machinery the framework
  carries; the \emph{ordering} --- per-bar Python engines far behind array/compiled
  ones, event-driven behind the bare loop --- is the invariant.
\item \textbf{Minimal, not adversarial, event engine.} Our event-driven engine
  is deliberately lean, so it \emph{understates} the tax of a full-featured
  framework. A heavier per-bar object graph would widen the inversion, not close
  it; the direction of the result is robust, its exact size is a floor.
\end{itemize}

\section{Conclusion}
\label{sec:conclusion}

Parameter search multiplies whatever an engine pays per bar across an entire
sweep, so the paradigm of the engine decides its search throughput. On one
identical, parity-locked workload --- $40{,}779$ trades, bit-identical across four
engines, over $80$ HMA-cross combinations of a seeded $30{,}000$-bar series --- a
minimal event-driven framework is $22.6\times$ slower than a vectorized NumPy
engine and, the headline inversion, $1.72\times$ slower than even the naive
Python loop it ought to beat: its per-bar bar objects, callback frames, and broker
round-trips are strictly more work than a bare interpreted loop. The tax is paid
to the paradigm and not the language, as the Numba engine proves by running the
loop's exact algorithm compiled and going $13.7\times$ faster --- enough to tie
the vectorized engine at $0.96\times$ its time. The equivalence gate is what makes
the conclusion clean: because every engine emits the identical trade list, the
timing gap is pure per-bar overhead and nothing else. If a backtest library is
your parameter-search bottleneck, the fix is probably not a bigger machine but a
different paradigm for the search phase --- and, as measured here, even the pandas
loop you were too embarrassed to keep would beat the framework.

\paragraph{Reproducibility.} All numbers derive from one seeded run.
\texttt{scripts/run\_all.py} regenerates \texttt{results/results.json} from seed
$0$ (Python 3.12.3, NumPy 2.4.6) with Numba present; the four engines are in
\texttt{scripts/engines.py} and the shared HMA/HMA3 and trade-extraction code in
\texttt{scripts/wma.py} and \texttt{scripts/strategy.py}. The
\emph{parity block} (identical trades, the integrity mechanism) is
byte-deterministic across runs; the \emph{timing block} is a CPU-relative
benchmark and is not bit-reproducible by construction, so the manuscript quotes
timings from the committed run and pins them within rounding. The synthetic series
is a geometric random walk from \texttt{numpy.random.default\_rng(0)} (initial
price $30{,}000$, per-bar log-return standard deviation $5\times10^{-4}$, zero
drift, $n=30{,}000$). \texttt{tests/} contains deterministic invariant tests for
the equivalence gate and the robust timing orderings, and
\texttt{scripts/check\_paper\_numbers.py} verifies every headline numeric literal
in this manuscript against \texttt{results/results.json}.

\label{sec:formulas-pointer}
The HMA/HMA3 and WMA definitions, the overhead-ratio definitions, and their
mapping to \texttt{results/results.json} fields are collected in
\texttt{paper/FORMULAS.md}.

\bibliographystyle{plainnat}
\bibliography{refs}

\end{document}
