plan_bench — a write-heavy-then-derive-read benchmark corpus

Area: Benchmarks Teaches: the two-phase benchmark profile — a large scripted series of mutations followed by a derived measurement that traverses all of it. A self-contained investment ledger whose mutation count is the scaling knob, sized so the headline configuration runs in the dozens of seconds (real benchmark signal) while a small configuration stays fast for CI. Prerequisites: double-entry accounting (mutate flows and aggregates); robot plan execution (rule-driven derivation). Run: ox run-scenario examples/plan_bench --scenario scenarios/plan.toml && ox test examples/plan_bench — and examples/plan_bench/bench.sh to time it at scale. (The --scenario pin matters after a bench.sh run: a bare run-scenario also picks up the locally generated scenarios/generated.toml.)

A repository-owned benchmark corpus with the two-phase profile a plan-search benchmark measures: do a large scripted series of mutations, then measure a derived quantity whose computation traverses everything the mutations built.

The domain is a self-contained, simplified investment ledger — lots are acquired and sold, and each disposition’s realized gain (proceeds - costBasis) is classified by holding period and asset class, then rolled up. Nothing here is tax law or an external model; it is entirely owned by this repository and formatter-canonical.

Shape

1. Mutation phase (planner.ar) — recordBuy + recordSale, one Lot/Sale/sold edge per pair. The number of pairs is the scaling knob.
2. Measurement phase (rules.ar) — the derived rollups totalRealizedGain, longTermGain, shortTermGain, and the grouped netByAssetClass. Reading them forces the derivation plane to traverse every accumulated disposition.

The OrphanDisposition check re-evaluates on every commit, so under ox run-scenario (one transaction per step) the write phase is deliberately super-linear in the pair count — that is what gives the benchmark real signal (dozens of seconds at the headline scale, versus the millisecond-scale rest of the repository).

Construct coverage

• mutate/apply flows with insert / update (planner);
• deductive reads feeding inserts through nested for loops over parameterized queries (planner::classifyRound);
• rule-driven derivation — joins, comparisons, global and grouped aggregates, and an integrity check (rules);
• a relation with constrained endpoints (catalog::heldIn);
• a concept (kind) hierarchy and a standpoint scope (catalog, seed).

Running it

Correctness golden (tiny, hand-checkable — 2 securities, 3 lots, 3 sales):

ox run-scenario examples/plan_bench      # runs scenarios/plan.toml, asserts the rollups
ox test examples/plan_bench              # the in-language goldens (tests/mod.ar)

Benchmark at a chosen scale:

cd examples/plan_bench
./bench.sh 25       # smoke scale (~0.1s)
./bench.sh 500      # headline scale (~67s), dozens of seconds of real signal

bench.sh calls gen.py <N> to emit scenarios/generated.toml (git-ignored), builds the package, and reports the ox check and ox run-scenario wall-clocks. Measured on an Apple-silicon dev reference machine with release ox (the timings below are Apple-silicon reference numbers):

ox check is ~0.01 s at every scale (the schema is fixed-size; only the scenario grows), so the corpus build gate stays fast.

Note: ox run-scenario examples/plan_bench with no --scenario discovers and runs every scenarios/*.toml. After a bench.sh run the git-ignored scenarios/generated.toml sits alongside plan.toml, so a bare run-scenario will also execute the generated scale scenario. Pass --scenario scenarios/plan.toml to run only the correctness golden.