giles
59bec68dcc
Replaces the watchdog-bump approach with an automated check. The next 5× (or
worse) substrate regression will trip the alarm at build time instead of
hiding behind a deadline bump and only being noticed weeks later.
Components:
* lib/perf-smoke.sx — four micro-benchmarks chosen for distinct substrate
failure modes: function-call dispatch (fib), env construction (let-chain),
HO-form dispatch + lambda creation (map-sq), TCO + primitive dispatch
(tail-loop). Warm-up pass populates JIT cache before the timed pass so we
measure the steady state.
* scripts/perf-smoke.sh — pipes lib/perf-smoke.sx to sx_server.exe, parses
per-bench wall-time, asserts each is within FACTOR× of the recorded
reference (default 5×). `--update` rewrites the reference in-place.
* scripts/sx-build-all.sh — perf-smoke wired in as a post-step after JS
tests. Hard fail if any benchmark regressed beyond budget.
Reference numbers: minimum across 6 back-to-back runs on this dev machine
under typical concurrent-loop contention (load ~9, 2 vCPU, 7.6 GiB RAM,
OCaml 5.2.0, architecture @ 92f6f187). Documented in
plans/jit-perf-regression.md including how to update them.
The 5× factor is chosen so contention noise (~1–2× variance) doesn't trigger
false alarms but a real ≥5× substrate regression — the kind that motivated
this whole investigation — fails the build immediately.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
57 lines
1.9 KiB
Plaintext
57 lines
1.9 KiB
Plaintext
;; lib/perf-smoke.sx — substrate perf smoke test
|
||
;;
|
||
;; Four micro-benchmarks exercising different substrate hot paths. Each
|
||
;; emits its own elapsed-ms via clock-milliseconds. A wrapper script
|
||
;; (scripts/perf-smoke.sh) parses the output and compares to reference
|
||
;; numbers, exiting non-zero on any 5× or worse regression.
|
||
;;
|
||
;; Workloads are chosen for distinct failure modes:
|
||
;; bench-fib — function-call dispatch (recursive arithmetic)
|
||
;; bench-let-chain — env construction (deep let bindings × N)
|
||
;; bench-map-sq — HO-form dispatch + lambda creation
|
||
;; bench-tail-loop — TCO + primitive dispatch in tight loop
|
||
|
||
(define (bench-fib n)
|
||
(let ((fib (fn (n) (if (< n 2) n (+ (fib (- n 1)) (fib (- n 2)))))))
|
||
(let ((s (clock-milliseconds)))
|
||
(fib n)
|
||
(- (clock-milliseconds) s))))
|
||
|
||
(define (bench-let-chain iters)
|
||
(let ((s (clock-milliseconds)))
|
||
(let loop ((i 0) (acc 0))
|
||
(if (= i iters)
|
||
(- (clock-milliseconds) s)
|
||
(loop
|
||
(+ i 1)
|
||
(let ((a 1) (b 2) (c 3) (d 4) (e 5) (f 6) (g 7) (h 8))
|
||
(+ a b c d e f g h acc)))))))
|
||
|
||
(define (bench-map-sq n)
|
||
(let ((s (clock-milliseconds)))
|
||
(map (fn (x) (* x x)) (range 1 (+ n 1)))
|
||
(- (clock-milliseconds) s)))
|
||
|
||
(define (bench-tail-loop iters)
|
||
(let ((s (clock-milliseconds)))
|
||
(let loop ((i 0))
|
||
(if (= i iters)
|
||
(- (clock-milliseconds) s)
|
||
(loop (+ i 1))))))
|
||
|
||
(define (perf-smoke)
|
||
;; Warm-up: populate JIT cache so the timed pass sees the steady state.
|
||
(bench-fib 12)
|
||
(bench-let-chain 200)
|
||
(bench-map-sq 100)
|
||
(bench-tail-loop 500)
|
||
;; Timed pass. Sizes tuned for ~50-200 ms each on a quiet machine.
|
||
(let ((r-fib (bench-fib 18))
|
||
(r-let (bench-let-chain 1000))
|
||
(r-map (bench-map-sq 500))
|
||
(r-tail (bench-tail-loop 5000)))
|
||
(str "perf-smoke fib18=" r-fib
|
||
" let1000=" r-let
|
||
" map500=" r-map
|
||
" tail5000=" r-tail)))
|