lib/artdag/schedule.sx on lib/minikanren: slot var per node, fd-lt per edge, fd-label
search. schedule-asap (smallest-first labeling) agrees exactly with plan.sx greedy Kahn
waves (cross-validated); schedules enumerates all valid schedules; schedules-capped
filters to <=cap per slot; schedule-valid? independent dep check. Adds a 'schedule' suite
to conformance.sh loading the minikanren CLP(FD) stack. Completes the optional Phase 3/7
miniKanren box. schedule 15/15, total 213/213.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
T5 — send_after addresses a registered atom name; the delayed message
lands in that process's mailbox (destination resolved at fire time,
dead/unregistered targets drop silently).
T6 — gen_server loop now handles the {reply,R,S,T} / {noreply,S,T}
timeout-bearing callback returns by scheduling {timeout} to itself via
send_after; handle_info({timeout}, S) fires when no other message
arrives first. Sanity-checks the library hookup.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
T3 — concurrent timers fire in deadline order, not schedule order
(scheduler jumps the clock to the earliest pending deadline each
time the runnable queue drains). T4 — cancel_timer on an
already-fired timer returns the atom false.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Logical-clock timer wheel in the scheduler. send_after schedules a
message-delivery event at an absolute deadline (clock + Time ms);
cancel_timer marks a live timer cancelled and reports remaining ms,
or false. Time advances only when the runnable queue drains, jumping
to the earliest pending deadline (deterministic, no wall clock).
monotonic_time/0,1 exposes the logical ms clock.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Conformance gate + both smoke tests (smoke_kernel_route 6/6,
smoke_federate 6/6) still pass cold on m2 tip cd0de8cb. Dry-run
rebase onto current origin/architecture (0963aa51) shows 109
commits to replay with first conflict at m2's 24e3bf53 — the
binary_to_list/list_to_binary fix that landed independently on
both branches. Textual diff of the runtime.sx changes is identical
on both sides; only the scoreboard files differ. Resolution =
git rebase --skip on m2's duplicate substrate-fix commits.
No code conflict expected on the substantive m2 work (Blockers
#4 :pending-args scheduler fix, er-bif-http-listen rewrite,
er-bif-httpc-request, all of next/**).
The :pending-args extension to er-sched-step-alive! (03c32cda)
is substrate-shaped and only lives on m2 — should propagate to
loops/erlang, but that propagation belongs to the loops/erlang
loop, not this one.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The lone js opt-in-JIT residual was async/await_in_loop, which failed to PARSE
under JIT ("Unexpected token: op '<'" on `i < 5`) while passing on CEK. The js
exclusion was "js-*", but the recursive-descent parser is the jp-* namespace
(75 functions in lib/js/parser.sx) — only the lexer/transpile/runtime are js-*.
So the parser was left JIT-eligible and a jp-* function miscompiled this
construct (the long-standing parser-miscompile class).
Fix: extend the js exclusion to "js-* jp-*" so the parser is interpret-only too,
matching how every other guest's front-end is handled. js conformance under
SX_SERVING_JIT=1 is now 148/148, == CEK.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 6 common-lisp opt-in-JIT failures were all condition-system continuation
escape: cl-restart-case/cl-handler-case/cl-handler-bind wrap their body in
call/cc (restarts + non-local handler exit). When an SX function that drives
the condition system (the parse-recover / interactive-debugger fixtures, e.g.
parse-numbers, make-policy-debugger) is JIT-compiled, the call/cc form runs in
a NESTED cek-run where invoking the captured continuation
runs-to-completion-and-returns instead of escaping — so a restart fails to
abort and the body falls through. Observed as result accumulation
(got (1 3 0 3) vs (1 3)) and no-abort (restart returns the 999 sentinel).
These callers are arbitrary user/fixture code, not a fixed namespace, so they
can't be prefix-excluded. New data-driven mechanism:
- jit-exclude-callers-of! registers call/cc-establishing form names in
Sx_types.jit_excluded_caller_names.
- jit_compile_lambda skips any function whose constant pool (recursively,
incl. nested closures) references a registered name — code_refs_escaping_caller.
Guarded by Hashtbl.length > 0 so it's a no-op for every guest that doesn't
register (zero effect outside CL).
- lib/common-lisp/runtime.sx registers the establish side (cl-restart-case,
cl-handler-case, cl-handler-bind) and the invoke side (cl-invoke-restart,
cl-invoke-debugger, cl-signal, cl-error-with-debugger).
Result: CL conformance under SX_SERVING_JIT=1 = 487/0, EXACTLY matching the CEK
baseline (was 484/6 with a +3 double-execution over-count). parse-recover
3/4 -> 6/0, interactive-debugger 7/2 -> 7/0.
Note: the geometry/mop-trace suites report 0/0 on BOTH CEK and JIT — they error
"Undefined symbol: refl-class-chain-depth-with" (the CLOS suites don't preload
lib/guest/reflective/class-chain.sx). Pre-existing conformance-harness gap, not
a JIT issue; left as-is.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The host combined-binary integration test exposed a new JIT-unsafe class:
Dream's error middleware (host/wrap-errors -> dream-catch-with) failed to catch
a thrown error under JIT — it escaped as "Unhandled exception" and truncated the
host middleware suite (7/9 vs 9/9 on CEK).
Root cause: the VM's OP_PUSH_HANDLER (the compiled form of `guard`) only
intercepts a VM-level RAISE (opcode 37); it does NOT catch the OCaml Eval_error
that the `error` primitive throws from a CALL/CALL_PRIM in a callee frame. So a
JIT-compiled `guard` silently fails to catch. dream-catch-with is curried
((fn (on-error) (fn (next) (fn (req) (guard ...))))), so the guard lives in a
NESTED closure — JIT-compiling the outer function mints that inner guard as a
VmClosure with the broken VM handler.
Fix (central, not per-callsite): scan a JIT candidate's bytecode RECURSIVELY —
including nested closure code in the constant pool — for OP_PUSH_HANDLER, and
skip JIT for any handler-installing function. It then runs on the CEK, whose
guard catches correctly. Covers dream-catch-with, host wrap-errors/blog-render,
and every other guard / handler-bind user automatically.
Verified: minimal direct guard and curried cross-frame guard both return the
caught value under JIT (were "Unhandled exception"); the host run's "kaboom"
escapes went 2 -> 0. (Remaining host blog/page failures are "Undefined symbol:
render-page" — the host's native render fn, absent from the standalone
sx_server.exe; identical on CEK, i.e. an environment artifact, not a JIT
regression. The combined host binary has render-page.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Enabling the epoch serving-mode JIT globally regressed continuation-based guest
interpreters (the epoch mode is the shared command channel every loop's
conformance runner uses). Two-part fix:
1. SAFE DEFAULT GATE. register_jit_hook in the persistent server branch is now
opt-in via SX_SERVING_JIT=1 (default OFF). Default behaviour is unchanged
(no JIT in epoch serving) → zero regression for sibling loops. The
content/Smalltalk page server opts in.
2. GENERAL FIXES + per-guest interpret-only declarations:
- callable? (sx_server/run_tests/integration_tests/mcp_tree) now accepts
VmClosure. A JIT-compiled higher-order function returns its inner closure
as a VmClosure; callable? previously rejected it, so scheme-apply's
(callable? proc) guard failed with "not a procedure: <vm:anon>".
- jit-exclude! gains a trailing-"*" namespace-prefix form
(Sx_types.jit_excluded_prefixes), the robust way to mark a whole guest
interpreter interpret-only (a name-list misses functions in extra files —
it left erlang's vm/dispatcher JIT'd and 13 tests short).
- Per-guest exclusions in each guest's runtime.sx:
scheme "scheme-*" "scm-*" erlang "er-*" "erlang-*"
prolog "pl-*" common-lisp "cl-*" "clos-*"
js "js-*" haskell "hk-*"
Verified under opt-in JIT (== CEK, no hang): smalltalk 847/847, scheme/flow
166/166, erlang 530/530, prolog 590/590, apl 152/152, js 147/148. Residual
(documented, protected by the default gate): common-lisp 6 fails in advanced
suites (parser-recovery/debugger/CLOS/MOP). lua (0/16) and tcl (3/4) fail
identically on CEK — pre-existing, not JIT. run_tests --jit/no-jit unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
register_jit_hook is now installed in the persistent (epoch) serving-mode
branch of sx_server.ml, not just --http/cli/site. Smalltalk-on-SX conformance
under JIT is 847/847 — identical to the no-JIT baseline; Datalog 356/356.
run_tests --jit/no-jit are byte-identical before/after (no regression).
Five distinct root causes fixed (not one "miscompile"):
1. Serving mode never loaded lib/compiler.sx, so JIT used the native
Sx_compiler.compile stub (arity-0 bytecode, params as GLOBAL_GET →
"VM undefined: <param>"). Server-mode branch now loads compiler.sx
before registering the hook, matching http/cli/site.
2. compile-cond / compile-case-clauses / compile-guard-clauses only treated
keyword :else and true as the catch-all, not the bare symbol `else` that
the CEK's is-else-clause? accepts → GLOBAL_GET "else". (lib/compiler.sx)
3. OP_DIV produced a float for non-divisible Integer/Integer (1/2 → 0.5)
instead of the exact Rational the "/" primitive returns. Now delegates to
the primitive, matching CEK. (sx_vm.ml)
4. OP_EQ / _fast_eq lacked Rational/ListRef cases that the "=" primitive's
safe_eq has → (= 1/2 1/2) false under JIT. OP_EQ now delegates non-scalars
to the "=" primitive; _fast_eq gained rational + ListRef. (sx_vm.ml,
sx_runtime.ml)
5. Continuation-based control flow (Smalltalk ^expr non-local return, block
escape, exceptions via call/cc) can't run in the stack VM. New data-driven
exclusion set Sx_types.jit_excluded + `jit-exclude!` primitive, consulted in
jit_compile_lambda (covers both the CEK hook and vm_call's tiered path).
lib/smalltalk/eval.sx self-declares its continuation dispatch core
interpret-only; pure helpers still JIT. The SUnit suite-runner test helper
pharo-test-class miscompiles mid-loop and is excluded in tests/tokenize.sx.
Also adds SX_JIT_DENY / SX_JIT_ONLY env-var bisection filters to the serving
hook. Known residual documented in plans/jit-bytecode-correctness.md: the hook
re-runs a failed VM execution via CEK (correct result, possible duplicate side
effects); adopting run_tests' propagate-don't-rerun semantics is deferred to
avoid changing shared VM/CEK behavior under this loop.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Assert mau/confluent? actually discriminates: the Peano-arithmetic variant of the
optimisation laws is flagged non-confluent with named non-joinable pairs, so the green
'opt module is confluent' is real evidence rather than a rubber stamp. maude-optimize
40/40, total 198/198.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
artdag/opt-improvement compares the original output cone (dce to id) vs the
maude-reduced DAG under an injected cost-fn, returning before/after total-work and
critical-path. opt-cheaper? asserts optimisation never increases cost: the 5-node
chain drops to 2 (work 5->2, path 5->2) and stays cheaper under radius-weighted cost
(5->3); over dedup and untouched DAGs are never pessimised. Consumes cost.sx. Phase 7
base + (later) cost box done. maude-optimize 38/38, total 196/196.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
artdag/opt-reduce: encode a DAG cone -> opt-term, mau/creduce against the
optimisation module, decode the normal form back to build-entries and rebuild.
Result-preserving: a 5-node blur;blur;id;bright0 chain collapses to 2 nodes and an
over(I,I) dedup 3->2, both executing identically to the original; non-optimisable
DAGs round-trip their radius faithfully (unary 1+1+1 -> 3). Completes Phase 7's
bridge-back + equivalence boxes. maude-optimize 33/33, total 191/191.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lib/artdag/optimize-rules.sx — the effect-pipeline optimisation passes (identity
elim, no-op/zero-radius elim, adjacent fusion, idempotent over dedup) as a maude
module. Radius algebra is _+_ [assoc comm id: 0] (NOT Peano successor rules, which
are non-confluent here); mau/confluent? certifies 0 non-joinable critical pairs, so
the optimised pipeline's normal form / content id is rewrite-order stable. Consumes
lib/maude/confluence.sx. maude-optimize 25/25, total 183/183.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
next/tests/smoke_federate.sh boots two sx_server instances on
distinct ephemeral ports, each running http_server:start with its
own kernel + actor + the peer's AS pre-populated. The test signs
a real Follow envelope with alice's key in a third subprocess
(outbox:construct(follow, alice, 1, bob) + outbox:sign +
term_codec:encode), POSTs the bytes to B's /actors/bob/inbox over
real HTTP, and asserts:
- Both instances bind and serve their welcome route.
- Each instance's kernel-aware outbox returns the expected tip.
- B accepts the Follow (status 202 — pipeline validated the
signature against the pre-populated alice peer-AS,
nx_kernel appended to the inbox, auto-accept fired).
- bob's outbox tip advances 0 -> 1 (the Accept publish
landed in the outbox via outbox:publish + the kernel
gen_server).
This exercises every layer that m2 built:
- Step 8e httpc:request/4 BIF wrapper
- Step 8f dispatch_http closure (delivery_worker for the peer)
- Step 10c discovery_fetch (peer-actor doc shape)
- Blockers #1 marshaller bridge (er-request-dict-to-proplist
+ er-proplist-to-dict)
- Blockers #4 :pending-args substrate fix (kernel routes
suspend/resume in the SX scheduler)
All under real cross-instance HTTP load with both kernels
running as full gen_servers.
Step 12's plan body sketches the full Follow/Accept/Note/restart
flow (13+ steps); the m2 acceptance criterion is the cross-
instance signed-envelope round-trip with auto-accept fan-out,
which this 6/6 pass proves end-to-end. Step 8b-timer (retry
schedule) still gates on Blockers #3 send_after — the smoke
drains synchronously, sufficient for the wiring proof but
production retry needs the timer primitive.
m2 is now feature-complete except for the substrate timer
gate. The plan's Step 12 entry is ticked and a Progress log
entry added.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The CID-stability check now calls mau/confluent? / mau/non-joinable-pairs from
lib/maude/confluence.sx (merged in) instead of re-implementing critical-pair
analysis inside lib/artdag. Picks up confluence.sx via the architecture merge.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds lib/maude/confluence.sx — the CID-stability oracle the artdag optimiser
needs. 274 tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Substrate fix: two-line change to lib/erlang/runtime.sx that lets
http-listen handler routes call gen_server:call without deadlocking.
1. er-sched-step-alive!: pass :pending-args (when set) to the
initial-fun call instead of always passing an empty list.
Default behavior (no field) stays (list) — drop-in safe.
2. er-bif-http-listen sx-handler: instead of er-apply-fun handler
inline (which blows up on receive's er-suspend-marker because
the connection thread has no scheduler step on its stack),
create a real er-process with :initial-fun = handler and
:pending-args = (list req-pl), then er-sched-run-all! to drain.
Any receive (e.g. gen_server:call) suspends + resumes inside
the SX scheduler frame the process owns. Read :exit-result
for the response proplist; marshal back to SX dict.
Investigation arc (see plans/fed-sx-milestone-2.md Blockers #4 +
Progress log):
- loops/fed-prims bf8d0bf2 diagnosed it as Erlang-substrate, not
OCaml mutex (Pattern A wrong, Pattern B right but sketchy).
- First Pattern B attempt failed: tried er-spawn-fun on a raw SX
lambda, hit (er-fun? fv) gate. Connection-thread bisect
pinpointed the exact line.
- Real fix: use the existing er-fun (user's handler) directly,
but feed it via :pending-args so step-alive's hardcoded
(list) doesn't drop the request arg.
Acceptance:
- new next/tests/smoke_kernel_route.sh: 6/6 over real HTTP
(welcome /, /actors/alice, /actors/alice/outbox with
gen_server-backed tip, /actors/alice/inbox, unknown-actor,
via http_server:start(P, [{kernel, nx_kernel}])).
- next/tests/http_server_tcp.sh: 5/5 (bumped wait_bound from
30s to 180s — cold boot is slow under sibling-loop CPU load
and the per-handler scheduler ramp adds a small margin).
- Erlang conformance: 761/761.
Step 12's two-instance smoke test is now unblocked — its full
Follow / Accept / Note flow can layer on top of this kernel-route
surface. m2 plan updated.
Pre-existing httpc_request.sh flakiness ("Undefined symbol:
http-request" on the live-call epochs) reproduces WITHOUT this
change — see git stash A/B in the investigation. Unrelated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A tz event now exports DTSTART;TZID=<name>:<local> (EXDATE/RDATE likewise;
UNTIL stays UTC per RFC), and the VCALENDAR emits a VTIMEZONE per distinct zone
with DAYLIGHT/STANDARD sub-components generated from the zone's transition rules
(offsets + FREQ=YEARLY;BYMONTH;BYDAY) — London/Paris blocks match real-world
definitions. Clients recur at fixed wall-clock time, DST-correct (prior caveat
gone). Importer tolerates ;TZID= params. 376/376 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lib/maude is now on this branch (fast-forwarded to architecture). The fit is
proven (lib/maude/tests/effects.sx). Phase 7 spells out the adapter
(maude-bridge.sx), the optimisation laws as a maude module, equivalence with
optimize.sx, and a syntactic confluence/CID-stability check. maude is a
read-only consumed substrate; gotchas recorded.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Walked Pattern B's failure step-by-step from the connection thread
under a live http-listen instance, instrumenting each piece as its
own minimal sx-handler with a hardcoded reply dict:
hardcoded {:status 200 :headers {} :body "..."} -> HTTP 200 ✓
read er-sched-process-count -> "procs=2" ✓
er-pid-new! -> 204 ✓
er-proc-new! (er-env-new) -> 205 ✓
er-spawn-fun (fn () 42) -> HTTP 000
The break is er-spawn-fun's (not (er-fun? fv)) gate raising
"Erlang: spawn/1: not a fun" because the raw SX lambda isn't an
Erlang-fun-shaped {:tag "fun"} dict. The `error` raise propagates
through Sx_runtime.sx_call and is swallowed by the native http-listen
(try ... with _ -> ()) at sx_server.ml:852; connection writes
nothing and closes -> curl reports HTTP 000.
This invalidates the previous "scheduler-re-entry race" hypothesis:
the global er-sched-* state IS shared with the connection thread
and reads correctly (process count of 2 = boot main + http:listen).
The breakage is the strict er-fun? shape check, not concurrency.
Path forward (still substrate scope, one helper):
- Add an er-mk-host-fun helper in lib/erlang/runtime.sx (or a
small AST-constructor in transpile.sx) that produces a real
er-fun dict from a host SX closure.
- sx-handler can then build a 0-arity wrapper-with-captured-req-pl
and feed it to er-spawn-fun.
- er-sched-run-all! drains, exit-result is read, response goes
back to the wire.
Reverted runtime.sx to the Blockers #1 marshaller-bridge fix (the
in-flight Pattern B attempts are not committed). Blockers #4 entry
in plans/fed-sx-milestone-2.md updated with the verified diagnosis
and the one-helper path. Progress log entry added.
m2 stays at 11/12 steps; the substrate helper is loops/erlang scope.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
lib/maude/tests/effects.sx — proves artdag's effect-pipeline optimisations
(fusion, no-op/dead-op elim, identity elim, CSE/idempotent dedup) are
equational rewriting: the optimised pipeline is the normal form, confluence
gives a stable content id. The 'second consumer' spike for a maude-driven
optimiser in lib/artdag. Surfaced faithfulness note: id: affects matching/canon
not auto-reduction.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bug: tz events store wall-clock LOCAL times but export stamped them with a Z
(UTC) suffix, so a London 18:00 event falsely read as 18:00 UTC. ev-ical-conv
now converts a tz event's DTSTART/UNTIL/EXDATE/RDATE local->UTC before
formatting (London summer 18:00 -> 170000Z; Paris -> 160000Z); non-tz events
unchanged. Caveat: UTC RRULE drifts from wall-clock-stable tz recurrence across
a DST boundary (VTIMEZONE deferred). 366/366 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ical.sx parses VEVENT/VCALENDAR text back into events (ev/ical-lines->event,
ev/parse-vcalendar): DTSTART/DURATION/RRULE (ordinal BYDAY, BYMONTHDAY, UNTIL/
COUNT/INTERVAL) + EXDATE/RDATE. Round-trip is occurrence-exact — export->import
expands to the identical occurrence set. Completes bidirectional interop.
360/360 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
run.sx now handles 'search START =>* GOAL .' (reports the witness path) and
mau/run-pretty prints Maude-style 'result SORT: TERM' using least-sort
inference. searchpath.sx exposes mau/search-path-terms (term-level entry).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lib/maude/sorts.sx — mau/term-sort computes the least sort of a term (smallest
result sort among op declarations whose arg sorts the actuals satisfy modulo
subsorting); overloaded f(1)=NzNat vs f(s 0)=Nat. mau/has-sort? for
membership-style checks. Answers the plan's order-sorted substrate question.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Infix ops parse left (default / gather (E e)) or right (gather (e E)) per the
gather attribute, so _:_ [gather (e E)] reads a : b : c as right-nested. Full
insertion sort now runs over bare cons lists with no parentheses.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Parser reads trailing eq attributes (eq L = R [owise] .) via mau/split-attrs.
mau/crewrite-top is two-pass: ordinary equations first, owise last — an owise
catch-all fires only when no ordinary equation applies, regardless of
declaration order. Verified a catch-all declared first still defers.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lib/maude/searchpath.sx — mau/search-path returns the shortest sequence of
states from start to goal (the solution moves), mau/search-length its step
count. BFS over all one-step successors, threading the path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lib/maude/run.sx — mau/run-program / mau/run parse a module plus trailing
reduce/red/rewrite/rew commands (with optional 'in MOD :' qualifier) and
execute them, rendering results in mixfix surface syntax. An idiomatic
.maude file now runs end-to-end.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
