db gains a parallel :facts-keys {<rel>: {<tuple-string>: true}}
index alongside :facts. dl-tuple-key derives a stable string via
(str lit) — (p 30) and (p 30.0) collide correctly because SX
prints them identically. dl-add-fact! membership is now O(1)
instead of O(n) list scan; insert sequences for relations sized
N drop from O(N²) to O(N).
Wall clock on chain-7 saturation halves (~12s → ~6s); chain-15
roughly halves (~50s → ~25s) under shared CPU. Larger chains
still slow due to body-join overhead in dl-find-bindings —
Blocker entry refreshed with proposed follow-ups.
dl-retract! keeps both indices consistent: kept-keys is rebuilt
during the EDB filter, IDB wipes clear both lists and key dicts.
Parser: open Path and include Path top-level decls; Path is Ctor (.Ctor)*.
Eval resolves via ocaml-resolve-module-path (same :con-as-module-lookup
escape hatch used by :field). open extends the env with the module's
bindings; include also merges into the surrounding module's exports
(when inside a struct...end).
Path resolver lets M.Sub.x work for nested modules. Phase 4 LOC ~165.
New lib/datalog/api.sx: dl-program-data facts rules takes SX data
lists. Rules accept either dict form or list form using <- as the
rule arrow (since SX parses :- as a keyword). dl-rule constructor
for the dict shape. dl-assert! adds a fact and re-saturates;
dl-retract! drops EDB matches, wipes all rule-headed IDB
relations, and re-saturates from scratch — simplest correct
semantics until provenance tracking arrives.
9 API tests cover ancestor closure via data, dict-rule form,
dl-rule constructor, incremental assert/retract, cyclic-graph
reach, assert into empty, fact-style rule (no arrow), dict
passthrough.
module M = struct DECLS end parsed by sub-tokenising the body source
between struct and the matching end (nesting tracked via struct/begin/
sig/end). Field access is a postfix layer above parse-atom, binding
tighter than application: f r.x -> (:app f (:field r "x")).
Eval (:module-def NAME DECLS) builds a dict via ocaml-eval-module
running decls in a sub-env. (:field EXPR NAME) looks up dict fields,
treating (:con NAME) heads as module-name lookups instead of nullary
ctors so M.x works with M as a module.
Phase 4 LOC so far: ~110 lines (well under 2000 budget).
New lib/datalog/aggregates.sx: (count R V Goal), (sum R V Goal),
(min R V Goal), (max R V Goal). dl-eval-aggregate runs
dl-find-bindings on the goal under the outer subst, collects
distinct values of V, applies the operator, binds R. Empty input:
count/sum return 0; min/max produce no binding (rule fails).
Group-by emerges naturally from outer-subst substitution into the
goal — `popular(P) :- post(P), count(N, U, liked(U, P)), >=(N, 3).`
counts per-post.
Stratifier extended: dl-aggregate-dep-edge contributes a
negation-like edge so the aggregate's goal relation is fully
derived before the aggregate fires (non-monotonicity respected).
Safety relaxed for aggregates: goal-internal vars are existentials,
only the result var becomes bound.
New lib/datalog/strata.sx: dl-build-dep-graph (relation -> deps with
:neg flag), Floyd-Warshall reachability, SCC-via-mutual-reach for
non-stratifiability detection, iterative dl-compute-strata, and
dl-group-rules-by-stratum.
eval.sx refactor:
- dl-saturate-rules! db rules — semi-naive worker over a rule subset
- dl-saturate! db — stratified driver. Rejects non-stratifiable
programs at saturation time, then iterates strata in order
- dl-match-negation — succeeds iff inner positive match is empty
Order-aware safety in dl-rule-check-safety (Phase 4) already
required negation vars to be bound by a prior positive literal.
Stratum dict keys are strings (SX dicts don't accept ints).
Phase 6 magic sets deferred — opt-in path, semi-naive default
suffices for current workloads.
Parser: try EXPR with | pat -> handler | ... -> (:try EXPR CLAUSES).
Eval delegates to SX guard with else matching the raised value against
clause patterns; re-raises on no-match. raise/failwith/invalid_arg
shipped as builtins. failwith "msg" raises ("Failure" msg) so
| Failure msg -> ... patterns match.
Sugar for fun + match. AST (:function CLAUSES) -> unary closure that
runs ocaml-match-clauses on its arg. let rec recognises :function as a
recursive rhs and ties the knot via cell, so
let rec map f = function | [] -> [] | h::t -> f h :: map f t
works. ocaml-match-eval refactored to share clause-walk with function.
dl-saturate! is now semi-naive: tracks a per-relation delta dict,
and on each iteration walks every positive body-literal position,
substituting the delta of its relation while joining the rest
against the previous-iteration DB. Candidates are collected before
mutating the DB so the "full" sides see a consistent snapshot.
Rules with no positive body literal (e.g. (p X) :- (= X 5).)
fall back to a one-shot naive pass via dl-collect-rule-candidates.
dl-saturate-naive! retained as the reference implementation; 8
differential tests compare per-relation tuple counts on every
recursive program. Switched dl-tuple-member? to indexed iteration
instead of recursive rest (eliminates per-step list copy). Larger
chains under bundled conformance trip O(n) membership × CPU
sharing — added a Blocker to swap relations to hash-set membership.
Parser: for i = lo to|downto hi do body done, while cond do body done.
AST: (:for NAME LO HI :ascend|:descend BODY) and (:while COND BODY).
Eval re-binds the loop var per iteration; both forms evaluate to unit.
ref is a builtin boxing its arg in a one-element list. Prefix ! parses
to (:deref ...) and reads via (nth cell 0). := joins the binop
precedence table at level 1 right-assoc and mutates via set-nth!.
Closures share the underlying cell.
Two-phase grammar: parse-expr-no-seq (prior entry) + parse-expr wraps
it with ;-chaining. List bodies keep parse-expr-no-seq so ; remains a
separator inside [...]. Match clause bodies use the seq variant and stop
at | — real OCaml semantics. Trailing ; before end/)/|/in/then/else/eof
permitted.
Patterns: wildcard, literal, var, ctor (nullary + arg, flattens tuple
args so Pair(a,b) -> (:pcon "Pair" PA PB)), tuple, list literal, cons
:: (right-assoc), unit. Match: leading | optional, (:match SCRUT
CLAUSES) with each clause (:case PAT BODY). Body parsed via parse-expr
because | is below level-1 binop precedence.
ocaml-parse-program: program = decls + bare exprs, ;;-separated.
Each decl is (:def …), (:def-rec …), or (:expr …). Body parsing
re-feeds the source slice through ocaml-parse — shared-state refactor
deferred.
lib/hyperscript/tokenizer.sx — added cursor + follow-set wrapper over
the existing flat-list tokenize output:
hs-stream src → {:tokens :pos :follows :last-match :last-ws}
hs-stream-current s → next non-WS token (skips WS, captures :last-ws)
hs-stream-match s value → consume if value matches & not in follow set
hs-stream-match-type s ...types → consume if upstream type name matches
hs-stream-match-any s ...names → consume if value matches any name
hs-stream-match-any-op s ...ops → consume if op token & value matches
hs-stream-peek s value n → look n non-WS tokens ahead, no consume
hs-stream-consume-until s marker → collect tokens until marker
hs-stream-consume-until-ws s → collect until next whitespace
hs-stream-push-follow! / pop-follow!
hs-stream-push-follows! / pop-follows! n
hs-stream-clear-follows! → saved / restore-follows! saved
hs-stream-last-match / last-ws
hs-stream-type-map maps our lowercase type names to upstream's
("ident" → "IDENTIFIER", "number" → "NUMBER", etc.) so type-based
matching works against upstream test expectations.
13 tokenizer-stream tests now pass; 30/30 in hs-upstream-core/tokenizer.
Skips remaining: 5 (down from 18).
- 2 template-component scope tests
- 1 async event dispatch (until event keyword works)
- left for later: needs more architectural work
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
new (new Object("")) hung because js-new-call called
js-get-ctor-proto -> js-ctor-id -> inspect, and inspect on a
wrapper-with-proto-chain recurses through the prototype's
lambdas forever. Added (js-function? ctor) precheck at the top
of js-new-call that raises a TypeError instance instead.
conformance.sh: 148/148.
parser.sx parse-toggle-cmd: when seeing 'toggle .foo for', peek the
following two tokens. If they are '<ident> in', it is a for-in loop
and toggle does NOT consume 'for' as a duration clause. Restores the
trailing for-in to the command list.
parser.sx parse-on (handler modifiers): recognize 'throttled at <ms>'
and 'debounced at <ms>' as handler modifiers. Captured as :throttle /
:debounce kwargs in the on-form parts list.
compiler.sx emit-on: pre-extract :throttle / :debounce from parts via
new _strip-throttle-debounce helper before scan-on, then wrap the built
handler with (hs-throttle! handler ms) or (hs-debounce! handler ms).
runtime.sx: hs-throttle! — closure with __hs-last-fire timestamp,
fires immediately and drops events arriving within ms of the last fire.
hs-debounce! — closure with __hs-timer, clears any pending timer and
schedules a new setTimeout(handler, ms) so only the last burst event
fires.
Both formerly-architectural skips now pass:
- "toggle does not consume a following for-in loop"
- "throttled at <time> drops events within the window"
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Brings the architecture branch (559 commits ahead — R7RS step 4-6, JIT
expansion, host_error wrapping, bytecode compiler, etc.) into the
loops/haskell line of work. Conflict in lib/haskell/conformance.sh:
architecture replaced the inline driver with a thin wrapper delegating
to lib/guest/conformance.sh + a config file. Resolved by taking the
wrapper and extending lib/haskell/conformance.conf with all programs
added under loops/haskell (caesar, runlength-str, showadt, showio,
partial, statistics, newton, wordfreq, mapgraph, uniquewords, setops,
shapes, person, config, counter, accumulate, safediv, trycatch) plus
adding map.sx and set.sx to PRELOADS.
plans/haskell-completeness.md gains three new follow-up phases:
- Phase 17 — parser polish (`(x :: Int)` annotations, mid-file imports)
- Phase 18 — one ambitious conformance program (lambda-calc / Dijkstra /
JSON parser candidate list)
- Phase 19 — conformance speed (batch all suites in one sx_server
process to compress the 25-min run to single-digit minutes)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SX strictly arity-checks lambdas; JS allows passing more args than
declared (extras accessible via arguments). Was raising "f expects
1 args, got 2" whenever Array.from passed (value, index) to a
1-arg mapFn. Fixed in js-build-param-list: every JS param list
now ends with &rest __extra_args__ unless an explicit rest is
present, so extras are silently absorbed.
conformance.sh: 148/148.
The 2^32-1 threshold still allowed indices like 2147483648 to pad
billions of undefineds. Without sparse-array support there's no
semantic value in >1M padding; lowering the bail turns those tests
into fast assertion fails instead of timeouts.
built-ins/Array timeouts: 2 → 1. conformance.sh: 148/148.
arr[4294967295] = 'x' and arr.length = 4294967295 were padding
the SX list with js-undefined for ~4 billion entries — instant
timeout. Per ES spec, indices >= 2^32-1 aren't array indices
anyway (regular properties, which we can't store on lists).
Added (>= i 4294967295) bail clauses to js-list-set! and the
length setter.
built-ins/Array: 21/45 → 23/45 (5 timeouts → 2).
conformance.sh: 148/148.
String.fromCharCode.length, Math.max.length, Array.from.length
were returning 0 because their SX lambdas use &rest args with no
required params — but spec assigns each a specific length.
Added js-builtin-fn-length mapping JS name to spec length (12
entries). js-fn-length consults the table first and falls back to
counting real params.
built-ins/String: 79/99 → 80/99, built-ins/Array: 20/45 → 21/45.
conformance.sh: 148/148.
Was hardcoded to "[object Object]" for everything; per ES it should
return "[object Array]", "[object Function]", "[object Number]",
etc. by class. Added js-object-tostring-class helper that switches
on type-of and dict-internal markers (__js_*_value__,
__callable__). Prototype-identity checks ensure
Object.prototype.toString.call(Number.prototype) returns
"[object Number]" (similar for String/Boolean/Array).
built-ins/Array: 18/45 → 20/45, built-ins/Number: 43/50 → 44/50.
conformance.sh: 148/148.
Per ES, every function instance's constructor slot points to the
Function global. Was returning undefined for (function () {})
.constructor. Added constructor to the function-property cond in
js-get-prop; returns js-function-global.
conformance.sh: 148/148.
new Object(func) should return func itself (per ES spec - "if value
is a native ECMAScript object, return it"), but js-new-call only
kept the ctor's return when it was dict or list — functions fell
through to the empty wrapper. Added (js-function? ret) to the
accept set.
built-ins/Object: 42/50 → 44/50. conformance.sh: 148/148.
JS var is function-scoped, but the transpiler only collected
top-level vars and re-emitted (define) everywhere; for-body var
shadowed the outer (un-hoisted) scope. Three-part fix:
1. js-collect-var-names recurses into js-block/js-for/js-while
/js-do-while/js-if/js-try/js-switch/js-for-of-in;
2. var-kind decls emit (set! ...) instead of (define ...) since
the binding is already created at function scope;
3. js-block uses js-transpile-stmt-list (no re-hoist) instead of
js-transpile-stmts.
built-ins/Array: 17/45 → 18/45, String: 77/99 → 78/99.
conformance.sh: 148/148.
js-list-set! was a no-op for the length key. Added a clause that
pads with js-undefined via js-pad-list! when target > current.
Truncation skipped: the pop-last! SX primitive doesn't actually
mutate the list (length unchanged after the call), so no clean
way to shrink in place from SX. Extension covers common cases.
built-ins/Array: 16/45 → 17/45. conformance.sh: 148/148.
js-get-prop for SX lists fell through to js-undefined for any key
not in its hardcoded method list, so Array.prototype.myprop and
Object.prototype.hasOwnProperty were invisible to arrays.
Switched the fallback to walk Array.prototype via js-dict-get-walk,
which already chains to Object.prototype.
built-ins/Array: 14/45 → 16/45. conformance.sh: 148/148.
JS arrays must treat string indices that look like numbers ("0",
"42") as the corresponding integer slot. js-get-prop and js-list-set!
only handled numeric key, falling through to undefined / no-op for
string keys. Added a (and (string-typed key) (numeric? key)) clause
that converts via js-string-to-number and recurses with the integer
key. built-ins/Array: 13/45 → 14/45. conformance.sh: 148/148.
hk-bind-exceptions! in eval.sx registers throwIO, throw, evaluate, catch,
try, handle, displayException. SomeException constructor pre-registered
in runtime.sx (arity 1, type SomeException).
throwIO and the existing error primitive both raise via SX `raise` with a
uniform "hk-error: msg" string. catch/try/handle parse it back into a
SomeException via hk-exception-of, which strips nested
'Unhandled exception: "..."' host wraps (CEK's host_error formatter) and
the "hk-error: " prefix.
catch and handle evaluate the handler outside the guard scope (build an
"ok"/"exn" outcome tag inside guard, then dispatch outside) so that a
re-throw from the handler propagates past this catch — matching Haskell
semantics rather than infinite-looping in the same guard.
14 unit tests in tests/exceptions.sx (catch success, catch error, try
Right/Left, handle, throwIO + catch/try, evaluate, nested catch, do-bind
through catch, branch on try result, IORef-mutating handler).
Conformance: safediv.hs (8/8) and trycatch.hs (8/8). Scoreboard now
285/285 tests, 36/36 programs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
JS top-level var was emitting (define <name> X) at SX top level,
permanently rebinding any SX primitive of that name (e.g. var list
= X broke (list ...) globally). Two-part fix:
1. wrap transpiled program in (let () ...) in js-eval so defines
scope to the eval and don't leak.
2. rename call-args constructor in js-transpile-args from list to
js-args (a variadic alias) so even within the eval's own scope,
JS vars named list don't shadow arg construction.
Array-literal transpile keeps list (arrays must be mutable).
built-ins/Object: 41/50 → 42/50. conformance.sh: 148/148.
New lib/datalog/builtins.sx: (< <= > >= = !=) and (is X expr) with
+ - * /. dl-eval-arith recursively evaluates nested compounds.
Safety analysis now walks body left-to-right tracking the bound
set: comparisons require all args bound, is RHS vars must be bound
(LHS becomes bound), = special-cases the var/non-var combos.
db.sx keeps the simple safety check as a forward-reference
fallback; builtins.sx redefines dl-rule-check-safety to the
comprehensive version. eval.sx dispatches built-ins through
dl-eval-builtin instead of erroring. 19 new tests.
Tokens → list of {:head :body} / {:query} clauses. SX symbols for
constants and variables (case-distinguished). not(literal) in body
desugars to {:neg literal}. Nested compounds permitted in arg
position for arithmetic; safety analysis (Phase 3) will gate them.
Conformance harness wraps lib/guest/conformance.sh; produces
lib/datalog/scoreboard.{json,md}.
js-new-call Object had set obj.__proto__ correctly, but then the
__callable__ returned a fresh (dict), which js-new-call's "use
returned dict over obj" rule honoured — losing the proto. Added
is-new check (this.__proto__ === Object.prototype) and return
this instead of a new dict when invoked as a constructor with
no/null args. Now new Object().__proto__ === Object.prototype.
built-ins/Object: 37/50 → 41/50. conformance.sh: 148/148.
Trivial wrapper: apl-run-file = apl-run ∘ file-read, where
file-read is built-in to OCaml SX.
Tests verify primes.apl, life.apl, quicksort.apl all parse
end-to-end (their last form is a :dfn AST). Source-then-call
test confirms the loaded file's defined fn is callable, even
when the algorithm itself can't fully execute (primes' inline
⍵ rebinding still missing — :glyph-token, not :name-token).
js-loose-eq only had a __js_string_value__ unwrap clause, so
Object(1.1) == 1.1 returned false. Added parallel clauses for
__js_number_value__ and __js_boolean_value__ in both directions.
Now new Number(5) == 5, Object(true) == true, etc.
built-ins/Object: 26/50 → 37/50. conformance.sh: 148/148.
Per ES spec, Object('s') instanceof String, Object(42).constructor
=== Number, etc. Was passing primitives through as-is. Added cond
clauses to Object.__callable__ that dispatch by type and call
(js-new-call String/Number/Boolean (list arg)). The wrapper
constructors already store __js_*_value__ on this.
built-ins/Object: 16/50 → 26/50. conformance.sh: 148/148.
