vwait used frame-lookup which doesn't honor `::` global routing. So
`vwait ::done` after `set ::done fired` (where set routes to root frame)
never saw the var change in the local frame, looping forever.
Added tcl-vwait-lookup helper that mirrors tcl-var-get's `::` routing
but returns nil instead of erroring on missing vars.
Was the deadlock that hung the full test suite past test 32.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Returns {<rel-name>: tuple-count} for relations with any tuples
or that are rule-headed (so empty IDB shows as :rel 0 rather than
disappearing). Skips placeholder entries from internal
dl-ensure-rel! calls. 4 tests cover basic, empty IDB, mixed
EDB+IDB, and empty-db cases.
db gains :facts-index {<rel>: {<first-arg-key>: tuples}} mirroring
the membership :facts-keys index. dl-add-fact! populates the index;
dl-match-positive walks the body literal's first arg under the
current subst — when it's bound to a non-var, look up by (str arg)
instead of scanning the full relation.
For chain-style recursive rules (parent X Y), (ancestor Y Z) the
inner Y has at most one parent, so the inner lookup returns 0–1
tuples instead of N. chain-25 saturation drops from ~33s to ~18s
real (~2x). chain-50 still long but tractable; next bottleneck is
subst dict copies during unification.
dl-retract! refreshed to keep the new index consistent: kept-index
rebuilt during EDB filter, IDB wipes clear all three slots.
Differential semi-naive test bumped to chain-12, semi-only count
test to chain-25.
Parser: { f = e; f = e; ... } -> (:record (F E)...). { base with f = e;
... } -> (:record-update BASE (F E)...). Eval builds a dict from field
bindings; record-update merges the new fields over the base dict — the
same dict representation already used for modules.
{ also added to at-app-start? so records are valid arg atoms. Field
access via the existing :field postfix unifies record/module access.
Record patterns deferred to a later iteration.
lib/ocaml/conformance.sh runs the full test suite, classifies each
result by description prefix into one of 14 suites (tokenize, parser,
eval-core, phase2-refs/loops/function/exn, phase3-adt, phase4-modules,
phase5-hm, phase6-stdlib, let-and, phase1-params, misc), and emits
scoreboard.json + scoreboard.md.
Per the briefing: "Once the scoreboard exists (Phase 5.1), it is your
north star." Real OCaml testsuite vendoring deferred — needs more
stdlib + ADT decls to make .ml files runnable.
Parser: try-consume-param! handles ident, wildcard _ (fresh __wild_N
name), unit () (fresh __unit_N), typed (x : T) (skips signature).
parse-fun and parse-let (inline) reuse the helper; top-level
parse-decl-let inlines a similar test.
test.sh timeout bumped from 60s to 180s — the growing suite was hitting
the cap and reporting spurious failures.
Adds (cotagged P T1 T2) — P has both T1 and T2 with T1 != T2 — and
(tag-pair-count T1 T2 N) which counts posts cotagged with each
distinct (T1, T2) pair. Demonstrates count aggregation against a
recursive-then-aggregated stream of derived tuples.
2 new demo tests: cooking + vegetarian co-occurrence on a small
data set, and a count-of-co-occurrences query.
js-to-boolean was returning true for NaN because NaN != 0 by IEEE
semantics — the (= v 0) test fell through to the truthy else.
Per ES, NaN is one of the falsy values. Added a
(js-number-is-nan v) clause.
built-ins/Boolean: 19/27 → 21/27. conformance.sh: 148/148.
dl-query now auto-dispatches on the first element's shape:
- positive literal (head is a symbol) or {:neg ...} dict → wrap
- list of literals → conjunctive query
dl-query-coerce normalizes; dl-query-user-vars collects the union
of user-named vars (deduped, '_' filtered) for projection. Old
single-literal callers unchanged.
(dl-query db '(p X)) ; single
(dl-query db '((p X) (q X))) ; conjunction
(dl-query db (list '(n X) '(> X 2))) ; with comparison
2 new api tests cover multi-goal AND and conjunction with comparison.
Bug: dl-check-stratifiable iterated body literals looking only for
explicit :neg literals, missing aggregate cycles. Now also walks
aggregates via dl-aggregate-dep-edge — q(N) :- count(N, X, q(X))
correctly errors out at saturation time.
3 new tests cover:
- recursion-through-aggregation rejected
- negation + aggregation coexist when in different strata
- min over empty derived relation produces no result
Adds the canonical Phase 10 example from the plan: "Posts about
cooking by people I follow (transitively)." dl-demo-cooking-rules
defines reach over the follow graph (recursive transitive closure)
and cooking-post-by-network joining reach + authored + (tagged P
cooking). 3 new demo tests cover transitive network, direct-only
follow, and empty-network cases.
(findall L V Goal) — bind L to the distinct V values for which Goal
holds, or the empty list when none. One-line addition to
dl-do-aggregate that returns the unreduced list. Tests cover EDB,
derived relation, and empty cases.
Useful for "give me all the X such that ..." queries without
scalar reduction.
OCaml-on-SX is the deferred second consumer for lib/guest/hm.sx step 8.
lib/ocaml/infer.sx assembles Algorithm W on top of the shipped algebra:
- Var: lookup + hm-instantiate.
- Fun: fresh-tv per param, auto-curried via recursion.
- App: unify against hm-arrow, fresh-tv for result.
- Let: generalize rhs over (ftv(t) - ftv(env)) — let-polymorphism.
- If: unify cond with Bool, both branches with each other.
- Op (+, =, <, etc.): builtin signatures (int*int->int monomorphic,
=/<> polymorphic 'a->'a->bool).
Tests pass for: literals, fun x -> x : 'a -> 'a, let id ... id 5/id true,
fun f x -> f (f x) : ('a -> 'a) -> 'a -> 'a (twice).
Pending: tuples, lists, pattern matching, let-rec, modules in HM.
(p X _), (p _ Y) — the two _ are now different variables, matching
standard Datalog semantics. Previously both _ symbols were the same
SX symbol, so unification across them gave wrong answers.
Fix in db.sx: dl-rename-anon-term + dl-rename-anon-lit walk a term
or literal and replace each '_' symbol with a fresh _anon<N>.
dl-make-anon-renamer returns a counter-based name generator scoped
per call. dl-rename-anon-rule applies it to head and body of a
rule. dl-add-rule! invokes the renamer before safety check.
eval.sx: dl-query renames anon vars in the goal before search and
filters '_' out of the projection so user-facing results aren't
polluted with internal _anon<N> bindings.
The previous "underscore in head ok" test now correctly rejects
(p X _) :- q(X) as unsafe (the head's fresh anon var has no body
binder). New "underscore in body only" test confirms the safe
case. Two regression tests for rule-level and goal-level
independence.
Parser collects multiple bindings via 'and', emitting (:def-rec-mut
BINDINGS) for let-rec chains and (:def-mut BINDINGS) for non-rec.
Single bindings keep the existing (:def …) / (:def-rec …) shapes.
Eval (def-rec-mut): allocate placeholder cell per binding, build joint
env where each name forwards through its cell, then evaluate each rhs
against the joint env and fill the cells. Even/odd mutual-rec works.
Previous version put (define _throttle-ms ...) (define _debounce-ms ...)
(define _strip-throttle-debounce ...) inside emit-on's body, redefining
them on every call to emit-on. The kernel JIT-compiled the helper fn
fresh each invocation, doubling compile time across the suite and
pushing many tests over their wall-clock deadline (35 cumulative-only
timeouts in the latest batched run, up from 0).
Move the three definitions to module-level. Use (set! _throttle-ms nil)
(set! _debounce-ms nil) at the top of emit-on to reset state for each
call. JIT compilation of _strip-throttle-debounce now happens once.
Verified: hs-upstream-expressions/dom-scope went from 18/20 (with two
state-related timeouts) back to 20/20, suite wall-time 232s → 75s.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
dl-find-bindings now uses dl-fb-aux lits db subst i n (indexed
iteration via nth) instead of recursive (rest lits). Eliminates
O(N²) list-copy per body of length N. chain-15 saturation 25s
→ 16s; chain-25 finishes in 33s real (vs. timeout previously).
Bumped semi_naive tests to chain-10 differential + chain-15
semi-only count (was chain-5/chain-5). Blocker entry refreshed.
lib/ocaml/runtime.sx defines the stdlib in OCaml syntax (not SX): every
function exercises the parser, evaluator, match engine, and module
machinery built in earlier phases. Loaded once via ocaml-load-stdlib!,
cached in ocaml-stdlib-env, layered under user code via ocaml-base-env.
List: length, rev, rev_append, map, filter, fold_left/right, append,
iter, mem, for_all, exists, hd, tl, nth.
Option: map, bind, value, get, is_none, is_some.
Result: map, bind, is_ok, is_error.
Substrate validation: this stdlib is a nontrivial OCaml program — its
mere existence proves the substrate works.
New lib/datalog/demo.sx with three Datalog-as-query-language demos
over synthetic rose-ash data:
Federation: (mutual A B), (reachable A B), (foaf A C) over a
follows graph.
Content: (post-likes P N) via count aggregation, (popular P)
for likes >= 3, (interesting Me P) joining follows
+ authored + popular.
Permissions: (in-group A G) over transitive subgroup chains,
(can-access A R).
10 tests run each program against in-memory EDB tuples loaded via
dl-program-data.
Wiring to PostgreSQL and exposing as a service endpoint (/internal
/datalog) is out of scope for this loop — both would require
edits outside lib/datalog/. Programs above document the EDB shape
a real loader would populate.
Parser: module F (M) (N) ... = struct DECLS end -> (:functor-def NAME
PARAMS DECLS). module N = expr (non-struct) -> (:module-alias NAME
BODY-SRC). Functor params accept (P) or (P : Sig) — signatures
parsed-and-skipped via skip-optional-sig.
Eval: ocaml-make-functor builds curried host-SX closures from module
dicts to a module dict. ocaml-resolve-module-path extended for :app so
F(A), F(A)(B), and Outer.Inner all resolve to dicts.
Phase 4 LOC ~290 cumulative (still well under 2000).
Was returning the input unchanged: eval('1+2') gave "1+2".
Per spec, eval(string) parses and evaluates as JS; non-string
passes through. Wired through js-eval (existing
lex/parse/transpile/eval pipeline).
built-ins/String fail count 13 → 11. conformance.sh: 148/148.
db gains a parallel :facts-keys {<rel>: {<tuple-string>: true}}
index alongside :facts. dl-tuple-key derives a stable string via
(str lit) — (p 30) and (p 30.0) collide correctly because SX
prints them identically. dl-add-fact! membership is now O(1)
instead of O(n) list scan; insert sequences for relations sized
N drop from O(N²) to O(N).
Wall clock on chain-7 saturation halves (~12s → ~6s); chain-15
roughly halves (~50s → ~25s) under shared CPU. Larger chains
still slow due to body-join overhead in dl-find-bindings —
Blocker entry refreshed with proposed follow-ups.
dl-retract! keeps both indices consistent: kept-keys is rebuilt
during the EDB filter, IDB wipes clear both lists and key dicts.
Parser: open Path and include Path top-level decls; Path is Ctor (.Ctor)*.
Eval resolves via ocaml-resolve-module-path (same :con-as-module-lookup
escape hatch used by :field). open extends the env with the module's
bindings; include also merges into the surrounding module's exports
(when inside a struct...end).
Path resolver lets M.Sub.x work for nested modules. Phase 4 LOC ~165.
New lib/datalog/api.sx: dl-program-data facts rules takes SX data
lists. Rules accept either dict form or list form using <- as the
rule arrow (since SX parses :- as a keyword). dl-rule constructor
for the dict shape. dl-assert! adds a fact and re-saturates;
dl-retract! drops EDB matches, wipes all rule-headed IDB
relations, and re-saturates from scratch — simplest correct
semantics until provenance tracking arrives.
9 API tests cover ancestor closure via data, dict-rule form,
dl-rule constructor, incremental assert/retract, cyclic-graph
reach, assert into empty, fact-style rule (no arrow), dict
passthrough.
module M = struct DECLS end parsed by sub-tokenising the body source
between struct and the matching end (nesting tracked via struct/begin/
sig/end). Field access is a postfix layer above parse-atom, binding
tighter than application: f r.x -> (:app f (:field r "x")).
Eval (:module-def NAME DECLS) builds a dict via ocaml-eval-module
running decls in a sub-env. (:field EXPR NAME) looks up dict fields,
treating (:con NAME) heads as module-name lookups instead of nullary
ctors so M.x works with M as a module.
Phase 4 LOC so far: ~110 lines (well under 2000 budget).
New lib/datalog/aggregates.sx: (count R V Goal), (sum R V Goal),
(min R V Goal), (max R V Goal). dl-eval-aggregate runs
dl-find-bindings on the goal under the outer subst, collects
distinct values of V, applies the operator, binds R. Empty input:
count/sum return 0; min/max produce no binding (rule fails).
Group-by emerges naturally from outer-subst substitution into the
goal — `popular(P) :- post(P), count(N, U, liked(U, P)), >=(N, 3).`
counts per-post.
Stratifier extended: dl-aggregate-dep-edge contributes a
negation-like edge so the aggregate's goal relation is fully
derived before the aggregate fires (non-monotonicity respected).
Safety relaxed for aggregates: goal-internal vars are existentials,
only the result var becomes bound.
New lib/datalog/strata.sx: dl-build-dep-graph (relation -> deps with
:neg flag), Floyd-Warshall reachability, SCC-via-mutual-reach for
non-stratifiability detection, iterative dl-compute-strata, and
dl-group-rules-by-stratum.
eval.sx refactor:
- dl-saturate-rules! db rules — semi-naive worker over a rule subset
- dl-saturate! db — stratified driver. Rejects non-stratifiable
programs at saturation time, then iterates strata in order
- dl-match-negation — succeeds iff inner positive match is empty
Order-aware safety in dl-rule-check-safety (Phase 4) already
required negation vars to be bound by a prior positive literal.
Stratum dict keys are strings (SX dicts don't accept ints).
Phase 6 magic sets deferred — opt-in path, semi-naive default
suffices for current workloads.
Parser: try EXPR with | pat -> handler | ... -> (:try EXPR CLAUSES).
Eval delegates to SX guard with else matching the raised value against
clause patterns; re-raises on no-match. raise/failwith/invalid_arg
shipped as builtins. failwith "msg" raises ("Failure" msg) so
| Failure msg -> ... patterns match.
Sugar for fun + match. AST (:function CLAUSES) -> unary closure that
runs ocaml-match-clauses on its arg. let rec recognises :function as a
recursive rhs and ties the knot via cell, so
let rec map f = function | [] -> [] | h::t -> f h :: map f t
works. ocaml-match-eval refactored to share clause-walk with function.
dl-saturate! is now semi-naive: tracks a per-relation delta dict,
and on each iteration walks every positive body-literal position,
substituting the delta of its relation while joining the rest
against the previous-iteration DB. Candidates are collected before
mutating the DB so the "full" sides see a consistent snapshot.
Rules with no positive body literal (e.g. (p X) :- (= X 5).)
fall back to a one-shot naive pass via dl-collect-rule-candidates.
dl-saturate-naive! retained as the reference implementation; 8
differential tests compare per-relation tuple counts on every
recursive program. Switched dl-tuple-member? to indexed iteration
instead of recursive rest (eliminates per-step list copy). Larger
chains under bundled conformance trip O(n) membership × CPU
sharing — added a Blocker to swap relations to hash-set membership.
Parser: for i = lo to|downto hi do body done, while cond do body done.
AST: (:for NAME LO HI :ascend|:descend BODY) and (:while COND BODY).
Eval re-binds the loop var per iteration; both forms evaluate to unit.
ref is a builtin boxing its arg in a one-element list. Prefix ! parses
to (:deref ...) and reads via (nth cell 0). := joins the binop
precedence table at level 1 right-assoc and mutates via set-nth!.
Closures share the underlying cell.
Two-phase grammar: parse-expr-no-seq (prior entry) + parse-expr wraps
it with ;-chaining. List bodies keep parse-expr-no-seq so ; remains a
separator inside [...]. Match clause bodies use the seq variant and stop
at | — real OCaml semantics. Trailing ; before end/)/|/in/then/else/eof
permitted.
Patterns: wildcard, literal, var, ctor (nullary + arg, flattens tuple
args so Pair(a,b) -> (:pcon "Pair" PA PB)), tuple, list literal, cons
:: (right-assoc), unit. Match: leading | optional, (:match SCRUT
CLAUSES) with each clause (:case PAT BODY). Body parsed via parse-expr
because | is below level-1 binop precedence.
ocaml-parse-program: program = decls + bare exprs, ;;-separated.
Each decl is (:def …), (:def-rec …), or (:expr …). Body parsing
re-feeds the source slice through ocaml-parse — shared-state refactor
deferred.
lib/hyperscript/tokenizer.sx — added cursor + follow-set wrapper over
the existing flat-list tokenize output:
hs-stream src → {:tokens :pos :follows :last-match :last-ws}
hs-stream-current s → next non-WS token (skips WS, captures :last-ws)
hs-stream-match s value → consume if value matches & not in follow set
hs-stream-match-type s ...types → consume if upstream type name matches
hs-stream-match-any s ...names → consume if value matches any name
hs-stream-match-any-op s ...ops → consume if op token & value matches
hs-stream-peek s value n → look n non-WS tokens ahead, no consume
hs-stream-consume-until s marker → collect tokens until marker
hs-stream-consume-until-ws s → collect until next whitespace
hs-stream-push-follow! / pop-follow!
hs-stream-push-follows! / pop-follows! n
hs-stream-clear-follows! → saved / restore-follows! saved
hs-stream-last-match / last-ws
hs-stream-type-map maps our lowercase type names to upstream's
("ident" → "IDENTIFIER", "number" → "NUMBER", etc.) so type-based
matching works against upstream test expectations.
13 tokenizer-stream tests now pass; 30/30 in hs-upstream-core/tokenizer.
Skips remaining: 5 (down from 18).
- 2 template-component scope tests
- 1 async event dispatch (until event keyword works)
- left for later: needs more architectural work
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
new (new Object("")) hung because js-new-call called
js-get-ctor-proto -> js-ctor-id -> inspect, and inspect on a
wrapper-with-proto-chain recurses through the prototype's
lambdas forever. Added (js-function? ctor) precheck at the top
of js-new-call that raises a TypeError instance instead.
conformance.sh: 148/148.
parser.sx parse-toggle-cmd: when seeing 'toggle .foo for', peek the
following two tokens. If they are '<ident> in', it is a for-in loop
and toggle does NOT consume 'for' as a duration clause. Restores the
trailing for-in to the command list.
parser.sx parse-on (handler modifiers): recognize 'throttled at <ms>'
and 'debounced at <ms>' as handler modifiers. Captured as :throttle /
:debounce kwargs in the on-form parts list.
compiler.sx emit-on: pre-extract :throttle / :debounce from parts via
new _strip-throttle-debounce helper before scan-on, then wrap the built
handler with (hs-throttle! handler ms) or (hs-debounce! handler ms).
runtime.sx: hs-throttle! — closure with __hs-last-fire timestamp,
fires immediately and drops events arriving within ms of the last fire.
hs-debounce! — closure with __hs-timer, clears any pending timer and
schedules a new setTimeout(handler, ms) so only the last burst event
fires.
Both formerly-architectural skips now pass:
- "toggle does not consume a following for-in loop"
- "throttled at <time> drops events within the window"
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Brings the architecture branch (559 commits ahead — R7RS step 4-6, JIT
expansion, host_error wrapping, bytecode compiler, etc.) into the
loops/haskell line of work. Conflict in lib/haskell/conformance.sh:
architecture replaced the inline driver with a thin wrapper delegating
to lib/guest/conformance.sh + a config file. Resolved by taking the
wrapper and extending lib/haskell/conformance.conf with all programs
added under loops/haskell (caesar, runlength-str, showadt, showio,
partial, statistics, newton, wordfreq, mapgraph, uniquewords, setops,
shapes, person, config, counter, accumulate, safediv, trycatch) plus
adding map.sx and set.sx to PRELOADS.
plans/haskell-completeness.md gains three new follow-up phases:
- Phase 17 — parser polish (`(x :: Int)` annotations, mid-file imports)
- Phase 18 — one ambitious conformance program (lambda-calc / Dijkstra /
JSON parser candidate list)
- Phase 19 — conformance speed (batch all suites in one sx_server
process to compress the 25-min run to single-digit minutes)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SX strictly arity-checks lambdas; JS allows passing more args than
declared (extras accessible via arguments). Was raising "f expects
1 args, got 2" whenever Array.from passed (value, index) to a
1-arg mapFn. Fixed in js-build-param-list: every JS param list
now ends with &rest __extra_args__ unless an explicit rest is
present, so extras are silently absorbed.
conformance.sh: 148/148.
