erlang: lists:sort/1,2 + lists:usort/1 with full term order (lists_ext suite, 788/788)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 1m6s
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 1m6s
New lib/erlang/lists-ext.sx (loaded after runtime.sx): stable merge sort over an SX-list bridge. sort/1 and usort/1 use full Erlang term order via a self-contained er-ext-lt? (deep tuple/list compare that the shared er-lt? lacks); sort/2 takes a fun(A,B)->bool comparator. Registration wraps er-register-builtin-bifs! so the BIFs survive the mid-run registry resets done by tests/runtime.sx. Roadmap is saturated within this loop's scope; this is forever-loop stdlib hardening. New file forced by the broken sx-tree write tools in this worktree (see Blockers) — authored via Write + sx_validate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -159,6 +159,8 @@ The Phase 9 opcodes are registered, tested, and bridged SX↔OCaml, but inert: n
|
||||
|
||||
_Newest first._
|
||||
|
||||
- **2026-06-30 stdlib hardening — `lists:sort/1,2` + `lists:usort/1`** — Roadmap is saturated within this loop's scope (every remaining `[ ]` is blocked: `httpc`/`sqlite` on absent host primitives, 10a/10c on out-of-scope `lib/compiler.sx`). Continued as forever-loop hardening by filling idiomatic-Erlang stdlib gaps. Added the `lists` sort family in a **new file `lib/erlang/lists-ext.sx`** (loaded after `runtime.sx`): stable merge sort over an SX-list bridge, registered via `er-register-pure-bif!`. `lists:sort/1` and `usort/1` use full Erlang term order; `sort/2` takes a `fun(A,B)->bool` comparator. **Two notable findings:** (1) the shared `er-lt?` (transpile.sx) only deep-compares numbers/atoms/strings and treats *any two tuples (or lists) as order-equal* — so `lists:sort` (and, latently, `min/2`/`max/2`) would not order compound terms. Fixed locally with a self-contained `er-ext-lt?` that compares tuples by arity-then-elementwise and lists elementwise (shorter proper prefix first), delegating cross-type cases to `er-lt?`. `er-lt?` itself left untouched (shared by the `<` operator; can't edit transpile.sx — see Blockers). (2) `tests/runtime.sx` resets the BIF registry mid-run via `er-register-builtin-bifs!`, which would wipe a one-shot registration; so `lists-ext.sx` **wraps** `er-register-builtin-bifs!` to re-add its BIFs on every rebuild. New `lists_ext` suite (17 tests: term order, dup-keeping, stability, descending comparator, usort dedup). Conformance **771 → 788/788** (12→13 suites). New-file workaround forced because every sx-tree write tool (incl. `sx_write_file`) raises yojson "Expected string, got null" in this worktree — authored via the `Write` fallback + `sx_validate`, the same pattern other loops use. loops/erlang only.
|
||||
|
||||
- **2026-05-18 Phase 8 host-primitive BIFs wired (crypto / cid / file:list_dir)** — `loops/fed-prims` (merged at architecture `380bc69f`) delivered the platform primitives; wired the 3 previously-BLOCKED Phase 8 BIF groups in `lib/erlang/runtime.sx` as `er-register-pure-bif!`/`er-register-bif!` entries with term marshalling at the boundary. **`crypto:hash/2`** → `crypto-sha256`/`crypto-sha512`/`crypto-sha3-256`; atom `Type` dispatch, `er-source-to-string` for `Data`, host hex result → raw bytes via new `er-hexval`/`er-hex->bytes`, returns Erlang binary; bad type/arg → `error:badarg`. **`cid:from_bytes/1`** → `cid-from-bytes` with raw codec `0x55` + sha2-256 multihash assembled in SX (`[0x12,0x20]++digest`); **`cid:to_string/1`** → `cid-from-sx` of `er-format-value` (cbor-encode rejects `er-to-sx`-marshalled symbols; the canonical string form is total + deterministic). **`file:list_dir/1`** → `file-list-dir`, `{ok,[Binary]}` via `er-of-sx` / `{error,Reason}` reusing `er-classify-file-error`. Test gotcha caught + fixed: this Erlang port's binary parser only supports integer/var segments — `<<"abc">>` string-binary literals silently produce **empty** binaries, so the first-cut distinct-input tests compared two empty inputs and failed; rewrote ffi inputs to integer-segment binaries (`<<97,98,99>>`). ffi suite 14→**28** (3 BLOCKED negative-asserts flipped to positive+negative functional tests; `httpc`/`sqlite` kept as deferred unregistered-asserts per fed-prims handoff). Built `sx_server.exe` (dune, opam 5.2.0) at `380bc69f`; full conformance **729/729** (eval 385/385, vm 78/78, **ffi 28/28**, all process suites green). loops/erlang only — not merged, not pushed to architecture.
|
||||
|
||||
- **2026-05-18 FIXED merge-blocking regression: cyclic-env hang in `er-env-derived-from?`** — A trial merge of loops/erlang → architecture regressed Erlang **715/715 → 0/0** on the architecture binary. Bisected: not loader semantics, not a uniform slowdown — pinpointed to the *single* Phase 7 capstone test (eval.sx lines 1314-1346; prefix-1313 was byte-identical speed on both binaries, 27s, prefix-1346 was 28s on loops vs >5min/hung on architecture). Isolated further: spawn+reload alone 0.6s, reload+purge alone 0.3s, but spawn+reload+**purge over forever-blocked procs** hung. Root cause: `er-env-derived-from?` (transpile.sx, used by `code:purge`/`soft_purge` via `er-procs-on-env`) compared closure envs with `(= env target-env)`. loops/erlang's evaluator implements dict `=` as **object identity**; architecture's 131-commit-newer evaluator changed it to **structural deep equality**. Erlang closure envs are large and **cyclic** (a module fun's `:env` transitively references the fun), so structural `=` over them never terminates. Fix: use `identical?` (pointer-identity predicate, present + consistent `(true false)` on *both* binaries) — the actually-intended semantics and host-independent. Verified: full eval.sx on the architecture binary >200s/hung → **59s**; full 10-suite conformance on the architecture binary now **715/715** (eval 385/385, vm 78/78, ffi 14/14, all process suites green). loops/erlang behaviour unchanged (`identical?` ≡ its old `=`-identity). One-file change (`lib/erlang/transpile.sx`, +7/-2). The merge can now be re-attempted; this was the sole blocker.
|
||||
@@ -251,6 +253,8 @@ _Newest first._
|
||||
|
||||
## Blockers
|
||||
|
||||
- **sx-tree WRITE tools broken in this worktree** (2026-06-30). Every sx-tree edit/write tool (`sx_replace_node`, `sx_insert_child`, `sx_insert_near`, …) **and even `sx_write_file`** raise `Yojson.Util.Type_error("Expected string, got null")` against the `mcp_tree.exe` bound in `.mcp.json` (the `/root/rose-ash/...` main-worktree binary). Read/comprehension tools (`sx_validate`, `sx_find_all`, `sx_eval`, `sx_read_*`) work fine. **Workaround:** author/edit `.sx` files with the plain `Write` tool, then `sx_validate` — the same fallback other loops document (see `project_host_on_sx.md`, `project_content_on_sx.md` memory). This is why new `lib/erlang` BIFs land as fresh files (e.g. `lists-ext.sx`) rather than in-place edits to the large `transpile.sx`/`runtime.sx`. Real fix: rebuild `mcp_tree.exe` from current `hosts/ocaml` (out of this loop's binary-build scope) or repoint `.mcp.json` at a fixed binary.
|
||||
|
||||
- **Phase 10a — opcode emission requires `lib/compiler.sx` (out of scope)** (2026-05-15). Architecture fully traced this iteration: the OCaml JIT (`sx_vm.ml` `jit_compile_lambda`, ref-set at line 1206) invokes the SX-level `compile` from **`lib/compiler.sx`** via the CEK machine; that is the sole SX→bytecode producer. Erlang's hot helpers (`er-match-tuple`, `er-bif-*`, …) are SX functions in `transpile.sx` that get JIT-compiled through this path. To emit `erlang.OP_*` they must be recognized as intrinsics inside `compiler.sx`'s `compile-call` (the file's own docstring already anticipates this: "Compilers call `extension-opcode-id` to emit extension opcodes" — designed, not yet implemented). `lib/compiler.sx` is **lib-root**, excluded by the ground rules ("Don't edit lib/ root") and absent from the widened `lib/erlang/** + hosts/ocaml/** (extension only)` scope — editing it changes every guest language's JIT, so it must be owned by a shared-compiler session, not this loop. **Fix path:** that session implements 10a.1 (intrinsic registry in `compiler.sx`) + 10a.2 (`compile-call` emits the opcode when registered & `extension-opcode-id` non-nil, else generic CALL). Erlang's BIF handlers (10b, ids 230-239, all real) light up the instant emission exists — zero further work here. The control opcodes (222-229) additionally need 10a.3 (operand contract) + OCaml↔SX runtime-state bridging (Erlang scheduler/mailbox live in `lib/erlang/runtime.sx`, not OCaml).
|
||||
|
||||
- **Phase 9g — Perf bench gated on 9a** (2026-05-14). The conformance half of 9g (709/709 with stub VM loaded) is satisfied; the perf-bench half requires 9a's bytecode compiler to actually emit the new opcodes at hot call sites. Until then a benchmark would measure today's `er-bif-*` / `er-match-*` numbers unchanged (since the stub handlers wrap them 1-to-1). Re-fire 9g after 9a lands.
|
||||
|
||||
Reference in New Issue
Block a user