erlang: wire file:list_dir/1 against file-list-dir (Phase 8, +4 ffi tests); 729/729, progress log

lib/erlang/runtime.sx

@@ -1561,6 +1561,7 @@
     (er-register-pure-bif! "crypto" "hash" 2 er-bif-crypto-hash)
     (er-register-pure-bif! "cid" "from_bytes" 1 er-bif-cid-from-bytes)
     (er-register-pure-bif! "cid" "to_string" 1 er-bif-cid-to-string)
+    (er-register-bif! "file" "list_dir" 1 er-bif-file-list-dir)
     (er-mk-atom "ok")))
 
 ;; Register everything at load time.

lib/erlang/scoreboard.json

@@ -1,7 +1,7 @@
 {
   "language": "erlang",
-  "total_pass": 715,
-  "total": 715,
+  "total_pass": 729,
+  "total": 729,
   "suites": [
     {"name":"tokenize","pass":62,"total":62,"status":"ok"},
     {"name":"parse","pass":52,"total":52,"status":"ok"},
@@ -12,7 +12,7 @@
     {"name":"bank","pass":8,"total":8,"status":"ok"},
     {"name":"echo","pass":7,"total":7,"status":"ok"},
     {"name":"fib","pass":8,"total":8,"status":"ok"},
-    {"name":"ffi","pass":14,"total":14,"status":"ok"},
+    {"name":"ffi","pass":28,"total":28,"status":"ok"},
     {"name":"vm","pass":78,"total":78,"status":"ok"}
   ]
 }

lib/erlang/scoreboard.md

@@ -1,6 +1,6 @@
 # Erlang-on-SX Scoreboard
 
-**Total: 715 / 715 tests passing**
+**Total: 729 / 729 tests passing**
 
 |  | Suite | Pass | Total |
 |---|---|---|---|
@@ -13,7 +13,7 @@
 | ✅ | bank | 8 | 8 |
 | ✅ | echo | 7 | 7 |
 | ✅ | fib | 8 | 8 |
-| ✅ | ffi | 14 | 14 |
+| ✅ | ffi | 28 | 28 |
 | ✅ | vm | 78 | 78 |
 
 

lib/erlang/tests/ffi.sx

@@ -141,9 +141,24 @@
   "true")
 
 (er-ffi-test
-  "file:list_dir unregistered"
-  (er-lookup-bif "file" "list_dir" 1)
-  nil)
+  "file:list_dir ok tag"
+  (ffi-nm (ffi-ev "element(1, file:list_dir(\"lib/erlang\"))"))
+  "ok")
+
+(er-ffi-test
+  "file:list_dir non-empty"
+  (ffi-nm (ffi-ev "case file:list_dir(\"lib/erlang\") of {ok, L} -> length(L) > 3 end"))
+  "true")
+
+(er-ffi-test
+  "file:list_dir entries are binaries"
+  (ffi-nm (ffi-ev "case file:list_dir(\"lib/erlang\") of {ok, L} -> is_binary(hd(L)) end"))
+  "true")
+
+(er-ffi-test
+  "file:list_dir missing enoent"
+  (ffi-nm (ffi-ev "element(2, file:list_dir(\"/no/such/dir/xyz\"))"))
+  "enoent")
 
 ;; ── Still deferred (no host primitive): httpc (HTTP client, v2),
 ;; sqlite-* (v2 indexes). Assert NOT registered so a future iteration

plans/erlang-on-sx.md

@@ -117,7 +117,7 @@ Replace today's hardcoded BIF dispatch (`er-apply-bif`/`er-apply-remote-bif` in
 - [x] Term-marshalling helpers: `er-of-sx` (SX → Erlang) and `er-to-sx` (Erlang → SX). atom ↔ symbol, nil ↔ `()`, cons → list, tuple → list (one-way; tuples flatten), binary ↔ SX string, integer / float / boolean passthrough. **+23 runtime tests** (623/623 total). Erlang maps (`dict ↔ map`) deferred — Erlang map term not implemented in this port; will land when `#{}` syntax does. Pids, refs, funs pass through unchanged. SX strings on the way back become Erlang binaries (most useful FFI return shape).
 - [x] `crypto:hash/2` — **WIRED 2026-05-18** against `crypto-sha256`/`crypto-sha512`/`crypto-sha3-256` (loops/fed-prims). `crypto:hash(Type, Data)`: `Type` ∈ `sha256|sha512|sha3_256` atom; `Data` an Erlang binary/string/charlist (→ SX byte-string via `er-source-to-string`). Returns the **raw digest as an Erlang binary** (host hex → bytes via `er-hex->bytes`). Bad type / non-binary → `error:badarg`. 6 ffi tests (digest sizes 32/64, sha3 is_binary, deterministic, distinct, badarg).
 - [x] `cid:from_bytes/1`, `cid:to_string/1` — **WIRED 2026-05-18**. `cid:from_bytes(Bin)` → CIDv1 raw-codec (0x55), sha2-256 multihash built in SX (`[0x12,0x20]++digest`) fed to `cid-from-bytes`; returned as an Erlang binary string. `cid:to_string(Term)` → canonical CIDv1 of the term's stable `er-format-value` string via `cid-from-sx` (cbor-encode rejects marshalled symbols, so `er-to-sx` is unencodable for compound terms — string form is total + deterministic). 7 ffi tests (is_binary, deterministic, distinct-inputs, non-binary badarg, to_string is_binary/deterministic/distinct).
-- [x] `file:read_file/1`, `file:write_file/2`, `file:delete/1` — **+10 eval tests** (633/633 total). Returns `{ok, Binary}` / `ok` / `{error, Reason}` where Reason is `enoent`/`eacces`/`enotdir`/`eisdir`/`posix_error` (classified from the SX `file-read`/`-write`/`-delete` exception string). Path accepts SX string, Erlang binary, or Erlang char-code list. `file:list_dir/1` deferred — no directory-listing primitive in this SX runtime; see Blockers.
+- [x] `file:read_file/1`, `file:write_file/2`, `file:delete/1` — **+10 eval tests** (633/633 total). Returns `{ok, Binary}` / `ok` / `{error, Reason}` where Reason is `enoent`/`eacces`/`enotdir`/`eisdir`/`posix_error` (classified from the SX `file-read`/`-write`/`-delete` exception string). Path accepts SX string, Erlang binary, or Erlang char-code list. **`file:list_dir/1` WIRED 2026-05-18** against `file-list-dir` → `{ok, [Binary]}` (entries marshalled via `er-of-sx`) / `{error, Reason}` (same `er-classify-file-error` mapping; missing dir → `enoent`). 4 ffi tests (ok-tag, non-empty, entries-are-binaries, missing-enoent).
 - [ ] `httpc:request/4` — **BLOCKED** (no HTTP client primitive). See Blockers.
 - [ ] `sqlite:open/1`, `sqlite:close/1`, `sqlite:exec/2`, `sqlite:query/2` — **BLOCKED** (no SQLite primitive). See Blockers.
 - [x] Tests: 1 round-trip per BIF; suite name `ffi`; conformance scoreboard auto-picks it up — **+14 ffi tests** at 637/637 total. Suite covers the 3 implemented file BIFs (9 tests: write-ok, read-ok-tag, payload-is-binary, byte_size content, missing-enoent, bad-path-enoent, binary-payload round-trip, delete-ok, read-after-delete-enoent) plus 5 negative asserts (one per blocked BIF — `crypto:hash`/`cid:from_bytes`/`file:list_dir`/`httpc:request`/`sqlite:exec`) so this suite fails fast if a future iteration adds a wrapper without registering proper tests. Target "+40 ffi tests" was relative to the original 5-BIF-family plan; with 5 of those families blocked on host primitives, the achievable count is 14 — the suite scaffolding is what matters and is ready to accept the remaining tests when the primitives land.
@@ -159,6 +159,8 @@ The Phase 9 opcodes are registered, tested, and bridged SX↔OCaml, but inert: n
 
 _Newest first._
 
+- **2026-05-18 Phase 8 host-primitive BIFs wired (crypto / cid / file:list_dir)** — `loops/fed-prims` (merged at architecture `380bc69f`) delivered the platform primitives; wired the 3 previously-BLOCKED Phase 8 BIF groups in `lib/erlang/runtime.sx` as `er-register-pure-bif!`/`er-register-bif!` entries with term marshalling at the boundary. **`crypto:hash/2`** → `crypto-sha256`/`crypto-sha512`/`crypto-sha3-256`; atom `Type` dispatch, `er-source-to-string` for `Data`, host hex result → raw bytes via new `er-hexval`/`er-hex->bytes`, returns Erlang binary; bad type/arg → `error:badarg`. **`cid:from_bytes/1`** → `cid-from-bytes` with raw codec `0x55` + sha2-256 multihash assembled in SX (`[0x12,0x20]++digest`); **`cid:to_string/1`** → `cid-from-sx` of `er-format-value` (cbor-encode rejects `er-to-sx`-marshalled symbols; the canonical string form is total + deterministic). **`file:list_dir/1`** → `file-list-dir`, `{ok,[Binary]}` via `er-of-sx` / `{error,Reason}` reusing `er-classify-file-error`. Test gotcha caught + fixed: this Erlang port's binary parser only supports integer/var segments — `<<"abc">>` string-binary literals silently produce **empty** binaries, so the first-cut distinct-input tests compared two empty inputs and failed; rewrote ffi inputs to integer-segment binaries (`<<97,98,99>>`). ffi suite 14→**28** (3 BLOCKED negative-asserts flipped to positive+negative functional tests; `httpc`/`sqlite` kept as deferred unregistered-asserts per fed-prims handoff). Built `sx_server.exe` (dune, opam 5.2.0) at `380bc69f`; full conformance **729/729** (eval 385/385, vm 78/78, **ffi 28/28**, all process suites green). loops/erlang only — not merged, not pushed to architecture.
+
 - **2026-05-18 FIXED merge-blocking regression: cyclic-env hang in `er-env-derived-from?`** — A trial merge of loops/erlang → architecture regressed Erlang **715/715 → 0/0** on the architecture binary. Bisected: not loader semantics, not a uniform slowdown — pinpointed to the *single* Phase 7 capstone test (eval.sx lines 1314-1346; prefix-1313 was byte-identical speed on both binaries, 27s, prefix-1346 was 28s on loops vs >5min/hung on architecture). Isolated further: spawn+reload alone 0.6s, reload+purge alone 0.3s, but spawn+reload+**purge over forever-blocked procs** hung. Root cause: `er-env-derived-from?` (transpile.sx, used by `code:purge`/`soft_purge` via `er-procs-on-env`) compared closure envs with `(= env target-env)`. loops/erlang's evaluator implements dict `=` as **object identity**; architecture's 131-commit-newer evaluator changed it to **structural deep equality**. Erlang closure envs are large and **cyclic** (a module fun's `:env` transitively references the fun), so structural `=` over them never terminates. Fix: use `identical?` (pointer-identity predicate, present + consistent `(true false)` on *both* binaries) — the actually-intended semantics and host-independent. Verified: full eval.sx on the architecture binary >200s/hung → **59s**; full 10-suite conformance on the architecture binary now **715/715** (eval 385/385, vm 78/78, ffi 14/14, all process suites green). loops/erlang behaviour unchanged (`identical?` ≡ its old `=`-identity). One-file change (`lib/erlang/transpile.sx`, +7/-2). The merge can now be re-attempted; this was the sole blocker.
 
 - **2026-05-15 Phase 10a — architecture traced, scoped, blocked on `lib/compiler.sx`** — Investigation-only iteration (correctly: faking compiler emission within scope is impossible and would be dishonest). Traced the full JIT path: `sx_vm.ml`'s `jit_compile_lambda` (the ref set at line 1206) invokes the SX-level `compile` from `lib/compiler.sx` via the CEK machine — that is the only SX→bytecode producer. Erlang's hot helpers are ordinary SX functions in `transpile.sx` that get JIT-compiled through exactly this path, so emitting `erlang.OP_*` means teaching `compiler.sx`'s `compile-call` to recognize them as intrinsics and emit the extension opcode (the file's own docstring already anticipates this — "Compilers call `extension-opcode-id` to emit extension opcodes" — designed but unimplemented; grep confirms zero `extension-opcode-id` uses in `compiler.sx`). `lib/compiler.sx` is lib-root: excluded by ground rules and the widened scope (editing it changes every guest's JIT — must be a shared-compiler session, not this loop). Recorded a precise Blockers entry + decomposed 10a into four numbered sub-steps (10a.1 intrinsic registry, 10a.2 `compile-call` emission with graceful CALL fallback, 10a.3 operand/stack contract for control opcodes, 10a.4 bytecode-emission tests) so the owning session can execute directly. Key payoff documented: all 10 BIF handlers (230-239) are already real, so they light up the instant 10a.1-10a.2 land — zero further Erlang-side work for the BIF speedup. No code changed; conformance unverified-but-untouched at **715/715** (no source touched). Phase 10's loop-reachable work (10b BIF half) is complete; the rest is correctly blocked and fully actionable elsewhere.