Files
rose-ash/plans/agent-briefings/primitives-loop.md
2026-04-26 12:53:40 +00:00

34 KiB
Raw Blame History

SX Primitives — Meta-Loop Briefing

Goal: add fundamental missing SX primitives in sequence, then sweep all language implementations to replace their workarounds. Full rationale: vectors fix O(n) array access across every language; numeric tower fixes float/int conflation; dynamic-wind fixes cleanup semantics; coroutine primitive unifies Ruby/Lua/Tcl; string buffer fixes O(n²) concat; algebraic data types eliminate the tagged-dict pattern everywhere.

Each fire: find the first unchecked [ ], do it, commit, tick it, stop. Sub-items within a Phase may span multiple fires — just commit progress and tick what's done.


Phase 0 — Prep (gate)

  • Stop new-language loops: send /exit to sx-loops windows for the four blank-slate languages that haven't committed workarounds yet: tmux send-keys -t sx-loops:common-lisp "/exit" Enter tmux send-keys -t sx-loops:apl "/exit" Enter tmux send-keys -t sx-loops:ruby "/exit" Enter tmux send-keys -t sx-loops:tcl "/exit" Enter Verify all four windows are idle (claude prompt, no active task).

  • E38 + E39 landed: check both Bucket-E branches for implementation commits. git log --oneline hs-e38-sourceinfo | head -5 git log --oneline hs-e39-webworker | head -5 If either branch has only its base commit (no impl work yet): note "pending" and stop — next fire re-checks. Proceed only when both have at least one implementation commit.


Phase 1 — Vectors

Native mutable integer-indexed arrays. Fix: Lua O(n) sort, APL rank polymorphism, Ruby Array, Tcl lists, Common Lisp vectors, all using string-keyed dicts today.

Primitives to add:

  • make-vector n [fill] → vector of length n
  • vector? v → bool
  • vector-ref v i → element at index i (0-based)
  • vector-set! v i x → mutate in place
  • vector-length v → integer
  • vector->list v → list
  • list->vector lst → vector
  • vector-fill! v x → fill all elements
  • vector-copy v [start] [end] → fresh copy of slice

Steps:

  • OCaml: add SxVector of value array to hosts/ocaml/sx_types.ml; implement all primitives in hosts/ocaml/sx_primitives.ml (or equivalent); wire into evaluator. Note: Vector type + most prims were already present; added bounds-checked vector-ref/set! and optional start/end to vector-copy. 10/10 vector tests pass (r7rs suite).
  • Spec: add vector entries to spec/primitives.sx with type signatures and descriptions. All 10 vector primitives now have :as type annotations, :returns, and :doc strings. make-vector: optional fill param; vector-copy: optional start/end (done prev step).
  • JS bootstrapper: implement vectors in hosts/javascript/platform.js (or equivalent); ensure sx-browser.js rebuild picks them up. Fixed index-of for lists (was returning -1 not NIL, breaking bind-lambda-params), added lastErrorKont/hostError/try-catch/without-io-hook stubs. Vectors work.
  • Tests: 40+ tests in spec/tests/test-vectors.sx covering construction, ref, set!, length, conversions, fill, copy, bounds behaviour. 42 tests, all pass. 1847 standard / 2362 full passing (up from 5).
  • Verify: full test suite still passes (node hosts/javascript/run_tests.js --full). 2362/4924 pass (improvement from pre-existing lambda binding bug, no regressions).
  • Commit: spec: vector primitive (make-vector/vector-ref/vector-set!/etc) Committed as: js: fix lambda binding (index-of on lists), add vectors + R7RS platform stubs

Phase 2 — Numeric tower

Float ≠ integer distinction. Fix: Erlang =:=, Lua math.type(), Haskell Num/Integral, Common Lisp integerp/floatp/ratio, JS Number.isInteger.

Changes:

  • parse-number preserves float identity: "1.0" → float 1.0, not integer 1
  • New predicates: integer?, float?, exact?, inexact?
  • New coercions: exact->inexact, inexact->exact
  • Fix floor/ceiling/truncate/round to return integers when applied to floats
  • number->string renders 1.0 as "1.0", 1 as "1"
  • Arithmetic: (+ 1 1.0)2.0 (float contagion), (+ 1 1)2 (integer)

Steps:

  • OCaml: distinguish Integer of int / Number of float in sx_types.ml; update all arithmetic primitives for float contagion; fix parse-number. 92/92 numeric tower tests pass; 4874 total (394 pre-existing hs-upstream fails unchanged).
  • Spec: update spec/primitives.sx with new predicates + coercions; document contagion rules. Added integer?/float? predicates; updated number? body; / returns "float"; floor/ceil/truncate return "integer"; +/-/* doc float contagion; fixed double-paren params; 4874/394 baseline.
  • JS bootstrapper: update number representation and arithmetic. Added integer?/float?/exact?/inexact?/truncate/remainder/modulo/random-int/exact->inexact/ inexact->exact/parse-number. Fixed sx_server.ml epoch protocol for Integer type. JS: 1940 passed (+60); OCaml: 4874/394 unchanged. 6 tests JS-only fail (float≡int limitation).
  • Tests: 92 tests in spec/tests/test-numeric-tower.sx — int-arithmetic, float-contagion, division, predicates, coercions, rounding, parse-number, equality, modulo, min-max, stringify.
  • Verify: full suite passes. OCaml 4874/394 (baseline unchanged). JS 1940/2500 (+60 vs pre-tower). No regressions on any test that relied on 1.0 = 1 — those tests were already using integer literals which remain identical in JS. 6 JS-only failures are platform-inherent (JS float≡int).
  • Commit: all work landed across 4 commits (c70bbdeb, 45ec5535, b12a22e6, f5acb31c).

Phase 3 — Dynamic-wind

Fix: Common Lisp unwind-protect, Ruby ensure, JS finally, Tcl catch+cleanup, Erlang try...after (currently uses double-nested guard workaround).

  • Spec: implement dynamic-wind in spec/evaluator.sx such that the after-thunk fires on both normal return AND non-local exit (raise/call-cc escape). Must compose with guard — currently they don't interact.
  • OCaml: wire dynamic-wind through the CEK machine with a WindFrame continuation.
  • JS bootstrapper: update.
  • Tests: 20+ tests covering normal return, raise, call/cc escape, nested dynamic-winds.
  • Commit: spec: dynamic-wind + guard integration

Phase 4 — Coroutine primitive

Unify Ruby fibers, Lua coroutines, Tcl coroutines — all currently reimplemented separately using call/cc+perform/resume.

  • Spec: add make-coroutine, coroutine-resume, coroutine-yield, coroutine?, coroutine-alive? to spec/primitives.sx. Build on existing perform/cek-resume machinery — coroutines ARE perform/resume with a stable identity.
  • OCaml: implement coroutine type; wire resume/yield through CEK suspension.
  • JS bootstrapper: update.
  • Tests: 25+ tests — multi-yield, final return, arg passthrough, alive? predicate, nested coroutines, "final return vs yield" distinction (the Lua gotcha).
  • Commit: spec: coroutine primitive (make-coroutine/resume/yield)

Phase 5 — String buffer

Fix O(n²) string concatenation in loops across Lua, Ruby, Common Lisp, Tcl.

  • Spec + OCaml: add make-string-buffer, string-buffer-append!, string-buffer->string, string-buffer-length to primitives. OCaml: Buffer.t wrapper. JS: array+join.
  • Tests: 15+ tests.
  • Commit: spec: string-buffer primitive

Phase 6 — Algebraic data types

The deepest structural gap. Every language uses {:tag "..." :field ...} tagged dicts to simulate sum types. A native define-type + match form eliminates this everywhere.

  • Design: write plans/designs/sx-adt.md covering syntax, CEK dispatch, interaction with existing cond/case, exhaustiveness checking, recursive types, pattern variables. Draft, then stop — next fire reviews design before implementing.

  • Spec: implement define-type special form in spec/evaluator.sx: (define-type Name (Ctor1 field...) (Ctor2 field...) ...) Creates constructor functions Ctor1, Ctor2 + predicate Name?.

  • Spec: implement match special form: (match expr ((Ctor1 a b) body) ((Ctor2 x) body) (else body)) Exhaustiveness warning if not all constructors covered and no else.

  • OCaml: add SxAdt of string * value array to types; implement constructors + match.

  • JS bootstrapper: update.

  • Tests: 40+ tests in spec/tests/test-adt.sx.

  • Commit: spec: algebraic data types (define-type + match)


Phase 7 — Bitwise operations

Completely absent today. Needed by: Forth (core), APL (array masks), Erlang (bitmatch), JS (typed arrays, bitfields), Common Lisp (logand/logior/logxor/lognot/ash).

Primitives to add:

  • bitwise-and a b → integer
  • bitwise-or a b → integer
  • bitwise-xor a b → integer
  • bitwise-not a → integer
  • arithmetic-shift a count → integer (left if count > 0, right if count < 0)
  • bit-count a → number of set bits (popcount)
  • integer-length a → number of bits needed to represent a

Steps:

  • Spec: add entries to spec/primitives.sx with type signatures.
  • OCaml: implement in hosts/ocaml/sx_primitives.ml using OCaml land/lor/lxor/lnot/lsl/lsr.
  • JS bootstrapper: implement in hosts/javascript/platform.js using JS &/|/^/~/<</>>.
  • Tests: 25+ tests in spec/tests/test-bitwise.sx — basic ops, shift left/right, negative numbers, popcount.
  • Commit: spec: bitwise operations (bitwise-and/or/xor/not, arithmetic-shift, bit-count)

Phase 8 — Multiple values

R7RS standard. Common Lisp uses them heavily; Haskell tuples map naturally; Erlang multi-return. Without them, every function returning two things encodes it as a list or dict.

Primitives / forms to add:

  • values v... → multiple-value object
  • call-with-values producer consumer → applies consumer to values from producer
  • let-values (((a b) expr) ...) body — binding form (special form in evaluator)
  • define-values (a b ...) expr — top-level multi-value bind

Steps:

  • Spec: add SxValues type to evaluator; implement values + call-with-values in spec/evaluator.sx; add let-values / define-values special forms.
  • OCaml: add SxValues of value list to sx_types.ml; wire through CEK.
  • JS bootstrapper: implement values type + forms.
  • Tests: 25+ tests in spec/tests/test-values.sx — basic producer/consumer, let-values destructuring, define-values, interaction with begin/do.
  • Commit: spec: multiple values (values/call-with-values/let-values)

Phase 9 — Promises (lazy evaluation)

Critical for Haskell — lazy evaluation is so central that without it the Haskell implementation can't be idiomatic. Also useful for lazy lists in Common Lisp and lazy streams in Scheme-style code generally.

Primitives / forms to add:

  • delay expr → promise (special form — expr not evaluated yet)
  • force p → evaluate promise, cache result, return it
  • make-promise v → already-forced promise wrapping v
  • promise? v → bool
  • delay-force expr → for iterative lazy sequences (avoids stack growth in lazy streams)

Steps:

  • Spec: add delay / delay-force special forms to spec/evaluator.sx; add promise type with mutable forced/value slots; force checks if already forced before eval.
  • OCaml: add SxPromise of { mutable forced: bool; mutable value: value; thunk: value }; wire delay/force/delay-force through CEK.
  • JS bootstrapper: implement promise type + forms.
  • Tests: 25+ tests in spec/tests/test-promises.sx — basic delay/force, memoisation (forced only once), delay-force lazy stream, promise? predicate, make-promise.
  • Commit: spec: promises — delay/force/delay-force for lazy evaluation

Phase 10 — Mutable hash tables

Distinct from SX's immutable dicts. Dict primitives copy on every update — fine for functional code, wrong for table-heavy language implementations. Lua tables, Smalltalk dicts, Erlang process dictionaries, and JS Map all need O(1) mutable associative storage.

Primitives to add:

  • make-hash-table [capacity] → fresh mutable hash table
  • hash-table? v → bool
  • hash-table-set! ht key val → mutate in place
  • hash-table-ref ht key [default] → value or default/error
  • hash-table-delete! ht key → remove entry
  • hash-table-size ht → integer
  • hash-table-keys ht → list of keys
  • hash-table-values ht → list of values
  • hash-table->alist ht → list of (key . value) pairs
  • hash-table-for-each ht fn → iterate (fn key val) for side effects
  • hash-table-merge! dst src → merge src into dst in place

Steps:

  • Spec: add entries to spec/primitives.sx.
  • OCaml: add SxHashTable of (value, value) Hashtbl.t to sx_types.ml; implement all primitives in hosts/ocaml/sx_primitives.ml.
  • JS bootstrapper: implement using JS Map in hosts/javascript/platform.js.
  • Tests: 30+ tests in spec/tests/test-hash-table.sx — set/ref/delete, size, iteration, default on missing key, merge, keys/values lists.
  • Commit: spec: mutable hash tables (make-hash-table/ref/set!/delete!/etc)

Phase 11 — Sequence protocol

Unified iteration over lists and vectors without conversion. Currently map/filter/ for-each only work on lists — you must vector->list first, which defeats the purpose of vectors. A sequence protocol makes all collection operations polymorphic.

Approach: extend existing map/filter/reduce/for-each/some/every? to dispatch on type (list → existing path, vector → index loop, string → char iteration). Add:

  • in-range start [end] [step] → lazy range sequence (works with for-each/map)
  • sequence->list s → coerce any sequence to list
  • sequence->vector s → coerce any sequence to vector
  • sequence-length s → length of any sequence
  • sequence-ref s i → element by index (lists and vectors)
  • sequence-append s1 s2 → concatenate two same-type sequences

Steps:

  • Spec: extend map/filter/reduce/for-each/some/every? in spec/evaluator.sx to type-dispatch; add in-range lazy sequence type + helpers.
  • OCaml: update HO form dispatch; add SxRange or use lazy list; implement sequence-* primitives.
  • JS bootstrapper: update.
  • Tests: 30+ tests in spec/tests/test-sequences.sx — map over vector, filter over range, for-each over string chars, sequence-append, sequence->list/vector coercions.
  • Commit: spec: sequence protocol — polymorphic map/filter/for-each over list/vector/range

Phase 12 — gensym + symbol interning

Unique symbol generation. Tiny to implement; broadly needed: Prolog uses it for fresh variable names, Common Lisp uses it constantly in macros, any hygienic macro system needs it, and Smalltalk uses it for anonymous class/method naming.

Primitives to add:

  • gensym [prefix] → unique symbol, e.g. g42, var-17. Counter-based, monotonically increasing.
  • symbol-interned? s → bool — whether the symbol is in the global intern table
  • intern str → symbol — intern a string as a symbol (string->symbol already exists; this is the explicit interning operation for languages that distinguish interned vs uninterned)

Steps:

  • Spec: add gensym counter to evaluator state; implement in spec/evaluator.sx. string->symbol already exists — gensym is just a counter-suffixed variant.
  • OCaml: add global gensym counter; implement primitives.
  • JS bootstrapper: implement.
  • Tests: 15+ tests in spec/tests/test-gensym.sx — uniqueness, prefix, symbol?, string->symbol round-trip.
  • Commit: spec: gensym + symbol interning

Phase 13 — Character type

Common Lisp and Haskell have a distinct Char type that is not a string. Without it both implementations are approximations — CL's #\a literal and Haskell's 'a' both need a real char value, not a length-1 string.

Primitives to add:

  • char? v → bool
  • char->integer c → Unicode codepoint integer
  • integer->char n → char
  • char=? char<? char>? char<=? char>=? → comparators
  • char-ci=? char-ci<? etc. → case-insensitive comparators
  • char-alphabetic? char-numeric? char-whitespace? → predicates
  • char-upper-case? char-lower-case? → predicates
  • char-upcase char-downcase → char → char
  • string->list extended to return chars (not length-1 strings)
  • list->string accepting chars

Also: #\a reader syntax for char literals (parser addition).

Steps:

  • Spec: add SxChar type to evaluator; add char literal syntax #\a/#\space/#\newline to spec/parser.sx; implement all predicates + comparators.
  • OCaml: add SxChar of char to sx_types.ml; implement primitives.
  • JS bootstrapper: implement char type wrapping a codepoint integer.
  • Tests: 30+ tests in spec/tests/test-chars.sx — literals, char->integer round-trip, comparators, predicates, upcase/downcase, string<->list with chars.
  • Commit: spec: character type (char? char->integer #\\a literals + predicates)

Phase 14 — String ports

Needed for any language with a reader protocol: Common Lisp's read, Prolog's term parser, Smalltalk's printString. Without string ports these all do their own character walking on raw strings rather than treating a string as an I/O stream.

Primitives to add:

  • open-input-string str → input port
  • open-output-string → output port
  • get-output-string port → string (flush output port to string)
  • input-port? output-port? port? → predicates
  • read-char [port] → char or eof-object
  • peek-char [port] → char or eof-object (non-consuming)
  • read-line [port] → string or eof-object
  • write-char char [port] → void
  • write-string str [port] → void
  • eof-object → the eof sentinel
  • eof-object? v → bool
  • close-port port → void

Steps:

  • Spec: add port type + eof-object to evaluator; implement all primitives. Ports are mutable objects with a position cursor (input) or accumulation buffer (output).
  • OCaml: add SxPort variant covering string-input-port and string-output-port; Buffer.t for output, string+offset for input.
  • JS bootstrapper: implement port type.
  • Tests: 25+ tests in spec/tests/test-ports.sx — open/read/peek/eof, output accumulation, read-line, write-char, close.
  • Commit: spec: string ports (open-input-string/open-output-string/read-char/etc)

Phase 15 — Math completeness

Filling specific gaps that multiple language implementations need.

15a — modulo / remainder / quotient distinction

They differ on negative numbers — critical for Erlang rem, Haskell mod/rem, CL mod/rem:

  • quotient a b → truncate toward zero (same sign as dividend)
  • remainder a b → sign follows dividend (truncation division)
  • modulo a b → sign follows divisor (floor division) — R7RS

15b — Trigonometry and transcendentals

Lua, Haskell, Erlang, CL all need: sin, cos, tan, asin, acos, atan, exp, log, sqrt, expt. Check which are already present; add missing ones.

15c — GCD / LCM

gcd a b → greatest common divisor; lcm a b → least common multiple. Needed by Haskell Rational, CL, and any language doing fraction arithmetic.

15d — Radix number parsing / formatting

(number->string n radix) → e.g. (number->string 255 16)"ff". (string->number s radix) → e.g. (string->number "ff" 16)255. Needed by: Common Lisp, Smalltalk, Erlang integer formatting.

Steps:

  • Audit which trig / math functions are already in spec/primitives.sx; note gaps.
  • Spec + OCaml + JS: implement missing trig (sin/cos/tan/asin/acos/atan/exp/log).
  • Spec + OCaml + JS: quotient/remainder/modulo with correct negative semantics.
  • Spec + OCaml + JS: gcd/lcm.
  • Spec + OCaml + JS: radix variants of number->string/string->number.
  • Tests: 40+ tests in spec/tests/test-math.sx.
  • Commit: spec: math completeness — trig, quotient/remainder/modulo, gcd/lcm, radix

Phase 16 — Rational numbers

Haskell's Rational type and Common Lisp ratios (1/3) both need this. Natural extension of the numeric tower (Phase 2) — rationals are the third numeric type alongside int and float.

Primitives to add:

  • make-rational numerator denominator → rational (auto-reduced by GCD)
  • rational? v → bool
  • numerator r → integer
  • denominator r → integer
  • Reader syntax: 1/3 parsed as rational literal
  • Arithmetic: (+ 1/3 1/6)1/2; (* 1/3 3)1; mixed int/rational → rational
  • exact->inexact on rational → float; inexact->exact on float → rational approximation
  • (number->string 1/3)"1/3"

Steps:

  • Spec: add SxRational type; add n/d reader syntax to spec/parser.sx; extend all arithmetic primitives for rational contagion (int op rational → rational, rational op float → float).
  • OCaml: add SxRational of int * int (stored in reduced form); implement all arithmetic.
  • JS bootstrapper: implement rational type.
  • Tests: 30+ tests in spec/tests/test-rationals.sx — literals, arithmetic, reduction, mixed numeric tower, exact<->inexact conversion.
  • Commit: spec: rational numbers — 1/3 literals, arithmetic, numeric tower integration

Phase 17 — read / write / display

Completes the I/O model. Builds on string ports (Phase 14) and char type (Phase 13). read parses any SX value from a port; write serializes with quoting (round-trippable); display serializes without quoting (human-readable). Common Lisp's read macro, Prolog term I/O, and Smalltalk's printString all need this.

Primitives to add:

  • read [port] → SX value or eof-object — full SX parser reading from a port
  • read-char already in Phase 14; read uses it internally
  • write val [port] → void — serializes with quotes: "hello", #\a, (1 2 3)
  • display val [port] → void — serializes without quotes: hello, a, (1 2 3)
  • newline [port] → void — writes \n
  • write-to-string val → string — convenience: (write val (open-output-string))
  • display-to-string val → string — convenience

Steps:

  • Spec: implement read in spec/evaluator.sx — wraps the existing parser to read one datum from a port cursor; handles eof gracefully.
  • Spec: implement write/display/newline — extend the existing serializer for port output; write quotes strings + uses #\ for chars, display does not.
  • OCaml: wire read through port type; implement write/display output path.
  • JS bootstrapper: implement.
  • Tests: 25+ tests in spec/tests/test-read-write.sx — read string literal, read list, read eof, write round-trip, display vs write quoting, newline, write-to-string.
  • Commit: spec: read/write/display — S-expression reader/writer on ports

Phase 18 — Sets

O(1) membership testing. Distinct from hash tables (unkeyed) and lists (O(n)). Erlang has sets as a stdlib staple, Haskell Data.Set, APL uses set operations constantly, Common Lisp has union/intersection on lists but a native set is O(1).

Primitives to add:

  • make-set [list] → fresh set, optionally seeded from list
  • set? v → bool
  • set-add! s val → void
  • set-member? s val → bool
  • set-remove! s val → void
  • set-size s → integer
  • set->list s → list (unspecified order)
  • list->set lst → set
  • set-union s1 s2 → new set
  • set-intersection s1 s2 → new set
  • set-difference s1 s2 → new set (elements in s1 not in s2)
  • set-for-each s fn → iterate for side effects
  • set-map s fn → new set of mapped values

Steps:

  • Spec: add entries to spec/primitives.sx.
  • OCaml: implement using Hashtbl.t with unit values (or a proper Set functor with a comparison function); add SxSet to sx_types.ml.
  • JS bootstrapper: implement using JS Set.
  • Tests: 30+ tests in spec/tests/test-sets.sx — add/member/remove, union/intersection/ difference, list conversion, for-each, size.
  • Commit: spec: sets (make-set/set-add!/set-member?/union/intersection/etc)

Phase 19 — Regular expressions as primitives

lib/js/regex.sx is a pure-SX regex engine already written. Promoting it to a primitive gives every language free regex without reinventing: Lua patterns, Tcl regexp, Ruby regex, JS regex, Erlang re module. Mostly a wiring job — the implementation exists.

Primitives to add:

  • make-regexp pattern [flags] → regexp object (flags: "i" case-insensitive, "g" global, "m" multiline)
  • regexp? v → bool
  • regexp-match re str → match dict {:match "..." :start N :end N :groups (...)} or nil
  • regexp-match-all re str → list of match dicts
  • regexp-replace re str replacement → string with first match replaced
  • regexp-replace-all re str replacement → string with all matches replaced
  • regexp-split re str → list of strings (split on matches)
  • Reader syntax: #/pattern/flags for regexp literals (parser addition)

Steps:

  • Audit lib/js/regex.sx — understand the API it already exposes; map to the primitive API above.
  • Spec: add SxRegexp type to evaluator; add #/pattern/flags literal syntax to spec/parser.sx; wire lib/js/regex.sx engine as the implementation.
  • OCaml: implement using OCaml Re library (or Str); add SxRegexp to types.
  • JS bootstrapper: use native JS RegExp; wrap in the primitive API.
  • Tests: 30+ tests in spec/tests/test-regexp.sx — basic match, groups, replace, replace-all, split, flags (case-insensitive), no-match nil return.
  • Commit: spec: regular expressions (make-regexp/regexp-match/regexp-replace + #/pat/ literals)

Phase 20 — Bytevectors

R7RS standard. Needed for WebSocket binary frames (E36), binary protocol parsing, and efficient string encoding. Also the foundation for proper Unicode: string->utf8 / utf8->string require a byte array type.

Primitives to add:

  • make-bytevector n [fill] → bytevector of n bytes (fill defaults to 0)
  • bytevector? v → bool
  • bytevector-length bv → integer
  • bytevector-u8-ref bv i → byte 0255
  • bytevector-u8-set! bv i byte → void
  • bytevector-copy bv [start] [end] → fresh copy
  • bytevector-copy! dst at src [start] [end] → in-place copy
  • bytevector-append bv... → concatenated bytevector
  • utf8->string bv [start] [end] → string decoded as UTF-8
  • string->utf8 str [start] [end] → bytevector UTF-8 encoded
  • bytevector->list / list->bytevector → conversion

Steps:

  • Spec: add SxBytevector type; implement all primitives in spec/evaluator.sx / spec/primitives.sx.
  • OCaml: add SxBytevector of bytes to sx_types.ml; implement primitives using OCaml Bytes.
  • JS bootstrapper: implement using Uint8Array.
  • Tests: 30+ tests in spec/tests/test-bytevectors.sx — construction, ref/set, copy, append, utf8 round-trip, slice.
  • Commit: spec: bytevectors (make-bytevector/u8-ref/u8-set!/utf8->string/etc)

Phase 21 — format

CL-style string formatting beyond str. (format "Hello ~a, age ~d" name age). Haskell printf, Erlang io:format, CL format, and general string templating all use this idiom.

Directives:

  • ~a — display (no quotes)
  • ~s — write (with quotes)
  • ~d — decimal integer
  • ~x — hexadecimal integer
  • ~o — octal integer
  • ~b — binary integer
  • ~f — fixed-point float
  • ~e — scientific notation float
  • ~% — newline
  • ~& — fresh line (newline only if not already at start of line)
  • ~~ — literal tilde
  • ~t — tab

Signature: (format template arg...) → string. Optional: (format port template arg...) — write to port directly.

Steps:

  • Spec: implement format as a pure SX function in spec/primitives.sx — parses ~X directives, dispatches to display/write/number->string as appropriate. Pure SX: no host calls needed. Self-hosting — uses string-buffer (Phase 5) internally.
  • OCaml: expose as a primitive (or let it run as SX through the evaluator).
  • JS bootstrapper: same.
  • Tests: 25+ tests in spec/tests/test-format.sx — each directive, multiple args, nested format, port variant, ~~ escape.
  • Commit: spec: format — CL-style string formatting (~a ~s ~d ~x ~% etc)

Phase 22 — Language sweep

Replace workarounds with primitives. One language per fire (or per sub-item for big ones). Start with blank slates (CL, APL, Ruby, Tcl) — they haven't committed to workarounds yet.

Scope per language: only lib/<lang>/**. Don't touch spec or other languages. Brief each language's loop agent (or do inline) after rebasing their branch onto architecture.

  • Restart CL/APL/Ruby/Tcl loops with updated briefing pointing to new primitives. Add a note to each plans/<lang>-on-sx.md under a ## SX primitive baseline section: "Use vectors for arrays; numeric tower + rationals for numbers; ADTs for tagged data; coroutines for fibers; string-buffer for mutable string building; bitwise ops for bit manipulation; multiple values for multi-return; promises for lazy evaluation; hash tables for mutable associative storage; sets for O(1) membership; sequence protocol for polymorphic iteration; gensym for unique symbols; char type for characters; string ports + read/write for reader protocols; regexp for pattern matching; bytevectors for binary data; format for string templating."

  • Common Lisp: char type (#\a); string ports + read/write for reader/printer; gensym for macros; rational numbers for CL ratios; multiple values; sets for CL set ops; modulo/remainder/quotient; radix formatting; format for cl:format.

  • Lua: vectors for arrays; hash tables for Lua tables; delay/force for lazy iterators; regexp for Lua pattern matching; trig from math completeness; bytevectors for binary I/O.

  • Erlang: numeric tower for float/int; bitwise ops for bitmatch; multiple values for multi-return; sets for Erlang sets; remainder for rem; regexp for re module.

  • Haskell: numeric tower for Num/Integral/Fractional; promises for lazy evaluation (critical); multiple values for tuples; rational numbers for Rational; char type for Char; gcd/lcm; sets for Data.Set; read/write for Show/Read instances.

  • JS: vectors for Array; hash tables for Map; sets for Set; bitwise ops for typed arrays; regexp for JS regex; bytevectors for Uint8Array; radix formatting.

  • Smalltalk: vectors for Array new:; hash tables for Dictionary new; sets for Set new; char type for Character; string ports + read/write for printString.

  • APL: vectors as core array type; bitwise ops for array masks; sets for APL set ops; sequence protocol for rank-polymorphic operations; format for APL output formatting.

  • Ruby: coroutines for fibers; hash tables for Hash; sets for Set; regexp for Ruby regex; string ports for StringIO; bytevectors for String binary encoding.

  • Tcl: string ports for Tcl channel abstraction; string-buffer for append; coroutines for Tcl coroutines; regexp for Tcl regexp; format for Tcl format.

  • Forth: bitwise ops (core); string-buffer for word-definition accumulation; bytevectors for Forth's raw memory model.


Ground rules

  • Work on the architecture branch in /root/rose-ash (main worktree).
  • Use sx-tree MCP for all .sx file edits. Never use raw Edit/Write/Read on .sx files.
  • Commit after each concrete unit of work. Never leave the branch broken.
  • Never push to main — only push to origin/architecture.
  • Update this checklist every fire: tick [x] done, add inline notes on blockers.

Progress log

Newest first.

  • 2026-04-26: Phase 2 complete — Verify+Commit done. OCaml 4874/394, JS 1940/2500 (+60). No regressions. 6 JS-only failures are float≡int platform-inherent. Phase 2 fully landed across 4 commits.
  • 2026-04-26: Phase 2 JS bootstrapper done — integer?/float?/exact?/inexact? added (Number.isInteger); truncate/remainder/modulo/random-int/exact->inexact/inexact->exact/parse-number added. Fixed sx_server.ml epoch+blob+io-response protocol for Integer type. JS: 1940/2500 (+60). OCaml: 4874/394 baseline. 6 JS tests fail (JS float≡int platform limit). Committed b12a22e6.
  • 2026-04-26: Phase 2 Spec done — integer?/float? predicates added to spec/primitives.sx; floor/ceil/truncate :returns updated to "integer"; / to "float"; exact->inexact/inexact->exact docs and returns updated; float contagion documented on +/-/*; 4874/394 baseline. Committed 45ec5535.
  • 2026-04-26: Phase 2 OCaml+Tests done — Integer of int / Number of float in sx_types.ml; float contagion across all arithmetic; floor/truncate/round → Integer; integer?/float?/exact?/inexact?/exact->inexact/inexact->exact; 92/92 numeric tower tests pass; 4874 total (394 pre-existing unchanged). Committed c70bbdeb.
  • 2026-04-26: Phase 1 complete — JS step done. Fixed fundamental lambda binding bug (index-of on arrays returned -1 not NIL, making bind-lambda-params mis-fire &rest branch). Added lastErrorKont/hostError/try-catch stubs. 42/42 vector tests pass. 1847 std / 2362 full passing (up from 5). Committed.
  • 2026-04-25: Phase 1 spec step done — all 10 vector primitives in spec/primitives.sx have full :as type annotations, :returns, :doc; make-vector optional fill param added.
  • 2026-04-25: Phase 1 OCaml step done — bounds-checked vector-ref/set!, vector-copy now accepts optional start/end, spec/primitives.sx doc updated. 10/10 r7rs vector tests pass, 4747 total (394 pre-existing hs-upstream fails unchanged).
  • 2026-04-25: Phase 0 complete — stopped CL/APL/Ruby/Tcl loops (all 4 idle at shell); confirmed E38 (tokenizer :end/:line) and E39 (WebWorker stub) both have implementation commits.