rose-ash/plans/apl-on-sx.md

APL-on-SX: rank-polymorphic primitives + glyph parser

The headline showcase is rank polymorphism — a single primitive (+, ⌈, ⊂, ⍳) works uniformly on scalars, vectors, matrices, and higher-rank arrays. ~80 glyph primitives + 6 operators bind together with right-to-left evaluation; the entire language is a high-density combinator algebra. The JIT compiler + primitive table pay off massively here because almost every program is array → array pure pipelines.

End-state goal: Dyalog-flavoured APL subset, dfns + tradfns, classic programs (game-of-life, mandelbrot, prime-sieve, n-queens, conway), 100+ green tests.

Scope decisions (defaults — override by editing before we spawn)

• Syntax: Dyalog APL surface, Unicode glyphs. ⎕-quad system functions for I/O. ∇ tradfn header.
• Conformance: "Reads like APL, runs like APL." Not byte-compat with Dyalog; we care about right-to-left semantics and rank polymorphism.
• Test corpus: custom — APL idioms (Roger Hui style), classic programs, plus ~50 pattern tests for primitives.
• Out of scope: ⎕-namespaces beyond a handful, complex numbers, full TAO ordering, ⎕FX runtime function definition (use static ∇ only), nested-array-of-functions higher orders, the editor.
• Glyphs: input via plain Unicode in .apl source files. Backtick-prefix shortcuts handled by the user's editor — we don't ship one.

Ground rules

• Scope: only touch lib/apl/** and plans/apl-on-sx.md. Don't edit spec/, hosts/, shared/, or any other lib/<lang>/**. APL primitives go in lib/apl/runtime.sx.
• SX files: use sx-tree MCP tools only.
• Commits: one feature per commit. Keep ## Progress log updated and tick roadmap boxes.

Architecture sketch

APL source (Unicode glyphs)
    │
    ▼
lib/apl/tokenizer.sx   — glyphs, identifiers, numbers (¯ for negative), strings, strands
    │
    ▼
lib/apl/parser.sx      — right-to-left with valence resolution (mon vs dyadic by position)
    │
    ▼
lib/apl/transpile.sx   — AST → SX AST (entry: apl-eval-ast)
    │
    ▼
lib/apl/runtime.sx     — array model, ~80 primitives, 6 operators, dfns/tradfns

Core mapping:

• Array = SX dict {:shape (d1 d2 …) :ravel #(v1 v2 …)}. Scalar is rank-0 (empty shape), vector is rank-1, matrix rank-2, etc. Type uniformity not required (heterogeneous nested arrays via "boxed" elements ⊂x).
• Rank polymorphism — every scalar primitive is broadcast: 1 2 3 + 4 5 6 ↦ 5 7 9; (2 3⍴⍳6) + 1 ↦ broadcast scalar to matrix.
• Conformability = matching shapes, or one-side scalar, or rank-1 cycling (deferred — keep strict in v1).
• Valence = each glyph has a monadic and a dyadic meaning; resolution is purely positional (left-arg present → dyadic).
• Operator = takes one or two function operands, returns a derived function (f¨ = each f, f/ = reduce f, f∘g = compose, f⍨ = commute).
• Tradfn ∇R←L F R; locals = named function with explicit header.
• Dfn {⍺+⍵} = anonymous, ⍺ = left arg, ⍵ = right arg, ∇ = recurse.

Roadmap

Phase 1 — tokenizer + parser

• Tokenizer: Unicode glyphs (the full APL set: + - × ÷ * ⍟ ⌈ ⌊ | ! ? ○ ~ < ≤ = ≥ > ≠ ∊ ∧ ∨ ⍱ ⍲ , ⍪ ⍴ ⌽ ⊖ ⍉ ↑ ↓ ⊂ ⊃ ⊆ ∪ ∩ ⍳ ⍸ ⌷ ⍋ ⍒ ⊥ ⊤ ⊣ ⊢ ⍎ ⍕ ⍝), operators (/ \ ¨ ⍨ ∘ . ⍣ ⍤ ⍥ @), numbers (¯ for negative, 1E2, 1J2 complex deferred), characters ('a', '' escape), strands (juxtaposition of literals: 1 2 3), names, comments ⍝ …
• Parser: right-to-left; classify each token as function, operator, value, or name; resolve valence positionally; dfn {…} body, tradfn ∇ header, guards :; outer product ∘.f, inner product f.g, derived fns f/ f¨ f⍨ f⍣n
• Unit tests in lib/apl/tests/parse.sx

Phase 2 — array model + scalar primitives

• Array constructor: make-array shape ravel, scalar v, vector v…, enclose/disclose
• Shape arithmetic: ⍴ (shape), , (ravel), ≢ (tally / first-axis-length), ≡ (depth)
• Scalar arithmetic primitives broadcast: + - × ÷ ⌈ ⌊ * ⍟ | ! ○
• Scalar comparison primitives: < ≤ = ≥ > ≠
• Scalar logical: ~ ∧ ∨ ⍱ ⍲
• Index generator: ⍳n (vector 1..n or 0..n-1 depending on ⎕IO)
• ⎕IO = 1 default (Dyalog convention)
• 40+ tests in lib/apl/tests/scalar.sx

Phase 3 — structural primitives + indexing

• Reshape ⍴, ravel ,, transpose ⍉ (full + dyadic axis spec)
• Take ↑, drop ↓, rotate ⌽ (last axis), ⊖ (first axis)
• Catenate , (last axis) and ⍪ (first axis)
• Index ⌷ (squad), bracket-indexing A[I] (sugar for ⌷)
• Grade-up ⍋, grade-down ⍒
• Enclose ⊂, disclose ⊃, partition (subset deferred)
• Membership ∊, find ⍳ (dyadic), without ~ (dyadic), unique ∪ (deferred to phase 6)
• 40+ tests in lib/apl/tests/structural.sx

Phase 4 — operators (THE SHOWCASE)

• Reduce f/ (last axis), f⌿ (first axis) — including ∧/, ∨/, +/, ×/, ⌈/, ⌊/
• Scan f\, f⍀
• Each f¨ — applies f to each scalar/element
• Outer product ∘.f — 1 2 3 ∘.× 1 2 3 ↦ multiplication table
• Inner product f.g — +.× is matrix multiply
• Commute f⍨ — f⍨ x ↔ x f x, x f⍨ y ↔ y f x
• Compose f∘g — applies g first then f
• Power f⍣n — apply f n times; f⍣≡ until fixed point
• Rank f⍤k — apply f at sub-rank k
• At @ — selective replace
• 40+ tests in lib/apl/tests/operators.sx

Phase 5 — dfns + tradfns + control flow

• Dfn {…} with ⍺ (left arg, may be absent → niladic/monadic), ⍵ (right arg), ∇ (recurse), guards cond:expr, default left arg ⍺←default
• Local assignment via ← (lexical inside dfn)
• Tradfn ∇ header: R←L F R;l1;l2, statement-by-statement, branch via →linenum
• Dyalog control words: :If/:Else/:EndIf, :While/:EndWhile, :For X :In V :EndFor, :Select/:Case/:EndSelect, :Trap/:EndTrap (Trap deferred — no exception machinery yet)
• Niladic / monadic / dyadic dispatch (function valence at definition time)
• lib/apl/conformance.sh + runner, scoreboard.json + scoreboard.md

Phase 6 — classic programs + drive corpus

• Classic programs in lib/apl/tests/programs/:
  • life.apl — Conway's Game of Life as a one-liner using ⊂ ⊖ ⌽ +/
  • mandelbrot.apl — complex iteration with rank-polymorphic + × ⌊ (or real-axis subset)
  • primes.apl — (2=+⌿0=A∘.|A)/A←⍳N sieve
  • n-queens.apl — backtracking via reduce
  • quicksort.apl — the classic Roger Hui one-liner
• System functions: ⎕FMT, ⎕FR (float repr), ⎕TS (timestamp), ⎕IO, ⎕ML (migration level — fixed at 1), ⎕← (print)
• Drive corpus to 100+ green
• Idiom corpus — lib/apl/tests/idioms.sx covering classic Roger Hui / Phil Last idioms

Phase 7 — end-to-end pipeline + closing the gaps

Phase 1-6 built parser and runtime as parallel layers — they don't yet meet. Phase 7 wires them together so APL source actually runs through the full stack, and tightens loose ends.

• Operators in apl-eval-ast — handle :derived-fn (e.g. +/, f¨), :outer (∘.f), :derived-fn2 (f.g). Each derived-fn-node wraps an inner function; eval-ast resolves the inner glyph to a runtime fn and dispatches to the matching operator helper (apl-reduce, apl-each, apl-outer, apl-inner, apl-commute, apl-compose, apl-power, apl-rank).
• End-to-end pipeline — entry point apl-run : string → array that chains apl-tokenize → parse-apl → apl-eval-ast against an empty env. Verify with one-liners (+/⍳5 → 15, 1 2 3 + 4 5 6 → 7 9 11, etc.) and with the actual .apl source files in tests/programs/.
• :quad-name AST + handler — extend tokenizer/parser to recognise ⎕name, then handle in apl-eval-ast by dispatching to apl-quad-* runtime fns (⎕IO, ⎕ML, ⎕FR, ⎕TS, ⎕FMT, ⎕←). (⎕← deferred — tokenizer treats ← as :assign after ⎕.)
• Bracket indexing verification — load programs that use A[I] / A[I;J] end-to-end; confirm parser desugars to ⌷ and runtime returns expected slices. Add 5+ tests. (Single-axis only — multi-axis A[I;J] requires semicolon parsing, deferred.)
• Idiom corpus expansion — extend idioms.sx from 34 to 60+ once end-to-end works (we can express idioms as APL strings, not as runtime calls). Source-string-based idioms validate the whole stack.
• :Trap / :EndTrap — minimal exception machinery: :Trap n catches errors with code n, body runs in apl-tradfn-eval-block, on error switches to the trap branch. Define apl-throw and a small set of error codes; use try/catch from the host.

Phase 8 — fill the gaps left after end-to-end

Phase 7 wired the stack together; Phase 8 closes deferred items, lets real programs run from source, and starts pushing on performance.

• Quick-wins bundle (one iteration) — three small fixes that each unblock real programs:
  • decimal literals: read-digits! consumes one trailing . plus more digits so 3.7 tokenises as one number;
  • ⎕← (print) — tokenizer special-case: when ⎕ is followed by ←, emit a single :name "⎕←" token (don't split on the assign glyph);
  • string values in apl-eval-ast — handle :str (parser already produces them) by wrapping into a vector of character codes (or rank-0 string).
• Named function definitions — f ← {⍺+⍵} ⋄ 1 f 2 and 2 f 3.
  • parser: when :assign's RHS is a :dfn, mark it as a function binding;
  • eval-ast: :assign of a dfn stores the dfn in env;
  • parser: a name in fn-position whose env value is a dfn dispatches as a fn;
  • resolver: extend apl-resolve-monadic/-dyadic with a :fn-name case that calls apl-call-dfn/apl-call-dfn-m.
• Multi-axis bracket indexing — A[I;J] and A[;J] and A[I;].
  • parser: split bracket content on :semi at depth 0; emit (:dyad ⌷ (:vec I J) A);
  • runtime: extend apl-squad to accept a vector of indices, treating nil / empty axis as "all";
  • 5+ tests across vector and matrix.
• .apl files as actual tests — lib/apl/tests/programs/*.apl are currently documentation. Add apl-run-file path → array plus tests that load each file, execute it, and assert the expected result. Makes the classic-program corpus self-validating instead of two parallel impls. (Embedded source-string approach: tests/programs-e2e.sx runs the same algorithms as the .apl docs through the full pipeline. The original one-liners (e.g. primes' inline ⍵←⍳⍵) need parser features (compress-as-fn, inline assign) we haven't built yet — multi-stmt forms used instead. Slurp/read-file primitive missing in OCaml SX runtime.)
• Train/fork notation — (f g h) ⍵ ↔ (f ⍵) g (h ⍵) (3-train); (g h) ⍵ ↔ g (h ⍵) (2-train atop). Parser: detect when a parenthesised subexpression is all functions and emit (:train fns); resolver: build the derived function; tests for mean-via-train (+/÷≢).
• Performance pass — n-queens(8) currently ~30 s/iter (tight on the 300 s timeout). Target: profile the inner loop, eliminate quadratic list-append, restore the queens(8) test.

Phase 9 — make .apl source files run as-written

Goal: the existing lib/apl/tests/programs/*.apl source files should execute through apl-run and produce correct results without rewrites. Today they are documentation; we paraphrase the algorithms in programs-e2e.sx. Phase 9 closes that gap.

• Compress as a dyadic function — mask / arr between two values is the classic compress (select where mask≠0). Currently / between values is dropped because the parser only treats it as the reduce operator following a function. Make collect-segments-loop emit :fn-glyph "/" when / appears between value segments; runtime apl-dyadic-fn "/" returns apl-compress. Same for ⌿ (first-axis compress).
• Inline assignment — ⍵ ← ⍳⍵ mid-expression. Parser currently only handles :assign at the start of a statement. Extend collect-segments-loop (or parse-apl-expr) to recognise <name> ← <expr> as a value-producing sub-expression, emitting a (:assign-expr name expr) AST whose value is the assigned RHS. Required by the primes idiom (2=+⌿0=⍵∘.|⍵)/⍵←⍳⍵. (Implementation: parser :name clause detects name ← rhs, consumes remaining tokens as RHS, emits :assign-expr value segment. Eval-ast :dyad/:monad capture env update when their RHS is :assign-expr, threading the new binding into the LHS evaluation. Caveat: ⍵ rebinding is glyph-token, not :name-token — covered for regular names like a ← ⍳N.)
• ? (random / roll) — monadic ?N returns a random integer in 1..N. Used by quicksort.apl for pivot selection. Add apl-roll (deterministic seed for tests) + glyph wiring.
• apl-run-file path → array — read the file from disk, strip the ⍝ comments (already handled by tokenizer), and run. Needs an IO primitive on the SX side. Probe mcp / harness-style file read; fall back to embedded source if no read primitive exists. (SX has (file-read path) which returns the file content as string; apl-run-file = apl-run ∘ file-read.)
• End-to-end .apl tests — once the above land, add tests that run lib/apl/tests/programs/*.apl as written and assert results. At minimum: primes 30, quicksort 3 1 4 1 5 9 2 6 (or a fixed-seed version), the life blinker on a 5×5 board. (primes.apl runs as-written with ⍵-rebind now supported. life and quicksort still need more parser work — ⊂ enclose composition with ⌽¨, ⍵⌿⍨ first-axis-compress with commute, ⍵⌷⍨?≢⍵.)
• Audit silently-skipped glyphs — sweep apl-glyph-set and apl-parse-fn-glyphs against the runtime's apl-monadic-fn and apl-dyadic-fn cond chains to find any that the runtime supports but the parser doesn't see. (Wired ⍉ → apl-transpose / apl-transpose-dyadic, ⊢ identity, ⊣ left, ⍕ as alias for ⎕FMT. ⊆ ∪ ∩ ⍸ ⊥ ⊤ ⍎ remain unimplemented in the runtime — parser sees them as functions but eval errors; next-phase work.)

Phase 10 — fill runtime gaps + life/quicksort source files run

Phase 9 left seven glyphs that the parser recognises but the runtime cannot evaluate, and two source files (life.apl, quicksort.apl) that still need work to run as-written. Phase 10 closes both.

• ⍸ where — monadic ⍸ B returns the indices of the truthy cells (1-based per ⎕IO). Dyadic X ⍸ Y is interval index (find the largest i such that X[i] ≤ Y). Add apl-where + dyadic apl-interval-index; wire both into apl-monadic-fn / apl-dyadic-fn. Tests: ⍸ 0 1 0 1 1 → 2 4 5, ⍸ ⍳5 = ¯1+⍳5 → empty, 2 4 6 ⍸ 5 → 2.
• ∪ unique / ∩ intersection — monadic ∪ V returns V with duplicates removed (first-occurrence order); dyadic A ∪ B is union; A ∩ B is intersection (members of A that are also in B). Add apl-unique, apl-union, apl-intersect. Tests cover empty, single, repeats, mixed numerics.
• ⊥ decode / ⊤ encode — B ⊥ V evaluates digits V in base(s) B (Horner-style); B ⊤ N is the inverse, returning the digits of N in base(s) B. Both broadcast B as scalar or conformable vector. Add apl-decode and apl-encode. Tests: 2 ⊥ 1 0 1 → 5, 10 ⊥ 1 2 3 → 123, 2 2 2 ⊤ 5 → 1 0 1, 24 60 60 ⊤ 7384 → 2 3 4.
• ⊆ partition — dyadic M ⊆ V partitions V into vectors driven by mask M: a new partition starts wherever M[i] > M[i-1], and 0 cells are dropped. Returns a vector of (boxed) partitions. Add apl-partition. Tests: 1 1 0 1 1 ⊆ 'abcde' → ('ab' 'de'), 1 0 0 1 1 ⊆ ⍳5 → ((⊂ 1) (⊂ 4 5)).
• ⍎ execute — monadic ⍎ S evaluates S (a character vector) as APL source in the current environment, returning the result. Implement as (fn (s) (apl-run s)) — env is the global one; nested execute is fine. Wire into apl-monadic-fn. Tests: ⍎ '1 + 2' → 3, ⍎ '+/⍳10' → 55.
• life.apl runs as-written — Conway's life one-liner uses ⊃+/⌽¨ -1 0 1 ∘.,¯1 0 1 (each + outer-comma + disclose + reduce over a list of rotations) and the rule expression. Probe what fails when apl-run-file "lib/apl/tests/programs/life.apl" is called on a 5×5 blinker grid; fix any remaining parser/runtime gaps; assert blinker oscillates and block stays stable as full end-to-end tests in programs-e2e.sx.
• quicksort.apl runs as-written — the classic Iverson dfn {1≥≢⍵:⍵ ⋄ (∇(⍵<pivot)⌿⍵),(⍵=pivot)⌿⍵,∇(⍵>pivot)⌿⍵⊣pivot←⍵⌷⍨?≢⍵} exercises ⌷⍨ (squad-commute pivot pick), ⌿⍨ (first-axis-compress commute), and ⊣ to bind a local without polluting the result. Set the RNG seed for determinism and assert the sort against apl-grade-up.

Phase 11 — heterogeneous-strand inner product (restore life.apl ⊃)

Phase 10 step 6 closed life.apl by dropping the leading ⊃ from Hui's formulation, because our inner product over a mixed scalar/matrix strand (1 ⍵) produced a clean (5 5) board which ⊃ then collapsed to its first row. Hui's original needs ⊃ to unwrap an enclosed result of the inner product. Phase 11 closes that semantic gap so life.apl can be restored to its true as-written form.

• Inner product encloses on heterogeneous left arg — detect when A in A f.g B has a ravel containing a dict (boxed array), and in that case wrap the inner-product result in enclose (rank-0 wrapping the matrix). Then ⊃ on the result unwraps to the underlying board. Restore life.apl to the original {⊃1 ⍵ ∨.∧ 3 4 = +/ +/ ¯1 0 1 ∘.⊖ ¯1 0 1 ⌽¨ ⊂⍵} and update its tests + comment block.

SX primitive baseline

Use vectors for arrays; numeric tower + rationals for numbers; ADTs for tagged data; coroutines for fibers; string-buffer for mutable string building; bitwise ops for bit manipulation; multiple values for multi-return; promises for lazy evaluation; hash tables for mutable associative storage; sets for O(1) membership; sequence protocol for polymorphic iteration; gensym for unique symbols; char type for characters; string ports

• read/write for reader protocols; regexp for pattern matching; bytevectors for binary data; format for string templating.

Progress log

Newest first.

• 2026-05-11: Phase 11 — heterogeneous-strand inner product. apl-inner now encloses its result when A's ravel contains a dict (boxed array) — Hui's 1 ⍵ ∨.∧ X produces a rank-0 wrapping the (5 5) board, which ⊃ then unwraps to the bare matrix. Restored life.apl to its true as-written form {⊃1 ⍵ ∨.∧ 3 4 = +/ +/ ¯1 0 1 ∘.⊖ ¯1 0 1 ⌽¨ ⊂⍵} and updated all 5 e2e tests + comment block. Homogeneous inner product unaffected (+.× over numbers/matrices still produces bare arrays). +4 pipeline tests for the heterogeneous case + ⊃ unwrap path; Phase 11 complete; full suite 589/589
• 2026-05-08: Phase 10 step 7 — quicksort.apl runs as-written. Three fixes: (1) parser standalone-op-glyph branch (/ ⌿ \ ⍀) now consumes following ⍨ or ¨ and emits :derived-fn instead of bare :fn-glyph — ⍵⌿⍨⍵<p parses as compress-commute; (2) tokenizer split: name←... (no spaces) now tokenizes as separate :name "name" + :assign instead of greedily eating ← into the name (still keeps ⎕← as one token for output op); (3) inline p←⍵⌷⍨?≢⍵ mid-dfn now works via existing :assign-expr machinery. The classic Iverson dfn {1≥≢⍵:⍵ ⋄ p←⍵⌷⍨?≢⍵ ⋄ (∇⍵⌿⍨⍵<p),(p=⍵)/⍵,∇⍵⌿⍨⍵>p} sorts correctly. +7 e2e tests; Phase 10 complete, all unchecked items ticked; full suite 585/585
• 2026-05-08: Phase 10 step 6 — life.apl runs as-written. Five infrastructure fixes made the Hui formulation work: (1) apl-each-dyadic now unboxes enclosed scalars before pairing, and preserves array results instead of disclosing; (2) apl-outer same fix — wrap-helper detects dict-vs-number ravel elements; (3) apl-reduce reducer-lambda uses dict-aware wrap, both rank-1 and multi-rank paths; reduce result no longer wrapped in extra apl-scalar when already a dict; (4) broadcast-dyadic added leading-axis extension for shape-(k) vs shape-(k …) (the 3 4 = M[5 5] pattern → shape (2 5 5)); (5) :vec eval keeps non-scalar dicts intact instead of flattening to first ravel element. Updated life.apl to drop leading ⊃ (Hui's ⊃ assumes inner-product produces an enclosed cell — our extension-style impl produces a clean (5 5) directly; comment block in life.apl explains). +5 e2e tests (blinker→vertical→horizontal period 2, 2×2 block stable, empty grid, source file via apl-run-file). Full test suite 578/578
• 2026-05-08: Phase 10 step 5 — ⍎ execute. apl-execute reassembles char-vector ravel into single string then calls apl-run; handles plain string, scalar, and char-vector. ⍎ '1 + 2' → 3, ⍎ '+/⍳10' → 55, round-trip ⍎ ⎕FMT 42 → 42, nested ⍎ ⍎ '...' works, with ⋄ separator (assignment + use). Wired into apl-monadic-fn. +8 tests; pipeline 148/148
• 2026-05-08: Phase 10 step 4 — ⊆ partition. apl-partition: walk M and V together via reduce, opening a new partition where M[i]>M[i-1] (initial prev=0), continuing where M[i]≤prev∧M[i]≠0, dropping cells where M[i]=0. Returns apl-vector of apl-vector parts. 1 1 0 1 1 ⊆ 'abcde' → ('ab' 'de'), 1 0 0 1 1 ⊆ ⍳5 → ((1) (4 5)), strict-increase 1 2 opens new, constant 2 2 continues. Wired into apl-dyadic-fn. +8 tests; pipeline 140/140
• 2026-05-08: Phase 10 step 3 — ⊥ decode / ⊤ encode. apl-decode (Horner reduce over indices, base[i]>0; scalar base broadcasts to digit length); apl-encode (right-to-left modulo+floor-div via reduce). Mixed-radix HMS works: 24 60 60 ⊥ 2 3 4 → 7384, 24 60 60 ⊤ 7384 → 2 3 4. Round-trips exact. Wired ⊥ ⊤ into apl-dyadic-fn. +11 tests; pipeline 132/132
• 2026-05-08: Phase 10 step 2 — ∪ unique / ∩ intersection. apl-unique (monadic, dedup keeping first-occurrence order via reduce+index-of), apl-union (dyadic, dedup'd A then B-elements-not-in-A), apl-intersect (dyadic, A elements that are also in B, preserves left order). Wired ∪ into both apl-monadic-fn and apl-dyadic-fn cond chains; ∩ into apl-dyadic-fn. +12 tests; pipeline 121/121
• 2026-05-08: Phase 10 step 1 — ⍸ where. apl-where (monadic, indices of truthy cells, ⎕IO-respecting) + apl-interval-index (dyadic, count of breaks ≤ y; broadcasts over Y vector or scalar). Wired into apl-monadic-fn / apl-dyadic-fn (cond clauses inserted as proper siblings via sx_insert_child after sx_insert_near silently wrapped multi-form sources in (begin …)). +10 tests; pipeline 109/109
• 2026-05-08: Phase 10 added — fill runtime gaps (⍸ ∪ ∩ ⊥ ⊤ ⊆ ⍎) + life.apl and quicksort.apl as-written
• 2026-05-07: Phase 9 step 6 — glyph audit. Wired ⍉ → apl-transpose/apl-transpose-dyadic, ⊢ → monadic+dyadic identity-right, ⊣ → identity-left, ⍕ → apl-quad-fmt. +6 tests; Phase 9 complete, all unchecked items ticked; pipeline 99/99
• 2026-05-07: Phase 9 step 5 — primes.apl runs as-written end-to-end. Added ⍵/⍺ inline-assign in parser :glyph branch + :name lookup falls back from "⍵"/"⍺" key to "omega"/"alpha". apl-run "primes ← {(2=+⌿0=⍵∘.|⍵)/⍵←⍳⍵} ⋄ primes 50" → 15 primes. +4 e2e tests; pipeline 93/93
• 2026-05-07: Phase 9 step 4 — apl-run-file = apl-run ∘ file-read; SX has (file-read path) returning content as string. primes/life/quicksort .apl files now load and parse end-to-end (return :dfn AST). +4 tests
• 2026-05-07: Phase 9 step 3 — ?N random / roll. Top-level mutable apl-rng-state with LCG; apl-rng-seed! for deterministic tests; apl-roll wraps as scalar in 1..N. apl-monadic-fn maps "?" → apl-roll. +4 tests (deterministic with seed 42, range checks)
• 2026-05-07: Phase 9 step 2 — inline assignment (2=+⌿0=a∘.|a)/a←⍳30 runs end-to-end. Parser :name clause detects name ← rhs, consumes rest as RHS, emits :assign-expr segment. Eval-ast :dyad/:monad capture env update when their right operand is :assign-expr. +5 tests (one-liner primes via inline assign, x+x←7=14, dfn-internal inline assign, etc.)
• 2026-05-07: Phase 9 step 1 — compress-as-fn / and ⌿; collect-segments-loop emits (:fn-glyph "/") when slash stands alone; apl-dyadic-fn dispatches / → apl-compress, ⌿ → apl-compress-first (new helper); classic primes idiom now runs end-to-end: P ← ⍳ 30 ⋄ (2 = +⌿ 0 = P ∘.| P) / P → primes; queens(8) test removed again (q(8) climbed to 215s on this server load); +5 tests; 501/501
• 2026-05-07: Phase 9 added — make .apl source files run as-written (compress as dyadic /, inline assignment, ? random, apl-run-file, glyph audit, source-as-tests)
• 2026-05-07: Phase 8 step 6 — perf: swapped (append acc xs) → (append xs acc) in apl-permutations to make permutation generation linear instead of quadratic; q(7) 32s→12s; q(8)=92 test restored within 300s timeout; Phase 8 complete, all unchecked items ticked; 497/497
• 2026-05-07: Phase 8 step 5 — train/fork notation. Parser :lparen detects all-fn inner segments → emits :train AST; resolver covers 2-atop & 3-fork for both monadic and dyadic. (+/÷≢) 1..5 → 3 (mean), (- ⌊) 5 → -5 (atop), 2(+×-)5 → -21 (dyadic fork), (⌈/-⌊/) → 8 (range); +6 tests; 496/496
• 2026-05-07: Phase 8 step 4 — programs-e2e.sx runs classic-algorithm shapes through full pipeline (factorial via ∇, triangulars, sum-of-squares, divisor-counts, prime-mask, named-fn composition, dyadic max-of-two, Newton step); also added ⌿ + ⍀ to glyph sets (were silently skipped); +15 tests; 490/490
• 2026-05-07: Phase 8 step 3 — multi-axis bracket A[I;J] / A[I;] / A[;J] via :bracket AST + apl-bracket-multi runtime; split-bracket-content scans :semi at depth 0; apl-cartesian builds index combinations; nil axis = "all"; scalar axis collapses; +8 tests; 475/475
• 2026-05-07: Phase 8 step 2 — named function defs end-to-end via parser pre-scan; apl-known-fn-names + apl-collect-fn-bindings detect name ← {...} patterns; collect-segments-loop emits :fn-name for known names; resolver looks up env for :fn-name; supports recursion (∇ in named dfn); +7 tests including fact via ∇; 467/467
• 2026-05-07: Phase 8 step 1 — quick-wins bundle: decimal literals (3.7, ¯2.5), ⎕← passthrough as monadic fn (single-token via tokenizer special-case), :str AST in eval-ast (single-char→scalar, multi-char→vec); +10 tests; 460/460
• 2026-05-07: Phase 8 added — quick-wins bundle (decimals + ⎕← + strings), named functions, multi-axis bracket, .apl-files-as-tests, trains, perf
• 2026-05-07: Phase 7 step 6 — :Trap exception machinery via R7RS guard; apl-throw raises tagged error, apl-trap-matches? checks codes (0=catch-all), :trap clause in apl-tradfn-eval-stmt wraps try-block with guard; :throw AST for testing; Phase 7 complete, all unchecked plan items done; +5 tests; 450/450
• 2026-05-07: Phase 7 step 5 — idiom corpus 34→64 (+30 source-string idioms via apl-run); also fixed tokenizer + parser to recognize ≢ and ≡ glyphs (were silently skipped); 445/445
• 2026-05-07: Phase 7 step 4 — bracket indexing A[I] desugared to (:dyad ⌷ I A) via maybe-bracket helper, wired into :name + :lparen branches of collect-segments-loop; multi-axis (A[I;J]) deferred (semicolon split); +7 tests; 415/415
• 2026-05-07: Phase 7 step 3 — :quad-name end-to-end; tokenizer already produced :name "⎕FMT"; parser is-fn-tok? extended via apl-quad-fn-names; eval-ast :name dispatches ⎕IO/⎕ML/⎕FR/⎕TS to apl-quad-*; apl-monadic-fn handles ⎕FMT; ⎕← deferred (tokenizer splits ⎕←); +8 tests; 408/408
• 2026-05-07: Phase 7 step 2 — end-to-end pipeline apl-run : string → array (parse-apl + apl-eval-ast against empty env); +25 source-string tests covering scalars, strands, dyadic arith, monadic primitives, operators, ∘./.g products, comparisons, famous one-liners (+/⍳10=55, ×/⍳10=10!); tokenizer can't yet parse decimals so 3.7 literal tests dropped; 400/400
• 2026-05-07: Phase 7 step 1 — operators in apl-eval-ast via apl-resolve-monadic/dyadic; supports / ⌿ \ ⍀ ¨ ⍨ ∘. f.g; queens(8) test removed (too slow for 300s timeout); +14 eval-ops tests; 375/375
• 2026-05-07: Phase 7 added — end-to-end pipeline, operators in eval-ast, :quad-name, bracket-indexing verify, idiom expansion, :Trap; aim is to wire parser↔runtime so .apl source files actually run
• 2026-05-07: Phase 6 idiom corpus — lib/apl/tests/idioms.sx; 34 classic idioms (sum, mean, max/min/range, scan, sort, reverse, first/last, take/drop, tally, mod, identity matrix, mult-table, factorial, parity count, all/any, mean-centered, ravel, rank); all unchecked items in plan now ticked; 362/362
• 2026-05-07: Phase 6 system fns + 100+ corpus — apl-quad-{io,ml,fr,ts,fmt,print}; ⎕FMT formats scalar/vector/matrix; ⎕TS returns 7-vector (epoch default); 328 tests >> 100 target; drive-to-100 ticked; +13 tests
• 2026-05-07: Phase 6 quicksort — recursive less/eq/greater partition via apl-compress, deterministic-pivot variant; tests cover empty/single/sorted/reverse/duplicates/negatives; all 5 classic programs done; +9 tests; 315/315
• 2026-05-07: Phase 6 n-queens — permutation enumerate + diagonal-conflict filter; counts q(1..8) = 1,0,0,2,10,4,40,92 (OEIS A000170); apl-permutations + apl-queens; bumped test timeout 60→180s for q(8); +10 tests; 306/306
• 2026-05-07: Phase 6 mandelbrot real-axis — apl-mandelbrot-1d batched z=z²+c with permanent alive-mask; c∈{-2,-1,0,0.25} bounded, c=1→3, c=0.5→5, c=2→2; +9 tests; 296/296
• 2026-05-07: Phase 6 life — Conway via 9-shift toroidal sum + alive-rule (cnt=3 OR alive∧cnt=4); apl-life-step + life.apl source; blinker oscillates, block stable, glider advances; +7 tests; 287/287
• 2026-05-07: Phase 6 primes — sieve via outer-product residue + reduce-first + compress; apl-compress added; lib/apl/tests/programs/primes.apl source; +11 tests; 280/280
• 2026-05-07: Phase 5 conformance.sh + scoreboard.{json,md} — per-suite runner; current snapshot 269/269; Phase 5 complete
• 2026-05-07: Phase 5 valence dispatch — apl-dfn-valence (AST scan for ⍺/⍵), apl-tradfn-valence (slot check), apl-call unified entry; +14 tests; 269/269 tests
• 2026-05-07: Phase 5 control words — :If/:Else, :While, :For/:In, :Select/:Case via apl-tradfn-eval-block/stmt threading env; :Trap deferred; +10 tests (sum loop, factorial, dispatch, nested); 255/255 tests
• 2026-05-07: Phase 5 tradfn — apl-call-tradfn + apl-tradfn-loop; line-numbered stmts, :branch goto, →0 exits, locals; +10 tests including loop sum; 245/245 tests
• 2026-05-07: Phase 5 dfn complete — apl-eval-stmts (guards, locals, ⍺←default), ∇ recursion via env "nabla"; +9 tests (factorial, guards, defaults, locals); 235/235 tests
• 2026-05-07: Phase 5 dfn foundation — lib/apl/transpile.sx with apl-eval-ast (handles :num :vec :name :monad :dyad :program :dfn) + glyph→fn lookup tables; apl-call-dfn / apl-call-dfn-m bind ⍺/⍵; ∇/guards/defaults/locals pending; 226/226 tests
• 2026-05-07: Phase 4 step 10 — at @ (apl-at-replace + apl-at-apply); linear-index lookup, scalar-vals broadcast; 211/211 tests
• 2026-05-07: Phase 4 step 9 — rank f⍤k (apl-rank); cell decomposition + reassembly via frame/cell shapes; 201/201 tests
• 2026-05-06: Phase 4 step 8 — power f⍣n (apl-power) + fixed-point f⍣≡ (apl-power-fixed); 191/191 tests
• 2026-05-06: Phase 4 step 7 — compose f∘g (apl-compose monadic f∘g x, apl-compose-dyadic dyadic f x (g y)); 182/182 tests
• 2026-05-06: Phase 4 step 6 — commute f⍨ (apl-commute monadic dup, apl-commute-dyadic swap); 173/173 tests
• 2026-05-06: Phase 4 step 5 — inner product f.g (apl-inner); +.× matrix multiply, ∧.= equal-vectors; 163/163 tests
• 2026-05-06: Phase 4 step 4 — outer product ∘.f (apl-outer); rank-doubling result shape = a-shape++b-shape; 151/151 tests
• 2026-05-06: Phase 4 step 3 — each f¨ (monadic apl-each + dyadic apl-each-dyadic); scalar broadcast both sides; 139/139 tests
• 2026-05-06: Phase 4 step 2 — scan f\ (last axis) + f⍀ (first axis); apl-scan/apl-scan-first; 125/125 tests
• 2026-05-06: Phase 4 step 1 — reduce f/ (last axis) + f⌿ (first axis); apl-reduce/apl-reduce-first; 110/110 tests
• 2026-05-06: Phase 3 complete — membership ∊, dyadic ⍳ (index-of), without ~ (index-of returns nil for not-found); 94/94 tests
• 2026-05-06: Phase 3 step 6 — enclose ⊂ / disclose ⊃ (box/unbox, rank-0 detect via type-of); 82/82 tests
• 2026-05-06: Phase 3 step 5 — grade-up ⍋ / grade-down ⍒ (stable insertion sort); 74/74 tests
• 2026-05-06: Phase 3 step 4 — squad ⌷ (scalar/multi-dim/partial-slice); 66/66 tests
• 2026-05-06: Phase 3 step 3 — catenate , (last axis, scalar promo) and first-axis; 59/59 tests
• 2026-05-06: Phase 3 step 2 — take ↑ (multi-axis, pad), drop ↓, reverse/rotate ⌽⊖ (last+first axis); 50/50 tests
• 2026-05-06: Phase 3 step 1 — reshape ⍴ (cycling), transpose ⍉ (monadic+dyadic); helpers apl-strides/flat->multi/multi->flat; 27/27 structural tests; lib/apl/tests/structural.sx
• 2026-04-26: Phase 2 complete — array model + 7 scalar primitive groups; 82/82 tests; lib/apl/runtime.sx + lib/apl/tests/scalar.sx
• 2026-04-26: parser (Phase 1 step 2) — 44/44 parser tests green (90/90 total); right-to-left segment algorithm; derived fns, outer/inner product, dfns with guards, strand handling; lib/apl/parser.sx + lib/apl/tests/parse.sx
• 2026-04-25: tokenizer (Phase 1 step 1) — 46/46 tests green; Unicode-aware starts-with? scanner for multi-byte APL glyphs; lib/apl/tokenizer.sx + lib/apl/tests/parse.sx

Blockers

• 2026-05-08: sx-tree MCP server disconnected at start of Phase 10. Path-based sx-tree tools error with Type_error("Expected string, got null") and the server then dropped entirely (45 tools unavailable). Loop paused at Phase 10 step 1 (⍸ where); resume once /mcp reconnects sx-tree.
• 2026-05-07: sx-tree MCP server disconnected mid-Phase-9. lib/apl/**.sx edits require sx-tree per CLAUDE.md — Edit/Read on .sx is hook-blocked. Loop paused at Phase 9 step 2 (inline assignment); resume once MCP restored.