rose-ash/plans/elixir-on-sx.md

# Elixir-on-SX: Elixir on the CEK/VM

Compile Elixir source to SX AST; the existing CEK evaluator runs it. The natural companion
to `lib/erlang/` — Elixir compiles to the BEAM and most of its runtime semantics are
Erlang's. The interesting parts are Elixir-specific: the macro system (`quote`/`unquote`),
the pipe operator `|>`, `with` expressions, `defmodule`/`def`/`defp`, protocol dispatch,
and the `Stream` lazy evaluation library.

End-state goal: **core Elixir programs running**, including modules, pattern matching, the
pipe operator, macros (`quote`/`unquote`/`defmacro`), protocols, and actor-style processes
reusing the Erlang runtime foundation.

## Ground rules

- **Scope:** only touch `lib/elixir/**` and `plans/elixir-on-sx.md`. Do **not** edit
  `spec/`, `hosts/`, `shared/`, or other `lib/<lang>/`. Reuse `lib/erlang/` runtime
  functions where possible — import them, don't duplicate.
- **Shared-file issues** go under "Blockers" below with a minimal repro; do not fix here.
- **SX files:** use `sx-tree` MCP tools only.
- **Architecture:** Elixir source → Elixir AST → SX AST. Reuse Erlang runtime for process/
  message/pattern primitives; add Elixir-specific surface in `lib/elixir/`.
- **Commits:** one feature per commit. Keep `## Progress log` updated and tick boxes.

## Architecture sketch

```
Elixir source text
    │
    ▼
lib/elixir/tokenizer.sx   — atoms (:atom), strings (""), charlists (''), sigils (~r, ~s etc.),
    │                        operators (|>, <>, ++, :::, etc.), do/end blocks
    ▼
lib/elixir/parser.sx      — Elixir AST: defmodule, def/defp/defmacro, @attribute,
    │                        pattern matching, |> pipe, with, for comprehension, quote/unquote,
    │                        case/cond/if/unless, fn, receive, try/rescue/catch/after
    ▼
lib/elixir/transpile.sx   — Elixir AST → SX AST
    │
    ├── lib/erlang/runtime.sx    (reused: processes, message passing, pattern match)
    └── lib/elixir/runtime.sx   — Elixir-specific: Kernel, String, Enum, Stream, Map,
                                   List, Tuple, IO, protocol dispatch, macro expansion
```

Key semantic mappings (differences from Erlang):
- `defmodule M do ... end` → SX `define-library` + module dict `{:module "M" :fns {...}}`
- `def f(args) do body end` → named function in module dict, with pattern-match dispatch
- `|>` pipe → left-to-right function composition; `a |> f(b)` = `f(a, b)`
- `with x <- expr, y <- expr2 do body else patterns end` → chained pattern match with early exit
- `for x <- list, filter, do: expr` → list comprehension (SX `map`/`filter`)
- `quote do expr end` → returns AST as SX list (homoiconic — Elixir AST IS SX-like)
- `unquote(expr)` → evaluate expr and splice into surrounding `quote`
- `defmacro` → macro in module; expanded at compile time by calling the SX macro
- Protocol → dict of implementations keyed by type name; `defprotocol` defines interface,
  `defimpl` registers an implementation
- `Stream` → lazy sequences using SX promises/coroutines (Phase 9/4 of primitives)
- `Agent`/`GenServer` → SX coroutine + message queue (similar to Erlang process model)

## Roadmap

### Phase 1 — tokenizer + parser
- [ ] Tokenizer: atoms (`:atom`, `:"atom with spaces"`), strings (`""`), charlists (`''`),
      numbers (int, float, hex `0xFF`, octal `0o77`, binary `0b11`), booleans (`true`/`false`/`nil`),
      operators (`|>`, `<>`, `++`, `--`, `:::`, `&&`, `||`, `!`, `..`, `<-`, `=~`),
      sigils (`~r/regex/`, `~s"string"`, `~w(word list)`), do/end blocks, keywords as args
      `f(key: val)`, `@module_attribute`
- [ ] Parser:
      - Module: `defmodule Name do ... end` → module AST with body
      - Functions: `def f(pat) do body end`, `def f(pat) when guard do body end`,
        multi-clause `def f(a) do ...; def f(b) do ...` → clause list
      - `defp` (private), `defmacro`, `defmacrop`
      - `@doc`, `@moduledoc`, `@spec`, `@type`, `@behaviour` module attributes
      - `case expr do patterns end`, `cond do clauses end`, `if`/`unless`
      - `with x <- e, y <- e2, do: body, else: [pattern -> body]`
      - `for x <- list, filter, into: acc, do: expr` comprehension
      - `fn pat -> body end` anonymous function; capture `&Module.fun/arity`, `&(&1 + 1)`
      - `receive do patterns after timeout -> body end`
      - `try do body rescue e -> ... catch type, val -> ... after ... end`
      - `quote do ... end`, `unquote(expr)`, `unquote_splicing(list)`
      - `|>` pipe chain: `a |> f |> g(b)` → `g(f(a), b)`
- [ ] Tests in `lib/elixir/tests/parse.sx`

### Phase 2 — transpile: basic Elixir (no macros, no processes)
- [ ] `ex-eval-ast` entry
- [ ] Arithmetic, string `<>`, list `++`/`--`, comparison, boolean (`and`/`or`/`not`)
- [ ] Pattern matching in `=`, function heads, `case` — reuse Erlang pattern engine
- [ ] `def`/`defp` → SX `define` with clause dispatch (like Erlang function clauses)
- [ ] Module as a dict of named functions; `ModuleName.function(args)` dispatch
- [ ] `|>` pipe: desugar `a |> f(b, c)` → `f(a, b, c)` at transpile time
- [ ] `with` expression: chain of `<-` bindings, short-circuit on mismatch to `else`
- [ ] `for` comprehension: `for x <- list, filter do body end` → `map`/`filter`
- [ ] `fn` anonymous functions, `&` capture forms
- [ ] `if`/`unless`/`cond`/`case`
- [ ] String interpolation: `"Hello #{name}"` → string concat
- [ ] Keyword lists `[key: val]` → SX list of `{:key val}` dicts; maps `%{key: val}` → SX dict
- [ ] Tuples `{a, b, c}` → SX list (or vector); `elem/2`, `put_elem/3`
- [ ] 40+ eval tests in `lib/elixir/tests/eval.sx`

### Phase 3 — macro system
- [ ] `quote do expr end` → returns Elixir AST as SX list structure
      (Elixir AST is 3-tuples `{name, meta, args}` — map to SX `(list name meta args)`)
- [ ] `unquote(expr)` → evaluate and splice into surrounding `quote`
- [ ] `unquote_splicing(list)` → splice list into surrounding `quote`
- [ ] `defmacro` → define a macro in the module; macro receives AST args, returns AST
- [ ] Macro expansion: expand macros before transpiling (two-pass: collect defs, then expand)
- [ ] `use Module` → calls `Module.__using__/1` macro, injects code into caller
- [ ] `import Module` → bring functions into scope without prefix
- [ ] `alias Module, as: M` → short name for module
- [ ] Tests: `defmacro unless`, `defmacro my_if`, `use` injection, `__MODULE__`, `__DIR__`

### Phase 4 — protocols
- [ ] `defprotocol P do @spec f(t) :: result end` → defines protocol dict + dispatch fn
- [ ] `defimpl P, for: Type do def f(t) do ... end end` → register implementation
- [ ] Protocol dispatch: `P.f(value)` → look up type of value, find implementation, call it
- [ ] Built-in protocols: `Enumerable`, `Collectable`, `String.Chars`, `Inspect`
- [ ] `Enumerable` implementation for lists, maps, ranges — enables `Enum.*` on custom types
- [ ] `derive` — automatic protocol implementation for simple structs
- [ ] Tests: custom type implementing `Enumerable`, `String.Chars`, protocol fallback

### Phase 5 — structs + behaviours
- [ ] `defstruct [:field1, field2: default]` → defines `%ModuleName{}` struct type
      Structs are maps with `__struct__: ModuleName` key + defined fields
- [ ] Struct pattern matching: `%User{name: n} = user`
- [ ] `@behaviour Module` → declares behaviour callbacks; compile-time check
- [ ] `@impl true` / `@impl BehaviourName` → marks function as behaviour implementation
- [ ] Built-in behaviours: `GenServer`, `Supervisor`, `Agent`, `Task`
- [ ] Tests: struct creation, update syntax `%{struct | field: val}`, behaviour callbacks

### Phase 6 — processes + OTP patterns (reuses Erlang runtime)
- [ ] `spawn(fn -> ... end)` / `spawn(M, f, args)` → SX coroutine on scheduler
      Reuse `lib/erlang/` process + message queue infrastructure
- [ ] `send(pid, msg)` / `receive do patterns end` — already in Erlang runtime
- [ ] `GenServer` behaviour: `start_link`, `call`, `cast`, `handle_call`, `handle_cast`,
      `handle_info`, `init` — implement as SX macros expanding to process + message loop
- [ ] `Agent` — simple state wrapper over GenServer; `Agent.start_link`, `get`, `update`
- [ ] `Task` — async computation; `Task.async`, `Task.await`
- [ ] `Supervisor` — child spec, restart strategy (`one_for_one`, `one_for_all`)
- [ ] Tests: counter GenServer, bank account Agent, parallel Task, supervised worker

### Phase 7 — standard library
- [ ] `Enum.*` — `map`, `filter`, `reduce`, `each`, `into`, `flat_map`, `zip`, `sort`,
      `sort_by`, `min_by`, `max_by`, `group_by`, `frequencies`, `count`, `any?`, `all?`,
      `find`, `take`, `drop`, `take_while`, `drop_while`, `chunk_every`, `chunk_by`,
      `flat_map_reduce`, `scan`, `uniq`, `uniq_by`, `member?`, `empty?`, `sum`, `product`
- [ ] `Stream.*` — lazy versions of Enum; `Stream.map`, `Stream.filter`, `Stream.take`,
      `Stream.cycle`, `Stream.iterate`, `Stream.unfold`, `Stream.resource`
      Uses SX promises (Phase 9) for laziness
- [ ] `String.*` — `length`, `upcase`, `downcase`, `trim`, `split`, `replace`, `contains?`,
      `starts_with?`, `ends_with?`, `slice`, `at`, `graphemes`, `codepoints`, `to_integer`,
      `to_float`, `pad_leading`, `pad_trailing`, `duplicate`, `match?`
- [ ] `Map.*` — `new`, `get`, `put`, `delete`, `update`, `merge`, `keys`, `values`,
      `to_list`, `from_struct`, `has_key?`, `filter`, `map`, `reject`, `take`, `drop`
- [ ] `List.*` — `first`, `last`, `flatten`, `zip`, `unzip`, `keystore`, `keyfind`,
      `wrap`, `duplicate`, `improper?`, `delete`, `insert_at`, `replace_at`
- [ ] `Tuple.*` — `to_list`, `from_list`, `append`, `insert_at`, `delete_at`
- [ ] `Integer.*` / `Float.*` — `parse`, `to_string`, `digits`, `pow`, `is_odd?`, `is_even?`
- [ ] `IO.*` — `puts`, `gets`, `inspect`, `write`, `read` → SX IO perform
- [ ] `Kernel.*` — built-in functions: `is_integer?`, `is_binary?`, `length`, `hd`, `tl`,
      `elem`, `put_elem`, `apply`, `raise`, `exit`, `inspect`
- [ ] `inspect/1` / `IO.inspect/2` — debug printing using `Inspect` protocol

### Phase 8 — conformance target
- [ ] Vendor or hand-build 100+ Elixir program tests in `lib/elixir/tests/programs/`
- [ ] Drive scoreboard

## Blockers

_(none yet)_

## Progress log

_Newest first._

_(awaiting phase 1)_