Recover agent-loop progress: lua/prolog/forth/erlang/haskell phases 1-2
Salvaged from worktree-agent-* branches killed during sx-tree MCP outage: - lua: tokenizer + parser + phase-2 transpile (~157 tests) - prolog: tokenizer + parser + unification (72 tests, plan update lost to WIP) - forth: phase-1 reader/interpreter + phase-2 colon/VARIABLE (134 tests) - erlang: tokenizer + parser (114 tests) - haskell: tokenizer + parse tests (43 tests) Cherry-picked file contents only, not branch history, to avoid pulling in unrelated ocaml-vm merge commits that were in those branches' bases.
This commit is contained in:
@@ -50,10 +50,11 @@ Core mapping:
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — tokenizer + parser
|
||||
- [ ] Tokenizer: atoms (bare + single-quoted), variables (Uppercase/`_`-prefixed), numbers (int, float, `16#HEX`), strings `"..."`, chars `$c`, punct `( ) { } [ ] , ; . : :: ->`
|
||||
- [ ] Parser: module declarations, `-module`/`-export`/`-import` attributes, function clauses with head patterns + guards + body
|
||||
- [ ] Expressions: literals, vars, calls, tuples `{...}`, lists `[...|...]`, binaries `<<...>>`, `if`, `case`, `receive`, `fun`, `try/catch`, operators
|
||||
- [ ] Unit tests in `lib/erlang/tests/parse.sx`
|
||||
- [x] Tokenizer: atoms (bare + single-quoted), variables (Uppercase/`_`-prefixed), numbers (int, float, `16#HEX`), strings `"..."`, chars `$c`, punct `( ) { } [ ] , ; . : :: ->` — **62/62 tests**
|
||||
- [x] Parser: module declarations, `-module`/`-export`/`-import` attributes, function clauses with head patterns + guards + body — **52/52 tests**
|
||||
- [x] Expressions: literals, vars, calls, tuples `{...}`, lists `[...|...]`, `if`, `case`, `receive`, `fun`, `try/catch`, operators, precedence
|
||||
- [ ] Binaries `<<...>>` — not yet parsed (deferred to Phase 6)
|
||||
- [x] Unit tests in `lib/erlang/tests/parse.sx`
|
||||
|
||||
### Phase 2 — sequential eval + pattern matching + BIFs
|
||||
- [ ] `erlang-eval-ast`: evaluate sequential expressions
|
||||
@@ -98,7 +99,8 @@ Core mapping:
|
||||
|
||||
_Newest first._
|
||||
|
||||
- _(not started)_
|
||||
- **parser green** — `lib/erlang/parser.sx` + `parser-core.sx` + `parser-expr.sx` + `parser-module.sx`. 52/52 in `tests/parse.sx`. Covers literals, tuples, lists (incl. `[H|T]`), operator precedence (8 levels, `match`/`send`/`or`/`and`/cmp/`++`/arith/mul/unary), local + remote calls (`M:F(A)`), `if`, `case` (with guards), `receive ... after ... end`, `begin..end` blocks, anonymous `fun`, `try..of..catch..after..end` with `Class:Pattern` catch clauses. Module-level: `-module(M).`, `-export([...]).`, multi-clause functions with guards. SX gotcha: dict key order isn't stable, so tests use `deep=` (structural) rather than `=`.
|
||||
- **tokenizer green** — `lib/erlang/tokenizer.sx` + `lib/erlang/tests/tokenize.sx`. Covers atoms (bare, quoted, `node@host`), variables, integers (incl. `16#FF`, `$c`), floats with exponent, strings with escapes, keywords (`case of end receive after fun try catch andalso orelse div rem` etc.), punct (`( ) { } [ ] , ; . : :: -> <- <= => << >> | ||`), ops (`+ - * / = == /= =:= =/= < > =< >= ++ -- ! ?`), `%` line comments. 62/62 green.
|
||||
|
||||
## Blockers
|
||||
|
||||
|
||||
@@ -49,24 +49,24 @@ Representation:
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — reader + interpret mode
|
||||
- [ ] `lib/forth/reader.sx`: whitespace-split, number parsing (base-aware)
|
||||
- [ ] `lib/forth/runtime.sx`: stack as SX list, push/pop/peek helpers
|
||||
- [ ] Core stack words: `DUP`, `DROP`, `SWAP`, `OVER`, `ROT`, `NIP`, `TUCK`, `PICK`, `ROLL`, `?DUP`, `2DUP`, `2DROP`, `2SWAP`, `2OVER`
|
||||
- [ ] Arithmetic: `+`, `-`, `*`, `/`, `MOD`, `/MOD`, `NEGATE`, `ABS`, `MIN`, `MAX`, `1+`, `1-`, `2*`, `2/`
|
||||
- [ ] Comparison: `=`, `<`, `>`, `<=`, `>=`, `0=`, `0<`, `0>`
|
||||
- [ ] Logical: `AND`, `OR`, `XOR`, `INVERT`
|
||||
- [ ] I/O: `.` (print), `.S` (show stack), `EMIT`, `CR`, `SPACE`, `SPACES`
|
||||
- [ ] Interpreter loop: read word, look up, execute, repeat
|
||||
- [ ] Unit tests in `lib/forth/tests/interp.sx`
|
||||
- [x] `lib/forth/reader.sx`: whitespace-split, number parsing (base-aware)
|
||||
- [x] `lib/forth/runtime.sx`: stack as SX list, push/pop/peek helpers
|
||||
- [x] Core stack words: `DUP`, `DROP`, `SWAP`, `OVER`, `ROT`, `-ROT`, `NIP`, `TUCK`, `PICK`, `ROLL`, `?DUP`, `DEPTH`, `2DUP`, `2DROP`, `2SWAP`, `2OVER`
|
||||
- [x] Arithmetic: `+`, `-`, `*`, `/`, `MOD`, `/MOD`, `NEGATE`, `ABS`, `MIN`, `MAX`, `1+`, `1-`, `2+`, `2-`, `2*`, `2/`
|
||||
- [x] Comparison: `=`, `<>`, `<`, `>`, `<=`, `>=`, `0=`, `0<>`, `0<`, `0>`
|
||||
- [x] Logical: `AND`, `OR`, `XOR`, `INVERT` (32-bit two's-complement sim)
|
||||
- [x] I/O: `.` (print), `.S` (show stack), `EMIT`, `CR`, `SPACE`, `SPACES`, `BL`
|
||||
- [x] Interpreter loop: read word, look up, execute, repeat
|
||||
- [x] Unit tests in `lib/forth/tests/test-phase1.sx` — 108/108 pass
|
||||
|
||||
### Phase 2 — colon definitions + compile mode
|
||||
- [ ] `:` opens compile mode and starts a definition
|
||||
- [ ] `;` closes it and installs into the dictionary
|
||||
- [ ] Compile mode: non-IMMEDIATE words get appended as SX references; numbers get compiled as literals; IMMEDIATE words (like `IF`) run now
|
||||
- [ ] `VARIABLE`, `CONSTANT`, `VALUE`, `TO`
|
||||
- [ ] `@` (fetch), `!` (store), `+!`
|
||||
- [ ] Compile a colon def into an SX lambda that the CEK runs directly
|
||||
- [ ] Tests: define words, call them, nest definitions
|
||||
- [x] `:` opens compile mode and starts a definition
|
||||
- [x] `;` closes it and installs into the dictionary
|
||||
- [x] Compile mode: non-IMMEDIATE words are compiled as late-binding call thunks; numbers are compiled as pushers; IMMEDIATE words run immediately
|
||||
- [x] `VARIABLE`, `CONSTANT`, `VALUE`, `TO`, `RECURSE`, `IMMEDIATE`
|
||||
- [x] `@` (fetch), `!` (store), `+!`
|
||||
- [x] Colon-def body is `(fn (s) (for-each op body))` — runs on CEK, inherits TCO
|
||||
- [x] Tests in `lib/forth/tests/test-phase2.sx` — 26/26 pass
|
||||
|
||||
### Phase 3 — control flow + first Hayes tests green
|
||||
- [ ] `IF`, `ELSE`, `THEN` — compile to SX `if`
|
||||
@@ -99,7 +99,25 @@ Representation:
|
||||
|
||||
_Newest first._
|
||||
|
||||
- _(not started)_
|
||||
- **Phase 2 complete — colon defs, compile mode, VARIABLE/CONSTANT/VALUE/TO, @/!/+! (+26).**
|
||||
`lib/forth/compiler.sx` plus `tests/test-phase2.sx`.
|
||||
Colon-def body is a list of ops (one per source token) wrapped in a single
|
||||
lambda. References are late-binding thunks so forward/recursive references
|
||||
work via `RECURSE`. Redefinitions take effect immediately.
|
||||
VARIABLE creates a pusher for a symbolic address stored in `state.vars`.
|
||||
CONSTANT compiles to `(fn (s) (forth-push s v))`. VALUE/TO share the vars dict.
|
||||
Compiler rewrites `forth-interpret` to drive from a token list stored on
|
||||
state so parsing words (`:`, `VARIABLE`, `TO` etc.) can consume the next
|
||||
token with `forth-next-token!`. 134/134 (Phase 1 + 2) green.
|
||||
|
||||
- **Phase 1 complete — reader + interpret mode + core words (+108).**
|
||||
`lib/forth/{runtime,reader,interpreter}.sx` plus `tests/test-phase1.sx`.
|
||||
Stack as SX list (TOS = first). Dict is `{lowercased-name -> {:kind :body :immediate?}}`.
|
||||
Data + return stacks both mutable. Output buffered in state (no host IO yet).
|
||||
BASE-aware number parsing with `$`, `%`, `#` prefixes and `'c'` char literals.
|
||||
Bitwise AND/OR/XOR/INVERT simulated over 32-bit two's-complement.
|
||||
Integer `/` is truncated-toward-zero (ANS symmetric), MOD matches.
|
||||
Case-insensitive lookup. 108/108 tests green.
|
||||
|
||||
## Blockers
|
||||
|
||||
|
||||
@@ -54,11 +54,11 @@ Key mappings:
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — tokenizer + parser + layout rule
|
||||
- [ ] Tokenizer: reserved words, qualified names, operators, numbers (int, float, Rational later), chars/strings, comments (`--` and `{-` nested)
|
||||
- [x] Tokenizer: reserved words, qualified names, operators, numbers (int, float, Rational later), chars/strings, comments (`--` and `{-` nested)
|
||||
- [ ] Layout algorithm: turn indentation into virtual `{`, `;`, `}` tokens per Haskell 98 §10.3
|
||||
- [ ] Parser: modules, imports (stub), top-level decls, type sigs, function clauses with patterns + guards + where-clauses, expressions with operator precedence, lambdas, `let`, `if`, `case`, `do`, list comp, sections
|
||||
- [ ] AST design modelled on GHC's HsSyn at a surface level
|
||||
- [ ] Unit tests in `lib/haskell/tests/parse.sx`
|
||||
- [x] Unit tests in `lib/haskell/tests/parse.sx` (43 tokenizer tests, all green)
|
||||
|
||||
### Phase 2 — desugar + eager-ish eval + ADTs (untyped)
|
||||
- [ ] Desugar: guards → nested `if`s; `where` → `let`; list comp → `concatMap`-based; do-notation stays for now (desugared in phase 3)
|
||||
@@ -107,7 +107,27 @@ Key mappings:
|
||||
|
||||
_Newest first._
|
||||
|
||||
- _(not started)_
|
||||
- **2026-04-24** — Phase 1: Haskell 98 tokenizer (`lib/haskell/tokenizer.sx`, 490 lines)
|
||||
covering idents (lower/upper/qvarid/qconid), 23 reserved words, 11 reserved ops,
|
||||
varsym/consym operator chains, integer/hex/octal/float literals incl. exponent
|
||||
notation, char + string literals with escape sequences, nested `{- ... -}` block
|
||||
comments with depth counter, `-- ... EOL` line comments (respecting the
|
||||
"followed by symbol = not a comment" Haskell 98 rule), backticks, punctuation,
|
||||
and explicit `newline` tokens for the upcoming layout pass. 43 structural tests
|
||||
in `lib/haskell/tests/parse.sx`, a lightweight `hk-deep=?` equality helper
|
||||
and a custom `lib/haskell/test.sh` runner (pipes through the OCaml epoch
|
||||
protocol, falls back to the main-repo build when run from a worktree). 43/43
|
||||
green.
|
||||
|
||||
Also peeked at `/root/rose-ash/sx-haskell/` per briefing: that directory is a
|
||||
Haskell program implementing an **SX interpreter** (Types.hs, Eval.hs,
|
||||
Primitives.hs, etc. — ~2800 lines of .hs) — the *opposite* direction from this
|
||||
project. Nothing to fold in.
|
||||
|
||||
Gotchas hit: `emit!` and `peek` are SX evaluator special forms, so every local
|
||||
helper uses the `hk-` prefix. `cond`/`when`/`let` clauses evaluate ONLY the
|
||||
last expression; multi-expression bodies MUST be wrapped in `(do ...)`. These
|
||||
two together account for all the tokenizer's early crashes.
|
||||
|
||||
## Blockers
|
||||
|
||||
|
||||
@@ -37,18 +37,18 @@ Runtime shims in `lib/lua/runtime.sx`: `lua-truthy?`, string coercion for `..`/a
|
||||
Each item: implement → tests → tick box → update progress log.
|
||||
|
||||
### Phase 1 — tokenizer + parser
|
||||
- [ ] Tokenizer: numbers (int, float, hex), strings (short + long `[[…]]`), idents, keywords, operators, comments (`--`, `--[[…]]`)
|
||||
- [ ] Parser: blocks, `local`, `if/elseif/else/end`, `while`, numeric `for`, `function`, `return`, expressions, table constructors, indexing (`.`, `[]`), calls (`f(…)`, `f:m(…)`)
|
||||
- [ ] Skip for phase 1: generic `for … in …`, goto/labels, nested varargs `...`
|
||||
- [ ] Unit tests in `lib/lua/tests/parse.sx`: source → expected AST
|
||||
- [x] Tokenizer: numbers (int, float, hex), strings (short + long `[[…]]`), idents, keywords, operators, comments (`--`, `--[[…]]`)
|
||||
- [x] Parser: blocks, `local`, `if/elseif/else/end`, `while`, numeric `for`, `function`, `return`, expressions, table constructors, indexing (`.`, `[]`), calls (`f(…)`, `f:m(…)`)
|
||||
- [x] Skip for phase 1: generic `for … in …`, goto/labels, nested varargs `...`
|
||||
- [x] Unit tests in `lib/lua/tests/parse.sx`: source → expected AST
|
||||
|
||||
### Phase 2 — transpile: control flow + arithmetic
|
||||
- [ ] `lua-eval-ast` entry
|
||||
- [ ] Arithmetic (Lua 5.1 semantics — `/` is float)
|
||||
- [ ] Comparison + logical (short-circuit, Lua truthy)
|
||||
- [ ] `..` concat with string/number coercion
|
||||
- [ ] `if`, `while`, numeric `for`, `local`, assignment, blocks
|
||||
- [ ] 30+ eval tests in `lib/lua/tests/eval.sx`
|
||||
- [x] `lua-eval-ast` entry
|
||||
- [x] Arithmetic (Lua 5.1 semantics — `/` is float)
|
||||
- [x] Comparison + logical (short-circuit, Lua truthy)
|
||||
- [x] `..` concat with string/number coercion
|
||||
- [x] `if`, `while`, numeric `for`, `local`, assignment, blocks
|
||||
- [x] 30+ eval tests in `lib/lua/tests/eval.sx`
|
||||
|
||||
### Phase 3 — tables + functions + first PUC-Rio slice
|
||||
- [ ] `function` (anon, local, top-level), closures
|
||||
@@ -82,7 +82,9 @@ Each item: implement → tests → tick box → update progress log.
|
||||
|
||||
_Newest first. Agent appends on every commit._
|
||||
|
||||
- _(awaiting phase 1)_
|
||||
- 2026-04-24: lua: phase 2 transpile — arithmetic, comparison, short-circuit logical, `..` concat, if/while/repeat/for-num/local/assign. 157 total tests green.
|
||||
- 2026-04-24: lua: parser (exprs with precedence, all phase-1 statements, funcbody, table ctors, method/chained calls) — 112 total tokenizer+parser tests
|
||||
- 2026-04-24: lua: tokenizer (numbers/strings/long-brackets/keywords/ops/comments) + 56 tests
|
||||
|
||||
## Blockers
|
||||
|
||||
|
||||
Reference in New Issue
Block a user