hs: query targets, prolog hook, loop scripts, new plans, WASM regen
Hyperscript compiler/runtime:
- query target support in set/fire/put commands
- hs-set-prolog-hook! / hs-prolog-hook / hs-prolog in runtime
- runtime log-capture cleanup
Scripts: sx-loops-up/down, sx-hs-e-up/down, sx-primitives-down
Plans: datalog, elixir, elm, go, koka, minikanren, ocaml, hs-bucket-f,
designs (breakpoint, null-safety, step-limit, tell, cookies, eval,
plugin-system)
lib/prolog/hs-bridge.sx: initial hook-based bridge draft
lib/common-lisp/tests/runtime.sx: CL runtime tests
WASM: regenerate sx_browser.bc.js from updated hs sources
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
145
plans/datalog-on-sx.md
Normal file
145
plans/datalog-on-sx.md
Normal file
@@ -0,0 +1,145 @@
|
||||
# Datalog-on-SX: Datalog on the CEK/VM
|
||||
|
||||
Datalog is a declarative query language: a restricted subset of Prolog with no function
|
||||
symbols, only relations. Programs are sets of facts and rules; queries ask what follows.
|
||||
Evaluation is bottom-up (fixpoint iteration) rather than Prolog's top-down DFS — which
|
||||
means no infinite loops, guaranteed termination, and efficient incremental updates.
|
||||
|
||||
The unique angle: Datalog is a natural companion to the Prolog implementation already in
|
||||
progress (`lib/prolog/`). The parser and term representation can share infrastructure;
|
||||
the evaluator is an entirely different fixpoint engine rather than a DFS solver.
|
||||
|
||||
End-state goal: **full core Datalog** (facts, rules, stratified negation, aggregation,
|
||||
recursion) with a clean SX query API, and a demonstration of Datalog as a query engine
|
||||
for rose-ash data (e.g. federation graph, content relationships).
|
||||
|
||||
## Ground rules
|
||||
|
||||
- **Scope:** only touch `lib/datalog/**` and `plans/datalog-on-sx.md`. Do **not** edit
|
||||
`spec/`, `hosts/`, `shared/`, `lib/prolog/**`, or other `lib/<lang>/`.
|
||||
- **Shared-file issues** go under "Blockers" below with a minimal repro; do not fix here.
|
||||
- **SX files:** use `sx-tree` MCP tools only.
|
||||
- **Architecture:** Datalog source → term AST → fixpoint evaluator. No transpiler to SX AST —
|
||||
the evaluator is written in SX and works directly on term structures.
|
||||
- **Reference:** Ramakrishnan & Ullman "A Survey of Deductive Database Systems";
|
||||
Dalmau "Datalog and Constraint Satisfaction".
|
||||
- **Commits:** one feature per commit. Keep `## Progress log` updated and tick boxes.
|
||||
|
||||
## Architecture sketch
|
||||
|
||||
```
|
||||
Datalog source text
|
||||
│
|
||||
▼
|
||||
lib/datalog/tokenizer.sx — atoms, variables, numbers, strings, punct (?- :- , . ( ) [ ])
|
||||
│
|
||||
▼
|
||||
lib/datalog/parser.sx — facts: atom(args). rules: head :- body. queries: ?- goal.
|
||||
│ No function symbols (only constants and variables in args).
|
||||
▼
|
||||
lib/datalog/db.sx — extensional DB (EDB): ground facts; IDB: derived relations;
|
||||
│ clause index by relation name/arity
|
||||
▼
|
||||
lib/datalog/eval.sx — bottom-up fixpoint: semi-naive evaluation with delta sets;
|
||||
│ stratification for negation; incremental update API
|
||||
▼
|
||||
lib/datalog/query.sx — query API: (datalog-query db goal) → list of substitutions;
|
||||
SX embedding: define facts/rules as SX data directly
|
||||
```
|
||||
|
||||
Key differences from Prolog:
|
||||
- **No function symbols** — args are atoms, numbers, strings, or variables only. No `f(a,b)`.
|
||||
- **No cuts** — no procedural control.
|
||||
- **Bottom-up** — derive all consequences of all rules before answering; no search tree.
|
||||
- **Termination guaranteed** — no infinite derivation chains (no function symbols → finite Herbrand base).
|
||||
- **Stratified negation** — `not(P)` legal iff P does not recursively depend on its own negation.
|
||||
- **Aggregation** — `count`, `sum`, `min`, `max` over derived tuples (Datalog+).
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — tokenizer + parser
|
||||
- [ ] Tokenizer: atoms (lowercase/quoted), variables (uppercase/`_`), numbers, strings,
|
||||
operators (`:- `, `?-`, `,`, `.`), comments (`%`, `/* */`)
|
||||
Note: no function symbol syntax (no nested `f(...)` in arg position).
|
||||
- [ ] Parser:
|
||||
- Facts: `parent(tom, bob).` → `{:head (parent tom bob) :body ()}`
|
||||
- Rules: `ancestor(X,Z) :- parent(X,Y), ancestor(Y,Z).`
|
||||
→ `{:head (ancestor X Z) :body ((parent X Y) (ancestor Y Z))}`
|
||||
- Queries: `?- ancestor(tom, X).` → `{:query (ancestor tom X)}`
|
||||
- Negation: `not(parent(X,Y))` in body position → `{:neg (parent X Y)}`
|
||||
- [ ] Tests in `lib/datalog/tests/parse.sx`
|
||||
|
||||
### Phase 2 — unification + substitution
|
||||
- [ ] Share or port unification from `lib/prolog/` — term walk, occurs check off by default
|
||||
- [ ] `dl-unify` `t1` `t2` `subst` → extended subst or nil (no function symbols means simpler)
|
||||
- [ ] `dl-ground?` `term` → bool — all variables bound in substitution
|
||||
- [ ] Tests: atom/atom, var/atom, var/var, list args
|
||||
|
||||
### Phase 3 — extensional DB + naive evaluation
|
||||
- [ ] EDB: `{:relation-name → set-of-ground-tuples}` using SX sets (Phase 18 of primitives)
|
||||
- [ ] `dl-add-fact!` `db` `relation` `args` → add ground tuple
|
||||
- [ ] `dl-add-rule!` `db` `head` `body` → add rule clause
|
||||
- [ ] Naive evaluation: iterate rules until fixpoint
|
||||
For each rule, for each combination of body tuples that unify, derive head tuple.
|
||||
Repeat until no new tuples added.
|
||||
- [ ] `dl-query` `db` `goal` → list of substitutions satisfying goal against derived DB
|
||||
- [ ] Tests: transitive closure (ancestor), sibling, same-generation — classic Datalog programs
|
||||
|
||||
### Phase 4 — semi-naive evaluation (performance)
|
||||
- [ ] Delta sets: track newly derived tuples per iteration
|
||||
- [ ] Semi-naive rule: only join against delta tuples from last iteration, not full relation
|
||||
- [ ] Significant speedup for recursive rules — avoids re-deriving known tuples
|
||||
- [ ] `dl-stratify` `db` → dependency graph + SCC analysis → stratum ordering
|
||||
- [ ] Tests: verify semi-naive produces same results as naive; benchmark on large ancestor chain
|
||||
|
||||
### Phase 5 — stratified negation
|
||||
- [ ] Dependency graph analysis: which relations depend on which (positively or negatively)
|
||||
- [ ] Stratification check: error if negation is in a cycle (non-stratifiable program)
|
||||
- [ ] Evaluation: process strata in order — lower stratum fully computed before using its
|
||||
complement in a higher stratum
|
||||
- [ ] `not(P)` in rule body: at evaluation time, check P is NOT in the derived EDB
|
||||
- [ ] Tests: non-member (`not(member(X,L))`), colored-graph (`not(same-color(X,Y))`),
|
||||
stratification error detection
|
||||
|
||||
### Phase 6 — aggregation (Datalog+)
|
||||
- [ ] `count(X, Goal)` → number of distinct X satisfying Goal
|
||||
- [ ] `sum(X, Goal)` → sum of X values satisfying Goal
|
||||
- [ ] `min(X, Goal)` / `max(X, Goal)` → min/max of X satisfying Goal
|
||||
- [ ] `group-by` semantics: `count(X, sibling(bob, X))` → count of bob's siblings
|
||||
- [ ] Aggregation breaks stratification — evaluate in a separate post-fixpoint pass
|
||||
- [ ] Tests: social network statistics, grade aggregation, inventory sums
|
||||
|
||||
### Phase 7 — SX embedding API
|
||||
- [ ] `(dl-program facts rules)` → database from SX data directly (no parsing required)
|
||||
```
|
||||
(dl-program
|
||||
'((parent tom bob) (parent tom liz) (parent bob ann))
|
||||
'((ancestor X Z :- (parent X Y) (ancestor Y Z))
|
||||
(ancestor X Y :- (parent X Y))))
|
||||
```
|
||||
- [ ] `(dl-query db '(ancestor tom ?X))` → `((ann) (bob) (liz) (pat))`
|
||||
- [ ] `(dl-assert! db '(parent ann pat))` → incremental fact addition + re-derive
|
||||
- [ ] `(dl-retract! db '(parent tom bob))` → fact removal + re-derive from scratch
|
||||
- [ ] Integration demo: federation graph query — `(ancestor actor1 actor2)` over
|
||||
rose-ash ActivityPub follow relationships
|
||||
|
||||
### Phase 8 — Datalog as a query language for rose-ash
|
||||
- [ ] Schema: map SQLAlchemy model relationships to Datalog EDB facts
|
||||
(e.g. `(follows user1 user2)`, `(authored user post)`, `(tagged post tag)`)
|
||||
- [ ] Loader: `dl-load-from-db!` — query PostgreSQL, populate Datalog EDB
|
||||
- [ ] Query examples:
|
||||
- `?- ancestor(me, X), authored(X, Post), tagged(Post, cooking).`
|
||||
→ posts about cooking by people I follow (transitively)
|
||||
- `?- popular(Post) :- tagged(Post, T), count(L, (liked(L, Post))) >= 10.`
|
||||
→ posts with 10+ likes
|
||||
- [ ] Expose as a rose-ash service endpoint: `POST /internal/datalog` with program + query
|
||||
|
||||
## Blockers
|
||||
|
||||
_(none yet)_
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
_(awaiting phase 1)_
|
||||
80
plans/designs/f-breakpoint.md
Normal file
80
plans/designs/f-breakpoint.md
Normal file
@@ -0,0 +1,80 @@
|
||||
# F-Breakpoint — `breakpoint` command (+2)
|
||||
|
||||
**Suite:** `hs-upstream-breakpoint`
|
||||
**Target:** Both tests are `SKIP (untranslated)`.
|
||||
|
||||
## 1. The 2 tests
|
||||
|
||||
- `parses as a top-level command`
|
||||
- `parses inside an event handler`
|
||||
|
||||
Both are untranslated — no test body exists. The test names say "parses" — these are parser tests, not runtime tests.
|
||||
|
||||
## 2. What upstream checks
|
||||
|
||||
From `test/core/breakpoint.js`:
|
||||
|
||||
```js
|
||||
it('parses as a top-level command', () => {
|
||||
expect(() => _hyperscript.evaluate("breakpoint")).not.toThrow();
|
||||
});
|
||||
it('parses inside an event handler', () => {
|
||||
const el = document.createElement('div');
|
||||
el.setAttribute('_', 'on click breakpoint');
|
||||
expect(() => _hyperscript.processNode(el)).not.toThrow();
|
||||
});
|
||||
```
|
||||
|
||||
Both tests verify that `breakpoint` is accepted by the parser without throwing. Neither test checks that the debugger actually fires. `breakpoint` is a no-op command in production builds — it calls `debugger` in JS, which is a no-op when devtools are closed.
|
||||
|
||||
## 3. What's needed
|
||||
|
||||
### Parser (`lib/hyperscript/parser.sx`)
|
||||
|
||||
Add `breakpoint` to the command dispatch — it should parse as a zero-argument command. The parser's command `cond` (wherever `add`, `remove`, `hide` etc. are dispatched) needs a branch:
|
||||
|
||||
```
|
||||
((= val "breakpoint") (hs-parse-breakpoint))
|
||||
```
|
||||
|
||||
`hs-parse-breakpoint` just returns a `{:cmd "breakpoint"}` AST node (or however commands are represented). It consumes no additional tokens.
|
||||
|
||||
### Compiler (`lib/hyperscript/compiler.sx`)
|
||||
|
||||
Add a compiler branch for `breakpoint` AST node. Emits a no-op or a `debugger` statement equivalent. Since we're in SX (not JS), a no-op `(do nil)` is correct.
|
||||
|
||||
### Generator (`tests/playwright/generate-sx-tests.py`)
|
||||
|
||||
The 2 tests are simple — hand-write them:
|
||||
|
||||
```lisp
|
||||
(deftest "parses as a top-level command"
|
||||
(let ((result (guard (e (true false))
|
||||
(hs-compile "breakpoint")
|
||||
true)))
|
||||
(assert result)))
|
||||
|
||||
(deftest "parses inside an event handler"
|
||||
(hs-cleanup!)
|
||||
(let ((el (dom-create-element "div")))
|
||||
(dom-set-attr el "_" "on click breakpoint")
|
||||
(let ((result (guard (e (true false))
|
||||
(hs-activate! el)
|
||||
true)))
|
||||
(assert result))))
|
||||
```
|
||||
|
||||
## 4. Implementation checklist
|
||||
|
||||
1. `sx_find_all` in `lib/hyperscript/parser.sx` for the command dispatch `cond`.
|
||||
2. Add `breakpoint` branch → `hs-parse-breakpoint` function returning minimal command node.
|
||||
3. `sx_find_all` in `lib/hyperscript/compiler.sx` for command compilation dispatch.
|
||||
4. Add `breakpoint` branch → emit no-op.
|
||||
5. Replace 2 `SKIP` bodies in `spec/tests/test-hyperscript-behavioral.sx` with translated tests above.
|
||||
6. Run `hs_test_run suite="hs-upstream-breakpoint"` — expect 2/2.
|
||||
7. Run smoke 0–195 — no regressions.
|
||||
8. Commit: `HS: breakpoint command — parser + no-op compiler (+2)`
|
||||
|
||||
## 5. Risk
|
||||
|
||||
Very low. Zero-argument no-op command. The only risk is mis-locating the command dispatch branch in the parser.
|
||||
68
plans/designs/f1-null-safety.md
Normal file
68
plans/designs/f1-null-safety.md
Normal file
@@ -0,0 +1,68 @@
|
||||
# F1 — Null Safety Reporting (+7)
|
||||
|
||||
**Suite:** `hs-upstream-core/runtimeErrors`
|
||||
**Target:** 7 currently-failing tests (decrement, default, increment, put, remove, settle, transition commands)
|
||||
|
||||
## 1. Failing tests
|
||||
|
||||
The suite has 18 tests total; 11 already pass. The 7 failures all share the pattern:
|
||||
|
||||
```
|
||||
Expected '#doesntExist' is null, got
|
||||
```
|
||||
|
||||
The `eval-hs-error` helper already exists (landed in null-safety piece 1). It compiles and runs a HS snippet and returns the error string. The problem is that the listed commands don't guard against null targets before operating, so they produce no error (or a cryptic one) instead of `"'#doesntExist' is null"`.
|
||||
|
||||
| Test | Command | Null target expression |
|
||||
|------|---------|----------------------|
|
||||
| decrement | `decrement #doesntExist's innerHTML` | `#doesntExist` |
|
||||
| default | `default #doesntExist's innerHTML to 'foo'` | `#doesntExist` |
|
||||
| increment | `increment #doesntExist's innerHTML` | `#doesntExist` |
|
||||
| put | `put 'foo' into/before/after/at start of/at end of #doesntExist` | `#doesntExist` |
|
||||
| remove | `remove .foo/.@foo/#doesntExist from #doesntExist` | `#doesntExist` |
|
||||
| settle | `settle #doesntExist` | `#doesntExist` |
|
||||
| transition | `transition #doesntExist's *visibility to 0` | `#doesntExist` |
|
||||
|
||||
Note: add, hide, measure, send, sets, show, toggle, trigger already pass — they already guard.
|
||||
|
||||
## 2. Required error format
|
||||
|
||||
```
|
||||
'#doesntExist' is null
|
||||
```
|
||||
|
||||
The apostrophe-quoted selector string followed by ` is null`. The selector text is the original source text of the element expression (e.g. `#doesntExist`, not a stringified DOM node).
|
||||
|
||||
This is the same format already used by passing commands. The null-safety piece 1 commit added `eval-hs-error` and `hs-null-error` helper — just need to call it at the right point in each missing command.
|
||||
|
||||
## 3. Where to add guards
|
||||
|
||||
All in `lib/hyperscript/runtime.sx`. Pattern for each command:
|
||||
|
||||
```
|
||||
(when (nil? target)
|
||||
(hs-null-error target-source-text))
|
||||
```
|
||||
|
||||
Where `hs-null-error` (or equivalent) raises with the formatted message.
|
||||
|
||||
### Per-command location
|
||||
|
||||
- **decrement / increment** — after resolving the target element, before reading/writing innerHTML
|
||||
- **default** — after resolving target element, before reading current value
|
||||
- **put** — after resolving destination element (covers all put variants: into, before, after, at start, at end)
|
||||
- **remove** — after resolving the `from` target element
|
||||
- **settle** — after resolving target element, before starting transition poll
|
||||
- **transition** — after resolving target element, before reading/setting style
|
||||
|
||||
## 4. Implementation checklist
|
||||
|
||||
1. Find each failing command's runtime function in `lib/hyperscript/runtime.sx` using `sx_find_all`.
|
||||
2. For each: `sx_read_subtree` on the function body, locate where target is resolved, insert null guard calling `hs-null-error` (or the equivalent raise form already used by passing commands).
|
||||
3. After all 7: run `hs_test_run suite="hs-upstream-core/runtimeErrors"` — expect 18/18.
|
||||
4. Run smoke range 0–195 — expect no regressions.
|
||||
5. Commit: `HS: null-safety guards on decrement/default/increment/put/remove/settle/transition (+7)`
|
||||
|
||||
## 5. Risk
|
||||
|
||||
Low. The pattern is established by the 11 already-passing tests. The only risk is finding the correct point in each command where the element is resolved and before it's first used.
|
||||
166
plans/designs/f13-step-limit-and-meta.md
Normal file
166
plans/designs/f13-step-limit-and-meta.md
Normal file
@@ -0,0 +1,166 @@
|
||||
# F13 — Step Limit + `meta.caller` (+5 → 100%)
|
||||
|
||||
Five tests currently timeout or produce wrong values due to two root causes:
|
||||
step budget exhaustion and a missing `meta` implementation.
|
||||
|
||||
## Tests
|
||||
|
||||
| # | Suite | Test | Failure |
|
||||
|---|-------|------|---------|
|
||||
| 198 | `hs-upstream-core/runtime` | `has proper stack from event handler` | wrong-value: `meta.caller` returns `""` instead of an object with `.meta.feature.type = "onFeature"` |
|
||||
| 200 | `hs-upstream-core/runtime` | `hypertrace is reasonable` | TIMEOUT (15s, step limit) |
|
||||
| 615 | `hs-upstream-expressions/in` | `query template returns values` | TIMEOUT (37s, step limit) |
|
||||
| 1197 | `hs-upstream-repeat` | `repeat forever works` | TIMEOUT (step limit) |
|
||||
| 1198 | `hs-upstream-repeat` | `repeat forever works w/o keyword` | TIMEOUT (step limit) |
|
||||
|
||||
---
|
||||
|
||||
## Root cause A — Step limit (tests 200, 615, 1197, 1198)
|
||||
|
||||
The runner sets `HS_STEP_LIMIT=200000`. Every CEK step consumed by any
|
||||
expression in a test — including the double compilation warm-up guard blocks
|
||||
that appear before the actual DOM test — counts against this shared budget.
|
||||
|
||||
### `repeat forever` (1197, 1198)
|
||||
|
||||
The loop body terminates in exactly **5 iterations** (`if retVal == 5 then return`).
|
||||
This is bounded, not infinite. The step budget is exhausted before the loop
|
||||
runs because two `eval-expr-cek` compilation warm-up calls each consume tens
|
||||
of thousands of steps.
|
||||
|
||||
Fix: each warm-up guard compiles and discards a HS function definition. Those
|
||||
calls are defensive (wrapped in `guard` that swallows errors). We do NOT need
|
||||
to run the compiled code — the warm-up's purpose is just to ensure the
|
||||
compiler doesn't crash, not to consume steps. The step counter should not tick
|
||||
during compilation (compilation is a pure transform, not evaluation). If that's
|
||||
impractical to gate, raise `HS_STEP_LIMIT` to `2000000` (10×).
|
||||
|
||||
### `hypertrace is reasonable` (200)
|
||||
|
||||
Defines `bar()` → calls `baz()` → throws. Simple call chain. The "hypertrace"
|
||||
in the test name implies the HS runtime trace recorder is active during the
|
||||
test. If trace recording is on globally, every CEK step generates a trace entry
|
||||
allocation. Fix: confirm whether trace recording is always-on in the test runner
|
||||
and disable it by default (trace should only be on when explicitly requested).
|
||||
Alternatively raise step limit.
|
||||
|
||||
### `query template returns values` (615)
|
||||
|
||||
Uses `<${"p"}/>` — a CSS query selector built from a template string. Takes 37
|
||||
seconds. Likely the template selector evaluation triggers repeated DOM scanning
|
||||
or expensive string construction per step. Fix: profile with `hs_test_run
|
||||
verbose=true` to identify which step is slow. If it's a regex compilation
|
||||
per-call, cache it. If step limit only, raise to 2M.
|
||||
|
||||
### Unified fix: raise `HS_STEP_LIMIT` to `2000000`
|
||||
|
||||
The simplest fix that unblocks all four timeout tests. In
|
||||
`tests/hs-run-filtered.js`, change the default step limit. Per-test overrides
|
||||
can still be set via `HS_STEP_LIMIT` env var for debugging.
|
||||
|
||||
If the `query template` test is still slow at 2M steps (37s × 10 = 370s, which
|
||||
would be unacceptable), that test needs a separate performance fix — cache the
|
||||
compiled regex/query from the template string rather than rebuilding it on every
|
||||
access.
|
||||
|
||||
---
|
||||
|
||||
## Root cause B — `meta.caller` not implemented (test 198)
|
||||
|
||||
The HS `meta` object is available inside any function call. It exposes:
|
||||
|
||||
- `meta.caller` — the calling context object
|
||||
- `meta.caller.meta.feature.type` — the HS feature type of the caller
|
||||
(e.g. `"onFeature"` when called from an `on click` handler)
|
||||
|
||||
Test script:
|
||||
```
|
||||
def bar()
|
||||
log meta.caller
|
||||
return meta.caller
|
||||
end
|
||||
```
|
||||
Triggered via `on click put bar().meta.feature.type into my.innerHTML`.
|
||||
Expects `"onFeature"` in innerHTML. Currently gets `""`.
|
||||
|
||||
### What `meta` needs
|
||||
|
||||
`meta` is a dict-like object injected into every function's execution context
|
||||
at call time. Minimum fields for this test:
|
||||
|
||||
```
|
||||
meta = {
|
||||
:caller <the calling context — a dict with its own :meta field>
|
||||
:element <the element the script is attached to>
|
||||
}
|
||||
```
|
||||
|
||||
`meta.caller.meta.feature.type` must return `"onFeature"` when called from an
|
||||
`on` event handler. The feature type string `"onFeature"` is already used
|
||||
internally (event handler features are tagged with this type).
|
||||
|
||||
### Implementation
|
||||
|
||||
In `lib/hyperscript/runtime.sx`, at the point where a HS `def` function is
|
||||
called:
|
||||
|
||||
1. Build a `meta` dict:
|
||||
```
|
||||
{:caller calling-context :element current-element}
|
||||
```
|
||||
where `calling-context` is the current runtime context dict (which includes
|
||||
its own `:meta` field with `:feature {:type "onFeature"}` for event handlers).
|
||||
|
||||
2. Bind `meta` in the function's execution env.
|
||||
|
||||
3. Ensure event handler contexts carry `{:meta {:feature {:type "onFeature"}}}`.
|
||||
|
||||
This is an additive change — nothing currently uses `meta`, so no regression
|
||||
risk.
|
||||
|
||||
---
|
||||
|
||||
## Implementation checklist
|
||||
|
||||
### Step A — Raise step limit
|
||||
1. In `tests/hs-run-filtered.js`, change default `HS_STEP_LIMIT` from `200000`
|
||||
to `2000000`.
|
||||
2. Run tests 1197–1198: `hs_test_run(start=1197, end=1199)` — expect 2/2.
|
||||
3. Run test 615: `hs_test_run(start=615, end=616)` — expect 1/1 or note if
|
||||
still too slow.
|
||||
4. Run test 200: `hs_test_run(start=200, end=201)` — expect 1/1.
|
||||
|
||||
### Step B — `meta.caller` (test 198)
|
||||
5. `sx_find_all` in `lib/hyperscript/runtime.sx` for where `def` functions are
|
||||
called / where event handler contexts are constructed.
|
||||
6. Add `meta` dict construction at call time; bind in function env.
|
||||
7. Ensure `on` handler context carries `{:meta {:feature {:type "onFeature"}}}`.
|
||||
8. Run test 198: `hs_test_run(start=198, end=199)` — expect 1/1.
|
||||
|
||||
### Step C — Query template performance (if still slow after step A)
|
||||
9. Profile `hs_test_run(start=615, end=616, step_limit=2000000, verbose=true)`.
|
||||
10. If the CSS template query `<${"p"}/>` rebuilds on every call, add a memoize
|
||||
cache keyed on the template result string.
|
||||
11. Rerun — expect < 5s.
|
||||
|
||||
### Step D — Full suite verification
|
||||
12. Run all ranges with raised step limit:
|
||||
- `hs_test_run(start=0, end=201, step_limit=2000000)`
|
||||
- `hs_test_run(start=201, end=616, step_limit=2000000)`
|
||||
- `hs_test_run(start=616, end=1200, step_limit=2000000)`
|
||||
- `hs_test_run(start=1200, end=1496, step_limit=2000000)`
|
||||
13. Confirm all previously-passing tests still pass.
|
||||
14. Commit: `HS: raise step limit to 2M + meta.caller for onFeature stack (+5)`
|
||||
|
||||
---
|
||||
|
||||
## Risk
|
||||
|
||||
- **Step limit raise:** May make test suite slower overall (more steps to exhaust
|
||||
before timeout). But if tests pass quickly the limit is never reached.
|
||||
The 37s query-template test is the only real concern — if it genuinely needs
|
||||
2M steps × (time per step), it needs a performance fix too.
|
||||
- **`meta.caller`:** Additive binding in function scope. Zero regression risk.
|
||||
The only complexity is constructing the right shape for the calling context
|
||||
chain — but since only one test exercises this and the shape is simple, the
|
||||
risk is low.
|
||||
81
plans/designs/f2-tell.md
Normal file
81
plans/designs/f2-tell.md
Normal file
@@ -0,0 +1,81 @@
|
||||
# F2 — `tell` Semantics Fix (+3)
|
||||
|
||||
**Suite:** `hs-upstream-tell`
|
||||
**Target:** 3 failing tests out of 10. 7 already pass.
|
||||
|
||||
## 1. Failing tests
|
||||
|
||||
### "attributes refer to the thing being told"
|
||||
```
|
||||
on click tell #d2 then put @foo into me
|
||||
```
|
||||
d2 has attribute `foo="bar"`. After click, d1's text content should be `"bar"`.
|
||||
`@foo` is an attribute ref — it should resolve against the **told element** (d2), not the event target (d1).
|
||||
Currently gets `""` — attribute resolves against d1, which has no `foo` attribute.
|
||||
|
||||
### "your symbol represents the thing being told"
|
||||
```
|
||||
on click tell #d2 then put your innerText into me
|
||||
```
|
||||
d2 has innerText `"foo"`. After click, d1's text content should be `"foo"`.
|
||||
`your` is the possessive of `you` — inside a `tell` block, `you`/`your` should bind to the told element.
|
||||
Currently gets `""`.
|
||||
|
||||
### "does not overwrite the me symbol"
|
||||
```
|
||||
on click add .foo then tell #d2 then add .bar to me
|
||||
```
|
||||
After click: d1 should have both `.foo` and `.bar`; d2 should have neither.
|
||||
`me` inside the `tell` block must still refer to d1 (the original event target).
|
||||
Currently: assertion fails — `.bar` is going to d2 instead of d1.
|
||||
|
||||
## 2. What the 7 passing tests reveal about current behaviour
|
||||
|
||||
The passing tests include:
|
||||
- `you symbol represents the thing being told` — `add .bar to you` adds to d2 ✓
|
||||
- `establishes a proper beingTold symbol` — bare `add .bar` (no target) adds to the told element ✓
|
||||
- `restores a proper implicit me symbol` — after `tell` block ends, bare commands target d1 again ✓
|
||||
- `yourself attribute also works` — `remove yourself` inside tell removes d2 ✓
|
||||
|
||||
So `you`, `yourself`, and bare implicit target all work. The three bugs are:
|
||||
1. Attribute refs (`@foo`) don't resolve against the told element
|
||||
2. `your` (possessive of `you`) doesn't resolve
|
||||
3. `me` is being rebound to the told element instead of kept as d1
|
||||
|
||||
## 3. Root cause analysis
|
||||
|
||||
Inside a `tell X` block, the runtime sets the implicit target to X. The three failures suggest:
|
||||
|
||||
**Bug A — attribute refs:** `@foo` resolves via a property-access path that reads from the *current event target* (`me`/`self`), not from the *implicit tell target*. The tell block sets implicit target but the attribute ref lookup skips it.
|
||||
|
||||
**Bug B — `your`:** `your` is parsed as a possessive modifier expecting `you` to be bound. If `you` is not bound in the tell scope (and only the implicit target is set), `your X` fails to resolve.
|
||||
|
||||
**Bug C — `me` rebinding:** The tell command saves/restores `me` but the save/restore is either not happening or is restoring the wrong value. `me` inside the block should remain d1 while the implicit default target is d2.
|
||||
|
||||
## 4. Fix
|
||||
|
||||
In `lib/hyperscript/runtime.sx`, find the `tell` command handler (search for `hs-tell` or the tell dispatch branch).
|
||||
|
||||
The correct semantics:
|
||||
- Save current `me` value
|
||||
- Set implicit target (used by bare commands like `add .bar`) to the told element
|
||||
- Bind `you` = told element (so `you`, `your`, `yourself` work)
|
||||
- Do **not** rebind `me` — keep it as the original event target
|
||||
- Restore implicit target and unbind `you` after the block
|
||||
|
||||
For attribute refs (`@foo`): resolve against the current *implicit target* (told element), not against `me`. Find where `@attr` expressions are evaluated and ensure they read from the implicit target when inside a tell block.
|
||||
|
||||
## 5. Implementation checklist
|
||||
|
||||
1. `sx_find_all` in `lib/hyperscript/runtime.sx` for tell handler.
|
||||
2. `sx_read_subtree` on the tell handler — verify save/restore of `me` vs implicit target.
|
||||
3. Fix `me` rebinding: save old implicit target, set new one, do NOT touch `me`.
|
||||
4. Bind `you`/`your`/`yourself` to told element in the tell scope env.
|
||||
5. Find attribute ref (`@`) evaluation — ensure it reads from implicit target.
|
||||
6. Run `hs_test_run suite="hs-upstream-tell"` — expect 10/10.
|
||||
7. Run smoke 0–195 — no regressions.
|
||||
8. Commit: `HS: tell — fix me rebinding, your/attribute-ref resolution (+3)`
|
||||
|
||||
## 6. Risk
|
||||
|
||||
Medium. The 7 passing tests constrain what can change — the fix must preserve `you`, `yourself`, bare implicit target, and restore-after-tell semantics. The three bugs are independent enough that they can be fixed one at a time and verified after each.
|
||||
128
plans/designs/f5-cookies.md
Normal file
128
plans/designs/f5-cookies.md
Normal file
@@ -0,0 +1,128 @@
|
||||
# F5 — Cookie API (+5)
|
||||
|
||||
**Suite:** `hs-upstream-expressions/cookies`
|
||||
**Target:** All 5 tests are `SKIP (untranslated)`.
|
||||
|
||||
## 1. The 5 tests
|
||||
|
||||
From upstream `test/expressions/cookies.js`:
|
||||
|
||||
| Test | What it checks |
|
||||
|------|---------------|
|
||||
| `length is 0 when no cookies are set` | `cookies.length == 0` with no cookies set |
|
||||
| `basic set cookie values work` | `set cookies.name to "value"` then `cookies.name == "value"` |
|
||||
| `update cookie values work` | set, then set again, value updates |
|
||||
| `basic clear cookie values work` | `set cookies.name to "value"` then `clear cookies.name`, then `cookies.name == undefined` |
|
||||
| `iterate cookies values work` | `for name in cookies` iterates cookie names |
|
||||
|
||||
## 2. HyperScript cookie syntax
|
||||
|
||||
`cookies` is a special global expression in HyperScript backed by `document.cookie`. The upstream implementation wraps `document.cookie` in a proxy:
|
||||
|
||||
- `cookies.name` → read cookie by name (returns string or `undefined`)
|
||||
- `set cookies.name to val` → write cookie (sets `document.cookie = "name=val"`)
|
||||
- `clear cookies.name` → delete cookie (sets max-age=-1)
|
||||
- `cookies.length` → number of cookies set
|
||||
- `for name in cookies` → iterate over cookie names
|
||||
|
||||
## 3. Test runner mock
|
||||
|
||||
All 5 tests are untranslated — no SX test bodies exist yet. The generator needs patterns for the cookie expressions, and `hs-run-filtered.js` needs a `document.cookie` mock.
|
||||
|
||||
### Mock in `tests/hs-run-filtered.js`
|
||||
|
||||
Add a simple in-memory cookie store to the `dom` mock:
|
||||
|
||||
```js
|
||||
let _cookieStore = {};
|
||||
Object.defineProperty(global.document, 'cookie', {
|
||||
get() {
|
||||
return Object.entries(_cookieStore)
|
||||
.map(([k,v]) => `${k}=${v}`)
|
||||
.join('; ');
|
||||
},
|
||||
set(str) {
|
||||
const [pair, ...attrs] = str.split(';');
|
||||
const [name, val] = pair.split('=').map(s => s.trim());
|
||||
const maxAge = attrs.find(a => a.trim().startsWith('max-age='));
|
||||
if (maxAge && parseInt(maxAge.split('=')[1]) < 0) {
|
||||
delete _cookieStore[name];
|
||||
} else {
|
||||
_cookieStore[name] = val;
|
||||
}
|
||||
},
|
||||
configurable: true
|
||||
});
|
||||
```
|
||||
|
||||
Add `_cookieStore = {}` reset to `hs-cleanup!` equivalent in the runner.
|
||||
|
||||
## 4. SX runtime additions in `lib/hyperscript/runtime.sx`
|
||||
|
||||
HS needs a `cookies` special expression that the compiler resolves. Two approaches:
|
||||
|
||||
**Option A (simpler):** Treat `cookies` as a built-in variable bound to a proxy dict at runtime. When property access `cookies.name` is evaluated, dispatch to cookie read/write helpers.
|
||||
|
||||
**Option B (upstream-faithful):** Parse `cookies` as a special primary expression, emit runtime calls `hs-cookie-get`, `hs-cookie-set`, `hs-cookie-delete`, `hs-cookie-length`, `hs-cookie-names`.
|
||||
|
||||
Option A is less invasive. The runtime env gets a `cookies` binding pointing to a special object; property access and assignment on it dispatch to the cookie helpers, which call `(platform-cookie-get name)` / `(platform-cookie-set name val)` / `(platform-cookie-delete name)`.
|
||||
|
||||
Platform cookie operations map to `document.cookie` reads/writes in JS.
|
||||
|
||||
## 5. Generator patterns (`tests/playwright/generate-sx-tests.py`)
|
||||
|
||||
The upstream tests use patterns like:
|
||||
|
||||
```js
|
||||
await page.evaluate(() => { _hyperscript.evaluate("set cookies.foo to 'bar'") });
|
||||
expect(await page.evaluate(() => _hyperscript.evaluate("cookies.foo"))).toBe("bar");
|
||||
```
|
||||
|
||||
In our SX harness these become direct `eval-hs` calls. Since all 5 tests are untranslated, hand-write them rather than extending the generator (similar to E39).
|
||||
|
||||
## 6. Translated test bodies
|
||||
|
||||
```lisp
|
||||
(deftest "length is 0 when no cookies are set"
|
||||
(hs-cleanup!)
|
||||
(assert= (eval-hs "cookies.length") 0))
|
||||
|
||||
(deftest "basic set cookie values work"
|
||||
(hs-cleanup!)
|
||||
(eval-hs "set cookies.foo to 'bar'")
|
||||
(assert= (eval-hs "cookies.foo") "bar"))
|
||||
|
||||
(deftest "update cookie values work"
|
||||
(hs-cleanup!)
|
||||
(eval-hs "set cookies.foo to 'bar'")
|
||||
(eval-hs "set cookies.foo to 'baz'")
|
||||
(assert= (eval-hs "cookies.foo") "baz"))
|
||||
|
||||
(deftest "basic clear cookie values work"
|
||||
(hs-cleanup!)
|
||||
(eval-hs "set cookies.foo to 'bar'")
|
||||
(eval-hs "clear cookies.foo")
|
||||
(assert= (eval-hs "cookies.foo") nil))
|
||||
|
||||
(deftest "iterate cookies values work"
|
||||
(hs-cleanup!)
|
||||
(eval-hs "set cookies.a to '1'")
|
||||
(eval-hs "set cookies.b to '2'")
|
||||
(let ((names (eval-hs "for name in cookies collect name")))
|
||||
(assert (contains? names "a"))
|
||||
(assert (contains? names "b"))))
|
||||
```
|
||||
|
||||
## 7. Implementation checklist
|
||||
|
||||
1. Add cookie mock to `tests/hs-run-filtered.js`. Wire reset into test cleanup.
|
||||
2. Add `hs-cookie-get`, `hs-cookie-set`, `hs-cookie-delete`, `hs-cookie-length`, `hs-cookie-names` to `lib/hyperscript/runtime.sx`.
|
||||
3. Add `cookies` as a special expression in the HS parser/evaluator that dispatches to the above.
|
||||
4. Replace 5 `SKIP` bodies in `spec/tests/test-hyperscript-behavioral.sx` with translated test bodies above.
|
||||
5. Run `hs_test_run suite="hs-upstream-expressions/cookies"` — expect 5/5.
|
||||
6. Run smoke 0–195 — no regressions.
|
||||
7. Commit: `HS: cookie API — document.cookie proxy + 5 tests`
|
||||
|
||||
## 8. Risk
|
||||
|
||||
Medium. The mock is simple. The main risk is the `cookies` expression integration in the parser — it needs to hook into property-access and assignment paths that are already well-exercised. Keep the implementation thin: `cookies` is a runtime value with a special type, not a new parse form.
|
||||
107
plans/designs/f8-eval-statically.md
Normal file
107
plans/designs/f8-eval-statically.md
Normal file
@@ -0,0 +1,107 @@
|
||||
# F8 — evalStatically (+3)
|
||||
|
||||
**Suite:** `hs-upstream-core/evalStatically`
|
||||
**Target:** 3 failing (untranslated) out of 8. 5 already pass.
|
||||
|
||||
## 1. Current state
|
||||
|
||||
5 passing tests use `(eval-hs expr)` and check the return value for literals: booleans, null, numbers, plain strings, time expressions. These call `_hyperscript.evaluate(src)` and return the result.
|
||||
|
||||
3 failing tests are named:
|
||||
- `throws on math expressions`
|
||||
- `throws on symbol references`
|
||||
- `throws on template strings`
|
||||
|
||||
All are `SKIP (untranslated)` — no test body has been generated.
|
||||
|
||||
## 2. What upstream checks
|
||||
|
||||
From `test/core/evalStatically.js`, the `throwErrors` mode:
|
||||
|
||||
```js
|
||||
expect(() => _hyperscript.evaluate("1 + 2")).toThrow();
|
||||
expect(() => _hyperscript.evaluate("x")).toThrow();
|
||||
expect(() => _hyperscript.evaluate(`"hello ${name}"`)).toThrow();
|
||||
```
|
||||
|
||||
`_hyperscript.evaluate(src)` in strict static mode throws when the expression is not a pure literal — math operators, symbol references, and template string interpolation all involve runtime evaluation that can't be statically resolved.
|
||||
|
||||
The "static" constraint: only literals that can be evaluated without any runtime context or side effects are allowed. `1 + 2` is not static (it's a math op). `x` is not static (symbol lookup). `"hello ${name}"` is not static (interpolation).
|
||||
|
||||
## 3. What `eval-hs` currently does
|
||||
|
||||
`eval-hs` in our harness calls `(hs-compile-and-run src)` or equivalent. It does NOT currently have a "static mode" — it runs everything with the full runtime.
|
||||
|
||||
We need a new harness helper `eval-hs-static-error` that:
|
||||
1. Calls `(hs-compile src)` with a flag that makes it throw on non-literal expressions
|
||||
2. Returns the caught error message, or raises if no error was thrown
|
||||
|
||||
## 4. Implementation options
|
||||
|
||||
### Option A — Static analysis pass (accurate)
|
||||
|
||||
Before evaluation, walk the AST and reject any node that isn't a literal:
|
||||
- Number literal ✓
|
||||
- String literal (no interpolation) ✓
|
||||
- Boolean literal ✓
|
||||
- Null literal ✓
|
||||
- Time expression (`200ms`, `2s`) ✓
|
||||
- Everything else → throw `"expression is not static"`
|
||||
|
||||
This is a pre-eval AST check, not a runtime change. Lives in `lib/hyperscript/compiler.sx` as `hs-check-static`.
|
||||
|
||||
### Option B — Generator translation (simpler)
|
||||
|
||||
The 3 tests are untranslated. All three just verify that `_hyperscript.evaluate(expr)` throws. In our SX harness we can test this with a `guard` form:
|
||||
|
||||
```lisp
|
||||
(deftest "throws on math expressions"
|
||||
(let ((result (guard (e (true true))
|
||||
(eval-hs "1 + 2")
|
||||
false)))
|
||||
(assert result)))
|
||||
```
|
||||
|
||||
But this only works if `eval-hs` actually throws on math expressions. Currently it doesn't — `eval-hs "1 + 2"` returns `3`. So we'd need the static analysis anyway to make the test pass.
|
||||
|
||||
### Chosen approach: Option A
|
||||
|
||||
Add `hs-static-check` to the compiler: a fast AST walker that throws on any non-literal node. Wire it as an optional mode. The test harness calls `eval-hs-static` which runs with static-check enabled.
|
||||
|
||||
Actually, reading the upstream more carefully: `_hyperscript.evaluate` already throws in static mode without additional flags — the "evaluate" API is documented as static-only. Our `eval-hs` in the passing tests works because booleans/numbers/strings/time ARE static. `1 + 2`, `x`, and template strings are NOT static and should throw.
|
||||
|
||||
So the fix is: make `hs-compile-and-run` (or whatever backs `eval-hs`) reject non-literal AST nodes. The 5 passing tests will continue to pass (they use literals). The 3 failing tests will get translated using `eval-hs-error` or a guard pattern.
|
||||
|
||||
## 5. Non-literal AST node types to reject
|
||||
|
||||
| Expression | AST node type | Reject? |
|
||||
|-----------|--------------|---------|
|
||||
| `1`, `3.14` | number literal | ✓ allow |
|
||||
| `"hello"`, `'world'` | string literal (no interpolation) | ✓ allow |
|
||||
| `true`, `false` | boolean literal | ✓ allow |
|
||||
| `null` | null literal | ✓ allow |
|
||||
| `200ms`, `2s` | time literal | ✓ allow |
|
||||
| `1 + 2` | math operator | ✗ throw |
|
||||
| `x` | symbol reference | ✗ throw |
|
||||
| `"hello ${name}"` | template string | ✗ throw |
|
||||
|
||||
## 6. Implementation checklist
|
||||
|
||||
1. In `lib/hyperscript/compiler.sx`, add `hs-static?` predicate: returns true only for literal AST node types.
|
||||
2. In the `eval-hs` path (wherever `hs-compile-and-run` is called for the evaluate API), call `hs-static?` on the parsed AST and throw `"expression is not statically evaluable"` if false.
|
||||
3. Replace 3 `SKIP` bodies in `spec/tests/test-hyperscript-behavioral.sx`:
|
||||
```lisp
|
||||
(deftest "throws on math expressions"
|
||||
(assert (string? (eval-hs-error "1 + 2"))))
|
||||
(deftest "throws on symbol references"
|
||||
(assert (string? (eval-hs-error "x"))))
|
||||
(deftest "throws on template strings"
|
||||
(assert (string? (eval-hs-error "\"hello ${name}\""))))
|
||||
```
|
||||
4. Run `hs_test_run suite="hs-upstream-core/evalStatically"` — expect 8/8.
|
||||
5. Run smoke 0–195 — verify the 5 passing tests still pass.
|
||||
6. Commit: `HS: evalStatically — static literal check, 3 tests (+3)`
|
||||
|
||||
## 7. Risk
|
||||
|
||||
Low-medium. The main risk is that `eval-hs` is used in many tests for non-static expressions and adding a static check to the shared path would break them. The fix must be gated — either a separate `eval-hs-static` helper or a flag parameter. The passing tests must not be affected.
|
||||
341
plans/designs/hs-plugin-system.md
Normal file
341
plans/designs/hs-plugin-system.md
Normal file
@@ -0,0 +1,341 @@
|
||||
# HyperScript Plugin / Extension System
|
||||
|
||||
Post-Bucket-F capability work. No conformance delta on its own — the payoff is
|
||||
clean architecture for language embeds (Lua, Prolog, Worker runtime) and
|
||||
alignment with real `_hyperscript`'s extension model.
|
||||
|
||||
---
|
||||
|
||||
## 1. Motivation
|
||||
|
||||
### 1a. Real `_hyperscript` has a plugin API
|
||||
|
||||
Stock `_hyperscript` ships a core bundle with feature stubs and a `use(ext)`
|
||||
hook that loads named extensions at runtime. The worker feature is the canonical
|
||||
example: the core parser has a stub that errors helpfully; loading the worker
|
||||
extension replaces the stub with a real implementation.
|
||||
|
||||
We currently have no equivalent. New grammar or compiler targets require editing
|
||||
`parse-feat`'s hardcoded `cond` or `hs-to-sx`'s hardcoded dispatch. This is
|
||||
fine for conformance work but wrong for language embeds.
|
||||
|
||||
### 1b. Ad-hoc hooks are accumulating
|
||||
|
||||
`runtime.sx` already has `hs-prolog-hook` / `hs-set-prolog-hook!` / `prolog`
|
||||
(nodes 140–142) — an informal plugin slot bolted on outside the parser and
|
||||
compiler. This pattern will repeat for Lua, and again for the Worker runtime.
|
||||
A proper registry prevents the drift.
|
||||
|
||||
### 1c. E39 worker stub is a placeholder
|
||||
|
||||
The stub added in E39 (`parse-feat` raises immediately on `"worker"`) was
|
||||
explicitly designed to be replaced by a real plugin at a single site. This plan
|
||||
is where that replacement happens.
|
||||
|
||||
### 1d. Bucket-F Group 10 needs a converter registry
|
||||
|
||||
`as MyType` via registered converter is already in the Bucket-F plan (Group 10).
|
||||
A `hs-register-converter!` registry is the natural home for it — and the plugin
|
||||
system is the right time to add registries generally.
|
||||
|
||||
---
|
||||
|
||||
## 2. Scope
|
||||
|
||||
**In scope:**
|
||||
- Parser feature registry (`parse-feat` dispatch)
|
||||
- Compiler command registry (`hs-to-sx` dispatch)
|
||||
- `as` converter registry (`hs-coerce` dispatch)
|
||||
- Migration of E39 worker stub to use the parser registry
|
||||
- Migration of `hs-prolog-hook` ad-hoc slot to a proper plugin
|
||||
- Worker full runtime plugin (first real plugin)
|
||||
- Lua embed plugin
|
||||
- Prolog embed plugin
|
||||
|
||||
**Out of scope:**
|
||||
- Changing the test runner or generator
|
||||
- Any conformance delta (this plan doesn't target failing tests)
|
||||
- Third-party plugin loading from external URLs (future)
|
||||
- Hot-reload of plugins (future)
|
||||
|
||||
---
|
||||
|
||||
## 3. Registry design
|
||||
|
||||
Three registries, all SX dicts. Checked before the hardcoded `cond` in each
|
||||
dispatch. Registration functions defined alongside the registries in their
|
||||
respective files.
|
||||
|
||||
### 3a. Parser feature registry (`lib/hyperscript/parser.sx`)
|
||||
|
||||
```lisp
|
||||
(define _hs-feature-registry (dict))
|
||||
|
||||
(define hs-register-feature!
|
||||
(fn (keyword parse-fn)
|
||||
(set! _hs-feature-registry
|
||||
(dict-set _hs-feature-registry keyword parse-fn))))
|
||||
```
|
||||
|
||||
In `parse-feat`, prepend a registry lookup before the existing `cond`:
|
||||
|
||||
```lisp
|
||||
(let ((registered (dict-get _hs-feature-registry val)))
|
||||
(if registered
|
||||
(registered) ;; call the registered parse-fn (no args; uses closure over adv!/tp-val etc.)
|
||||
(cond ;; existing dispatch unchanged below
|
||||
...)))
|
||||
```
|
||||
|
||||
`parse-fn` is a zero-arg thunk that has access to the parser's internal state
|
||||
via the same closure that the existing `parse-*` helpers use. Since `parse-feat`
|
||||
is itself defined inside the big `let` in `hs-parse`, all the parser helpers
|
||||
(`adv!`, `tp-val`, `tp-typ`, `parse-cmd-list`, etc.) are in scope.
|
||||
|
||||
### 3b. Compiler command registry (`lib/hyperscript/compiler.sx`)
|
||||
|
||||
```lisp
|
||||
(define _hs-compiler-registry (dict))
|
||||
|
||||
(define hs-register-compiler!
|
||||
(fn (head compile-fn)
|
||||
(set! _hs-compiler-registry
|
||||
(dict-set _hs-compiler-registry (str head) compile-fn))))
|
||||
```
|
||||
|
||||
In `hs-to-sx`, before the existing `cond` on `head`, check the registry:
|
||||
|
||||
```lisp
|
||||
(let ((registered (dict-get _hs-compiler-registry (str head))))
|
||||
(if registered
|
||||
(registered ast)
|
||||
(cond ...)))
|
||||
```
|
||||
|
||||
`compile-fn` receives the full AST node and returns an SX expression.
|
||||
|
||||
### 3c. `as` converter registry (`lib/hyperscript/runtime.sx`)
|
||||
|
||||
```lisp
|
||||
(define _hs-converters (dict))
|
||||
|
||||
(define hs-register-converter!
|
||||
(fn (type-name converter-fn)
|
||||
(set! _hs-converters
|
||||
(dict-set _hs-converters type-name converter-fn))))
|
||||
```
|
||||
|
||||
In `hs-coerce`, add a registry lookup as the last `cond` clause before the
|
||||
fallthrough error:
|
||||
|
||||
```lisp
|
||||
((dict-get _hs-converters type-name)
|
||||
((dict-get _hs-converters type-name) value))
|
||||
```
|
||||
|
||||
This is also the hook that Bucket-F Group 10 (`can accept custom conversions`)
|
||||
hangs on — so implementing it here kills two birds.
|
||||
|
||||
---
|
||||
|
||||
## 4. First-party plugins
|
||||
|
||||
Each plugin is a `.sx` file in `lib/hyperscript/plugins/`. Plugins call the
|
||||
registration functions at load time (top-level `do` forms). The host loads
|
||||
plugins explicitly after the core files.
|
||||
|
||||
### 4a. Worker plugin (`lib/hyperscript/plugins/worker.sx`)
|
||||
|
||||
**Phase 1 — stub migration (immediate):**
|
||||
Remove the inline error branch from `parse-feat` (the E39 stub). Replace with:
|
||||
|
||||
```lisp
|
||||
(hs-register-feature! "worker"
|
||||
(fn ()
|
||||
(error "worker plugin is not installed — see https://hyperscript.org/features/worker")))
|
||||
```
|
||||
|
||||
This is identical behaviour to E39 but routed through the registry. The stub
|
||||
lives in the plugin file, not the core parser. No test regression.
|
||||
|
||||
**Phase 2 — full runtime:**
|
||||
|
||||
Parser: `parse-worker-feat` — consumes `worker <Name> [(<url>*)] <def|js>* end`,
|
||||
returns `(worker Name urls defs)` AST node.
|
||||
|
||||
Compiler: registered under `"worker"` head:
|
||||
- Emits `(hs-worker-define! "Name" urls defs)` call.
|
||||
|
||||
Runtime additions in the plugin file:
|
||||
- `hs-worker-define!` — creates a `{:_hs-worker true :name N :handle H :exports (...)}` record,
|
||||
binds it in the HS top-level env under `Name`.
|
||||
- `hs-method-call` (existing) detects `:_hs-worker` and dispatches via `postMessage`.
|
||||
- Worker script body compiled to a standalone SX bundle posted to a Blob URL.
|
||||
- Return values are promise-wrapped; async-transparent via `perform`/IO suspension.
|
||||
|
||||
Mock env additions for the test runner: `Worker` constructor + synchronous
|
||||
message loop for the 7 sibling `test.skip(...)` upstream tests (the ones
|
||||
deferred in E39).
|
||||
|
||||
### 4b. Prolog plugin (`lib/hyperscript/plugins/prolog.sx`)
|
||||
|
||||
Replaces the ad-hoc `hs-prolog-hook` in `runtime.sx`.
|
||||
|
||||
**Parser:** Register `"prolog"` feature — parses
|
||||
`prolog(<db-expr>, <goal-expr>)` at feature level (alternative: keep as an
|
||||
expression, register a compiler extension only).
|
||||
|
||||
**Compiler:** Registered under `"prolog"` head — emits `(prolog db goal)`.
|
||||
|
||||
**Runtime:** The existing `prolog` function in `runtime.sx` moves here.
|
||||
`hs-prolog-hook` and `hs-set-prolog-hook!` are removed from `runtime.sx` and
|
||||
the hook mechanism is replaced by the plugin loading `lib/prolog/runtime.sx`
|
||||
and wiring the solver directly.
|
||||
|
||||
Remove from `runtime.sx` nodes 140–142 once the plugin is live.
|
||||
|
||||
### 4c. Lua plugin (`lib/hyperscript/plugins/lua.sx`)
|
||||
|
||||
**Parser:** Register `"lua"` feature — parses `lua ... end` block, captures
|
||||
the body as a raw string.
|
||||
|
||||
**Compiler:** Registered under `"lua"` head — emits `(lua-eval <body-string>)`.
|
||||
|
||||
**Runtime:** `lua-eval` calls `lib/lua/runtime.sx`'s eval entry point, returns
|
||||
result as an SX value via `hs-host-to-sx`. Errors surface as HS `catch`-able
|
||||
exceptions.
|
||||
|
||||
This enables inline Lua in HyperScript:
|
||||
|
||||
```
|
||||
on click
|
||||
lua
|
||||
return document.title:upper()
|
||||
end
|
||||
put it into me
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Load order
|
||||
|
||||
```
|
||||
lib/hyperscript/parser.sx ;; defines _hs-feature-registry, hs-register-feature!
|
||||
lib/hyperscript/compiler.sx ;; defines _hs-compiler-registry, hs-register-compiler!
|
||||
lib/hyperscript/runtime.sx ;; defines _hs-converters, hs-register-converter!
|
||||
lib/hyperscript/plugins/worker.sx
|
||||
lib/hyperscript/plugins/prolog.sx
|
||||
lib/hyperscript/plugins/lua.sx
|
||||
```
|
||||
|
||||
The test runner (`tests/hs-run-filtered.js`) loads plugins after core. The
|
||||
browser WASM bundle includes all three by default (plugins are small; no
|
||||
reason to lazy-load them).
|
||||
|
||||
---
|
||||
|
||||
## 6. Migration checklist
|
||||
|
||||
The work below is ordered to keep main green at every commit. Each step is
|
||||
independently committable.
|
||||
|
||||
### Step 1 — Registries (infrastructure, no behaviour change)
|
||||
|
||||
1. Add `_hs-feature-registry` + `hs-register-feature!` to `parser.sx`.
|
||||
Thread the registry check into `parse-feat`. No entries yet → behaviour
|
||||
unchanged.
|
||||
2. Add `_hs-compiler-registry` + `hs-register-compiler!` to `compiler.sx`.
|
||||
Thread into `hs-to-sx`. No entries yet → behaviour unchanged.
|
||||
3. Add `_hs-converters` + `hs-register-converter!` to `runtime.sx`. Thread
|
||||
into `hs-coerce`. No entries yet → behaviour unchanged.
|
||||
4. `sx_validate` all three files. Run full HS suite — expect zero regressions.
|
||||
5. Commit: `HS: plugin registry infrastructure (parser + compiler + converter)`.
|
||||
|
||||
### Step 2 — Worker stub migration
|
||||
|
||||
6. Create `lib/hyperscript/plugins/worker.sx`. Register the worker stub error.
|
||||
7. Remove the inline `((= val "worker") ...)` branch from `parse-feat` in
|
||||
`parser.sx`.
|
||||
8. Update the test runner to load `worker.sx` after core.
|
||||
9. Run `HS_SUITE=hs-upstream-worker` — expect 1/1. Run full suite — expect no
|
||||
regressions.
|
||||
10. Commit: `HS: migrate E39 worker stub to plugin registry`.
|
||||
|
||||
### Step 3 — Prolog plugin
|
||||
|
||||
11. Create `lib/hyperscript/plugins/prolog.sx`. Wire to `lib/prolog/runtime.sx`.
|
||||
12. Remove `hs-prolog-hook`, `hs-set-prolog-hook!`, `prolog` from `runtime.sx`
|
||||
nodes 140–142.
|
||||
13. Update test runner to load `prolog.sx`.
|
||||
14. Validate and run full suite.
|
||||
15. Commit: `HS: prolog plugin replaces ad-hoc hook`.
|
||||
|
||||
### Step 4 — `as` converter registry (bridges Bucket-F Group 10)
|
||||
|
||||
16. Confirm `hs-register-converter!` satisfies the Group 10 test
|
||||
`can accept custom conversions`. If yes, this step may be pulled into
|
||||
Bucket-F Group 10 instead (no duplication — just move step 3 of §6 there).
|
||||
17. Commit: `HS: as-converter registry wired into hs-coerce`.
|
||||
|
||||
### Step 5 — Lua plugin
|
||||
|
||||
18. Create `lib/hyperscript/plugins/lua.sx`.
|
||||
19. Add `lua-eval` to `runtime.sx` or directly in the plugin file.
|
||||
20. Parser: `parse-lua-feat` consuming `lua … end`.
|
||||
21. Compiler: registered `"lua"` head.
|
||||
22. Write 3–5 tests in `spec/tests/test-hyperscript-lua.sx`:
|
||||
- Lua returns a string → HS uses it.
|
||||
- Lua error → HS catch.
|
||||
- Lua reads a passed argument.
|
||||
23. Commit: `HS: Lua plugin — inline lua...end blocks`.
|
||||
|
||||
### Step 6 — Worker full runtime plugin
|
||||
|
||||
24. Extend `worker.sx`: implement `parse-worker-feat`, compiler entry,
|
||||
`hs-worker-define!`, `hs-method-call` worker branch.
|
||||
25. Extend test runner: `Worker` constructor + synchronous message loop.
|
||||
26. Un-skip the 7 sibling worker tests from upstream.
|
||||
27. Target: 7/7 worker suite.
|
||||
28. Commit: `HS: Worker plugin full runtime (+7 tests)`.
|
||||
|
||||
---
|
||||
|
||||
## 7. Risks
|
||||
|
||||
- **`parse-feat` closure scope** — `hs-register-feature!` stores parse-fns
|
||||
that need access to parser-internal helpers (`adv!`, `tp-val`, etc.). These
|
||||
are only in scope inside `hs-parse`'s big `let`. Two options:
|
||||
(a) the registry stores fns that receive a parser-context dict as arg, or
|
||||
(b) the registry is checked *inside* `parse-feat` where helpers are in scope
|
||||
and fns are zero-arg closures captured at registration time.
|
||||
Option (b) is simpler but requires plugins to be loaded while the parser
|
||||
`let` is being evaluated — i.e., plugins must be defined *inside* the parser
|
||||
file or the context dict must be exposed. **Recommended:** expose a
|
||||
`_hs-parser-ctx` dict at the module level that parse-fns receive as their
|
||||
sole argument. This makes the API explicit and plugins independent files.
|
||||
|
||||
- **Worker Blob URL in WASM** — `URL.createObjectURL` is available in browsers
|
||||
but not in the OCaml WASM host. Worker full runtime is browser-only; flag it
|
||||
with a capability check and graceful fallback.
|
||||
|
||||
- **Lua/Prolog mutual recursion** — a Lua block calling back into HS calling
|
||||
back into Lua is theoretically possible via the IO suspension machinery.
|
||||
Don't try to support it initially; raise a clear error if detected.
|
||||
|
||||
- **Plugin load-order sensitivity** — `hs-register-feature!` must be called
|
||||
before any source is parsed. If a plugin is loaded lazily (future), a
|
||||
`worker MyWorker` in the page would hit the stub before the full plugin
|
||||
registers. Acceptable for now; document that plugins must be loaded at boot.
|
||||
|
||||
- **`runtime.sx` cleanup for prolog** — nodes 140–142 are referenced nowhere
|
||||
else in the codebase (grep confirms). Safe to delete once the plugin is live.
|
||||
|
||||
---
|
||||
|
||||
## 8. Non-goals
|
||||
|
||||
- Runtime `use(ext)` API (JS-style dynamic plugin install) — future.
|
||||
- Plugin namespacing / versioning — future.
|
||||
- Any conformance tests other than the 7 worker tests in step 6.
|
||||
- Changing how the WASM bundle is built or split.
|
||||
173
plans/elixir-on-sx.md
Normal file
173
plans/elixir-on-sx.md
Normal file
@@ -0,0 +1,173 @@
|
||||
# Elixir-on-SX: Elixir on the CEK/VM
|
||||
|
||||
Compile Elixir source to SX AST; the existing CEK evaluator runs it. The natural companion
|
||||
to `lib/erlang/` — Elixir compiles to the BEAM and most of its runtime semantics are
|
||||
Erlang's. The interesting parts are Elixir-specific: the macro system (`quote`/`unquote`),
|
||||
the pipe operator `|>`, `with` expressions, `defmodule`/`def`/`defp`, protocol dispatch,
|
||||
and the `Stream` lazy evaluation library.
|
||||
|
||||
End-state goal: **core Elixir programs running**, including modules, pattern matching, the
|
||||
pipe operator, macros (`quote`/`unquote`/`defmacro`), protocols, and actor-style processes
|
||||
reusing the Erlang runtime foundation.
|
||||
|
||||
## Ground rules
|
||||
|
||||
- **Scope:** only touch `lib/elixir/**` and `plans/elixir-on-sx.md`. Do **not** edit
|
||||
`spec/`, `hosts/`, `shared/`, or other `lib/<lang>/`. Reuse `lib/erlang/` runtime
|
||||
functions where possible — import them, don't duplicate.
|
||||
- **Shared-file issues** go under "Blockers" below with a minimal repro; do not fix here.
|
||||
- **SX files:** use `sx-tree` MCP tools only.
|
||||
- **Architecture:** Elixir source → Elixir AST → SX AST. Reuse Erlang runtime for process/
|
||||
message/pattern primitives; add Elixir-specific surface in `lib/elixir/`.
|
||||
- **Commits:** one feature per commit. Keep `## Progress log` updated and tick boxes.
|
||||
|
||||
## Architecture sketch
|
||||
|
||||
```
|
||||
Elixir source text
|
||||
│
|
||||
▼
|
||||
lib/elixir/tokenizer.sx — atoms (:atom), strings (""), charlists (''), sigils (~r, ~s etc.),
|
||||
│ operators (|>, <>, ++, :::, etc.), do/end blocks
|
||||
▼
|
||||
lib/elixir/parser.sx — Elixir AST: defmodule, def/defp/defmacro, @attribute,
|
||||
│ pattern matching, |> pipe, with, for comprehension, quote/unquote,
|
||||
│ case/cond/if/unless, fn, receive, try/rescue/catch/after
|
||||
▼
|
||||
lib/elixir/transpile.sx — Elixir AST → SX AST
|
||||
│
|
||||
├── lib/erlang/runtime.sx (reused: processes, message passing, pattern match)
|
||||
└── lib/elixir/runtime.sx — Elixir-specific: Kernel, String, Enum, Stream, Map,
|
||||
List, Tuple, IO, protocol dispatch, macro expansion
|
||||
```
|
||||
|
||||
Key semantic mappings (differences from Erlang):
|
||||
- `defmodule M do ... end` → SX `define-library` + module dict `{:module "M" :fns {...}}`
|
||||
- `def f(args) do body end` → named function in module dict, with pattern-match dispatch
|
||||
- `|>` pipe → left-to-right function composition; `a |> f(b)` = `f(a, b)`
|
||||
- `with x <- expr, y <- expr2 do body else patterns end` → chained pattern match with early exit
|
||||
- `for x <- list, filter, do: expr` → list comprehension (SX `map`/`filter`)
|
||||
- `quote do expr end` → returns AST as SX list (homoiconic — Elixir AST IS SX-like)
|
||||
- `unquote(expr)` → evaluate expr and splice into surrounding `quote`
|
||||
- `defmacro` → macro in module; expanded at compile time by calling the SX macro
|
||||
- Protocol → dict of implementations keyed by type name; `defprotocol` defines interface,
|
||||
`defimpl` registers an implementation
|
||||
- `Stream` → lazy sequences using SX promises/coroutines (Phase 9/4 of primitives)
|
||||
- `Agent`/`GenServer` → SX coroutine + message queue (similar to Erlang process model)
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — tokenizer + parser
|
||||
- [ ] Tokenizer: atoms (`:atom`, `:"atom with spaces"`), strings (`""`), charlists (`''`),
|
||||
numbers (int, float, hex `0xFF`, octal `0o77`, binary `0b11`), booleans (`true`/`false`/`nil`),
|
||||
operators (`|>`, `<>`, `++`, `--`, `:::`, `&&`, `||`, `!`, `..`, `<-`, `=~`),
|
||||
sigils (`~r/regex/`, `~s"string"`, `~w(word list)`), do/end blocks, keywords as args
|
||||
`f(key: val)`, `@module_attribute`
|
||||
- [ ] Parser:
|
||||
- Module: `defmodule Name do ... end` → module AST with body
|
||||
- Functions: `def f(pat) do body end`, `def f(pat) when guard do body end`,
|
||||
multi-clause `def f(a) do ...; def f(b) do ...` → clause list
|
||||
- `defp` (private), `defmacro`, `defmacrop`
|
||||
- `@doc`, `@moduledoc`, `@spec`, `@type`, `@behaviour` module attributes
|
||||
- `case expr do patterns end`, `cond do clauses end`, `if`/`unless`
|
||||
- `with x <- e, y <- e2, do: body, else: [pattern -> body]`
|
||||
- `for x <- list, filter, into: acc, do: expr` comprehension
|
||||
- `fn pat -> body end` anonymous function; capture `&Module.fun/arity`, `&(&1 + 1)`
|
||||
- `receive do patterns after timeout -> body end`
|
||||
- `try do body rescue e -> ... catch type, val -> ... after ... end`
|
||||
- `quote do ... end`, `unquote(expr)`, `unquote_splicing(list)`
|
||||
- `|>` pipe chain: `a |> f |> g(b)` → `g(f(a), b)`
|
||||
- [ ] Tests in `lib/elixir/tests/parse.sx`
|
||||
|
||||
### Phase 2 — transpile: basic Elixir (no macros, no processes)
|
||||
- [ ] `ex-eval-ast` entry
|
||||
- [ ] Arithmetic, string `<>`, list `++`/`--`, comparison, boolean (`and`/`or`/`not`)
|
||||
- [ ] Pattern matching in `=`, function heads, `case` — reuse Erlang pattern engine
|
||||
- [ ] `def`/`defp` → SX `define` with clause dispatch (like Erlang function clauses)
|
||||
- [ ] Module as a dict of named functions; `ModuleName.function(args)` dispatch
|
||||
- [ ] `|>` pipe: desugar `a |> f(b, c)` → `f(a, b, c)` at transpile time
|
||||
- [ ] `with` expression: chain of `<-` bindings, short-circuit on mismatch to `else`
|
||||
- [ ] `for` comprehension: `for x <- list, filter do body end` → `map`/`filter`
|
||||
- [ ] `fn` anonymous functions, `&` capture forms
|
||||
- [ ] `if`/`unless`/`cond`/`case`
|
||||
- [ ] String interpolation: `"Hello #{name}"` → string concat
|
||||
- [ ] Keyword lists `[key: val]` → SX list of `{:key val}` dicts; maps `%{key: val}` → SX dict
|
||||
- [ ] Tuples `{a, b, c}` → SX list (or vector); `elem/2`, `put_elem/3`
|
||||
- [ ] 40+ eval tests in `lib/elixir/tests/eval.sx`
|
||||
|
||||
### Phase 3 — macro system
|
||||
- [ ] `quote do expr end` → returns Elixir AST as SX list structure
|
||||
(Elixir AST is 3-tuples `{name, meta, args}` — map to SX `(list name meta args)`)
|
||||
- [ ] `unquote(expr)` → evaluate and splice into surrounding `quote`
|
||||
- [ ] `unquote_splicing(list)` → splice list into surrounding `quote`
|
||||
- [ ] `defmacro` → define a macro in the module; macro receives AST args, returns AST
|
||||
- [ ] Macro expansion: expand macros before transpiling (two-pass: collect defs, then expand)
|
||||
- [ ] `use Module` → calls `Module.__using__/1` macro, injects code into caller
|
||||
- [ ] `import Module` → bring functions into scope without prefix
|
||||
- [ ] `alias Module, as: M` → short name for module
|
||||
- [ ] Tests: `defmacro unless`, `defmacro my_if`, `use` injection, `__MODULE__`, `__DIR__`
|
||||
|
||||
### Phase 4 — protocols
|
||||
- [ ] `defprotocol P do @spec f(t) :: result end` → defines protocol dict + dispatch fn
|
||||
- [ ] `defimpl P, for: Type do def f(t) do ... end end` → register implementation
|
||||
- [ ] Protocol dispatch: `P.f(value)` → look up type of value, find implementation, call it
|
||||
- [ ] Built-in protocols: `Enumerable`, `Collectable`, `String.Chars`, `Inspect`
|
||||
- [ ] `Enumerable` implementation for lists, maps, ranges — enables `Enum.*` on custom types
|
||||
- [ ] `derive` — automatic protocol implementation for simple structs
|
||||
- [ ] Tests: custom type implementing `Enumerable`, `String.Chars`, protocol fallback
|
||||
|
||||
### Phase 5 — structs + behaviours
|
||||
- [ ] `defstruct [:field1, field2: default]` → defines `%ModuleName{}` struct type
|
||||
Structs are maps with `__struct__: ModuleName` key + defined fields
|
||||
- [ ] Struct pattern matching: `%User{name: n} = user`
|
||||
- [ ] `@behaviour Module` → declares behaviour callbacks; compile-time check
|
||||
- [ ] `@impl true` / `@impl BehaviourName` → marks function as behaviour implementation
|
||||
- [ ] Built-in behaviours: `GenServer`, `Supervisor`, `Agent`, `Task`
|
||||
- [ ] Tests: struct creation, update syntax `%{struct | field: val}`, behaviour callbacks
|
||||
|
||||
### Phase 6 — processes + OTP patterns (reuses Erlang runtime)
|
||||
- [ ] `spawn(fn -> ... end)` / `spawn(M, f, args)` → SX coroutine on scheduler
|
||||
Reuse `lib/erlang/` process + message queue infrastructure
|
||||
- [ ] `send(pid, msg)` / `receive do patterns end` — already in Erlang runtime
|
||||
- [ ] `GenServer` behaviour: `start_link`, `call`, `cast`, `handle_call`, `handle_cast`,
|
||||
`handle_info`, `init` — implement as SX macros expanding to process + message loop
|
||||
- [ ] `Agent` — simple state wrapper over GenServer; `Agent.start_link`, `get`, `update`
|
||||
- [ ] `Task` — async computation; `Task.async`, `Task.await`
|
||||
- [ ] `Supervisor` — child spec, restart strategy (`one_for_one`, `one_for_all`)
|
||||
- [ ] Tests: counter GenServer, bank account Agent, parallel Task, supervised worker
|
||||
|
||||
### Phase 7 — standard library
|
||||
- [ ] `Enum.*` — `map`, `filter`, `reduce`, `each`, `into`, `flat_map`, `zip`, `sort`,
|
||||
`sort_by`, `min_by`, `max_by`, `group_by`, `frequencies`, `count`, `any?`, `all?`,
|
||||
`find`, `take`, `drop`, `take_while`, `drop_while`, `chunk_every`, `chunk_by`,
|
||||
`flat_map_reduce`, `scan`, `uniq`, `uniq_by`, `member?`, `empty?`, `sum`, `product`
|
||||
- [ ] `Stream.*` — lazy versions of Enum; `Stream.map`, `Stream.filter`, `Stream.take`,
|
||||
`Stream.cycle`, `Stream.iterate`, `Stream.unfold`, `Stream.resource`
|
||||
Uses SX promises (Phase 9) for laziness
|
||||
- [ ] `String.*` — `length`, `upcase`, `downcase`, `trim`, `split`, `replace`, `contains?`,
|
||||
`starts_with?`, `ends_with?`, `slice`, `at`, `graphemes`, `codepoints`, `to_integer`,
|
||||
`to_float`, `pad_leading`, `pad_trailing`, `duplicate`, `match?`
|
||||
- [ ] `Map.*` — `new`, `get`, `put`, `delete`, `update`, `merge`, `keys`, `values`,
|
||||
`to_list`, `from_struct`, `has_key?`, `filter`, `map`, `reject`, `take`, `drop`
|
||||
- [ ] `List.*` — `first`, `last`, `flatten`, `zip`, `unzip`, `keystore`, `keyfind`,
|
||||
`wrap`, `duplicate`, `improper?`, `delete`, `insert_at`, `replace_at`
|
||||
- [ ] `Tuple.*` — `to_list`, `from_list`, `append`, `insert_at`, `delete_at`
|
||||
- [ ] `Integer.*` / `Float.*` — `parse`, `to_string`, `digits`, `pow`, `is_odd?`, `is_even?`
|
||||
- [ ] `IO.*` — `puts`, `gets`, `inspect`, `write`, `read` → SX IO perform
|
||||
- [ ] `Kernel.*` — built-in functions: `is_integer?`, `is_binary?`, `length`, `hd`, `tl`,
|
||||
`elem`, `put_elem`, `apply`, `raise`, `exit`, `inspect`
|
||||
- [ ] `inspect/1` / `IO.inspect/2` — debug printing using `Inspect` protocol
|
||||
|
||||
### Phase 8 — conformance target
|
||||
- [ ] Vendor or hand-build 100+ Elixir program tests in `lib/elixir/tests/programs/`
|
||||
- [ ] Drive scoreboard
|
||||
|
||||
## Blockers
|
||||
|
||||
_(none yet)_
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
_(awaiting phase 1)_
|
||||
131
plans/elm-on-sx.md
Normal file
131
plans/elm-on-sx.md
Normal file
@@ -0,0 +1,131 @@
|
||||
# Elm-on-SX: Elm 0.19 on the CEK/VM
|
||||
|
||||
Compile Elm source to SX AST; the existing CEK evaluator runs it. The unique angle: SX's
|
||||
reactive island system (`defisland`, signals, `provide`/`context`) is a natural host for
|
||||
The Elm Architecture — Model/Update/View maps almost directly onto SX's reactive runtime.
|
||||
This is the only language in the set that targets SX's browser-side reactivity rather than
|
||||
the server-side evaluator.
|
||||
|
||||
End-state goal: **core Elm programs running in the browser via SX islands**, with The Elm
|
||||
Architecture wired to SX signals. Not a full Elm compiler — no exhaustiveness checking, no
|
||||
module system, no type inference — but a faithful runtime that can run Elm programs written
|
||||
in idiomatic style.
|
||||
|
||||
## Ground rules
|
||||
|
||||
- **Scope:** only touch `lib/elm/**` and `plans/elm-on-sx.md`. Do **not** edit `spec/`,
|
||||
`hosts/`, `shared/`, or other `lib/<lang>/`.
|
||||
- **Shared-file issues** go under "Blockers" below with a minimal repro; do not fix here.
|
||||
- **SX files:** use `sx-tree` MCP tools only.
|
||||
- **Architecture:** Elm source → Elm AST → SX AST. No standalone Elm evaluator.
|
||||
- **Type system:** defer. Focus on runtime semantics. Type errors surface at eval time.
|
||||
- **Commits:** one feature per commit. Keep `## Progress log` updated and tick boxes.
|
||||
|
||||
## Architecture sketch
|
||||
|
||||
```
|
||||
Elm source text
|
||||
│
|
||||
▼
|
||||
lib/elm/tokenizer.sx — numbers, strings, idents, operators, indentation-sensitive lexer
|
||||
│
|
||||
▼
|
||||
lib/elm/parser.sx — Elm AST: module, import, type alias, type, let, case, lambda,
|
||||
│ if, list/tuple/record literals, pipe operator |>
|
||||
▼
|
||||
lib/elm/transpile.sx — Elm AST → SX AST
|
||||
│
|
||||
▼
|
||||
lib/elm/runtime.sx — TEA runtime: Program, sandbox, element; Cmd/Sub wrappers;
|
||||
│ Html.* shims; Browser.* shims
|
||||
▼
|
||||
SX island / reactive runtime (browser)
|
||||
```
|
||||
|
||||
Key semantic mappings:
|
||||
- `Model` → SX signal (`make-signal`)
|
||||
- `update : Msg -> Model -> Model` → SX signal updater (called on each message)
|
||||
- `view : Model -> Html Msg` → SX component (re-renders on model signal change)
|
||||
- `Cmd` → SX `perform` IO request
|
||||
- `Sub` → SX event listener registered via `dom-listen`
|
||||
- `Maybe a` → `nil` (Nothing) or value (Just a) — uses ADTs from Phase 6 of primitives
|
||||
- `Result a b` → ADT `(Ok val)` / `(Err err)`
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — tokenizer + parser
|
||||
- [ ] Tokenizer: keywords (`module`, `import`, `type`, `alias`, `let`, `in`, `if`, `then`,
|
||||
`else`, `case`, `of`, `port`), indentation tokens (indent/dedent/newline), string
|
||||
literals, number literals, operators (`|>`, `>>`, `<<`, `<|`, `++`, `::`), type vars
|
||||
- [ ] Parser: module declaration, imports, type aliases, union types, function definitions
|
||||
with pattern matching, `let`/`in`, `case`/`of`, `if`/`then`/`else`, lambda `\x -> e`,
|
||||
list literals `[1,2,3]`, tuple literals `(a,b)`, record literals `{x=1, y=2}`,
|
||||
record update `{ r | x = 1 }`, pipe operator `|>`
|
||||
- [ ] Skip for phase 1: ports, subscriptions, effects manager, type annotations
|
||||
- [ ] Tests in `lib/elm/tests/parse.sx`
|
||||
|
||||
### Phase 2 — transpile: expressions + pattern matching
|
||||
- [ ] `elm-eval-ast` entry
|
||||
- [ ] Arithmetic, string `++`, comparison, boolean ops
|
||||
- [ ] Lambda → SX `fn`; function application
|
||||
- [ ] `let`/`in` → SX `let`
|
||||
- [ ] `if`/`then`/`else` → SX `if`
|
||||
- [ ] `case`/`of` with constructor, literal, tuple, list, wildcard patterns → SX `cond`
|
||||
using ADT match (Phase 6 primitives)
|
||||
- [ ] List ops: `List.map`, `List.filter`, `List.foldl`, `List.foldr`
|
||||
- [ ] `Maybe` and `Result` as ADTs
|
||||
- [ ] 30+ eval tests in `lib/elm/tests/eval.sx`
|
||||
|
||||
### Phase 3 — The Elm Architecture runtime
|
||||
- [ ] `Browser.sandbox` — pure TEA loop (no Cmds, no Subs)
|
||||
`{ init : model, update : msg -> model -> model, view : model -> Html msg }`
|
||||
Wires to: SX signal for model, SX component for view, message dispatch on user events
|
||||
- [ ] `Html.*` shims: `div`, `p`, `button`, `input`, `text`, `h1`–`h6`, `ul`, `li`, `a`,
|
||||
`span`, `img` — emit SX component calls
|
||||
- [ ] `Html.Attributes.*`: `class`, `id`, `href`, `src`, `type_`, `placeholder`, `value`
|
||||
- [ ] `Html.Events.*`: `onClick`, `onInput`, `onSubmit`, `onBlur`, `onFocus`
|
||||
- [ ] `Browser.element` — adds `init` returning `(model, Cmd msg)`, `subscriptions`
|
||||
- [ ] Demo: counter app (`init=0`, `update Increment m = m+1`, `view` shows count + button)
|
||||
|
||||
### Phase 4 — Cmds and Subs
|
||||
- [ ] `Cmd` — mapped to SX `perform` IO requests. `Cmd.none`, `Cmd.batch`
|
||||
- [ ] `Http.get`/`Http.post` → SX fetch IO
|
||||
- [ ] `Sub` — mapped to SX `dom-listen`. `Sub.none`, `Sub.batch`
|
||||
- [ ] `Browser.Events.onClick`, `onKeyPress`, `onAnimationFrame`
|
||||
- [ ] `Time.every` — periodic subscription via SX timer IO
|
||||
- [ ] `Task.perform`/`Task.attempt` — single-shot async operations
|
||||
|
||||
### Phase 5 — standard library
|
||||
- [ ] `String.*` — `length`, `append`, `concat`, `split`, `join`, `trim`, `toUpper`, `toLower`,
|
||||
`contains`, `startsWith`, `endsWith`, `replace`, `toInt`, `toFloat`, `fromInt`, `fromFloat`
|
||||
- [ ] `List.*` — `map`, `filter`, `foldl`, `foldr`, `head`, `tail`, `isEmpty`, `length`,
|
||||
`reverse`, `append`, `concat`, `member`, `sort`, `sortBy`, `indexedMap`, `range`
|
||||
- [ ] `Dict.*` — SX immutable dict; `fromList`, `toList`, `get`, `insert`, `remove`, `update`,
|
||||
`member`, `keys`, `values`, `map`, `filter`, `foldl`
|
||||
- [ ] `Set.*` — SX set primitive (Phase 18); `fromList`, `toList`, `member`, `insert`,
|
||||
`remove`, `union`, `intersect`, `diff`
|
||||
- [ ] `Maybe.*` — `withDefault`, `map`, `andThen`, `map2`
|
||||
- [ ] `Result.*` — `withDefault`, `map`, `andThen`, `mapError`, `toMaybe`
|
||||
- [ ] `Tuple.*` — `first`, `second`, `pair`, `mapFirst`, `mapSecond`
|
||||
- [ ] `Basics.*` — `identity`, `always`, `not`, `xor`, `modBy`, `remainderBy`, `clamp`,
|
||||
`min`, `max`, `abs`, `sqrt`, `logBase`, `e`, `pi`, `floor`, `ceiling`, `round`,
|
||||
`truncate`, `toFloat`, `isNaN`, `isInfinite`, `compare`
|
||||
- [ ] `Random.*` — seed-based PRNG via SX IO perform
|
||||
|
||||
### Phase 6 — full browser integration
|
||||
- [ ] `Browser.application` — URL routing, `onUrlChange`, `onUrlRequest`
|
||||
- [ ] `Browser.Navigation.*` — `pushUrl`, `replaceUrl`, `back`, `forward`
|
||||
- [ ] `Url.Parser.*` — path segment parsing
|
||||
- [ ] `Json.Decode.*` — JSON decoder combinators
|
||||
- [ ] `Json.Encode.*` — JSON encoder
|
||||
- [ ] `Ports` — `port` keyword; JS interop via SX `host-call`
|
||||
|
||||
## Blockers
|
||||
|
||||
_(none yet)_
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
_(awaiting phase 1)_
|
||||
145
plans/go-on-sx.md
Normal file
145
plans/go-on-sx.md
Normal file
@@ -0,0 +1,145 @@
|
||||
# Go-on-SX: Go on the CEK/VM
|
||||
|
||||
Compile Go source to SX AST; the existing CEK evaluator runs it. The unique angle: Go's
|
||||
goroutines and channels map cleanly onto SX's IO suspension machinery (`perform`/`cek-resume`)
|
||||
— a goroutine is a `cek-step-loop` running in a cooperative scheduler, a channel send/receive
|
||||
is a `perform` that suspends until the other end is ready.
|
||||
|
||||
End-state goal: **core Go programs running**, including goroutines, channels, defer/panic/recover,
|
||||
interfaces, and structs. Not a full Go compiler — no generics, no CGo, no full stdlib — but
|
||||
a faithful runtime for idiomatic Go concurrent programs.
|
||||
|
||||
## Ground rules
|
||||
|
||||
- **Scope:** only touch `lib/go/**` and `plans/go-on-sx.md`. Do **not** edit `spec/`,
|
||||
`hosts/`, `shared/`, or other `lib/<lang>/`.
|
||||
- **Shared-file issues** go under "Blockers" below with a minimal repro; do not fix here.
|
||||
- **SX files:** use `sx-tree` MCP tools only.
|
||||
- **Architecture:** Go source → Go AST → SX AST. No standalone Go evaluator.
|
||||
- **Concurrency model:** cooperative, not preemptive. Goroutines yield at channel ops and
|
||||
`time.Sleep`. A round-robin scheduler in SX drives them.
|
||||
- **Commits:** one feature per commit. Keep `## Progress log` updated and tick boxes.
|
||||
|
||||
## Architecture sketch
|
||||
|
||||
```
|
||||
Go source text
|
||||
│
|
||||
▼
|
||||
lib/go/tokenizer.sx — Go tokens: keywords, idents, string/rune/number literals,
|
||||
│ operators, semicolon insertion rules
|
||||
▼
|
||||
lib/go/parser.sx — Go AST: package, import, var, const, type, func, struct,
|
||||
│ interface, goroutine, channel ops, defer, select, for range
|
||||
▼
|
||||
lib/go/transpile.sx — Go AST → SX AST
|
||||
│
|
||||
▼
|
||||
lib/go/runtime.sx — goroutine scheduler, channel primitives, defer stack,
|
||||
│ panic/recover, interface dispatch, slice/map ops
|
||||
▼
|
||||
CEK / VM
|
||||
```
|
||||
|
||||
Key semantic mappings:
|
||||
- `go fn()` → spawn new coroutine (SX coroutine primitive, Phase 4 of primitives)
|
||||
- `ch <- v` (send) → `perform` that suspends until receiver ready; scheduler picks next goroutine
|
||||
- `v := <-ch` (receive) → `perform` that suspends until sender ready
|
||||
- `select { case ... }` → scheduler checks all channel readiness, picks first ready
|
||||
- `defer fn()` → push onto a per-goroutine defer stack; run on return/panic
|
||||
- `panic(v)` → `raise` the value; `recover()` catches it in deferred function
|
||||
- `interface{}` → any SX value (duck typed)
|
||||
- `struct { ... }` → SX hash table with field names as keys
|
||||
- `slice` → SX vector with length + capacity metadata
|
||||
- `map[K]V` → SX mutable hash table (Phase 10 of primitives)
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — tokenizer + parser
|
||||
- [ ] Tokenizer: keywords (`package`, `import`, `func`, `var`, `const`, `type`, `struct`,
|
||||
`interface`, `go`, `chan`, `select`, `defer`, `return`, `if`, `else`, `for`, `range`,
|
||||
`switch`, `case`, `default`, `break`, `continue`, `goto`, `fallthrough`, `map`,
|
||||
`make`, `new`, `nil`, `true`, `false`), automatic semicolon insertion, string literals
|
||||
(interpreted + raw `` `...` ``), rune literals `'a'`, number literals (int, float, hex,
|
||||
octal, binary, complex), operators, slices `[:]`
|
||||
- [ ] Parser: package clause, imports, top-level `func`/`var`/`const`/`type`; function
|
||||
bodies: short variable decl `:=`, assignments, `if`/`else`, `for`/`range`, `switch`,
|
||||
`return`, struct literals, slice literals, map literals, composite literals, type
|
||||
assertions `v.(T)`, method calls `v.Method(args)`, goroutine `go`, channel ops
|
||||
`<-ch`, `ch <- v`, `defer`, `select`
|
||||
- [ ] Tests in `lib/go/tests/parse.sx`
|
||||
|
||||
### Phase 2 — transpile: basic Go (no goroutines)
|
||||
- [ ] `go-eval-ast` entry
|
||||
- [ ] Arithmetic, string ops, comparison, boolean
|
||||
- [ ] Variables, short decl, assignment, multiple assignment
|
||||
- [ ] `if`/`else if`/`else`
|
||||
- [ ] `for` (C-style), `for range` over slice/map/string
|
||||
- [ ] Functions: named + anonymous, multiple return values (SX multiple values, Phase 8)
|
||||
- [ ] Structs → SX hash tables; field access `.field`; struct literals `T{f: v}`
|
||||
- [ ] Slices → SX vectors; `len`, `cap`, `append`, `copy`, slice expressions `s[a:b]`
|
||||
- [ ] Maps → SX hash tables; `make(map[K]V)`, `m[k]`, `m[k] = v`, `delete(m, k)`,
|
||||
comma-ok `v, ok := m[k]`
|
||||
- [ ] Pointers — modelled as single-element mutable vectors; `&x` creates wrapper, `*p` dereferences
|
||||
- [ ] `fmt.Println`/`fmt.Printf`/`fmt.Sprintf` → SX IO perform (print)
|
||||
- [ ] 40+ eval tests in `lib/go/tests/eval.sx`
|
||||
|
||||
### Phase 3 — defer / panic / recover
|
||||
- [ ] Defer stack per function frame — SX list of thunks, run LIFO on return
|
||||
- [ ] `defer` statement pushes thunk; transpiler wraps function body in try/finally equivalent
|
||||
- [ ] `panic(v)` → `raise` with Go panic wrapper
|
||||
- [ ] `recover()` → catches panic value inside a deferred function; returns nil otherwise
|
||||
- [ ] Panic propagation across call stack until recovered or fatal
|
||||
- [ ] Tests: defer ordering, panic/recover, panic in goroutine without recover
|
||||
|
||||
### Phase 4 — goroutines + channels
|
||||
- [ ] Coroutine-based goroutine type using SX coroutine primitive (Phase 4 of primitives)
|
||||
- [ ] Round-robin scheduler in `lib/go/runtime.sx`: maintains run queue, steps each
|
||||
goroutine one turn at a time, suspends at channel ops
|
||||
- [ ] Unbuffered channels: `make(chan T)` → rendezvous point; send suspends until receive
|
||||
and vice versa. Implemented as a pair of waiting queues + `cek-resume`.
|
||||
- [ ] Buffered channels: `make(chan T, n)` → circular buffer; send only blocks when full,
|
||||
receive only blocks when empty
|
||||
- [ ] `close(ch)` — mark channel closed; receivers drain then get zero value + `false`
|
||||
- [ ] `select` — scheduler inspects all cases, picks a ready one (random if multiple),
|
||||
blocks if none ready until at least one becomes ready
|
||||
- [ ] `go fn(args)` — spawns new goroutine on run queue
|
||||
- [ ] `time.Sleep(d)` — yields current goroutine, re-queues after d milliseconds
|
||||
(simulated with IO perform timer)
|
||||
- [ ] Tests: ping-pong, fan-out, fan-in, select with default, range over channel
|
||||
|
||||
### Phase 5 — interfaces
|
||||
- [ ] Interface type → SX dict `{:type "T" :methods {...}}` dispatch table
|
||||
- [ ] `interface{}` / `any` → any SX value (already implicit)
|
||||
- [ ] Type assertion `v.(T)` → check `:type` field, panic if mismatch
|
||||
- [ ] Type switch `switch v.(type) { case T: ... }` → dispatches on `:type`
|
||||
- [ ] Method sets — structs implement interfaces implicitly if they have the right methods
|
||||
- [ ] Value vs pointer receivers — pointer receiver gets the mutable vector wrapper
|
||||
- [ ] Built-in interfaces: `error` (`Error() string`), `Stringer` (`String() string`)
|
||||
- [ ] Tests: interface satisfaction, type assertion, type switch, error interface
|
||||
|
||||
### Phase 6 — standard library subset
|
||||
- [ ] `fmt` — `Println`, `Printf`, `Sprintf`, `Fprintf`, `Errorf`, `Stringer` dispatch
|
||||
- [ ] `strings` — `Contains`, `HasPrefix`, `HasSuffix`, `Split`, `Join`, `TrimSpace`,
|
||||
`ToUpper`, `ToLower`, `Replace`, `Index`, `Count`, `Repeat`
|
||||
- [ ] `strconv` — `Itoa`, `Atoi`, `FormatFloat`, `ParseFloat`, `ParseInt`, `FormatInt`
|
||||
- [ ] `math` — full surface via SX math primitives (Phase 15)
|
||||
- [ ] `sort` — `sort.Slice`, `sort.Ints`, `sort.Strings`
|
||||
- [ ] `errors` — `errors.New`, `errors.Is`, `errors.As`
|
||||
- [ ] `sync` — `sync.Mutex` (cooperative — just a boolean flag + goroutine queue),
|
||||
`sync.WaitGroup`, `sync.Once`
|
||||
- [ ] `io` — `io.Reader`/`io.Writer` interfaces; `io.ReadAll`; `strings.NewReader`
|
||||
|
||||
### Phase 7 — full conformance target
|
||||
- [ ] Vendor a Go test suite or hand-build 100+ program tests in `lib/go/tests/programs/`
|
||||
- [ ] Drive scoreboard
|
||||
|
||||
## Blockers
|
||||
|
||||
_(none yet)_
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
_(awaiting phase 1)_
|
||||
351
plans/hs-bucket-f.md
Normal file
351
plans/hs-bucket-f.md
Normal file
@@ -0,0 +1,351 @@
|
||||
# HS Conformance — Bucket F Plan
|
||||
|
||||
Based on a full suite run on 2026-04-26. Current score: **~1297/1489 covered** (~87%).
|
||||
Skipped from runs: tests 197–200 (hypertrace, slow), 615 (slow), 1197–1198 (repeat-forever timeouts).
|
||||
|
||||
**⚠ Updated 2026-04-26:** The hs-loop completed significant Bucket D work before being stopped.
|
||||
`hs-f` branches from `loops/hs` HEAD which already includes:
|
||||
- MutationObserver mock + `on mutation` dispatch (+7) → **Group 4 likely done**
|
||||
- Cookie API partial (+3/5) → **Group 5 partially done**
|
||||
- `elsewhere`/`from elsewhere` + count filters (+7) → **Group 3a/3c partially done**
|
||||
- Namespaced `def` (+3) → already done
|
||||
- SourceInfo E38 (+4) + WebWorker E39 (+1) → already merged
|
||||
|
||||
**The Bucket F agent must run `hs_test_run` on each group's suite before implementing,
|
||||
to verify what's actually still failing. Skip any group that already passes.**
|
||||
|
||||
Total remaining failures: ~193. Broken into groups below.
|
||||
|
||||
---
|
||||
|
||||
## Group 0 — Bucket E payoff (~47 tests, will land automatically)
|
||||
|
||||
These are already implemented or in-flight on Bucket E branches. Once merged they close ~47 tests.
|
||||
|
||||
| Suite | Tests | Status |
|
||||
|-------|------:|-------|
|
||||
| `hs-upstream-core/tokenizer` | 17 | E37 in progress |
|
||||
| `hs-upstream-socket` | 16 | E36 in progress |
|
||||
| `hs-upstream-fetch` | 8 | E40 in progress |
|
||||
| `hs-upstream-core/sourceInfo` | 4 | E38 done, not yet merged |
|
||||
| `hs-upstream-worker` | 1 | E39 done, not yet merged |
|
||||
| E37 string interpolation bug | 1 | E37 |
|
||||
|
||||
**Do not plan these — they resolve when Bucket E merges.**
|
||||
|
||||
---
|
||||
|
||||
## Group 1 — Null safety reporting (+7)
|
||||
|
||||
**Suite:** `hs-upstream-core/runtimeErrors`
|
||||
**Failures:** 7 tests, all "Expected `'#doesntExist' is null`, got ``"
|
||||
**What's needed:** When a command like `put`, `increment`, `decrement`, `default`, `remove`, `settle`, `transition` receives a null element (e.g. `#doesntExist`), HS must throw a structured null-safety error with the element reference in the message. The null check + error format is already designed in Bucket D #31 (cluster 31 of `hs-conformance-to-100.md`).
|
||||
|
||||
**Estimate:** +7. Straightforward — null guard at command dispatch entry.
|
||||
|
||||
---
|
||||
|
||||
## Group 2 — `tell` semantics (+3)
|
||||
|
||||
**Suite:** `hs-upstream-tell`
|
||||
**Failures:**
|
||||
- `attributes refer to the thing being told` — Expected `bar`, got ``
|
||||
- `your symbol represents the thing being told` — Expected `foo`, got ``
|
||||
- `does not overwrite the me symbol` — assertion fail
|
||||
|
||||
**What's needed:** Inside a `tell X` block, `you`/`your` must resolve to X, attribute refs must resolve against X, and `me` must retain its original value (not be rebound to X). Currently `tell` rebinds `me` instead of introducing a separate `you` binding.
|
||||
|
||||
**Estimate:** +3. Scoping fix in the `tell` command handler.
|
||||
|
||||
---
|
||||
|
||||
## Group 3 — `on` event handler features (+19, skip-list)
|
||||
|
||||
**Suite:** `hs-upstream-on`
|
||||
**34 tests on skip-list.** Prioritise tractable subsets:
|
||||
|
||||
### 3a — Event filtering by count (+6)
|
||||
- `can filter events based on count`
|
||||
- `can filter events based on count range`
|
||||
- `can filter events based on unbounded count range`
|
||||
- `can mix ranges`
|
||||
- `on first click fires only once`
|
||||
- `multiple event handlers at a time are allowed to execute with the every keyword`
|
||||
|
||||
The `on (N)`, `on (N to M)`, `on first`, `every` modifiers. Parser + runtime counter state per handler.
|
||||
|
||||
### 3b — `finally` blocks (+6)
|
||||
- `basic finally blocks work`
|
||||
- `async basic finally blocks work`
|
||||
- `exceptions in finally block don't kill the event queue`
|
||||
- `async exceptions in finally block don't kill the event queue`
|
||||
- `finally blocks work when exception thrown in catch`
|
||||
- `async finally blocks work when exception thrown in catch`
|
||||
|
||||
`on … catch … finally` analogous to JS try/catch/finally. Needs a finally-frame in the CEK machine (similar to dynamic-wind).
|
||||
|
||||
### 3c — `elsewhere` modifier (+2)
|
||||
- `supports "elsewhere" modifier`
|
||||
- `supports "from elsewhere" modifier`
|
||||
|
||||
`on click elsewhere` = click outside the element. Needs a global listener + target exclusion check.
|
||||
|
||||
### 3d — Exception events (+3)
|
||||
- `rethrown exceptions trigger 'exception' event`
|
||||
- `uncaught exceptions trigger 'exception' event`
|
||||
- `can catch exceptions thrown in hyperscript functions`
|
||||
- `can catch exceptions thrown in js functions`
|
||||
|
||||
When an unhandled exception escapes an `on` handler, HS must dispatch an `exception` CustomEvent on the element.
|
||||
|
||||
### 3e — Element removal cleanup (+2)
|
||||
- `listeners on other elements are removed when the registering element is removed`
|
||||
- `listeners on self are not removed when the element is removed`
|
||||
|
||||
Cleanup hook via MutationObserver watching for element removal.
|
||||
|
||||
### Deferred (skip-list, complex):
|
||||
- `can be in a top level script tag` — requires script tag re-initialisation
|
||||
- `can ignore when target doesn't exist` — target null guard
|
||||
- `can handle an or after a from clause` — parser edge case
|
||||
- `each behavior installation has its own event queue` — behavior isolation
|
||||
|
||||
---
|
||||
|
||||
## Group 4 — MutationObserver / `on mutation` (+10)
|
||||
|
||||
**Suite:** `hs-upstream-on` (mutation subset, skip-list)
|
||||
**Tests:**
|
||||
- `can listen for attribute mutations`
|
||||
- `can listen for attribute mutations on other elements`
|
||||
- `can listen for childList mutations`
|
||||
- `can listen for general mutations`
|
||||
- `can listen for multiple mutations`
|
||||
- `can listen for multiple mutations 2`
|
||||
- `can listen for specific attribute mutations`
|
||||
- `can pick event properties out by name`
|
||||
- `can pick detail fields out by name`
|
||||
- `attribute observers are persistent (not recreated on re-run)` (hs-upstream-when)
|
||||
|
||||
**What's needed:** MutationObserver mock in the test runner (`hs-run-filtered.js`) + `on mutation` command in the parser/runtime. Already prototyped in Bucket D #32.
|
||||
|
||||
**Estimate:** +10.
|
||||
|
||||
---
|
||||
|
||||
## Group 5 — Cookie API (+5)
|
||||
|
||||
**Suite:** `hs-upstream-expressions/cookies`
|
||||
All 5 tests untranslated. Cookie read/write as an expression: `cookies.name`, `set cookies.name to val`, `cookies.name is undefined`. Needs `document.cookie` mock in runner + cookie-expression parse path.
|
||||
|
||||
**Estimate:** +5. Self-contained.
|
||||
|
||||
---
|
||||
|
||||
## Group 6 — Block literals (+4)
|
||||
|
||||
**Suite:** `hs-upstream-expressions/blockLiteral`
|
||||
All 4 untranslated. Syntax: `[x | x + 1]` — an inline lambda. Used as a first-class value passable to `map`, `filter` etc.
|
||||
|
||||
- `basic block literals work`
|
||||
- `basic identity works`
|
||||
- `basic two arg identity works`
|
||||
- `can map an array`
|
||||
|
||||
**Estimate:** +4. Parser addition + runtime callable wrapping.
|
||||
|
||||
---
|
||||
|
||||
## Group 7 — Async logical operators (+5)
|
||||
|
||||
**Suite:** `hs-upstream-expressions/logicalOperator`
|
||||
Promise-aware `and`/`or`:
|
||||
- `and short-circuits when lhs promise resolves to false`
|
||||
- `or short-circuits when lhs promise resolves to true`
|
||||
- `or evaluates rhs when lhs promise resolves to false`
|
||||
- `should short circuit with and expression`
|
||||
- `should short circuit with or expression`
|
||||
|
||||
**What's needed:** `and`/`or` must await promise operands before short-circuiting. Currently they evaluate eagerly without awaiting.
|
||||
|
||||
**Estimate:** +5. Async await integration in logical operator eval.
|
||||
|
||||
---
|
||||
|
||||
## Group 8 — `evalStatically` (+3)
|
||||
|
||||
**Suite:** `hs-upstream-core/evalStatically`
|
||||
- `throws on math expressions`
|
||||
- `throws on symbol references`
|
||||
- `throws on template strings`
|
||||
|
||||
`_hyperscript.evaluate(src, {}, { throwErrors: true })` must throw synchronously for expressions with side-effects or unresolved symbols. Currently the static evaluator doesn't gate on `throwErrors`.
|
||||
|
||||
**Estimate:** +3. Flag-gated error throw path.
|
||||
|
||||
---
|
||||
|
||||
## Group 9 — Parse error API (+6)
|
||||
|
||||
**Suite:** `hs-upstream-core/parser` + `hs-upstream-core/bootstrap`
|
||||
- `basic parse error messages work`
|
||||
- `fires hyperscript:parse-error event with all errors`
|
||||
- `parse error at EOF on trailing newline does not crash`
|
||||
- `_hyperscript() evaluate API still throws on first error`
|
||||
- `fires hyperscript:before:init and hyperscript:after:init` (bootstrap)
|
||||
- `hyperscript:before:init can cancel initialization` (bootstrap)
|
||||
|
||||
**What's needed:**
|
||||
- Parser must emit a `hyperscript:parse-error` CustomEvent on `document` when compilation fails, with the error list as detail.
|
||||
- `hyperscript:before:init` / `hyperscript:after:init` lifecycle events dispatched around element initialization.
|
||||
- `before:init` can cancel (return false / `event.preventDefault()`).
|
||||
|
||||
**Estimate:** +6. Event dispatch hooks in the bootstrap/init path.
|
||||
|
||||
---
|
||||
|
||||
## Group 10 — `as` expression conversions (+8)
|
||||
|
||||
**Suite:** `hs-upstream-expressions/asExpression`
|
||||
Currently 30/42 = 12 failures. Tractable subset:
|
||||
|
||||
- `converts a NodeList into HTML` — NodeList → outerHTML join
|
||||
- `converts strings into fragments` — string → DocumentFragment
|
||||
- `converts elements into fragments` — element → DocumentFragment
|
||||
- `converts arrays into fragments` — array of elements → DocumentFragment
|
||||
- `converts array as Set` — array → Set (dedup)
|
||||
- `converts object as Map` — object → Map
|
||||
- `can accept custom conversions` — `as MyType` via registered converter
|
||||
- `can use the a modifier if you like` — `as a Number` synonym
|
||||
|
||||
Two already-broken non-skip failures:
|
||||
- `converts a complete form into Values` — Expected `dog`, got ``
|
||||
- `converts multiple selects with programmatically changed selections` — Expected `cat`, got `dog`
|
||||
|
||||
**Estimate:** +8 for the tractable subset. Custom converters and Map/Set require runtime additions.
|
||||
|
||||
---
|
||||
|
||||
## Group 11 — Miscellaneous runtime bugs (+12)
|
||||
|
||||
Small scattered failures, each 1–3 tests:
|
||||
|
||||
| Suite | Failure | Likely cause |
|
||||
|-------|---------|-------------|
|
||||
| `hs-upstream-put` | `properly processes hyperscript` ×3 (got 40, expected 42) | Off-by-one in `put ... before/after` reprocessing |
|
||||
| `hs-upstream-put` | `waits on promises` | Promise await missing from put target eval |
|
||||
| `hs-upstream-js` | `can return values to _hyperscript` | JS block return value not threaded back |
|
||||
| `hs-upstream-js` | `can do both of the above` | Same |
|
||||
| `hs-upstream-js` | `handles rejected promises without hanging` | Rejected promise in js block uncaught |
|
||||
| `hs-upstream-set` | `set waits on promises` | Same as put |
|
||||
| `hs-upstream-set` | `can set into indirect style ref 3` | Indirect style ref path bug |
|
||||
| `hs-upstream-hide` | `retain original display` | `none` vs `block` display tracking |
|
||||
| `hs-upstream-toggle` | `toggle for fixed time` | Timed toggle assertion timing |
|
||||
| `hs-upstream-transition` | `initial value` | `initial` keyword not restoring computed value |
|
||||
| `hs-upstream-expressions/arrayLiteral` | `objects with _order` | `_order` internal key leaking into equality check |
|
||||
| `hs-upstream-core/bootstrap` | 4 bugs | Event handler bugs in reinit, cleanup, respond |
|
||||
| `hs-upstream-expressions/closest` | `where clause` | `where` consumed by `closest` instead of outer |
|
||||
| `hs-upstream-core/scoping` | 2 bugs | Pseudo-possessive, built-in variable clash |
|
||||
|
||||
**Estimate:** +12 once individually triaged.
|
||||
|
||||
---
|
||||
|
||||
## Group 12 — Formerly "hard floor" — now in scope
|
||||
|
||||
Initial assessment was wrong — these are medium difficulty, not genuinely hard. All 16 are worth attempting.
|
||||
|
||||
| Suite | Tests | Actual difficulty | What's needed |
|
||||
|-------|------:|-------------------|---------------|
|
||||
| `hs-upstream-breakpoint` | 2 | **Trivial** | No-op parser command + generator translation. Design: `plans/designs/f-breakpoint.md` |
|
||||
| `hs-upstream-expressions/logicalOperator` (unparenthesized error) | 2 | Low | Parser strictness: `1 + 2 + 3` should throw "ambiguous operator precedence" |
|
||||
| `hs-upstream-core/security` | 1 | Medium | `_hyperscript.config.disableScripting = true` guard at `hs-activate!` time |
|
||||
| `hs-upstream-expressions/asExpression` (Date, custom dynamic) | 3 | Medium | `as a Date` → `new Date(val)`; custom converters via `_hyperscript.addType` registry |
|
||||
| `hs-upstream-on` (remaining skip-list) | ~8 | Medium | Script tag reinit (MutationObserver on `<script>` changes); behavior isolation queue |
|
||||
|
||||
**Breakpoint** — both tests just check that `breakpoint` *parses* without throwing. No devtools. See design doc.
|
||||
|
||||
**Security** — test creates a div with `_="on click add .foo"`, activates it, clicks, asserts `.foo` is NOT added. This is a `disableScripting` config flag: when set, `hs-activate!` skips initialisation. One guard at activation entry.
|
||||
|
||||
**Unparenthesized operator error** — `1 + 2 + 3` in HS is ambiguous (no defined associativity for chained operators). Parser should throw a parse error rather than silently picking left-associativity. Needs a "multiple operators at same precedence level" check after parsing a binary expression.
|
||||
|
||||
**Sequence these last** — after groups 1–11 are done. Breakpoint is a 30-min job and should be pulled into the quick-wins batch.
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
| Group | Tests | Difficulty | Design doc |
|
||||
|-------|------:|-----------|-----------|
|
||||
| 0 — Bucket E payoff | ~47 | Free | (E branches) |
|
||||
| 1 — Null safety | +7 | Low | `f1-null-safety.md` |
|
||||
| 2 — `tell` semantics | +3 | Low | `f2-tell.md` |
|
||||
| 3 — `on` event features | +19 | Medium | (TBD) |
|
||||
| 4 — MutationObserver | +10 | Medium | (TBD) |
|
||||
| 5 — Cookie API | +5 | Low | `f5-cookies.md` |
|
||||
| 6 — Block literals | +4 | Medium | (TBD) |
|
||||
| 7 — Async logical ops | +5 | Medium | (TBD) |
|
||||
| 8 — evalStatically | +3 | Low | `f8-eval-statically.md` |
|
||||
| 9 — Parse error API | +6 | Medium | (TBD) |
|
||||
| 10 — `as` conversions | +8 | Medium | (TBD) |
|
||||
| 11 — Misc bugs | +12 | Low–Medium | (TBD) |
|
||||
| 12 — Breakpoint | +2 | Trivial | `f-breakpoint.md` |
|
||||
| 12 — Security config | +1 | Medium | (TBD) |
|
||||
| 12 — Unparenthesized op error | +2 | Low | (TBD) |
|
||||
| 12 — `as` Date + custom | +3 | Medium | (TBD) |
|
||||
| 12 — `on` remaining | +8 | Medium | (TBD) |
|
||||
| **Total recoverable** | **~145** | | |
|
||||
|
||||
## Group 13 — Step limit + `meta.caller` (+5 → 100%)
|
||||
|
||||
Design doc: `plans/designs/f13-step-limit-and-meta.md`
|
||||
|
||||
| Test | Failure | Fix |
|
||||
|------|---------|-----|
|
||||
| `repeat forever works` (×2) | Step limit — loop terminates in 5 iterations but two compilation warm-up guards eat the budget first | Raise `HS_STEP_LIMIT` to 2,000,000 in `hs-run-filtered.js` |
|
||||
| `hypertrace is reasonable` | Step limit — trace recorder may be on globally inflating step count | Raise step limit; disable global trace if on |
|
||||
| `query template returns values` | Step limit (37s) — CSS template query `<${"p"}/>` may rebuild on every call | Raise step limit; cache compiled template query if still slow |
|
||||
| `has proper stack from event handler` | Wrong value — `meta.caller.meta.feature.type` returns `""` instead of `"onFeature"` | Implement `meta` dict in `def` function call scope; wire `{:feature {:type "onFeature"}}` into event handler contexts |
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
| Group | Tests | Difficulty | Design doc |
|
||||
|-------|------:|-----------|-----------|
|
||||
| 0 — Bucket E payoff | ~47 | Free | (E branches) |
|
||||
| 1 — Null safety | +7 | Low | `f1-null-safety.md` |
|
||||
| 2 — `tell` semantics | +3 | Low | `f2-tell.md` |
|
||||
| 3 — `on` event features | +19 | Medium | (TBD) |
|
||||
| 4 — MutationObserver | +10 | Medium | (TBD) |
|
||||
| 5 — Cookie API | +5 | Low | `f5-cookies.md` |
|
||||
| 6 — Block literals | +4 | Medium | (TBD) |
|
||||
| 7 — Async logical ops | +5 | Medium | (TBD) |
|
||||
| 8 — evalStatically | +3 | Low | `f8-eval-statically.md` |
|
||||
| 9 — Parse error API | +6 | Medium | (TBD) |
|
||||
| 10 — `as` conversions | +8 | Medium | (TBD) |
|
||||
| 11 — Misc bugs | +12 | Low–Medium | (TBD) |
|
||||
| 12 — Breakpoint | +2 | Trivial | `f-breakpoint.md` |
|
||||
| 12 — Security config | +1 | Medium | (TBD) |
|
||||
| 12 — Unparenthesized op error | +2 | Low | (TBD) |
|
||||
| 12 — `as` Date + custom | +3 | Medium | (TBD) |
|
||||
| 12 — `on` remaining | +8 | Medium | (TBD) |
|
||||
| 13 — Step limit + meta.caller | +5 | Low | `f13-step-limit-and-meta.md` |
|
||||
| **Total recoverable** | **~150** | | |
|
||||
|
||||
**Projected ceiling: ~1299 + 47 + 150 = 1496/1496 = 100%**
|
||||
|
||||
---
|
||||
|
||||
## Suggested sequencing for Bucket F loop
|
||||
|
||||
1. Groups 1, 2, 5, 8 + breakpoint — quick wins, design docs ready, ~20 tests
|
||||
2. Groups 11 misc bugs — isolate and fix one suite at a time
|
||||
3. Group 9 parse error API — hooks into bootstrap, needs care
|
||||
4. Groups 3a, 3b (on-count + finally) — medium, self-contained
|
||||
5. Groups 4 (MutationObserver) + 3c/3d/3e (elsewhere, exceptions, cleanup)
|
||||
6. Groups 6, 7 (block literals, async logical ops) — new syntax
|
||||
7. Group 10 (as conversions) — additive, low regression risk
|
||||
8. Group 12 remainder — security config, unparenthesized op error, as-Date, on remaining
|
||||
|
||||
Each group should get a design doc in `plans/designs/f<N>-<name>.md` before implementation starts.
|
||||
229
plans/koka-on-sx.md
Normal file
229
plans/koka-on-sx.md
Normal file
@@ -0,0 +1,229 @@
|
||||
# Koka-on-SX: Koka on the CEK/VM
|
||||
|
||||
Implement a Koka interpreter on SX. The unique angle: Koka's algebraic effects and
|
||||
handlers map directly onto SX's `perform`/`cek-resume` machinery — this is the language
|
||||
that will stress-test whether SX's effect system is principled enough, and expose any
|
||||
gaps. Every other language in the set works around effects ad-hoc; Koka makes them the
|
||||
primary abstraction.
|
||||
|
||||
End-state goal: **core Koka programs running on the SX CEK evaluator**, with algebraic
|
||||
effect handlers wired through `perform`/`cek-resume`. Not a full Koka compiler — no type
|
||||
inference, no row-polymorphic effect types, no LLVM backend — but a faithful runtime for
|
||||
idiomatic Koka programs.
|
||||
|
||||
## What Koka adds that nothing else covers
|
||||
|
||||
- **Structured effect declarations**: `effect state<s> { fun get() : s; fun set(s) : () }`
|
||||
— named, typed effect operations, not just untyped `perform` tokens
|
||||
- **Resumable handlers**: `handler { return(x) -> x; get() -> resume(0); set(x) -> resume(()) }`
|
||||
— multi-shot continuations, handlers as first-class values
|
||||
- **Effect polymorphism**: functions declare their effect set (`a -> <state<int>,console> b`)
|
||||
— exposes whether SX can track which effects are in scope
|
||||
- **Tail-resumptive handlers**: most practical handlers resume exactly once, which should
|
||||
be optimisable — tests whether the CEK machine can detect and collapse this
|
||||
- **Algebraic data types as the foundation**: `type maybe<a> { Nothing; Just(value: a) }`
|
||||
— exercises the Phase 6 ADT primitive directly
|
||||
|
||||
## Ground rules
|
||||
|
||||
- **Scope:** only touch `lib/koka/**` and `plans/koka-on-sx.md`. Do **not** edit `spec/`,
|
||||
`hosts/`, `shared/`, or other `lib/<lang>/`.
|
||||
- **Shared-file issues** go under "Blockers" below with a minimal repro; do not fix here.
|
||||
- **SX files:** use `sx-tree` MCP tools only.
|
||||
- **Architecture:** Koka source → Koka AST → interpret directly via CEK. No separate
|
||||
Koka evaluator — host the semantics in SX, run on the existing CEK machine.
|
||||
- **Effect types:** defer type inference entirely. Track effects at runtime only — an
|
||||
unhandled effect at the top level raises a runtime error, not a type error.
|
||||
- **Commits:** one feature per commit. Keep `## Progress log` updated and tick boxes.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Koka source text
|
||||
│
|
||||
▼
|
||||
lib/koka/tokenizer.sx — keywords, operators, indent-sensitivity, type-level syntax
|
||||
│
|
||||
▼
|
||||
lib/koka/parser.sx — Koka AST: fun, val, effect, handler, with, match, resume,
|
||||
│ return clause, ADT definitions, basic type expressions
|
||||
▼
|
||||
lib/koka/eval.sx — Koka AST → CEK evaluation via SX primitives:
|
||||
│ ADT (define-type/match from Phase 6)
|
||||
│ Effects (perform/cek-resume from spec/evaluator.sx)
|
||||
│ Coroutines optional (Phase 4 primitives)
|
||||
▼
|
||||
SX CEK evaluator (both JS and OCaml hosts)
|
||||
```
|
||||
|
||||
Key semantic mappings:
|
||||
|
||||
| Koka construct | SX mapping |
|
||||
|---------------|-----------|
|
||||
| `fun f(x) body` | `(define (f x) body)` |
|
||||
| `val x = expr` | `(let ((x expr)) ...)` |
|
||||
| `effect E { fun op() : t }` | register effect tag `E/op` in effect env |
|
||||
| `op()` inside handler scope | `(perform (list "E" "op" args))` |
|
||||
| `handler { return(x)->e; op()->resume(v) }` | `(guard ...)` + `cek-resume` |
|
||||
| `with handler { body }` | install handler for duration of body |
|
||||
| `match x { Nothing -> e1; Just(v) -> e2 }` | SX `(match x ...)` via Phase 6 ADT |
|
||||
| `type maybe<a> { Nothing; Just(value:a) }` | `(define-type maybe (Nothing) (Just value))` |
|
||||
| `resume(v)` in handler | `(cek-resume k v)` where k is captured continuation |
|
||||
| `return(x) -> expr` | final-value clause when no effect fires |
|
||||
|
||||
## Koka semantics in brief
|
||||
|
||||
### Effects and handlers
|
||||
|
||||
```koka
|
||||
effect console
|
||||
fun println(s : string) : ()
|
||||
|
||||
fun greet(name : string) : <console> ()
|
||||
println("Hello, " ++ name)
|
||||
|
||||
fun main()
|
||||
with handler
|
||||
return(x) -> x
|
||||
println(s) -> { print-string(s ++ "\n"); resume(()) }
|
||||
greet("world")
|
||||
```
|
||||
|
||||
- `effect console` declares an effect with one operation `println`
|
||||
- `greet` uses `console` — any call to `println` inside will look up the nearest
|
||||
enclosing handler
|
||||
- `with handler { ... }` installs a handler; `resume(())` continues the suspended
|
||||
computation
|
||||
|
||||
### Multi-shot resumption
|
||||
|
||||
```koka
|
||||
effect choice
|
||||
fun choose() : bool
|
||||
|
||||
fun xor(p : bool, q : bool) : <choice> bool
|
||||
val a = choose()
|
||||
val b = choose()
|
||||
(a || b) && !(a && b)
|
||||
|
||||
fun all-results()
|
||||
with handler
|
||||
return(x) -> [x]
|
||||
choose() -> resume(True) ++ resume(False)
|
||||
xor(True, False)
|
||||
// => [True, True, False, True]
|
||||
```
|
||||
|
||||
This is the test that exposes whether `cek-resume` supports multi-shot (calling the
|
||||
same continuation twice). SX's delimited continuations do support this — Koka will
|
||||
verify it end-to-end.
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — Tokenizer + parser (core expressions)
|
||||
|
||||
- [ ] Tokenizer: keywords (`fun`, `val`, `effect`, `handler`, `with`, `match`, `return`,
|
||||
`resume`, `type`, `alias`, `if`, `then`, `else`, `fn`), operators (`++`, `->`,
|
||||
`|>`, `:`, `<`, `>`, `,`), identifiers, numbers, strings, booleans
|
||||
- [ ] Parser — expressions:
|
||||
- literals: int, float, bool (`True`/`False`), string
|
||||
- `val x = e` bindings
|
||||
- `fun f(x, y) body` definitions
|
||||
- `if c then e1 else e2`
|
||||
- `match x { Pat -> e; ... }`
|
||||
- lambda `fn(x) -> e`
|
||||
- function application `f(x, y)`
|
||||
- infix operators: `++`, `+`, `-`, `*`, `/`, `==`, `!=`, `<`, `>`, `&&`, `||`
|
||||
- pipe `|>`: `x |> f` = `f(x)`
|
||||
- [ ] Tests: `lib/koka/tests/parse.sx` — 40+ parse round-trip tests
|
||||
|
||||
### Phase 2 — ADT definitions + match
|
||||
|
||||
- [ ] Parser: `type name<a> { Ctor1; Ctor2(field: t); ... }` declarations
|
||||
- [ ] Eval: map to SX `define-type` + `match` (requires Phase 6 primitives)
|
||||
- [ ] Built-in: `maybe<a>` (Nothing / Just), `result<a,e>` (Ok / Error), `list<a>` (Nil / Cons)
|
||||
- [ ] Tests: ADT construction, matching, nested patterns — 25+ tests
|
||||
|
||||
### Phase 3 — Core evaluator
|
||||
|
||||
- [ ] `koka-eval` entry: walks Koka AST, evaluates in SX env
|
||||
- [ ] Arithmetic, string `++`, comparison, boolean ops
|
||||
- [ ] `val`/`let` binding
|
||||
- [ ] Function definitions and application (first-class functions)
|
||||
- [ ] `if`/`then`/`else`
|
||||
- [ ] `match` with constructor, literal, variable, wildcard patterns
|
||||
- [ ] Basic list ops: `map`, `filter`, `foldl`, `length`, `head`, `tail`
|
||||
- [ ] Tests: `lib/koka/tests/eval.sx` — 40+ tests, pure expressions only
|
||||
|
||||
### Phase 4 — Effect system
|
||||
|
||||
- [ ] Effect declaration: `(koka-declare-effect! "console" (list "println"))`
|
||||
registers operations in a global effect registry
|
||||
- [ ] Effect operation call: when `println(s)` is evaluated inside a handler scope,
|
||||
emit `(perform (list :effect "console" :op "println" :args (list s)))`
|
||||
- [ ] Handler installation: `with handler { return(x)->e; println(s)->resume(v) }`
|
||||
installs a `guard`-like frame that catches `perform` signals matching the effect,
|
||||
binds arguments, and exposes `resume` as a callable that invokes `cek-resume`
|
||||
- [ ] `resume(v)`: calls `(cek-resume captured-k v)` where `captured-k` is the
|
||||
continuation captured at the `perform` point
|
||||
- [ ] `return(x) -> e` clause: handles the normal return value when no effect fires
|
||||
- [ ] Tests: `lib/koka/tests/effects.sx` — 30+ tests:
|
||||
- basic handler (state, console, exception)
|
||||
- unhandled effect → runtime error
|
||||
- nested handlers (inner shadows outer)
|
||||
- multi-shot resumption (choice effect — the key test)
|
||||
- tail-resumptive handler (resumes exactly once — verify no extra allocation)
|
||||
|
||||
### Phase 5 — Standard effect library
|
||||
|
||||
- [ ] `console` effect: `println`, `print`, `readline` (mock)
|
||||
- [ ] `exn` effect: `throw`, `catch` wrappers
|
||||
- [ ] `state<s>` effect: `get`, `set`, `modify`
|
||||
- [ ] `async` effect: `await` mapped to SX `perform` IO suspension
|
||||
- [ ] Tests: programs using each stdlib effect — 20+ tests
|
||||
|
||||
### Phase 6 — Classic Koka programs as integration tests
|
||||
|
||||
- [ ] `counter.koka` — stateful counter via state effect
|
||||
- [ ] `choice.koka` — multi-shot choice generating all results
|
||||
- [ ] `iterator.koka` — yield-based iteration via a custom effect
|
||||
- [ ] `exception.koka` — structured exception handling
|
||||
- [ ] `coroutine.koka` — producer/consumer via two interleaved effects
|
||||
- [ ] Each as a self-contained test in `lib/koka/tests/programs.sx`
|
||||
|
||||
## Key blockers / dependencies
|
||||
|
||||
- **Phase 6 ADT primitive** (`define-type`/`match`) — required before Phase 2.
|
||||
Track: `plans/agent-briefings/primitives-loop.md` Phase 6.
|
||||
- **Multi-shot continuations** — `cek-resume` must support calling the same
|
||||
continuation multiple times. Verify with: `(let ((k #f)) (perform 'x) ...)` called
|
||||
twice. This should already work given the multi-shot delimited continuation work.
|
||||
- **Effect handler stack** — SX's `guard` is not quite the right primitive for
|
||||
deep-handler semantics. May need `(with-handler effect-tag handler-fn body)` as a
|
||||
new evaluator form, or can be emulated via `guard` + `perform` reshaping.
|
||||
|
||||
## Comparison to other languages in the set
|
||||
|
||||
| Language | Effect model |
|
||||
|----------|-------------|
|
||||
| Lua | none (errors only) |
|
||||
| Prolog | none (cuts only) |
|
||||
| Erlang | message-passing (not algebraic) |
|
||||
| Haskell | IO monad (monadic, not algebraic) |
|
||||
| JS | promise/async-await (one-shot) |
|
||||
| Ruby | exceptions + fibers |
|
||||
| **Koka** | **algebraic effects + multi-shot handlers** |
|
||||
|
||||
Koka is the only language that uses SX's effect system as its *primary* computational
|
||||
model. It will expose whether `perform`/`cek-resume` is sufficient or needs typed effect
|
||||
tagging, scoping rules, and a handler stack distinct from `guard`.
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
- _(none yet)_
|
||||
|
||||
## Blockers
|
||||
|
||||
- ADT primitive (Phase 6 of primitives-loop) must land before Phase 2 starts.
|
||||
138
plans/minikanren-on-sx.md
Normal file
138
plans/minikanren-on-sx.md
Normal file
@@ -0,0 +1,138 @@
|
||||
# miniKanren-on-SX: relational programming on the CEK/VM
|
||||
|
||||
miniKanren is not a language to parse — it is an **embedded DSL** implemented as a library
|
||||
of SX functions. No tokenizer, no transpiler. The entire system is a set of `define` forms
|
||||
in `lib/minikanren/`. Programs are SX expressions using the miniKanren API.
|
||||
|
||||
The unique angle: SX's delimited continuation machinery (`perform`/`cek-resume`, call/cc)
|
||||
maps almost perfectly to the search monad miniKanren needs. Backtracking is cooperative
|
||||
suspension, not a separate trail machine. This is the cleanest possible host for miniKanren.
|
||||
|
||||
End-state goal: **full core miniKanren** (`run`, `fresh`, `==`, `conde`, `condu`, `onceo`,
|
||||
`project`, `matche`) + **core.logic-style relations** (`appendo`, `membero`, `listo`,
|
||||
`numbero`, etc.) + **arithmetic constraints** (`fd` domain, `CLP(FD)` subset).
|
||||
|
||||
## Ground rules
|
||||
|
||||
- **Scope:** only touch `lib/minikanren/**` and `plans/minikanren-on-sx.md`. Do **not**
|
||||
edit `spec/`, `hosts/`, `shared/`, or other `lib/<lang>/`.
|
||||
- **Shared-file issues** go under "Blockers" below with a minimal repro; do not fix here.
|
||||
- **SX files:** use `sx-tree` MCP tools only.
|
||||
- **Architecture:** pure library — no source parser. Programs are written in SX using the API.
|
||||
- **Reference:** *The Reasoned Schemer* (Friedman/Byrd/Kiselyov) + Byrd's dissertation.
|
||||
- **Commits:** one feature per commit. Keep `## Progress log` updated and tick boxes.
|
||||
|
||||
## Architecture sketch
|
||||
|
||||
```
|
||||
SX program using miniKanren API
|
||||
│
|
||||
├── lib/minikanren/unify.sx — terms, variables, walk, unification, occurs check
|
||||
├── lib/minikanren/substitution.sx — substitution as association list / hash table
|
||||
├── lib/minikanren/stream.sx — lazy streams of substitutions (via delay/force)
|
||||
├── lib/minikanren/goals.sx — == / fresh / conde / condu / onceo / project / matche
|
||||
├── lib/minikanren/run.sx — run* / run n — drive the search, extract answers
|
||||
├── lib/minikanren/relations.sx — standard relations: appendo, membero, listo, etc.
|
||||
└── lib/minikanren/clpfd.sx — arithmetic constraints (CLP(FD) subset)
|
||||
```
|
||||
|
||||
Key semantic mappings:
|
||||
- **Logic variable** → SX vector of length 1 (mutable box); `make-var` creates fresh one;
|
||||
`walk` follows the substitution chain
|
||||
- **Substitution** → SX association list (or hash table for performance) mapping var → term
|
||||
- **Stream of substitutions** → lazy list using `delay`/`force` (Phase 9 of primitives)
|
||||
- **Goal** → SX function `substitution → stream-of-substitutions`
|
||||
- **`==`** → unifies two terms, extending substitution or failing (empty stream)
|
||||
- **`fresh`** → introduces new logic variables; `(fresh (x y) goal)` → goal with x, y bound
|
||||
- **`conde`** → interleave streams from multiple goal clauses (depth-first with interleaving)
|
||||
- **`run n`** → drive the stream, collect first n substitutions, reify answers
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — variables + unification
|
||||
- [ ] `make-var` → fresh logic variable (unique mutable box)
|
||||
- [ ] `var?` `v` → bool — is this a logic variable?
|
||||
- [ ] `walk` `term` `subst` → follow substitution chain to ground term or unbound var
|
||||
- [ ] `walk*` `term` `subst` → deep walk (recurse into lists/dicts)
|
||||
- [ ] `unify` `u` `v` `subst` → extended substitution or `#f` (failure)
|
||||
Handles: var/var, var/term, term/var, list unification, number/string/symbol equality.
|
||||
No occurs check by default; `unify-check` with occurs check as opt-in.
|
||||
- [ ] Empty substitution `empty-s` = `(list)` (empty assoc list)
|
||||
- [ ] Tests in `lib/minikanren/tests/unify.sx`: ground terms, vars, lists, failure, occurs
|
||||
|
||||
### Phase 2 — streams + goals
|
||||
- [ ] Stream type: `mzero` (empty stream = `nil`), `unit s` (singleton = `(list s)`),
|
||||
`mplus` (interleave two streams), `bind` (apply goal to stream)
|
||||
- [ ] Lazy streams via `delay`/`force` — mature pairs for depth-first, immature for lazy
|
||||
- [ ] `==` goal: `(fn (s) (let ((s2 (unify u v s))) (if s2 (unit s2) mzero)))`
|
||||
- [ ] `succeed` / `fail` — trivial goals
|
||||
- [ ] `fresh` — `(fn (f) (fn (s) ((f (make-var)) s)))` — introduces one var; `fresh*` for many
|
||||
- [ ] `conde` — interleaving disjunction of goal lists
|
||||
- [ ] `condu` — committed choice (soft-cut): only explores first successful clause
|
||||
- [ ] `onceo` — succeeds at most once
|
||||
- [ ] Tests: basic goal composition, backtracking, interleaving
|
||||
|
||||
### Phase 3 — run + reification
|
||||
- [ ] `run*` `goal` → list of all answers (reified)
|
||||
- [ ] `run n` `goal` → list of first n answers
|
||||
- [ ] `reify` `term` `subst` → replace unbound vars with `_0`, `_1`, ... names
|
||||
- [ ] `reify-s` → builds reification substitution for naming unbound vars consistently
|
||||
- [ ] `fresh` with multiple variables: `(fresh (x y z) goal)` sugar
|
||||
- [ ] Query variable conventions: `q` as canonical query variable
|
||||
- [ ] Tests: classic miniKanren programs — `(run* q (== q 1))` → `(1)`,
|
||||
`(run* q (conde ((== q 1)) ((== q 2))))` → `(1 2)`,
|
||||
Peano arithmetic, `appendo` preview
|
||||
|
||||
### Phase 4 — standard relations
|
||||
- [ ] `appendo` `l` `s` `ls` — list append, runs forwards and backwards
|
||||
- [ ] `membero` `x` `l` — x is a member of l
|
||||
- [ ] `listo` `l` — l is a proper list
|
||||
- [ ] `nullo` `l` — l is empty
|
||||
- [ ] `pairo` `p` — p is a pair (cons cell)
|
||||
- [ ] `caro` `p` `a` — car of pair
|
||||
- [ ] `cdro` `p` `d` — cdr of pair
|
||||
- [ ] `conso` `a` `d` `p` — cons
|
||||
- [ ] `firsto` / `resto` — aliases for caro/cdro
|
||||
- [ ] `reverseo` `l` `r` — reverse of list
|
||||
- [ ] `flatteno` `l` `f` — flatten nested lists
|
||||
- [ ] `permuteo` `l` `p` — permutation of list
|
||||
- [ ] `lengtho` `l` `n` — length as a relation (Peano or integer)
|
||||
- [ ] Tests: run each relation forwards and backwards; generate from partial inputs
|
||||
|
||||
### Phase 5 — `project` + `matche` + negation
|
||||
- [ ] `project` `(x ...) body` — access reified values of logic vars inside a goal;
|
||||
escapes to ground values for arithmetic or string ops
|
||||
- [ ] `matche` — pattern matching over logic terms (extension from core.logic)
|
||||
`(matche l ((head . tail) goal) (() goal))`
|
||||
- [ ] `conda` — soft-cut disjunction (like Prolog `->`)
|
||||
- [ ] `condu` — committed choice (already in phase 2; refine semantics here)
|
||||
- [ ] `nafc` — negation as finite failure with constraint
|
||||
- [ ] Tests: Zebra puzzle, N-queens, Sudoku via `project`, family relations via `matche`
|
||||
|
||||
### Phase 6 — arithmetic constraints CLP(FD)
|
||||
- [ ] Finite domain variables: `fd-var` with domain `[lo..hi]`
|
||||
- [ ] `in` `x` `domain` — constrain x to domain
|
||||
- [ ] `fd-eq` `x` `y` — x = y (constraint propagation)
|
||||
- [ ] `fd-neq` `x` `y` — x ≠ y
|
||||
- [ ] `fd-lt` `fd-lte` `fd-gt` `fd-gte` — ordering constraints
|
||||
- [ ] `fd-plus` `x` `y` `z` — x + y = z (constraint)
|
||||
- [ ] `fd-times` `x` `y` `z` — x * y = z
|
||||
- [ ] Arc consistency propagation — when domain narrows, propagate to constrained vars
|
||||
- [ ] Labelling: `fd-run` drives search by splitting domains when propagation stalls
|
||||
- [ ] Tests: send-more-money, N-queens with CLP(FD), map coloring, cryptarithmetic
|
||||
|
||||
### Phase 7 — tabling (memoization of relations)
|
||||
- [ ] `tabled` annotation: memoize calls to a relation using a hash table
|
||||
- [ ] Prevents infinite loops in recursive relations like `patho` on cyclic graphs
|
||||
- [ ] Producer/consumer scheduling for tabled relations (variant of SLG resolution)
|
||||
- [ ] Tests: cyclic graph reachability, mutual recursion, Fibonacci via tabling
|
||||
|
||||
## Blockers
|
||||
|
||||
_(none yet)_
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
_(awaiting phase 1)_
|
||||
315
plans/ocaml-on-sx.md
Normal file
315
plans/ocaml-on-sx.md
Normal file
@@ -0,0 +1,315 @@
|
||||
# OCaml-on-SX: OCaml + ReasonML + Dream on the CEK/VM
|
||||
|
||||
The meta-circular demo: SX's native evaluator is OCaml, so implementing OCaml on top of
|
||||
SX closes the loop — the source language of the host is running inside the host it
|
||||
compiles to. Beyond the elegance, it's practically useful: once OCaml expressions run on
|
||||
the SX CEK/VM you get Dream (a clean OCaml web framework) almost for free, and ReasonML
|
||||
is a syntax variant that shares the same transpiler output.
|
||||
|
||||
End-state goal: **OCaml programs running on the SX CEK/VM**, with enough of the standard
|
||||
library to support Dream's middleware model. Dream-on-SX is the integration target —
|
||||
a `handler`/`middleware`/`router` API that feels idiomatic while running purely in SX.
|
||||
ReasonML (Phase 8) adds an alternative syntax frontend that targets the same transpiler.
|
||||
|
||||
## What this covers that nothing else in the set does
|
||||
|
||||
- **Strict ML semantics** — unlike Haskell, OCaml is call-by-value with explicit `Lazy.t`
|
||||
for laziness. Pattern match is exhaustive. Polymorphic variants. Structural equality.
|
||||
- **First-class modules and functors** — modules as values (phase 4); functors as SX
|
||||
higher-order functions over module records. Unlike Haskell typeclasses, OCaml's module
|
||||
system is explicit and compositional.
|
||||
- **Mutable state without monads** — `ref`, `:=`, `!` are primitives. Arrays. `Hashtbl`.
|
||||
The IO model is direct; `Lwt`/Dream map to `perform`/`cek-resume` for async.
|
||||
- **Dream's composable HTTP model** — `handler = request -> response promise`,
|
||||
`middleware = handler -> handler`. Algebraically clean; `@@` composition maps to SX
|
||||
function composition trivially.
|
||||
- **ReasonML** — same semantics, JS-friendly surface syntax. JSX variant pairs with SX
|
||||
component rendering.
|
||||
|
||||
## Ground rules
|
||||
|
||||
- **Scope:** only touch `lib/ocaml/**`, `lib/dream/**`, `lib/reasonml/**`, and
|
||||
`plans/ocaml-on-sx.md`. Do **not** edit `spec/`, `hosts/`, `shared/`, or other
|
||||
`lib/<lang>/`.
|
||||
- **Shared-file issues** go under "Blockers" below with a minimal repro; do not fix here.
|
||||
- **SX files:** use `sx-tree` MCP tools only.
|
||||
- **Architecture:** OCaml source → AST → SX AST → CEK. No standalone OCaml evaluator.
|
||||
The OCaml AST is walked by an `ocaml-eval` function in SX that produces SX values.
|
||||
- **Type system:** deferred until Phase 5. Phases 1–4 are intentionally untyped —
|
||||
get the evaluator right first, then layer HM inference on top.
|
||||
- **Dream:** implemented as a library in Phase 7; no separate build step. `Dream.run`
|
||||
wraps SX's existing HTTP server machinery via `perform`/`cek-resume`.
|
||||
- **Commits:** one feature per commit. Keep `## Progress log` updated and tick boxes.
|
||||
|
||||
## Architecture sketch
|
||||
|
||||
```
|
||||
OCaml source text
|
||||
│
|
||||
▼
|
||||
lib/ocaml/tokenizer.sx — keywords, operators, string/char literals, comments
|
||||
│
|
||||
▼
|
||||
lib/ocaml/parser.sx — OCaml AST: let/let rec, fun, match, if, begin/end,
|
||||
│ module/struct/functor, type decls, expressions
|
||||
▼
|
||||
lib/ocaml/desugar.sx — surface → core: tuple patterns, or-patterns,
|
||||
│ sequence (;) → (do), when guards, field punning
|
||||
▼
|
||||
lib/ocaml/transpile.sx — OCaml AST → SX AST
|
||||
│
|
||||
▼
|
||||
lib/ocaml/runtime.sx — ADT constructors, module primitives, ref/array ops,
|
||||
│ Stdlib shims, Dream server (phase 7)
|
||||
▼
|
||||
SX CEK evaluator (both JS and OCaml hosts)
|
||||
```
|
||||
|
||||
## Semantic mappings
|
||||
|
||||
| OCaml construct | SX mapping |
|
||||
|----------------|-----------|
|
||||
| `let x = e` (top-level) | `(define x e)` |
|
||||
| `let f x y = e` | `(define (f x y) e)` |
|
||||
| `let rec f x = e` | `(define (f x) e)` — SX define is already recursive |
|
||||
| `fun x -> e` | `(fn (x) e)` |
|
||||
| `e1 \|> f` | `(f e1)` — pipe desugars to reverse application |
|
||||
| `e1; e2` | `(do e1 e2)` |
|
||||
| `begin e1; e2; e3 end` | `(do e1 e2 e3)` |
|
||||
| `if c then e1 else e2` | `(if c e1 e2)` |
|
||||
| `match x with \| P -> e` | `(match x (P e) ...)` via Phase 6 ADT primitive |
|
||||
| `type t = A \| B of int` | `(define-type t (A) (B v))` |
|
||||
| `module M = struct ... end` | SX dict `{:let-bindings ...}` — module as record |
|
||||
| `functor (M : S) -> ...` | `(fn (M) ...)` — functor as SX lambda over module record |
|
||||
| `open M` | inject M's bindings into scope via `env-merge` |
|
||||
| `M.field` | `(get M :field)` |
|
||||
| `{ r with f = v }` | `(dict-set r :f v)` |
|
||||
| `ref x` | `(make-ref x)` — mutable cell |
|
||||
| `!r` | `(deref-ref r)` |
|
||||
| `r := v` | `(set-ref! r v)` |
|
||||
| `(a, b, c)` | tagged list `(:tuple a b c)` |
|
||||
| `[1; 2; 3]` | `(list 1 2 3)` |
|
||||
| `[| 1; 2; 3 |]` | `(make-array 1 2 3)` (Phase 6) |
|
||||
| `try e with \| Ex -> h` | `(guard (fn (ex) h) e)` via SX exception system |
|
||||
| `raise Ex` | `(perform (:raise Ex))` |
|
||||
| `Printf.printf "%d" x` | `(perform (:print (format "%d" x)))` |
|
||||
|
||||
## Dream semantic mappings (Phase 7)
|
||||
|
||||
| Dream construct | SX mapping |
|
||||
|----------------|-----------|
|
||||
| `handler = request -> response promise` | `(fn (req) (perform (:http-respond ...)))` |
|
||||
| `middleware = handler -> handler` | `(fn (next) (fn (req) ...))` |
|
||||
| `Dream.router [routes]` | `(ocaml-dream-router routes)` — dispatch on method+path |
|
||||
| `Dream.get "/path" h` | route record `{:method "GET" :path "/path" :handler h}` |
|
||||
| `Dream.scope "/p" [ms] [rs]` | prefix mount with middleware chain |
|
||||
| `Dream.param req "name"` | path param extracted during routing |
|
||||
| `m1 @@ m2 @@ handler` | `(m1 (m2 handler))` — left-fold composition |
|
||||
| `Dream.session_field req "k"` | `(perform (:session-get req "k"))` |
|
||||
| `Dream.set_session_field req "k" v` | `(perform (:session-set req "k" v))` |
|
||||
| `Dream.flash req` | `(perform (:flash-get req))` |
|
||||
| `Dream.form req` | `(perform (:form-parse req))` — returns Ok/Error ADT |
|
||||
| `Dream.websocket handler` | `(perform (:websocket handler))` |
|
||||
| `Dream.run handler` | starts SX HTTP server with handler as root |
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — Tokenizer + parser
|
||||
|
||||
- [ ] **Tokenizer:** keywords (`let`, `rec`, `in`, `fun`, `function`, `match`, `with`,
|
||||
`type`, `of`, `module`, `struct`, `end`, `functor`, `sig`, `open`, `include`,
|
||||
`if`, `then`, `else`, `begin`, `try`, `exception`, `raise`, `mutable`,
|
||||
`for`, `while`, `do`, `done`, `and`, `as`, `when`), operators (`->`, `|>`,
|
||||
`<|`, `@@`, `@`, `:=`, `!`, `::`, `**`, `:`, `;`, `;;`), identifiers (lower,
|
||||
upper/ctor, labels `~label:`, optional `?label:`), char literals `'c'`,
|
||||
string literals (escaped + heredoc `{|...|}`), int/float literals,
|
||||
line comments `(*` nested block comments `*)`.
|
||||
- [ ] **Parser:** top-level `let`/`let rec`/`type`/`module`/`exception`/`open`/`include`
|
||||
declarations; expressions: literals, identifiers, constructor application,
|
||||
lambda, application (left-assoc), binary ops with precedence table,
|
||||
`if`/`then`/`else`, `match`/`with`, `try`/`with`, `let`/`in`, `begin`/`end`,
|
||||
`fun`/`function`, tuples, list literals, record literals/updates, field access,
|
||||
sequences `;`, unit `()`.
|
||||
- [ ] **Patterns:** constructor, literal, variable, wildcard `_`, tuple, list cons `::`,
|
||||
list literal, record, `as`, or-pattern `P1 | P2`, `when` guard.
|
||||
- [ ] OCaml is **not** indentation-sensitive — no layout algorithm needed.
|
||||
- [ ] Tests in `lib/ocaml/tests/parse.sx` — 50+ round-trip parse tests.
|
||||
|
||||
### Phase 2 — Core evaluator (untyped)
|
||||
|
||||
- [ ] `ocaml-eval` entry: walks OCaml AST, produces SX values.
|
||||
- [ ] `let`/`let rec`/`let ... in` (mutually recursive with `and`).
|
||||
- [ ] Lambda + application (curried by default — auto-curry multi-param defs).
|
||||
- [ ] `fun`/`function` (single-arg lambda with immediate match on arg).
|
||||
- [ ] `if`/`then`/`else`, `begin`/`end`, sequence `;`.
|
||||
- [ ] Arithmetic, comparison, boolean ops, string `^`, `mod`.
|
||||
- [ ] Unit `()` value; `ignore`.
|
||||
- [ ] References: `ref`, `!`, `:=`.
|
||||
- [ ] Mutable record fields.
|
||||
- [ ] `for i = lo to hi do ... done` loop; `while cond do ... done`.
|
||||
- [ ] `try`/`with` — maps to SX `guard`; `raise` via perform.
|
||||
- [ ] Tests in `lib/ocaml/tests/eval.sx` — 50+ tests, pure + imperative.
|
||||
|
||||
### Phase 3 — ADTs + pattern matching
|
||||
|
||||
- [ ] `type` declarations: `type t = A | B of t1 * t2 | C of { x: int }`.
|
||||
- [ ] Constructors as tagged lists: `A` → `(:A)`, `B(1, "x")` → `(:B 1 "x")`.
|
||||
- [ ] `match`/`with`: constructor, literal, variable, wildcard, tuple, list cons/nil,
|
||||
`as` binding, or-patterns, nested patterns, `when` guard.
|
||||
- [ ] Exhaustiveness: runtime error on incomplete match (no compile-time check yet).
|
||||
- [ ] Built-in types: `option` (`None`/`Some`), `result` (`Ok`/`Error`),
|
||||
`list` (nil/cons), `bool`, `unit`, `exn`.
|
||||
- [ ] `exception` declarations; built-in: `Not_found`, `Invalid_argument`,
|
||||
`Failure`, `Match_failure`.
|
||||
- [ ] Polymorphic variants (surface syntax `\`Tag value`; runtime same tagged list).
|
||||
- [ ] Tests in `lib/ocaml/tests/adt.sx` — 40+ tests: ADTs, match, option/result.
|
||||
|
||||
### Phase 4 — Modules + functors
|
||||
|
||||
- [ ] `module M = struct let x = 1 let f y = x + y end` → SX dict `{:x 1 :f <fn>}`.
|
||||
- [ ] `module type S = sig val x : int val f : int -> int end` → interface record
|
||||
(runtime stub; typed checking in Phase 5).
|
||||
- [ ] `module M : S = struct ... end` — coercive sealing (runtime: pass-through).
|
||||
- [ ] `functor (M : S) -> struct ... end` → SX `(fn (M) ...)`.
|
||||
- [ ] `module F = Functor(Base)` — functor application.
|
||||
- [ ] `open M` — merge M's dict into current env (`env-merge`).
|
||||
- [ ] `include M` — same as open at structure level.
|
||||
- [ ] `M.name` — dict get via `:name` key.
|
||||
- [ ] First-class modules (pack/unpack) — deferred to Phase 5.
|
||||
- [ ] Standard module hierarchy: `List`, `Option`, `Result`, `String`, `Char`,
|
||||
`Int`, `Float`, `Bool`, `Unit`, `Printf`, `Format` (stubs, filled in Phase 6).
|
||||
- [ ] Tests in `lib/ocaml/tests/modules.sx` — 30+ tests.
|
||||
|
||||
### Phase 5 — Hindley-Milner type inference
|
||||
|
||||
- [ ] Algorithm W: `gen`/`inst`, `unify`, `infer-expr`, `infer-decl`.
|
||||
- [ ] Type variables: `'a`, `'b`; unification with occur-check.
|
||||
- [ ] Let-polymorphism: generalise at let-bindings.
|
||||
- [ ] ADT types: `type 'a option = None | Some of 'a`.
|
||||
- [ ] Function types, tuple types, record types.
|
||||
- [ ] Type signatures: `val f : int -> int` — verify against inferred type.
|
||||
- [ ] Module type checking: seal against `sig` (Phase 4 stubs become real checks).
|
||||
- [ ] Error reporting: position-tagged errors with expected vs actual types.
|
||||
- [ ] First-class modules: `(module M : S)` pack; `(val m : (module S))` unpack.
|
||||
- [ ] No rank-2 polymorphism, no GADTs (out of scope).
|
||||
- [ ] Tests in `lib/ocaml/tests/types.sx` — 60+ inference tests.
|
||||
|
||||
### Phase 6 — Standard library
|
||||
|
||||
- [ ] `List`: `map`, `filter`, `fold_left`, `fold_right`, `length`, `rev`, `append`,
|
||||
`concat`, `flatten`, `iter`, `iteri`, `mapi`, `for_all`, `exists`, `find`,
|
||||
`find_opt`, `mem`, `assoc`, `assq`, `sort`, `stable_sort`, `nth`, `hd`, `tl`,
|
||||
`init`, `combine`, `split`, `partition`.
|
||||
- [ ] `Option`: `map`, `bind`, `fold`, `get`, `value`, `join`, `iter`, `to_list`,
|
||||
`to_result`, `is_none`, `is_some`.
|
||||
- [ ] `Result`: `map`, `bind`, `fold`, `get_ok`, `get_error`, `map_error`,
|
||||
`to_option`, `is_ok`, `is_error`.
|
||||
- [ ] `String`: `length`, `get`, `sub`, `concat`, `split_on_char`, `trim`,
|
||||
`uppercase_ascii`, `lowercase_ascii`, `contains`, `starts_with`, `ends_with`,
|
||||
`index_opt`, `replace_all` (non-stdlib but needed).
|
||||
- [ ] `Char`: `code`, `chr`, `escaped`, `lowercase_ascii`, `uppercase_ascii`.
|
||||
- [ ] `Int`/`Float`: arithmetic, `to_string`, `of_string_opt`, `min_int`, `max_int`.
|
||||
- [ ] `Hashtbl`: `create`, `add`, `replace`, `find`, `find_opt`, `remove`, `mem`,
|
||||
`iter`, `fold`, `length` — backed by SX mutable dict.
|
||||
- [ ] `Map.Make` functor — balanced BST backed by SX sorted dict.
|
||||
- [ ] `Set.Make` functor.
|
||||
- [ ] `Printf`: `sprintf`, `printf`, `eprintf` — format strings via `(format ...)`.
|
||||
- [ ] `Sys`: `argv`, `getenv_opt`, `getcwd` — via `perform` IO.
|
||||
- [ ] Scoreboard runner: `lib/ocaml/conformance.sh` + `scoreboard.json`.
|
||||
- [ ] Target: 150+ tests across all stdlib modules.
|
||||
|
||||
### Phase 7 — Dream web framework (`lib/dream/`)
|
||||
|
||||
The five types: `request`, `response`, `handler = request -> response`,
|
||||
`middleware = handler -> handler`, `route`. Everything else is a function over these.
|
||||
|
||||
- [ ] **Core types** in `lib/dream/types.sx`: request/response records, route record.
|
||||
- [ ] **Router** in `lib/dream/router.sx`:
|
||||
- `dream-get path handler`, `dream-post path handler`, etc. for all HTTP methods.
|
||||
- `dream-scope prefix middlewares routes` — prefix mount with middleware chain.
|
||||
- `dream-router routes` — dispatch tree, returns handler; no match → 404.
|
||||
- Path param extraction: `:name` segments, `**` wildcard.
|
||||
- `dream-param req name` — retrieve matched path param.
|
||||
- [ ] **Middleware** in `lib/dream/middleware.sx`:
|
||||
- `dream-pipeline middlewares handler` — compose middleware left-to-right.
|
||||
- `dream-no-middleware` — identity.
|
||||
- Logger: `(dream-logger next req)` — logs method, path, status, timing.
|
||||
- Content-type sniffer.
|
||||
- [ ] **Sessions** in `lib/dream/session.sx`:
|
||||
- Cookie-backed session middleware.
|
||||
- `dream-session-field req key`, `dream-set-session-field req key val`.
|
||||
- `dream-invalidate-session req`.
|
||||
- [ ] **Flash messages** in `lib/dream/flash.sx`:
|
||||
- `dream-flash-middleware` — single-request cookie store.
|
||||
- `dream-add-flash-message req category msg`.
|
||||
- `dream-flash-messages req` — returns list of `(category, msg)`.
|
||||
- [ ] **Forms + CSRF** in `lib/dream/form.sx`:
|
||||
- `dream-form req` — returns `(Ok fields)` or `(Err :csrf-token-invalid)`.
|
||||
- `dream-multipart req` — streaming multipart form data.
|
||||
- CSRF middleware: stateless signed tokens, session-scoped.
|
||||
- `dream-csrf-tag req` — returns hidden input fragment for SX templates.
|
||||
- [ ] **WebSockets** in `lib/dream/websocket.sx`:
|
||||
- `dream-websocket handler` — upgrades request; handler `(fn (ws) ...)`.
|
||||
- `dream-send ws msg`, `dream-receive ws`, `dream-close ws`.
|
||||
- [ ] **Static files:** `dream-static root-path` — serves files, ETags, range requests.
|
||||
- [ ] **`dream-run`**: wires root handler into SX's `perform (:http-listen ...)`.
|
||||
- [ ] **Demos** in `lib/dream/demos/`:
|
||||
- `hello.ml` → `lib/dream/demos/hello.sx`: "Hello, World!" route.
|
||||
- `counter.ml` → `lib/dream/demos/counter.sx`: in-memory counter with sessions.
|
||||
- `chat.ml` → `lib/dream/demos/chat.sx`: multi-room WebSocket chat.
|
||||
- `todo.ml` → `lib/dream/demos/todo.sx`: CRUD list with forms + CSRF.
|
||||
- [ ] Tests in `lib/dream/tests/`: routing dispatch, middleware composition,
|
||||
session round-trip, CSRF accept/reject, flash read-after-write — 60+ tests.
|
||||
|
||||
### Phase 8 — ReasonML syntax variant (`lib/reasonml/`)
|
||||
|
||||
ReasonML is OCaml with a JS-friendly surface: semicolons, `let` with `=` everywhere,
|
||||
`=>` for lambdas, `switch` for match, `{j|...|j}` string interpolation. Same semantics —
|
||||
different tokenizer + parser, same `lib/ocaml/transpile.sx` output.
|
||||
|
||||
- [ ] **Tokenizer** in `lib/reasonml/tokenizer.sx`:
|
||||
- `let x = e;` binding syntax (semicolons required).
|
||||
- `(x, y) => e` arrow function syntax.
|
||||
- `switch (x) { | Pat => e | ... }` for match.
|
||||
- JSX: `<Comp prop=val />`, `<div>children</div>`.
|
||||
- String interpolation: `{j|hello $(name)|j}`.
|
||||
- Type annotations: `x : int`, `let f : int => int = x => x + 1`.
|
||||
- [ ] **Parser** in `lib/reasonml/parser.sx`:
|
||||
- Produce same OCaml AST nodes as `lib/ocaml/parser.sx`.
|
||||
- JSX → SX component calls: `<Comp x=1 />` → `(~comp :x 1)`.
|
||||
- Multi-arg functions: `(x, y) => e` → auto-curried pair.
|
||||
- [ ] Shared transpiler: `lib/reasonml/transpile.sx` delegates to
|
||||
`lib/ocaml/transpile.sx` (parse → ReasonML AST → OCaml AST → SX AST).
|
||||
- [ ] Tests in `lib/reasonml/tests/`: tokenizer, parser, eval, JSX — 40+ tests.
|
||||
- [ ] ReasonML Dream demos: translate Phase 7 demos to ReasonML syntax.
|
||||
|
||||
## The meta-circular angle
|
||||
|
||||
SX is bootstrapped to OCaml (`hosts/ocaml/`). Running OCaml inside SX running on OCaml is
|
||||
the "mother tongue" closure: OCaml → SX → OCaml. This means:
|
||||
|
||||
- The OCaml host's native pattern matching and ADTs are exact reference semantics for
|
||||
the SX-level implementation — any mismatch is a bug.
|
||||
- The SX `match` / `define-type` primitives (Phase 6 of the primitives roadmap) were
|
||||
built knowing OCaml was the intended target.
|
||||
- When debugging the transpiler, the OCaml REPL is always available as oracle.
|
||||
- Dream running in SX can serve the sx.rose-ash.com docs site — the framework that
|
||||
describes the runtime it runs on.
|
||||
|
||||
## Key dependencies
|
||||
|
||||
- **Phase 6 ADT primitive** (`define-type`/`match`) — required before Phase 3.
|
||||
- **`perform`/`cek-resume`** IO suspension — required before Phase 7 (Dream async).
|
||||
- **HO forms** and first-class lambdas — already in spec, no blocker.
|
||||
- **Module system** (Phase 4) is independent of type inference (Phase 5) — can overlap.
|
||||
- **ReasonML** (Phase 8) can start once OCaml parser is stable (after Phase 2).
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
_(awaiting phase 1)_
|
||||
|
||||
## Blockers
|
||||
|
||||
_(none yet)_
|
||||
@@ -72,7 +72,7 @@ Representation choices (finalise in phase 1, document here):
|
||||
- [ ] String/atom predicates
|
||||
|
||||
### Phase 5 — Hyperscript integration
|
||||
- [ ] `prolog-query` primitive callable from SX/Hyperscript
|
||||
- [x] `prolog-query` primitive callable from SX/Hyperscript — `prolog(db, goal)` hook in runtime
|
||||
- [ ] Hyperscript DSL: `when allowed(user, :edit) then …`
|
||||
- [ ] Integration suite
|
||||
|
||||
|
||||
Reference in New Issue
Block a user