coop/rose-ash
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

hs: query targets, prolog hook, loop scripts, new plans, WASM regen

Browse Source 
Hyperscript compiler/runtime:
- query target support in set/fire/put commands
- hs-set-prolog-hook! / hs-prolog-hook / hs-prolog in runtime
- runtime log-capture cleanup

Scripts: sx-loops-up/down, sx-hs-e-up/down, sx-primitives-down
Plans: datalog, elixir, elm, go, koka, minikanren, ocaml, hs-bucket-f,
       designs (breakpoint, null-safety, step-limit, tell, cookies, eval,
       plugin-system)
lib/prolog/hs-bridge.sx: initial hook-based bridge draft
lib/common-lisp/tests/runtime.sx: CL runtime tests

WASM: regenerate sx_browser.bc.js from updated hs sources

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
giles
2026-05-06 09:19:56 +00:00
parent c311d4ebc4
commit 985671cd76
31 changed files with 16041 additions and 7056 deletions
Download Patch File Download Diff File Expand all files Collapse all files

341
plans/designs/hs-plugin-system.md Normal file
View File

@@ -0,0 +1,341 @@
# HyperScript Plugin / Extension System
Post-Bucket-F capability work. No conformance delta on its own — the payoff is
clean architecture for language embeds (Lua, Prolog, Worker runtime) and
alignment with real `_hyperscript`'s extension model.
---
## 1. Motivation
### 1a. Real `_hyperscript` has a plugin API
Stock `_hyperscript` ships a core bundle with feature stubs and a `use(ext)`
hook that loads named extensions at runtime. The worker feature is the canonical
example: the core parser has a stub that errors helpfully; loading the worker
extension replaces the stub with a real implementation.
We currently have no equivalent. New grammar or compiler targets require editing
`parse-feat`'s hardcoded `cond` or `hs-to-sx`'s hardcoded dispatch. This is
fine for conformance work but wrong for language embeds.
### 1b. Ad-hoc hooks are accumulating
`runtime.sx` already has `hs-prolog-hook` / `hs-set-prolog-hook!` / `prolog`
(nodes 140–142) — an informal plugin slot bolted on outside the parser and
compiler. This pattern will repeat for Lua, and again for the Worker runtime.
A proper registry prevents the drift.
### 1c. E39 worker stub is a placeholder
The stub added in E39 (`parse-feat` raises immediately on `"worker"`) was
explicitly designed to be replaced by a real plugin at a single site. This plan
is where that replacement happens.
### 1d. Bucket-F Group 10 needs a converter registry
`as MyType` via registered converter is already in the Bucket-F plan (Group 10).
A `hs-register-converter!` registry is the natural home for it — and the plugin
system is the right time to add registries generally.
---
## 2. Scope
**In scope:**
- Parser feature registry (`parse-feat` dispatch)
- Compiler command registry (`hs-to-sx` dispatch)
- `as` converter registry (`hs-coerce` dispatch)
- Migration of E39 worker stub to use the parser registry
- Migration of `hs-prolog-hook` ad-hoc slot to a proper plugin
- Worker full runtime plugin (first real plugin)
- Lua embed plugin
- Prolog embed plugin
**Out of scope:**
- Changing the test runner or generator
- Any conformance delta (this plan doesn't target failing tests)
- Third-party plugin loading from external URLs (future)
- Hot-reload of plugins (future)
---
## 3. Registry design
Three registries, all SX dicts. Checked before the hardcoded `cond` in each
dispatch. Registration functions defined alongside the registries in their
respective files.
### 3a. Parser feature registry (`lib/hyperscript/parser.sx`)
```lisp
(define _hs-feature-registry (dict))
(define hs-register-feature!
(fn (keyword parse-fn)
(set! _hs-feature-registry
(dict-set _hs-feature-registry keyword parse-fn))))
```
In `parse-feat`, prepend a registry lookup before the existing `cond`:
```lisp
(let ((registered (dict-get _hs-feature-registry val)))
(if registered
(registered) ;; call the registered parse-fn (no args; uses closure over adv!/tp-val etc.)
(cond ;; existing dispatch unchanged below
...)))
```
`parse-fn` is a zero-arg thunk that has access to the parser's internal state
via the same closure that the existing `parse-*` helpers use. Since `parse-feat`
is itself defined inside the big `let` in `hs-parse`, all the parser helpers
(`adv!`, `tp-val`, `tp-typ`, `parse-cmd-list`, etc.) are in scope.
### 3b. Compiler command registry (`lib/hyperscript/compiler.sx`)
```lisp
(define _hs-compiler-registry (dict))
(define hs-register-compiler!
(fn (head compile-fn)
(set! _hs-compiler-registry
(dict-set _hs-compiler-registry (str head) compile-fn))))
```
In `hs-to-sx`, before the existing `cond` on `head`, check the registry:
```lisp
(let ((registered (dict-get _hs-compiler-registry (str head))))
(if registered
(registered ast)
(cond ...)))
```
`compile-fn` receives the full AST node and returns an SX expression.
### 3c. `as` converter registry (`lib/hyperscript/runtime.sx`)
```lisp
(define _hs-converters (dict))
(define hs-register-converter!
(fn (type-name converter-fn)
(set! _hs-converters
(dict-set _hs-converters type-name converter-fn))))
```
In `hs-coerce`, add a registry lookup as the last `cond` clause before the
fallthrough error:
```lisp
((dict-get _hs-converters type-name)
((dict-get _hs-converters type-name) value))
```
This is also the hook that Bucket-F Group 10 (`can accept custom conversions`)
hangs on — so implementing it here kills two birds.
---
## 4. First-party plugins
Each plugin is a `.sx` file in `lib/hyperscript/plugins/`. Plugins call the
registration functions at load time (top-level `do` forms). The host loads
plugins explicitly after the core files.
### 4a. Worker plugin (`lib/hyperscript/plugins/worker.sx`)
**Phase 1 — stub migration (immediate):**
Remove the inline error branch from `parse-feat` (the E39 stub). Replace with:
```lisp
(hs-register-feature! "worker"
(fn ()
(error "worker plugin is not installed — see https://hyperscript.org/features/worker")))
```
This is identical behaviour to E39 but routed through the registry. The stub
lives in the plugin file, not the core parser. No test regression.
**Phase 2 — full runtime:**
Parser: `parse-worker-feat` — consumes `worker <Name> [(<url>*)] <def|js>* end`,
returns `(worker Name urls defs)` AST node.
Compiler: registered under `"worker"` head:
- Emits `(hs-worker-define! "Name" urls defs)` call.
Runtime additions in the plugin file:
- `hs-worker-define!` — creates a `{:_hs-worker true :name N :handle H :exports (...)}` record,
binds it in the HS top-level env under `Name`.
- `hs-method-call` (existing) detects `:_hs-worker` and dispatches via `postMessage`.
- Worker script body compiled to a standalone SX bundle posted to a Blob URL.
- Return values are promise-wrapped; async-transparent via `perform`/IO suspension.
Mock env additions for the test runner: `Worker` constructor + synchronous
message loop for the 7 sibling `test.skip(...)` upstream tests (the ones
deferred in E39).
### 4b. Prolog plugin (`lib/hyperscript/plugins/prolog.sx`)
Replaces the ad-hoc `hs-prolog-hook` in `runtime.sx`.
**Parser:** Register `"prolog"` feature — parses
`prolog(<db-expr>, <goal-expr>)` at feature level (alternative: keep as an
expression, register a compiler extension only).
**Compiler:** Registered under `"prolog"` head — emits `(prolog db goal)`.
**Runtime:** The existing `prolog` function in `runtime.sx` moves here.
`hs-prolog-hook` and `hs-set-prolog-hook!` are removed from `runtime.sx` and
the hook mechanism is replaced by the plugin loading `lib/prolog/runtime.sx`
and wiring the solver directly.
Remove from `runtime.sx` nodes 140–142 once the plugin is live.
### 4c. Lua plugin (`lib/hyperscript/plugins/lua.sx`)
**Parser:** Register `"lua"` feature — parses `lua ... end` block, captures
the body as a raw string.
**Compiler:** Registered under `"lua"` head — emits `(lua-eval <body-string>)`.
**Runtime:** `lua-eval` calls `lib/lua/runtime.sx`'s eval entry point, returns
result as an SX value via `hs-host-to-sx`. Errors surface as HS `catch`-able
exceptions.
This enables inline Lua in HyperScript:
```
on click
lua
return document.title:upper()
end
put it into me
end
```
---
## 5. Load order
```
lib/hyperscript/parser.sx ;; defines _hs-feature-registry, hs-register-feature!
lib/hyperscript/compiler.sx ;; defines _hs-compiler-registry, hs-register-compiler!
lib/hyperscript/runtime.sx ;; defines _hs-converters, hs-register-converter!
lib/hyperscript/plugins/worker.sx
lib/hyperscript/plugins/prolog.sx
lib/hyperscript/plugins/lua.sx
```
The test runner (`tests/hs-run-filtered.js`) loads plugins after core. The
browser WASM bundle includes all three by default (plugins are small; no
reason to lazy-load them).
---
## 6. Migration checklist
The work below is ordered to keep main green at every commit. Each step is
independently committable.
### Step 1 — Registries (infrastructure, no behaviour change)
1. Add `_hs-feature-registry` + `hs-register-feature!` to `parser.sx`.
Thread the registry check into `parse-feat`. No entries yet → behaviour
unchanged.
2. Add `_hs-compiler-registry` + `hs-register-compiler!` to `compiler.sx`.
Thread into `hs-to-sx`. No entries yet → behaviour unchanged.
3. Add `_hs-converters` + `hs-register-converter!` to `runtime.sx`. Thread
into `hs-coerce`. No entries yet → behaviour unchanged.
4. `sx_validate` all three files. Run full HS suite — expect zero regressions.
5. Commit: `HS: plugin registry infrastructure (parser + compiler + converter)`.
### Step 2 — Worker stub migration
6. Create `lib/hyperscript/plugins/worker.sx`. Register the worker stub error.
7. Remove the inline `((= val "worker") ...)` branch from `parse-feat` in
`parser.sx`.
8. Update the test runner to load `worker.sx` after core.
9. Run `HS_SUITE=hs-upstream-worker` — expect 1/1. Run full suite — expect no
regressions.
10. Commit: `HS: migrate E39 worker stub to plugin registry`.
### Step 3 — Prolog plugin
11. Create `lib/hyperscript/plugins/prolog.sx`. Wire to `lib/prolog/runtime.sx`.
12. Remove `hs-prolog-hook`, `hs-set-prolog-hook!`, `prolog` from `runtime.sx`
nodes 140–142.
13. Update test runner to load `prolog.sx`.
14. Validate and run full suite.
15. Commit: `HS: prolog plugin replaces ad-hoc hook`.
### Step 4 — `as` converter registry (bridges Bucket-F Group 10)
16. Confirm `hs-register-converter!` satisfies the Group 10 test
`can accept custom conversions`. If yes, this step may be pulled into
Bucket-F Group 10 instead (no duplication — just move step 3 of §6 there).
17. Commit: `HS: as-converter registry wired into hs-coerce`.
### Step 5 — Lua plugin
18. Create `lib/hyperscript/plugins/lua.sx`.
19. Add `lua-eval` to `runtime.sx` or directly in the plugin file.
20. Parser: `parse-lua-feat` consuming `lua … end`.
21. Compiler: registered `"lua"` head.
22. Write 3–5 tests in `spec/tests/test-hyperscript-lua.sx`:
- Lua returns a string → HS uses it.
- Lua error → HS catch.
- Lua reads a passed argument.
23. Commit: `HS: Lua plugin — inline lua...end blocks`.
### Step 6 — Worker full runtime plugin
24. Extend `worker.sx`: implement `parse-worker-feat`, compiler entry,
`hs-worker-define!`, `hs-method-call` worker branch.
25. Extend test runner: `Worker` constructor + synchronous message loop.
26. Un-skip the 7 sibling worker tests from upstream.
27. Target: 7/7 worker suite.
28. Commit: `HS: Worker plugin full runtime (+7 tests)`.
---
## 7. Risks
- **`parse-feat` closure scope** — `hs-register-feature!` stores parse-fns
that need access to parser-internal helpers (`adv!`, `tp-val`, etc.). These
are only in scope inside `hs-parse`'s big `let`. Two options:
(a) the registry stores fns that receive a parser-context dict as arg, or
(b) the registry is checked *inside* `parse-feat` where helpers are in scope
and fns are zero-arg closures captured at registration time.
Option (b) is simpler but requires plugins to be loaded while the parser
`let` is being evaluated — i.e., plugins must be defined *inside* the parser
file or the context dict must be exposed. **Recommended:** expose a
`_hs-parser-ctx` dict at the module level that parse-fns receive as their
sole argument. This makes the API explicit and plugins independent files.
- **Worker Blob URL in WASM** — `URL.createObjectURL` is available in browsers
but not in the OCaml WASM host. Worker full runtime is browser-only; flag it
with a capability check and graceful fallback.
- **Lua/Prolog mutual recursion** — a Lua block calling back into HS calling
back into Lua is theoretically possible via the IO suspension machinery.
Don't try to support it initially; raise a clear error if detected.
- **Plugin load-order sensitivity** — `hs-register-feature!` must be called
before any source is parsed. If a plugin is loaded lazily (future), a
`worker MyWorker` in the page would hit the stub before the full plugin
registers. Acceptable for now; document that plugins must be loaded at boot.
- **`runtime.sx` cleanup for prolog** — nodes 140–142 are referenced nowhere
else in the codebase (grep confirms). Safe to delete once the plugin is live.
---
## 8. Non-goals
- Runtime `use(ext)` API (JS-style dynamic plugin install) — future.
- Plugin namespacing / versioning — future.
- Any conformance tests other than the 7 worker tests in step 6.
- Changing how the WASM bundle is built or split.