# Kernel-on-SX: first-class everything

The natural successor to SX's recently-completed env-as-value work (sx-improvements Phase 4). Kernel — John Shutt's reformulation of Lisp from his 2010 PhD — pushes *first-class* all the way: environments, evaluators, special forms (operatives), lambda variants are all runtime values, manipulable by programs. SX already has env-as-value; Kernel is what env-as-value looks like *all the way*.

**The chisel:** *reflection*. Every language in the current set treats some part of itself as fixed and ineffable — Common Lisp's special forms, Erlang's process model, OCaml's modules. Kernel reifies more of itself than any other language does. Implementing it stresses the substrate's *self-knowledge*: which parts of evaluation does SX expose to user programs, and which stay opaque?

**What this exposes about the substrate:**
- Whether `eval-expr` can be called as a primitive on user-supplied environments without breaking invariants.
- Whether CEK frames can be reified as values (they currently aren't).
- Whether special-form dispatch can be table-driven and user-extensible at runtime.
- Whether the macro hygiene story extends to Shutt's "hygienic operatives" (operatives that don't capture).

**End-state goal:** Kernel's R-1RK core — `$vau`/`$lambda`/`wrap`/`unwrap`, first-class environments, the applicative–operative distinction, the standard environment, encapsulations.

## Ground rules
- Scope: `lib/kernel/**` and `plans/kernel-on-sx.md` only. Substrate work belongs to `sx-improvements.md` — if a feature is missing, file it there, don't fix from this plan.
- Consumes from `lib/guest/`: `core/lex.sx`, `core/pratt.sx` (s-expression-shaped, minimal demand), `core/ast.sx`, `core/match.sx`.
- **May propose** a new sub-layer `lib/guest/reflective/` — environment reification helpers, applicative-vs-operative dispatch, evaluator continuation protocols. A second consumer would be needed; candidates are a hypothetical "MetaScheme" or a Common-Lisp port that exposes its evaluator.
- Branch: `loops/kernel`. Standard worktree pattern.

## Architecture sketch

```
Kernel source text (S-expression syntax)
    │
    ▼
lib/kernel/parser.sx       — bog-standard s-expr reader
    │
    ▼
lib/kernel/eval.sx         — kernel-eval: walks the AST, threads first-class env
    │                        dispatches to operatives via env-bound bindings, not
    │                        a hardcoded switch
    ▼
lib/kernel/runtime.sx      — applicative/operative tagged values, wrap/unwrap,
    │                        standard environment construction, encapsulations
    ▼
SX CEK evaluator
```

## Semantic mappings

| Kernel construct | SX mapping |
|------------------|-----------|
| `($lambda (x) body)` | applicative: `(make-applicative (fn (x) body))` — args evaluated |
| `($vau (x) e body)` | operative: `(make-operative (fn (x e) body))` — args UN-evaluated, dynamic env passed as `e` |
| `(wrap op)` | applicative wrapping an operative: evaluate args, then call op |
| `(unwrap app)` | get the underlying operative of an applicative |
| `($define! x v)` | operative: bind `x` to `v` in dynamic env |
| `(eval expr env)` | call `kernel-eval` on `expr` in `env` — first-class |
| `(make-environment)` | fresh empty env |
| `(get-current-environment)` | reify the calling env (via SX env-as-value) |
| `($if c t e)` | operative: evaluate `c`, then `t` or `e` in dynamic env |

The whole interesting thing: there are no special forms hardcoded in the evaluator. `$if`, `$define!`, `$lambda` are all *operatives* bound in the standard environment. User code can rebind them. The evaluator is just `lookup-and-call`.

## Roadmap

### Phase 1 — Parser
- [ ] S-expression reader with the standard atoms (number, string, symbol, boolean, nil) and lists.
- [ ] Reader macros optional; defer to Phase 6.
- [ ] Tests in `lib/kernel/tests/parse.sx`.

### Phase 2 — Core evaluator with first-class environments
- [ ] `kernel-eval expr env` — primary entry, walks AST, threads env as a value.
- [ ] Symbol lookup → environment value (using SX env-as-value primitives).
- [ ] List → look up head, dispatch on tag (applicative vs operative).
- [ ] No hardcoded special forms — even `if`/`define`/`lambda` are env-bound.
- [ ] Tests in `lib/kernel/tests/eval.sx`.

### Phase 3 — `$vau` / `$lambda` / `wrap` / `unwrap`
- [ ] Operative tagged value: `{:type :operative :params :env-param :body :static-env}`.
- [ ] Applicative tagged value wraps an operative + the "evaluate args first" contract.
- [ ] `$vau` builds operatives; `$lambda` is `wrap` ∘ `$vau`.
- [ ] `wrap` / `unwrap` round-trip cleanly.
- [ ] Tests: define a custom operative, define a custom applicative on top of it.

### Phase 4 — Standard environment
- [ ] Standard env construction: bind `$if`, `$define!`, `$lambda`, `$vau`, `wrap`, `unwrap`, `eval`, `make-environment`, `get-current-environment`, plus arithmetic and list primitives.
- [ ] Tests: classic Kernel programs (factorial, list operations, environment manipulation).

### Phase 5 — Encapsulations
- [ ] `make-encapsulation-type` returns three operatives: encapsulator, predicate, decapsulator. Standard Kernel idiom for opaque types.
- [ ] Tests: implement promises, streams, or simple modules via encapsulations.

### Phase 6 — Hygienic operatives (Shutt's later work)
- [ ] Operatives that don't capture caller bindings — uses scope sets / frame stamps to track provenance.
- [ ] Bridge to SX's hygienic macro story; possibly extends `lib/guest/reflective/` with hygiene primitives.
- [ ] Tests: write an operative that introduces a binding and verify it doesn't shadow caller's same-named bindings.

### Phase 7 — Propose `lib/guest/reflective/`
- [ ] Once Phase 3 lands and stabilises, identify which env-reification + dispatch primitives are reusable. Candidate API: `make-operative`, `make-applicative`, `with-current-env`, `eval-in-env`.
- [ ] Find a second consumer (Common-Lisp's macro-expansion evaluator? a metacircular Scheme variant? a future plan).
- [ ] Only extract once two consumers exist (per stratification rule).

## lib/guest feedback loop

**Consumes:** `core/lex`, `core/pratt`, `core/ast`, `core/match`.

**Stresses substrate:** env-as-value (Phase 4 of sx-improvements) under heavy use; `eval` as a primitive on user environments; potentially CEK frame reification.

**May propose:** `lib/guest/reflective/` sub-layer — environment manipulation, evaluator-as-value, applicative/operative dispatch protocols.

**What it teaches:** whether SX's recent env-as-value direction generalises to "evaluator-as-value." If Kernel implements cleanly in <2000 lines, env-as-value is real. If it requires substrate fixes at every turn, env-as-value was incomplete and the substrate is telling us what's missing.

## References
- Shutt, "Fexprs as the basis of Lisp function application" (PhD thesis, 2010).
- Kernel Report (R-1RK): https://web.cs.wpi.edu/~jshutt/kernel.html
- Klisp implementation (Andres Navarro) — pragmatic reference.

## Progress log
_(awaiting Phase 1 — depends on stable env-as-value substrate state)_

## Blockers
_(none yet — main risk is substrate gap discovery during Phase 2)_