rose-ash/plans/scheme-on-sx.md

Scheme-on-SX: the reflective-kit second-consumer port

The kernel-on-sx loop documented six reflective API candidates; two are now live (env.sx, class-chain.sx). Three more — evaluator.sx, hygiene.sx, quoting.sx — wait on a guest with operative-free lexical scope, hygienic syntax-transformer infrastructure, and quasiquote. Scheme is exactly that guest.

A correct R7RS-small implementation acts as second consumer for those three kits in one stroke. It also confirms a third independent consumer for env.sx (after Kernel + Tcl + Smalltalk), and a candidate fourth consumer for class-chain.sx (Scheme's record types have parent fields — though OO is non-core in Scheme so the fit is weaker).

Strategic note on combiner.sx

Scheme has no fexprs. combiner.sx's applicative/operative split is Kernel-specific machinery. Scheme is not a second consumer for combiner.sx — that file stays Kernel-only until a Maru, Klisp, or CL-fexpr port arrives. The current session's earlier claim that Scheme "unlocks four more reflective kits" was over-counted; the correct number is three.

Scope decisions

• Target dialect: R7RS-small. Source-only — no images, no FFI, no C extensions, no JIT.
• Numbers: integers + floats. Rationals optional (defer to phase N+1). Complex out.
• Tail-call optimisation: required. Implemented via the existing SX CEK machinery — call recursion in the evaluator uses iterative cek-call rather than host recursion.
• Continuations: call/cc required for R7RS. Use SX's call/cc primitive directly.
• Hygienic macros: syntax-rules required. syntax-case deferred.
• Char/string semantics: Unicode codepoints; surface API matches R7RS section 6.
• I/O: minimal stub (display, write, newline, read) on SX's IO surface.
• define-library: required for module testing; implementation reuses SX's define-library if it's exposed, else hand-rolls a flat module registry.

Architecture sketch

lib/scheme/parser.sx     — reader: numbers, strings, symbols, booleans,
                            chars #\c, vectors #(...), dotted-pairs (a . b),
                            quasi-quote sugar, datum comments #;, block
                            comments #| ... |#

lib/scheme/eval.sx       — eval-expr ENV: walks AST. Symbols → env-lookup.
                            Lists → look up head; if syntactic operator
                            (if/lambda/define/set!/quote/quasiquote/
                             let/let*/letrec/begin/cond/case/and/or/when/
                             unless/do), dispatch to native handler. Else
                            apply combiner (always applicative).

                          ENV is `lib/guest/reflective/env.sx` directly
                          — Scheme is the third consumer for env.sx with
                          NO adapter cfg (canonical wire shape).

lib/scheme/runtime.sx    — Standard environment, primitives, R7RS base.
                            Variadic arithmetic, list ops, string ops,
                            char ops, vector ops, define-record-type,
                            syntax-rules, etc.

lib/scheme/tests/        — Standard pattern: parse, eval, lambda+closure,
                            macros (syntax-rules), call/cc, define-library,
                            classic programs (factorial, Y, tree-walking,
                            named let, do-loop), R7RS conformance subset.

Roadmap

Phase 1 — Parser

• Reader for R7RS lexical syntax: integers, floats, strings (with escapes), symbols (extended-identifier-character set), booleans #t/#f/#true/#false, characters #\c #\space #\newline, vectors #(...), dotted pairs (a . b), quote/quasiquote/unquote/unquote-splicing sugar (same reader macros as Kernel).
• Datum comments #;<datum> (skip one whole expression).
• Block comments #| ... |# (nestable).
• Tests in lib/scheme/tests/parse.sx.

Phase 2 — Evaluator + env

• scheme-eval EXPR ENV — primary entry, uses lib/guest/reflective/env.sx directly as the canonical scope chain. Third consumer for env.sx.
• Self-evaluating: numbers, booleans, strings, chars, vectors.
• Symbol lookup → refl-env-lookup-with.
• List → look up head; syntactic operators dispatch natively; otherwise applicative call with evaluated args.
• Tests in lib/scheme/tests/eval.sx.

Phase 3 — Syntactic operators

• if, quote, set!, define (top-level + internal).
• lambda — fixed-arity, rest-arg via dot, multi-body via implicit begin.
• let, let*, letrec, letrec* — including named-let.
• begin — implicit + explicit.
• cond, case, when, unless, and, or, do.
• Tests for each.

Phase 4 — Standard environment

• Variadic + - * / and chained comparison.
• Type predicates (R7RS number?, pair?, null?, symbol?, string?, procedure?, vector?, char?, boolean?).
• List ops: cons car cdr caar cadr ... cddddr (or just a subset), list length reverse append map filter fold-left fold-right for-each.
• String ops: string-length string-ref substring string-append string=? string<? char->integer integer->char.
• Char ops: char->integer integer->char char-alphabetic? char-numeric? etc.
• Vector ops: vector make-vector vector-length vector-ref vector-set! vector->list list->vector.
• I/O: display write newline read.
• Numerical: abs floor ceiling round truncate min max modulo quotient remainder gcd lcm expt.
• Classic programs: factorial, fib, list reversal, tree map.

Phase 5 — call/cc + dynamic-wind

• call-with-current-continuation / call/cc.
• dynamic-wind.
• with-exception-handler, raise, error.
• Tests: escape continuations, multi-shot via call/cc (chosen via host SX call/cc).

Phase 6 — syntax-rules + hygiene

• define-syntax, let-syntax, letrec-syntax.
• syntax-rules pattern matching, ellipsis, template instantiation.
• Hygiene: scope-set / lifted-symbol implementation. Second consumer for lib/guest/reflective/hygiene.sx extraction once that kit's API surface stabilises.
• Tests: hygienic identifier capture, ellipsis patterns, recursive macros.

Phase 7 — Reflection: eval, interaction-environment, etc.

• eval EXPR ENV — applicative form of the evaluator. Second consumer for lib/guest/reflective/evaluator.sx extraction.
• interaction-environment, null-environment, scheme-report-environment.
• environment? predicate.

Phase 8 — define-library + module hygiene

• define-library, import, export.
• cond-expand for feature-flag conditionals.
• Tests: cross-library imports, identifier renaming.

Phase 9 — Records

• define-record-type with constructor/predicate/accessors/mutators.
• Tests: typical record idioms.

Phase 10 — Quasiquote runtime

• Backquote walker with depth tracking. Second consumer for lib/guest/reflective/quoting.sx extraction.
• Tests including nested quasiquote.

Phase 11 — Conformance + scoreboard

• Curated R7RS test slice (Chibi, Larceny, or hand-picked).
• lib/scheme/conformance.sh + scoreboard.
• Drive conformance toward 100% on chosen slice.

Reflective kit consumption — explicit mapping

Kit	When it lands	How Scheme uses it
lib/guest/reflective/env.sx	Phase 2	Direct — canonical wire shape, no cfg needed. Third consumer.
lib/guest/reflective/evaluator.sx	Phase 7 (will trigger the extraction)	Scheme's eval/interaction-environment/null-environment mirror the proposed refl-eval/refl-make-environment/refl-current-env triple. Second consumer → extraction unblocked.
lib/guest/reflective/hygiene.sx	Phase 6	Scheme's hygienic syntax-rules is the canonical implementation of scope sets / lifted symbols. Second consumer for the deferred Shutt-style hygiene work — Scheme's hygiene goes BEYOND Kernel's by-default-static-env-extension into proper scope-set lifting. Drives the deferred research-grade kit.
lib/guest/reflective/quoting.sx	Phase 10	Scheme's backquote walker is structurally identical to Kernel's knl-quasi-walk, with depth tracking added. Second consumer → extraction unblocked.
lib/guest/reflective/combiner.sx	NEVER (no fexprs)	Not applicable. Stays Kernel-only until a fexpr-having consumer arrives.
lib/guest/reflective/short-circuit.sx	Possibly Phase 3	Scheme's and/or are syntactic, not operative; could be second consumer but adapter would need to bridge "macro that short-circuits" vs "operative that short-circuits". Marginal.

Ground rules

• Scope: only lib/scheme/** and plans/scheme-on-sx.md and lib/guest/reflective/** (for extraction work). Don't edit spec/, hosts/, shared/, or other lib/<lang>/ directories.
• Consume: lib/guest/lex.sx (character predicates), lib/guest/reflective/env.sx (scope chain), eventually evaluator.sx/hygiene.sx/quoting.sx once extracted with Scheme as second consumer.
• Commits: one feature per commit. Short factual messages.
• Tests: every phase ends with a test file. Conformance scoreboard at the end.
• Branch: loops/scheme. Worktree pattern (already set up at /root/rose-ash-loops/scheme).
• Substrate gaps: filed to sx-improvements.md, not fixed in this loop.

References

• R7RS-small: https://small.r7rs.org/attachment/r7rs.pdf
• Chibi Scheme — a small, readable R7RS implementation.
• Dybvig, "Three Implementation Models for Scheme" — for the hygiene story.
• Existing kernel-on-sx code in lib/kernel/ — much of the parser, evaluator structure, and env handling carries over near-verbatim because Kernel and Scheme share lexical scope.

Progress log

• 2026-05-14 — Phases 1, 2, 3, 3.5, 4, 5abc, 6ab, 7, 8, 9, 10, 11 landed in one loop session. 296 Scheme tests across 9 suites; ~1830 LoC of substrate. Test runner + scoreboard at lib/scheme/test.sh and lib/scheme/scoreboard.md. Three reflective kits unlocked: env.sx extracted directly as third consumer, evaluator.sx and quoting.sx second-consumer-ready for the kit-extraction commits (kit code is documented in plans/kernel-on-sx.md; Scheme consumer code is in place).

Phase-by-phase outcomes

• Phase 1 (Parser, 62 tests): R7RS lexical syntax with reader macros, three comment flavours (;, #;, #| |#).
• Phase 2 (Eval + env third-consumer, 23 tests): scheme-make-env etc. are thin aliases for refl-env-* from lib/guest/reflective/env.sx. No adapter cfg needed — Scheme uses the canonical wire shape directly.
• Phase 3 (if/define/set!/begin/lambda + closures, 24 tests): factorial 10 → 3628800, counter via closed-over set!, curried lambda.
• Phase 3.5 (let/let*/cond/when/unless/and/or, 21 tests).
• Phase 4 (standard env + set! bugfix, 82 tests): variadic arithmetic, type predicates, list/string/char/vector ops, higher-order combinators. Found and fixed an SX cond multi-expression branch bug affecting set!. Bugfix unblocked 4 silently-failing tests in Phase 3.
• Phase 5a (call/cc, 8 tests): single-shot escape continuations.
• Phase 5b (raise/guard/with-exception-handler/error, 12 tests): catch-once-then-rehandle-outside pattern avoids handler-self-raise loops.
• Phase 5c (dynamic-wind, 5 tests): basic before-thunk-after with raise propagation. call/cc-escape tracking deferred.
• Phase 6a (define-syntax + syntax-rules, 12 tests): pattern matching with literals + pattern variables + list structure; template substitution.
• Phase 6b (syntax-rules ellipsis, 8 tests): tail-rest single-variable form. (my-and 1 2 3) etc. work.
• Phase 7 (eval / interaction-environment, 13 tests): second consumer for evaluator.sx. interaction-environment closes over the env being built, so user-side defines via (eval ... ie) persist across calls.
• Phase 8 (define-library + import, 7 tests): minimal module system. Private definitions stay in library env; only exports are visible after import.
• Phase 9 (define-record-type, 9 tests): tagged-dict records with optional mutators.
• Phase 10 (quasiquote runtime, 10 tests): second consumer for quoting.sx. Identical algorithm to Kernel's knl-quasi-walk — universal across reflective Lisps.
• Phase 11 (test.sh + scoreboard): single-process aggregating runner, scoreboard markdown.

Deferred phases

• Phase 6c — full hygiene. Dybvig-style scope-sets / lifted-symbol algorithm. Would be the second consumer for the deferred lib/guest/reflective/hygiene.sx. Current macros work for common patterns but don't prevent introduced-binding capture. Research-grade work; warrants its own loop iteration.
• Nested quasiquote depth tracking.
• R7RS module rich features (cond-expand, include, import sets like only/except/prefix/rename).
• Dotted-pair (a b . rest) parser syntax + lambda rest-args.
• Full call/cc + dynamic-wind interaction: dynamic-extent re-entry/re-exit tracking.

Chisel ledger update

This Scheme port satisfies the two-consumer rule for three reflective kits documented in the kernel-on-sx loop:

Kit	Status
env.sx	Extracted — Scheme is the third consumer (after Kernel + Tcl/Smalltalk), uses the canonical shape directly with no cfg
evaluator.sx	Second consumer ready — Scheme eval/interaction-environment/null-environment/scheme-report-environment mirror the proposed refl-eval/refl-current-env/refl-make-environment triple
quoting.sx	Second consumer ready — Scheme scm-quasi-walk is structurally identical to Kernel's knl-quasi-walk; the only difference is the unquote keyword names (cfg parameterisation)
hygiene.sx	Still awaiting (needs Phase 6c)
combiner.sx	N/A — Scheme has no fexprs
short-circuit.sx	N/A — Scheme and/or are syntactic, not operative

The kit-extraction commits themselves are follow-on work; this Scheme port is the consumer-side foundation.