rose-ash/plans/common-lisp-on-sx.md

Common-Lisp-on-SX: conditions + restarts on delimited continuations

The headline showcase is the condition system. Restarts are resumable exceptions — every other Lisp implementation reinvents this on host-stack unwind tricks. On SX restarts are textbook delimited continuations: signal walks the handler chain; invoke-restart resumes the captured continuation at the restart point. Same delcc primitive that powers Erlang actors, expressed as a different surface.

End-state goal: ANSI Common Lisp subset with a working condition/restart system, CLOS multimethods (with :before/:after/:around), the LOOP macro, packages, and ~150 hand-written + classic programs.

Scope decisions (defaults — override by editing before we spawn)

• Syntax: ANSI Common Lisp surface. Read tables, dispatch macros (#', #(, #\, #:, #x, #b, #o, ratios 1/3).
• Conformance: ANSI X3.226 as a target, not bug-for-bug SBCL/CCL. "Reads like CL, runs like CL."
• Test corpus: custom + a curated slice of ansi-test. Plus classic programs: condition-system demo, restart-driven debugger, multiple-dispatch geometry, LOOP corpus.
• Out of scope: compilation to native, FFI, sockets, threads, MOP class redefinition, full pathname/logical-pathname machinery, structures with :include deep customization.
• Packages: simple — defpackage/in-package/export/use-package/:cl/:cl-user. No nicknames, no shadowing-import edge cases.

Ground rules

• Scope: only touch lib/common-lisp/** and plans/common-lisp-on-sx.md. Don't edit spec/, hosts/, shared/, or any other lib/<lang>/**. CL primitives go in lib/common-lisp/runtime.sx.
• SX files: use sx-tree MCP tools only.
• Commits: one feature per commit. Keep ## Progress log updated and tick roadmap boxes.

Architecture sketch

Common Lisp source
    │
    ▼
lib/common-lisp/reader.sx     — tokenizer + reader (read macros, dispatch chars)
    │
    ▼
lib/common-lisp/parser.sx     — AST: forms, declarations, lambda lists
    │
    ▼
lib/common-lisp/transpile.sx  — AST → SX AST (entry: cl-eval-ast)
    │
    ▼
lib/common-lisp/runtime.sx    — special forms, condition system, CLOS, packages, BIFs

Core mapping:

• Symbol = SX symbol with package prefix; package table is a flat dict.
• Cons cell = SX pair via cons/car/cdr; lists native.
• Multiple values = thread through values/multiple-value-bind; primary-value default for one-context callers.
• Block / return-from = captured continuation; return-from name v invokes the block-named ^k.
• Tagbody / go = each tag is a continuation; go tag invokes it.
• Unwind-protect = scope frame with a cleanup thunk fired on any non-local exit.
• Conditions / restarts = layered handler chain on top of handler-bind + delcc. signal walks handlers; invoke-restart resumes a captured continuation.
• CLOS = generic functions are dispatch tables on argument-class lists; method combination computed lazily; call-next-method is a continuation.
• Macros = SX macros (sentinel-body) — defmacro lowers directly.

Roadmap

Phase 1 — reader + parser

• Tokenizer: symbols (with package qualification pkg:sym / pkg::sym), numbers (int, float, ratio 1/3, #xFF, #b1010, #o17), strings "…" with \ escapes, characters #\Space #\Newline #\a, comments ;, block comments #| … |#
• Reader: list, dotted pair, quote ', function #', quasiquote `, unquote ,, splice ,@, vector #(…), uninterned #:foo, nil/t literals
• Parser: lambda lists with &optional &rest &key &aux &allow-other-keys, defaults, supplied-p variables
• Unit tests in lib/common-lisp/tests/read.sx

Phase 2 — sequential eval + special forms

• cl-eval-ast: quote, if, progn, let, let*, flet, labels, setq, setf (subset), function, lambda, the, locally, eval-when
• block + return-from via captured continuation
• tagbody + go via per-tag continuations
• unwind-protect cleanup frame
• multiple-value-bind, multiple-value-call, multiple-value-prog1, values, nth-value
• defun, defparameter, defvar, defconstant, declaim, proclaim (no-op)
• Dynamic variables — defvar/defparameter produce specials; let rebinds via parameterize-style scope
• 182 tests in lib/common-lisp/tests/eval.sx

Phase 3 — conditions + restarts (THE SHOWCASE)

• define-condition — class hierarchy rooted at condition/error/warning/simple-error/simple-warning/type-error/arithmetic-error/division-by-zero
• signal, error, cerror, warn — all walk the handler chain
• handler-bind — non-unwinding handlers, may decline by returning normally
• handler-case — unwinding handlers (call/cc escape)
• restart-case, with-simple-restart, restart-bind
• find-restart, invoke-restart, compute-restarts
• with-condition-restarts — associate restarts with a specific condition
• invoke-restart-interactively, *break-on-signals*, *debugger-hook* (basic)
• Classic programs in lib/common-lisp/tests/programs/:
  • restart-demo.sx — division with use-zero and retry restarts (7 tests)
  • parse-recover.sx — parser with skipped-token restart (6 tests)
  • interactive-debugger.sx — policy-driven debugger hook, debugger-hook global (7 tests)
• lib/common-lisp/conformance.sh + runner, scoreboard.json + scoreboard.md (363 total tests)

Phase 4 — CLOS

• defclass with :initarg/:initform/:accessor/:reader/:writer/:allocation
• make-instance, slot-value, (setf slot-value), with-slots, with-accessors
• defgeneric with :method-combination (standard, plus +, and, or)
• defmethod with :before / :after / :around qualifiers
• call-next-method (continuation), next-method-p
• class-of, find-class, slot-boundp, change-class (basic)
• Multiple dispatch — method specificity by argument-class precedence list
• Built-in classes registered for tagged values (integer, float, string, symbol, cons, null, t)
• Classic programs:
  • geometry.sx — intersect generic dispatching on (point line), (line line), (line plane) — 12 tests
  • mop-trace.sx — :before + :after printing call trace — 13 tests

Phase 5 — macros + LOOP + reader macros

• defmacro, macrolet, symbol-macrolet, macroexpand-1, macroexpand
• gensym, gentemp
• set-macro-character, set-dispatch-macro-character, get-macro-character
• The LOOP macro — iteration drivers (for … in/across/from/upto/downto/by, while, until, repeat), accumulators (collect, append, nconc, count, sum, maximize, minimize), conditional clauses (if/when/unless/else), termination (finally/thereis/always/never), named blocks
• LOOP test corpus: 27 tests covering all clause types

Phase 6 — packages + stdlib drive

• defpackage, in-package, export, use-package, import, find-package
• Package qualification at the reader level — cl:car, mypkg::internal
• :common-lisp (:cl) and :common-lisp-user (:cl-user) packages
• Sequence functions — mapcar, mapc, mapcan, reduce, find, find-if, position, count, every, some, notany, notevery, remove, remove-if, subst
• List ops — assoc, getf, nth, last, butlast, nthcdr, tailp, ldiff
• String ops — string=, string-upcase, string-downcase, subseq, concatenate
• FORMAT — basic directives ~A, ~S, ~D, ~F, ~%, ~&, ~T, ~{...~} (iteration), ~[...~] (conditional), ~^ (escape), ~P (plural)
• Drive corpus to 200+ green

SX primitive baseline

Use vectors for arrays; numeric tower + rationals for numbers; ADTs for tagged data; coroutines for fibers; string-buffer for mutable string building; bitwise ops for bit manipulation; multiple values for multi-return; promises for lazy evaluation; hash tables for mutable associative storage; sets for O(1) membership; sequence protocol for polymorphic iteration; gensym for unique symbols; char type for characters; string ports

• read/write for reader protocols; regexp for pattern matching; bytevectors for binary data; format for string templating.

Progress log

Newest first.

• 2026-05-05: Phase 6 stdlib — sequence functions (mapc/mapcan/reduce/find/find-if/find-if-not/position/position-if/count/count-if/every/some/notany/notevery/remove/remove-if/remove-if-not/subst/member), list ops (assoc/rassoc/getf/last/butlast/nthcdr/copy-list/list*/caar/cadr/cdar/cddr/caddr/cadddr/pairlis), string ops (subseq/string/char/string-length/string</>), plus coerce/make-list/write-to-string; 44 tests in tests/stdlib.sx; Phase 6 sequence+list+string boxes ticked. Total: 508 tests, 0 failed.

• 2026-05-05: Phase 4 CLOS fully complete — lib/common-lisp/clos.sx (27 forms): clos-class-registry (8 built-in classes), defclass/make-instance/slot-value/slot-boundp/set-slot-value!/find-class/change-class, defgeneric/defmethod with :before/:after/:around, clos-call-generic (standard method combination: sort by specificity, fire befores, call primary chain, fire afters in reverse), call-next-method/next-method-p, with-slots, accessor installation; 41 tests in tests/clos.sx; classic programs geometry.sx (12 tests, multi-dispatch intersect on P/L/Plane) and mop-trace.sx (13 tests, :before/:after tracing). Dynamic variables in eval.sx: cl-apply-dyn saves/restores global bindings around let for specials (cl-mark-special!/cl-special?/cl-dyn-unbound). Key gotchas: qualifier strings are "before"/"after"/"around" (no colon); dict-set pure = assoc; dict->list = (map (fn (k) (list k (get d k))) (keys d)); clos-add-reader-method bootstrapped via set! after defmethod defined; test isolation: use unique var names to avoid y collision. 437 total tests, 0 failed.

• 2026-05-05: Phase 3 fully complete — conformance.sh runner + scoreboard.json/scoreboard.md; 363 total tests across all suites (79 reader, 31 parser, 174 eval, 59 conditions, 7+6+7 classic programs).

• 2026-05-05: Phase 3 complete — cl-debugger-hook/cl-invoke-debugger in runtime.sx (cl-error routes through hook), cl-break-on-signals (fires hook before handlers on type match), cl-invoke-restart-interactively (calls fn with no args); 4 new tests (147 total). Phase 3 all boxes ticked.

• 2026-05-05: Phase 3 interactive-debugger.sx — cl-debugger-hook global, cl-invoke-debugger, cl-error-with-debugger, make-policy-debugger; 7 tests (143 total). Tests wired into test.sh program suite runner. Phase 3 condition core complete.

• 2026-05-05: Phase 3 classic programs — tests/programs/restart-demo.sx (7 tests: safe-divide with use-zero + retry restarts) and tests/programs/parse-recover.sx (6 tests: token parser with skip-token + use-zero restarts, handler-case abort). Key gotcha: use = not equal? for list comparison in sx_server.

• 2026-05-05: Phase 3 conditions + restarts — cl-condition-classes hierarchy (15 types), cl-condition?/cl-condition-of-type?, cl-make-condition, cl-define-condition, cl-signal/cl-error/cl-warn/cl-cerror, cl-handler-bind (non-unwinding), cl-handler-case (call/cc escape), cl-restart-case/cl-with-simple-restart, cl-find-restart/cl-invoke-restart/cl-compute-restarts, cl-with-condition-restarts; 55 new tests in tests/conditions.sx (123 total runtime tests). Key gotcha: cl-condition-classes must be captured at define-time via let in cl-condition-of-type? — free-variable lookup at call-time fails through env_merge parent chain.

• 2026-05-05: multiple values — VALUES returns {:cl-type "mv"} wrapper for 2+ values; cl-mv-primary/cl-mv-vals helpers; MULTIPLE-VALUE-BIND binds vars to value list; MULTIPLE-VALUE-CALL/PROG1/NTH-VALUE; cl-mv-primary applied in IF/AND/OR/COND/cl-call-fn for single-value contexts; 15 new tests (174 eval, 346 total green).

• 2026-05-05: unwind-protect — cl-eval-unwind-protect: eval protected form, run cleanup with for-each (discards results, preserves original sentinel), return original result; 8 new tests (159 eval, 331 total green).

• 2026-05-05: tagbody + go — cl-go-tag? sentinel; cl-eval-tagbody runs body with tag-index map (keys str-normalised for integer tags); go-tag propagation in cl-eval-body alongside block-return; 11 new tests (151 eval, 323 total green).

• 2026-05-05: block + return-from — sentinel propagation in cl-eval-body; cl-eval-block catches matching sentinels; BLOCK/RETURN-FROM/RETURN dispatch in cl-eval-list; 13 new tests (140 eval, 312 total green). Parser: CL strings → {:cl-type "string"} dicts.

• 2026-04-25: Phase 2 eval — 127 tests, 299 total green. lib/common-lisp/eval.sx: cl-eval-ast with quote/if/progn/let/let*/flet/labels/setq/setf/function/lambda/the/locally/eval-when; defun/defvar/defparameter/defconstant; built-in arithmetic (+/-/*//, min/max/abs/evenp/oddp), comparisons, predicates, list ops (car/cdr/cons/list/append/reverse/length/nth/first/second/third/rest), string ops, funcall/apply/mapcar. Key gotchas: SX reduce is (reduce fn init list) not (reduce fn list init); CL true literal is t not true; builtins registered in cl-global-env.fns via wrapper dicts for #' syntax.

• 2026-04-25: Phase 1 lambda-list parser — 31 new tests, 172 total green. cl-parse-lambda-list in parser.sx + tests/lambda.sx. Handles &optional/&rest/&body/&key/&aux/&allow-other-keys, defaults, supplied-p. Key gotchas: (when (> (len items) 0) ...) not (when items ...) (empty list is truthy); custom cl-deep= needed for dict/list structural equality in tests.

• 2026-04-25: Phase 1 reader/parser — 62 new tests, 141 total green. lib/common-lisp/parser.sx: cl-read/cl-read-all, lists, dotted pairs, quote/backquote/unquote/splice/#', vectors, #:uninterned, NIL→nil, T→true, reader macro wrappers.

• 2026-04-25: Phase 1 tokenizer — 79 tests green. lib/common-lisp/reader.sx + tests/read.sx + test.sh. Handles symbols (pkg:sym, pkg::sym), integers, floats, ratios, hex/binary/octal, strings, #\ chars, reader macros (#' #( #: ,@), line/block comments. Key gotcha: SX str for string concat (not concat), substring-based read-while.

Blockers

• (none yet)