rose-ash/plans/common-lisp-on-sx.md

Common-Lisp-on-SX: conditions + restarts on delimited continuations

The headline showcase is the condition system. Restarts are resumable exceptions — every other Lisp implementation reinvents this on host-stack unwind tricks. On SX restarts are textbook delimited continuations: signal walks the handler chain; invoke-restart resumes the captured continuation at the restart point. Same delcc primitive that powers Erlang actors, expressed as a different surface.

End-state goal: ANSI Common Lisp subset with a working condition/restart system, CLOS multimethods (with :before/:after/:around), the LOOP macro, packages, and ~150 hand-written + classic programs.

Scope decisions (defaults — override by editing before we spawn)

• Syntax: ANSI Common Lisp surface. Read tables, dispatch macros (#', #(, #\, #:, #x, #b, #o, ratios 1/3).
• Conformance: ANSI X3.226 as a target, not bug-for-bug SBCL/CCL. "Reads like CL, runs like CL."
• Test corpus: custom + a curated slice of ansi-test. Plus classic programs: condition-system demo, restart-driven debugger, multiple-dispatch geometry, LOOP corpus.
• Out of scope: compilation to native, FFI, sockets, threads, MOP class redefinition, full pathname/logical-pathname machinery, structures with :include deep customization.
• Packages: simple — defpackage/in-package/export/use-package/:cl/:cl-user. No nicknames, no shadowing-import edge cases.

Ground rules

• Scope: only touch lib/common-lisp/** and plans/common-lisp-on-sx.md. Don't edit spec/, hosts/, shared/, or any other lib/<lang>/**. CL primitives go in lib/common-lisp/runtime.sx.
• SX files: use sx-tree MCP tools only.
• Commits: one feature per commit. Keep ## Progress log updated and tick roadmap boxes.

Architecture sketch

Common Lisp source
    │
    ▼
lib/common-lisp/reader.sx     — tokenizer + reader (read macros, dispatch chars)
    │
    ▼
lib/common-lisp/parser.sx     — AST: forms, declarations, lambda lists
    │
    ▼
lib/common-lisp/transpile.sx  — AST → SX AST (entry: cl-eval-ast)
    │
    ▼
lib/common-lisp/runtime.sx    — special forms, condition system, CLOS, packages, BIFs

Core mapping:

• Symbol = SX symbol with package prefix; package table is a flat dict.
• Cons cell = SX pair via cons/car/cdr; lists native.
• Multiple values = thread through values/multiple-value-bind; primary-value default for one-context callers.
• Block / return-from = captured continuation; return-from name v invokes the block-named ^k.
• Tagbody / go = each tag is a continuation; go tag invokes it.
• Unwind-protect = scope frame with a cleanup thunk fired on any non-local exit.
• Conditions / restarts = layered handler chain on top of handler-bind + delcc. signal walks handlers; invoke-restart resumes a captured continuation.
• CLOS = generic functions are dispatch tables on argument-class lists; method combination computed lazily; call-next-method is a continuation.
• Macros = SX macros (sentinel-body) — defmacro lowers directly.

Roadmap

Phase 1 — reader + parser

• Tokenizer: symbols (with package qualification pkg:sym / pkg::sym), numbers (int, float, ratio 1/3, #xFF, #b1010, #o17), strings "…" with \ escapes, characters #\Space #\Newline #\a, comments ;, block comments #| … |#
• Reader: list, dotted pair, quote ', function #', quasiquote `, unquote ,, splice ,@, vector #(…), uninterned #:foo, nil/t literals
• Parser: lambda lists with &optional &rest &key &aux &allow-other-keys, defaults, supplied-p variables
• Unit tests in lib/common-lisp/tests/read.sx

Phase 2 — sequential eval + special forms

• cl-eval-ast: quote, if, progn, let, let*, flet, labels, setq, setf (subset), function, lambda, the, locally, eval-when
• block + return-from via captured continuation
• tagbody + go via per-tag continuations
• unwind-protect cleanup frame
• multiple-value-bind, multiple-value-call, multiple-value-prog1, values, nth-value
• defun, defparameter, defvar, defconstant, declaim, proclaim (no-op)
• Dynamic variables — defvar/defparameter produce specials; let rebinds via parameterize-style scope
• 127 tests in lib/common-lisp/tests/eval.sx

Phase 3 — conditions + restarts (THE SHOWCASE)

• define-condition — class hierarchy rooted at condition/error/warning/simple-error/simple-warning/type-error/arithmetic-error/division-by-zero
• signal, error, cerror, warn — all walk the handler chain
• handler-bind — non-unwinding handlers, may decline by returning normally
• handler-case — unwinding handlers (call/cc escape)
• restart-case, with-simple-restart, restart-bind
• find-restart, invoke-restart, compute-restarts
• with-condition-restarts — associate restarts with a specific condition
• invoke-restart-interactively, *break-on-signals*, *debugger-hook* (basic)
• Classic programs in lib/common-lisp/tests/programs/:
  • restart-demo.sx — division with use-zero and retry restarts (7 tests)
  • parse-recover.sx — parser with skipped-token restart (6 tests)
  • interactive-debugger.lisp — ASCII REPL using :debugger-hook
• lib/common-lisp/conformance.sh + runner, scoreboard.json + scoreboard.md

Phase 4 — CLOS

• defclass with :initarg/:initform/:accessor/:reader/:writer/:allocation
• make-instance, slot-value, (setf slot-value), with-slots, with-accessors
• defgeneric with :method-combination (standard, plus +, and, or)
• defmethod with :before / :after / :around qualifiers
• call-next-method (continuation), next-method-p
• class-of, find-class, slot-boundp, change-class (basic)
• Multiple dispatch — method specificity by argument-class precedence list
• Built-in classes registered for tagged values (integer, float, string, symbol, cons, null, t)
• Classic programs:
  • geometry.lisp — intersect generic dispatching on (point line), (line line), (line plane)…
  • mop-trace.lisp — :before + :after printing call trace

Phase 5 — macros + LOOP + reader macros

• defmacro, macrolet, symbol-macrolet, macroexpand-1, macroexpand
• gensym, gentemp
• set-macro-character, set-dispatch-macro-character, get-macro-character
• The LOOP macro — iteration drivers (for … in/across/from/upto/downto/by, while, until, repeat), accumulators (collect, append, nconc, count, sum, maximize, minimize), conditional clauses (if/when/unless/else), termination (finally/thereis/always/never), named blocks
• LOOP test corpus: 30+ tests covering all clause types

Phase 6 — packages + stdlib drive

• defpackage, in-package, export, use-package, import, find-package
• Package qualification at the reader level — cl:car, mypkg::internal
• :common-lisp (:cl) and :common-lisp-user (:cl-user) packages
• Sequence functions — mapcar, mapc, mapcan, reduce, find, find-if, position, count, every, some, notany, notevery, remove, remove-if, subst
• List ops — assoc, getf, nth, last, butlast, nthcdr, tailp, ldiff
• String ops — string=, string-upcase, string-downcase, subseq, concatenate
• FORMAT — basic directives ~A, ~S, ~D, ~F, ~%, ~&, ~T, ~{...~} (iteration), ~[...~] (conditional), ~^ (escape), ~P (plural)
• Drive corpus to 200+ green

SX primitive baseline

Use vectors for arrays; numeric tower + rationals for numbers; ADTs for tagged data; coroutines for fibers; string-buffer for mutable string building; bitwise ops for bit manipulation; multiple values for multi-return; promises for lazy evaluation; hash tables for mutable associative storage; sets for O(1) membership; sequence protocol for polymorphic iteration; gensym for unique symbols; char type for characters; string ports

• read/write for reader protocols; regexp for pattern matching; bytevectors for binary data; format for string templating.

Progress log

Newest first.

• 2026-05-05: Phase 3 classic programs — tests/programs/restart-demo.sx (7 tests: safe-divide with use-zero + retry restarts) and tests/programs/parse-recover.sx (6 tests: token parser with skip-token + use-zero restarts, handler-case abort). Key gotcha: use = not equal? for list comparison in sx_server.
• 2026-05-05: Phase 3 conditions + restarts — cl-condition-classes hierarchy (15 types), cl-condition?/cl-condition-of-type?, cl-make-condition, cl-define-condition, cl-signal/cl-error/cl-warn/cl-cerror, cl-handler-bind (non-unwinding), cl-handler-case (call/cc escape), cl-restart-case/cl-with-simple-restart, cl-find-restart/cl-invoke-restart/cl-compute-restarts, cl-with-condition-restarts; 55 new tests in tests/conditions.sx (123 total runtime tests). Key gotcha: cl-condition-classes must be captured at define-time via let in cl-condition-of-type? — free-variable lookup at call-time fails through env_merge parent chain.
• 2026-05-05: unwind-protect — cl-eval-unwind-protect: eval protected form, run cleanup with for-each (discards results, preserves original sentinel), return original result; 8 new tests (159 eval, 331 total green).
• 2026-05-05: tagbody + go — cl-go-tag? sentinel; cl-eval-tagbody runs body with tag-index map (keys str-normalised for integer tags); go-tag propagation in cl-eval-body alongside block-return; 11 new tests (151 eval, 323 total green).
• 2026-05-05: block + return-from — sentinel propagation in cl-eval-body; cl-eval-block catches matching sentinels; BLOCK/RETURN-FROM/RETURN dispatch in cl-eval-list; 13 new tests (140 eval, 312 total green). Parser: CL strings → {:cl-type "string"} dicts.
• 2026-04-25: Phase 2 eval — 127 tests, 299 total green. lib/common-lisp/eval.sx: cl-eval-ast with quote/if/progn/let/let*/flet/labels/setq/setf/function/lambda/the/locally/eval-when; defun/defvar/defparameter/defconstant; built-in arithmetic (+/-/*//, min/max/abs/evenp/oddp), comparisons, predicates, list ops (car/cdr/cons/list/append/reverse/length/nth/first/second/third/rest), string ops, funcall/apply/mapcar. Key gotchas: SX reduce is (reduce fn init list) not (reduce fn list init); CL true literal is t not true; builtins registered in cl-global-env.fns via wrapper dicts for #' syntax.
• 2026-04-25: Phase 1 lambda-list parser — 31 new tests, 172 total green. cl-parse-lambda-list in parser.sx + tests/lambda.sx. Handles &optional/&rest/&body/&key/&aux/&allow-other-keys, defaults, supplied-p. Key gotchas: (when (> (len items) 0) ...) not (when items ...) (empty list is truthy); custom cl-deep= needed for dict/list structural equality in tests.
• 2026-04-25: Phase 1 reader/parser — 62 new tests, 141 total green. lib/common-lisp/parser.sx: cl-read/cl-read-all, lists, dotted pairs, quote/backquote/unquote/splice/#', vectors, #:uninterned, NIL→nil, T→true, reader macro wrappers.
• 2026-04-25: Phase 1 tokenizer — 79 tests green. lib/common-lisp/reader.sx + tests/read.sx + test.sh. Handles symbols (pkg:sym, pkg::sym), integers, floats, ratios, hex/binary/octal, strings, #\ chars, reader macros (#' #( #: ,@), line/block comments. Key gotcha: SX str for string concat (not concat), substring-based read-while.

Blockers

• (none yet)