rose-ash/plans/ruby-on-sx.md

Ruby-on-SX: fibers + blocks + open classes on delimited continuations

The headline showcase is fibers — Ruby's Fiber.new { … Fiber.yield v … } / Fiber.resume are textbook delimited continuations with sugar. MRI implements them by swapping C stacks; on SX they fall out of the existing perform/cek-resume machinery for free. Plus blocks/yield (lexical escape continuations, same shape as Smalltalk's non-local return), method_missing, and singleton classes.

End-state goal: Ruby 2.7-flavoured subset, Enumerable mixin, fibers + threads-via-fibers (no real OS threads), method_missing-driven DSLs, ~150 hand-written + classic programs.

Scope decisions (defaults — override by editing before we spawn)

• Syntax: Ruby 2.7. No 3.x pattern matching, no rightward assignment, no endless methods. We pick 2.7 because it's the biggest semantic surface that still parses cleanly.
• Conformance: "Reads like Ruby, runs like Ruby." Slice of RubySpec (Core + Library subset), not full RubySpec.
• Test corpus: custom + curated RubySpec slice. Plus classic programs: fiber-based generator, internal DSL with method_missing, mixin-based Enumerable on a custom class.
• Out of scope: real threads, GIL, refinements, binding_of_caller from non-Ruby contexts, Encoding object beyond UTF-8/ASCII-8BIT, RubyVM::* introspection beyond bytecode-disassembly placeholder, IO subsystem beyond puts/gets/File.read.
• Symbols: SX symbols. Strings are mutable copies; symbols are interned.

Ground rules

• Scope: only touch lib/ruby/** and plans/ruby-on-sx.md. Don't edit spec/, hosts/, shared/, or any other lib/<lang>/**. Ruby primitives go in lib/ruby/runtime.sx.
• SX files: use sx-tree MCP tools only.
• Commits: one feature per commit. Keep ## Progress log updated and tick roadmap boxes.

Architecture sketch

Ruby source
    │
    ▼
lib/ruby/tokenizer.sx   — keywords, ops, %w[], %i[], heredocs (deferred), regex (deferred)
    │
    ▼
lib/ruby/parser.sx      — AST: classes, modules, methods, blocks, calls
    │
    ▼
lib/ruby/transpile.sx   — AST → SX AST (entry: rb-eval-ast)
    │
    ▼
lib/ruby/runtime.sx     — class table, MOP, dispatch, fibers, primitives

Core mapping:

• Object = SX dict {:class :ivars :singleton-class?}. Instance variables live in ivars keyed by symbol.
• Class = SX dict {:name :superclass :methods :class-methods :metaclass :includes :prepends}. Class table is flat.
• Method dispatch = lookup walks ancestor chain (prepended → class → included modules → superclass → …). Falls back to method_missing with a Symbol+args.
• Block = lambda + escape continuation. yield invokes the block in current context. return from within a block invokes the enclosing-method's escape continuation.
• Proc = lambda without strict arity. Proc.new + proc {}.
• Lambda = lambda with strict arity + return-returns-from-lambda semantics.
• Fiber = pair of continuations (resume-k, yield-k) wrapped in a record. Fiber.new { … } builds it; Fiber.resume invokes the resume-k; Fiber.yield invokes the yield-k. Built directly on perform/cek-resume.
• Module = class without instance allocation. include puts it in the chain; prepend puts it earlier; extend puts it on the singleton.
• Singleton class = lazily allocated per-object class for def obj.foo definitions.
• Symbol = interned SX symbol. :foo reads as (quote foo) flavour.

Roadmap

Phase 1 — tokenizer + parser

• Tokenizer: keywords (def end class module if unless while until do return yield begin rescue ensure case when then else elsif), identifiers (lowercase = local/method, @ = ivar, @@ = cvar, $ = global, uppercase = constant), numbers (int, float, 0x 0o 0b, _ separators), strings ("…" interpolation, '…' literal, %w[a b c], %i[a b c]), symbols :foo :"…", operators (+ - * / % ** == != < > <= >= <=> === =~ !~ << >> & | ^ ~ ! && || and or not), :: . , ; ( ) [ ] { } -> => |, comments #
• Parser: program is sequence of statements separated by newlines or ;; method def def name(args) … end; class class Foo < Bar … end; module module M … end; block do |a, b| … end and { |a, b| … }; call sugar (no parens), obj.method, Mod::Const; arg shapes (positional, default, splat *args, double-splat **opts, block &blk)
• If/while/case expressions (return values), unless/until, postfix modifiers
• Begin/rescue/ensure/retry, raise, raise with class+message
• Unit tests in lib/ruby/tests/parse.sx

Phase 2 — object model + sequential eval

• Class table bootstrap: BasicObject, Object, Kernel, Module, Class, Numeric, Integer, Float, String, Symbol, Array, Hash, Range, NilClass, TrueClass, FalseClass, Proc, Method
• rb-eval-ast: literals, variables (local, ivar, cvar, gvar, constant), assignment (single and parallel a, b = 1, 2, splat receive), method call, message dispatch
• Method lookup walks ancestor chain; cache hit-class per (class, selector)
• method_missing fallback constructing args list
• super and super(args) — lookup in defining class's superclass
• Singleton class allocation on first def obj.foo or class << obj
• nil, true, false are singletons of their classes; tagged values aren't boxed
• Constant lookup (lexical-then-inheritance) with Module.nesting
• 60+ tests in lib/ruby/tests/eval.sx

Phase 3 — blocks + procs + lambdas

• Method invocation captures escape continuation ^k for return; binds it as block's escape
• yield invokes implicit block
• block_given?, &blk parameter, &proc arg unpacking
• Proc.new, proc { }, lambda { } (or ->(x) { x })
• Lambda strict arity + lambda-local return semantics
• Proc lax arity (a, b, c unpacks Array; missing args nil)
• break, next, redo — break is escape-from-loop-or-block; next is escape-from-block-iteration; redo re-runs current iteration
• 30+ tests in lib/ruby/tests/blocks.sx

Phase 4 — fibers (THE SHOWCASE)

• Fiber.new { |arg| … Fiber.yield v … } allocates a fiber record with paired continuations
• Fiber.resume(args…) resumes the fiber, returning the value passed to Fiber.yield
• Fiber.yield(v) from inside the fiber suspends and returns control to the resumer
• Fiber.current from inside the fiber
• Fiber#alive?, Fiber#raise (deferred)
• Fiber.transfer — symmetric coroutines (resume from any side)
• Classic programs in lib/ruby/tests/programs/:
  • generator.rb — pull-style infinite enumerator built on fibers
  • producer-consumer.rb — bounded buffer with Fiber.transfer
  • tree-walk.rb — recursive tree walker that yields each node, driven by Fiber.resume
• lib/ruby/conformance.sh + runner, scoreboard.json + scoreboard.md

Phase 5 — modules + mixins + metaprogramming

• include M — appends M's methods after class methods in chain
• prepend M — prepends M before class methods
• extend M — adds M to singleton class
• Module#ancestors, Module#included_modules
• define_method, class_eval, instance_eval, module_eval
• respond_to?, respond_to_missing?, method_missing
• Object#send, Object#public_send, Object#__send__
• Module#method_added, singleton_method_added hooks
• Hooks: included, extended, inherited, prepended
• Internal-DSL classic program: lib/ruby/tests/programs/dsl.rb

Phase 6 — stdlib drive

• Enumerable mixin: each (abstract), map, select/filter, reject, reduce/inject, each_with_index, each_with_object, take, drop, take_while, drop_while, find/detect, find_index, any?, all?, none?, one?, count, min, max, min_by, max_by, sort, sort_by, group_by, partition, chunk, each_cons, each_slice, flat_map, lazy
• Comparable mixin: <=>, <, <=, >, >=, ==, between?, clamp
• Array: indexing, slicing, push/pop/shift/unshift, concat, flatten, compact, uniq, sort, reverse, zip, dig, pack/unpack (deferred)
• Hash: [], []=, delete, merge, each_pair, keys, values, to_a, dig, fetch, default values, default proc
• Range: each, step, cover?, include?, size, min, max
• String: indexing, slicing, split, gsub (string-arg version, regex deferred), sub, upcase, downcase, strip, chomp, chars, bytes, to_i, to_f, to_sym, *, +, <<, format with %
• Integer: times, upto, downto, step, digits, gcd, lcm
• Drive corpus to 200+ green

Progress log

Newest first.

• (none yet)

Blockers

• (none yet)