giles 96019e9fe8 ruby: Phase 1 tokenizer (+107 tests) ...

lib/ruby/tokenizer.sx — rb-tokenize: keywords, identifiers (@/@~/$/const),
numbers (dec/hex/oct/bin/float), strings (dq with raw interpolation, sq),
symbols, %w/%i, operators (all compound forms), punctuation, comments,
line/col tracking. Plus test runner test.sh and 107 passing tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-04-25 18:13:05 +00:00

23 KiB

Raw Blame History

View Raw