go: lex.sx — hex/octal/binary integer literals + underscores, +14 tests [consumes-lex]
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 29s

Adds prefixed integer forms per Go spec § Integer literals:
0x.. / 0X.. (hex), 0b.. / 0B.. (binary), 0o.. / 0O.. (octal),
legacy 0123 octal also accepted. Underscores allowed between digits
in any run; lexer is permissive (parser/types phase can enforce
strict placement).

Dispatch lives in gl-read-number! against the first 1-2 chars;
hex digit run consumes lex-hex-digit? from lib/guest/lex.sx. Octal
and binary use local gl-oct-digit?/gl-bin-digit? — narrow enough
that promoting them to the kit is premature.

lex 92/92.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-27 06:57:47 +00:00
parent 4fc73a97f4
commit fe614fc531
5 changed files with 80 additions and 15 deletions

View File

@@ -6,7 +6,8 @@
;; Types:
;; "ident" — identifiers (foo, _bar, mixedCase)
;; "keyword" — one of the 25 Go keywords
;; "int" — integer literals (decimal only this iteration)
;; "int" — integer literals (decimal, 0x.. hex, 0b.. binary, 0o.. octal,
;; legacy 0123 octal; underscores between digits allowed)
;; "string" — interpreted string literals "..."
;; "rune" — rune literals 'x' (single char + simple escapes)
;; "op" — operators & punctuation; :value is the literal text
@@ -100,6 +101,10 @@
(fn
(at)
(when (go-asi-trigger? (gl-last)) (gl-emit! "semi" "\n" at))))
(define
gl-oct-digit?
(fn (c) (and (not (= c nil)) (>= c "0") (<= c "7"))))
(define gl-bin-digit? (fn (c) (or (= c "0") (= c "1"))))
(define
gl-skip-line!
(fn
@@ -131,13 +136,37 @@
(gl-read-ident! start))
(slice src start pos)))
(define
gl-read-digits!
gl-read-digit-run!
(fn
(digit?)
(when
(and (< pos src-len) (or (digit? (gl-cur)) (= (gl-cur) "_")))
(gl-advance! 1)
(gl-read-digit-run! digit?))))
(define
gl-read-number!
(fn
()
(when
(and (< pos src-len) (lex-digit? (gl-cur)))
(gl-advance! 1)
(gl-read-digits!))))
(cond
(and
(= (gl-cur) "0")
(or
(= (gl-peek 1) "x")
(= (gl-peek 1) "X")))
(do (gl-advance! 2) (gl-read-digit-run! lex-hex-digit?))
(and
(= (gl-cur) "0")
(or
(= (gl-peek 1) "b")
(= (gl-peek 1) "B")))
(do (gl-advance! 2) (gl-read-digit-run! gl-bin-digit?))
(and
(= (gl-cur) "0")
(or
(= (gl-peek 1) "o")
(= (gl-peek 1) "O")))
(do (gl-advance! 2) (gl-read-digit-run! gl-oct-digit?))
:else (gl-read-digit-run! lex-digit?))))
(define
gl-read-string!
(fn
@@ -343,7 +372,7 @@
(do
(let
((start pos))
(gl-read-digits!)
(gl-read-number!)
(gl-emit! "int" (slice src start pos) start))
(gl-scan!))
(= (gl-cur) "\"")

View File

@@ -1,9 +1,9 @@
{
"language": "go",
"total_pass": 78,
"total": 78,
"total_pass": 92,
"total": 92,
"suites": [
{"name":"lex","pass":78,"total":78,"status":"ok"},
{"name":"lex","pass":92,"total":92,"status":"ok"},
{"name":"parse","pass":0,"total":0,"status":"pending"},
{"name":"types","pass":0,"total":0,"status":"pending"},
{"name":"eval","pass":0,"total":0,"status":"pending"},

View File

@@ -1,10 +1,10 @@
# Go-on-SX Scoreboard
**Total: 78 / 78 tests passing**
**Total: 92 / 92 tests passing**
| | Suite | Pass | Total |
|---|---|---|---|
| ✅ | lex | 78 | 78 |
| ✅ | lex | 92 | 92 |
| ⬜ | parse | 0 | 0 |
| ⬜ | types | 0 | 0 |
| ⬜ | eval | 0 | 0 |

View File

@@ -68,12 +68,43 @@
(go-test "kw: type" (tok-types "type") (list "keyword" "eof"))
(go-test "kw: var" (tok-types "var") (list "keyword" "eof"))
;; ── integer literals ──────────────────────────────────────────────
;; ── integer literals — decimal ────────────────────────────────────
(go-test "int: zero" (tok-values "0") (list "0" "\n" nil))
(go-test "int: small" (tok-values "42") (list "42" "\n" nil))
(go-test "int: bigger" (tok-values "123456") (list "123456" "\n" nil))
(go-test "int: type" (tok-types "42") (list "int" "semi" "eof"))
;; ── integer literals — prefixed + underscores (Go spec § Integer literals)
(go-test "int: hex lower" (tok-values "0x1f") (list "0x1f" "\n" nil))
(go-test "int: hex upper-x" (tok-values "0X1F") (list "0X1F" "\n" nil))
(go-test
"int: hex mixed digits"
(tok-values "0xDEADbeef")
(list "0xDEADbeef" "\n" nil))
(go-test "int: binary lower" (tok-values "0b1010") (list "0b1010" "\n" nil))
(go-test "int: binary upper" (tok-values "0B1101") (list "0B1101" "\n" nil))
(go-test "int: octal modern" (tok-values "0o755") (list "0o755" "\n" nil))
(go-test "int: octal upper" (tok-values "0O17") (list "0O17" "\n" nil))
(go-test "int: octal legacy" (tok-values "0755") (list "0755" "\n" nil))
(go-test "int: hex type" (tok-types "0x1F") (list "int" "semi" "eof"))
(go-test "int: bin type" (tok-types "0b101") (list "int" "semi" "eof"))
(go-test
"int: dec underscore"
(tok-values "1_000_000")
(list "1_000_000" "\n" nil))
(go-test
"int: hex underscore"
(tok-values "0xDEAD_BEEF")
(list "0xDEAD_BEEF" "\n" nil))
(go-test
"int: bin underscore"
(tok-values "0b1010_1010")
(list "0b1010_1010" "\n" nil))
(go-test
"int: hex then +"
(tok-types "0xFF + 1")
(list "int" "op" "int" "semi" "eof"))
;; ── string literals ───────────────────────────────────────────────
(go-test "string: empty" (tok-values "\"\"") (list "" "\n" nil))
(go-test "string: hello" (tok-values "\"hello\"") (list "hello" "\n" nil))

View File

@@ -143,9 +143,10 @@ Progress-log line → push `origin/loops/go`.
ident/int/string/rune/{break,continue,fallthrough,return}/{++,--,),],}}.
- [ ] Float / imaginary literals
- [ ] Raw string literals `` `...` ``
- [ ] Hex/octal/binary integer literals (0x… 0o… 0b…) + underscores
- [x] Hex/octal/binary integer literals (0x… 0o… 0b…) + underscores
(legacy 0123 octal also accepted; consumes lex-hex-digit?)
- [ ] Full operator set audit (47 distinct per Go spec)
- **Acceptance:** lex/ suite at 50+ tests. Current: 78/78.
- **Acceptance:** lex/ suite at 50+ tests. Current: 92/92.
### Phase 2 — Parser (`lib/go/parse.sx`) ⬜
- Consume `lib/guest/core/pratt.sx` + `lib/guest/core/ast.sx`. Chisel notes
@@ -404,6 +405,10 @@ _(none yet)_
_Newest first. Append one dated entry per commit._
- 2026-05-27 — Phase 1 cont.: prefixed integer literals (`0x..`, `0X..`,
`0b..`, `0B..`, `0o..`, `0O..`, legacy `0123`) + underscore separators
in any digit run. Dispatch in `gl-read-number!`; consumes
`lex-hex-digit?` from the kit. +14 tests, lex 92/92. `[consumes-lex]`.
- 2026-05-26 — Phase 1 first slice: `lib/go/lex.sx` tokenizer consuming
`lib/guest/lex.sx` predicates. 25 keywords, ident/int/string/rune lits,
line+block comments, common operators, automatic semicolon insertion per