datalog: tokenizer raises on unterminated block comment (249/249)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 43s

Bug: `/* unclosed` was silently consumed to EOF, swallowing any
Datalog code that followed inside the (never-closing) comment.
Programs would produce empty parses with no error.

Fix: skip-block-comment! now raises when it hits EOF without
finding `*/`. Error message includes the position where the
problem was first detected. Line comments (`%`) and properly
closed block comments (`/* ... */`) are unaffected.

1 new tokenize test verifies the error path.
This commit is contained in:
2026-05-10 20:59:33 +00:00
parent 2a1d8eeab2
commit 4b5e75dc3e
4 changed files with 19 additions and 7 deletions

View File

@@ -118,6 +118,16 @@
"block comment"
(dl-tk-types (dl-tokenize "/* a\nb */ x."))
(list "atom" "punct" "eof"))
;; Unterminated block comment must raise — previously it was
;; silently consumed to EOF.
(dl-tk-test!
"unterminated block comment raises"
(let ((threw false))
(do
(guard (e (#t (set! threw true)))
(dl-tokenize "/* unclosed comment"))
threw))
true)
(dl-tk-test!
"whitespace"
(dl-tk-types (dl-tokenize " foo ,\t bar ."))