datalog: tokenizer raises on unterminated string + quoted atom (251/251)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 35s

Bug: read-quoted ran to EOF silently when the closing quote was
missing. The token's value was whatever ran-to-end string had been
accumulated; the parser later saw an unexpected EOF, but the error
message blamed the wrong location ("expected `)` got eof") and
hid the real problem.

Fix: read-quoted now raises with a message that distinguishes
strings from quoted atoms, including the position where the
opening quote was lost. The escape-sequence handling and proper
closing are unaffected.

2 new tokenize tests.
This commit is contained in:
2026-05-10 21:05:28 +00:00
parent 4b5e75dc3e
commit 5bcda5c88c
4 changed files with 30 additions and 7 deletions

View File

@@ -123,7 +123,11 @@
(fn
()
(cond
((>= pos src-len) nil)
((>= pos src-len)
(error
(str "Tokenizer: unterminated "
(if (= quote-char "'") "quoted atom" "string")
" (started near position " pos ")")))
((= (cur) "\\")
(do
(advance! 1)