Compare commits

...

6 Commits

Author SHA1 Message Date
f68591456e content: Phase 5 — rich inline text via structured runs (861/861)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 40s
CtText.text may be a list of runs (text marks href); CtHeading/CtQuote rich,
CtCode verbatim. New runs.sx overrides render/markdown/text methods (byte-
identical for plain strings, opt-in). 4 modes: HTML tags / markdown / nested
SX / plain asText (drift-proof). find-replace per-run marks-preserving;
search across run boundaries; CRDT block-granularity LWW; data+wire round-trip.
Runs are a Smalltalk-renderable list (not a dict — substrate can't read dict
fields under nested render dispatch). +36 tests (44 suites). Phase 6 (char-
level inline CRDT) recorded as future.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 21:10:56 +00:00
160d0f2dd0 content: content/block-path + block-depth — locate a block in the tree (825/825)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 1m1s
Read-side companion to doc-find-deep + reparent: ancestor-section chain
(root-first) for a block id, nil if absent (distinct from () top-level path);
block-depth is the path length. For breadcrumbs / scoping. New suite +13.
Probe this pass found no bugs: clone/remap tree-wide, all block types have
asMarkdown:.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 19:45:29 +00:00
e9316b37c2 content: tree reparent — move-into section + promote (812/812)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 41s
insert/move were top-level only; a block could never move into/out of a
section. content/move-into (relocate to a section child at index, tree-wide)
+ content/promote (lift nested block to top level, subtree intact). Pure
tree transforms like the rest of move.sx; cycle-safe (rejects moving a block
into its own descendant). +13 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 18:28:36 +00:00
29954689bc content: content/sanitize — drop invalid blocks tree-wide (799/799)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 45s
Enforcement counterpart to validate: removes blocks failing validate's own
per-block id/field checks (reused via content/-block-issues, single-sourced)
so federated/imported input can render safely. Tree-wide; distinct from
normalize (empty vs invalid); keeps valid-shell sections, drops invalid ones.
New suite +12 tests (42 suites total).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 17:18:49 +00:00
f31c7a4002 content: document markdown table-pipe round-trip limitation + fix sketch
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 33s
Probed the Markdown boundary; found table cells containing | don't round-trip
(asMarkdown emits raw |, md/import splits on every |). Recorded under Known
limitations with repro + two-sided fix sketch. Fix blocked: md-import.sx is
449 lines and all sx-tree edit tools error in this worktree (only
sx_write_file works) — deferred rather than risk a full manual rewrite.
Engine SATURATED at 787/787.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 16:08:41 +00:00
c5d9e1480d content: validation vets list items + table cells element-deep (787/787)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 19s
validate only checked that list items / table rows-headers ARE lists; a
non-string item or non-list/non-string-cell row passed yet crashes asText/
render/find-replace/search. Added ct-all-str?/ct-all-rows? + deepened list/
table branches (guarded against double-reporting). +9 validate tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 15:29:54 +00:00
15 changed files with 1085 additions and 33 deletions

45
lib/content/block-path.sx Normal file
View File

@@ -0,0 +1,45 @@
;; content-on-sx — locate a block in the tree (ancestor section path).
;;
;; The read-side companion to doc-find-deep (which returns the block) and the
;; move/reparent ops (which relocate it): content/block-path returns the list of
;; ancestor section ids, root-first, leading to a block id — i.e. where the
;; block sits in the tree. A top-level block has an empty path; a block one
;; section deep has a one-element path; a missing id returns nil (distinct from
;; the empty-list path of a present top-level block). content/block-depth is the
;; path length (0 = top level, -1 = absent). Useful for breadcrumbs and for
;; scoping an edit to a block's enclosing section. Pure traversal; descends into
;; any block carrying a children list, like the rest of the tree helpers.
;;
;; Requires (loaded by harness): block.sx, doc.sx.
(define
bp-in-blocks
(fn
(blocks id trail)
(if
(= (len blocks) 0)
nil
(let
((b (first blocks)))
(if
(= (blk-id b) id)
trail
(let
((ch (st-iv-get b "children")))
(let
((found (if (list? ch) (bp-in-blocks ch id (append trail (list (blk-id b)))) nil)))
(if (= found nil) (bp-in-blocks (rest blocks) id trail) found))))))))
;; ancestor section ids (root-first) for `id`, or nil if the block is absent.
(define
content/block-path
(fn (doc id) (bp-in-blocks (doc-blocks doc) id (list))))
;; depth of `id`: 0 at top level, n nested n sections deep, -1 if absent.
(define
content/block-depth
(fn
(doc id)
(let
((p (content/block-path doc id)))
(if (= p nil) -1 (len p)))))

View File

@@ -15,7 +15,7 @@ if [ ! -x "$SX_SERVER" ]; then
fi fi
fi fi
SUITES=(block doc render api meta page page-full markdown text section compose tree-edit move clone query toc anchor outline flatten transform normalize find-replace stats summary index table callout media data wire validate store snapshot crdt crdt-tree crdt-blocks crdt-store sync md-import md-doc fed) SUITES=(block doc render api meta page page-full markdown runs text section compose tree-edit move block-path clone query toc anchor outline flatten transform normalize find-replace stats summary index table callout media data wire validate sanitize store snapshot crdt crdt-tree crdt-blocks crdt-store sync md-import md-doc fed)
OUT_JSON="lib/content/scoreboard.json" OUT_JSON="lib/content/scoreboard.json"
OUT_MD="lib/content/scoreboard.md" OUT_MD="lib/content/scoreboard.md"
@@ -48,6 +48,7 @@ run_suite() {
(load "lib/content/compose.sx") (load "lib/content/compose.sx")
(load "lib/content/tree-edit.sx") (load "lib/content/tree-edit.sx")
(load "lib/content/move.sx") (load "lib/content/move.sx")
(load "lib/content/block-path.sx")
(load "lib/content/clone.sx") (load "lib/content/clone.sx")
(load "lib/content/query.sx") (load "lib/content/query.sx")
(load "lib/content/toc.sx") (load "lib/content/toc.sx")
@@ -68,7 +69,9 @@ run_suite() {
(load "lib/content/page.sx") (load "lib/content/page.sx")
(load "lib/content/page-full.sx") (load "lib/content/page-full.sx")
(load "lib/content/markdown.sx") (load "lib/content/markdown.sx")
(load "lib/content/runs.sx")
(load "lib/content/validate.sx") (load "lib/content/validate.sx")
(load "lib/content/sanitize.sx")
(load "lib/content/store.sx") (load "lib/content/store.sx")
(load "lib/content/snapshot.sx") (load "lib/content/snapshot.sx")
(load "lib/content/crdt.sx") (load "lib/content/crdt.sx")

View File

@@ -10,6 +10,11 @@
;; via content/find-replace and a word count over asText stay consistent. ;; via content/find-replace and a word count over asText stay consistent.
;; Immutable; case-sensitive. ;; Immutable; case-sensitive.
;; ;;
;; A text field may be a plain string OR a list of rich-text runs (Phase 5,
;; run = (text marks href)). fr-rep-text rewrites per run, preserving each run's
;; marks/href; a match that physically straddles two runs is not joined (the
;; replacement would have no single mark set) — each run is rewritten in place.
;;
;; Requires (loaded by harness): block.sx, transform.sx (content/map-blocks), ;; Requires (loaded by harness): block.sx, transform.sx (content/map-blocks),
;; table.sx (CtTable ivars). ;; table.sx (CtTable ivars).
@@ -24,6 +29,23 @@
(define fr-rep (fn (s from to) (replace (str s) from to))) (define fr-rep (fn (s from to) (replace (str s) from to)))
;; rewrite a text-bearing field that is either a plain string or a runs list
(define
fr-rep-text
(fn
(v from to)
(if
(list? v)
(map
(fn
(r)
(list
(fr-rep (nth r 0) from to)
(nth r 1)
(nth r 2)))
v)
(fr-rep v from to))))
;; Blocks whose prose content find/replace rewrites (matches asText's set). ;; Blocks whose prose content find/replace rewrites (matches asText's set).
(define (define
fr-has-text? fr-has-text?
@@ -66,7 +88,7 @@
(if (list? r) (map (fn (c) (fr-rep c from to)) r) r)) (if (list? r) (map (fn (c) (fr-rep c from to)) r) r))
rs)) rs))
b1)))) b1))))
(else (blk-set b "text" (fr-rep (blk-get b "text") from to))))))) (else (blk-set b "text" (fr-rep-text (blk-get b "text") from to)))))))
(define (define
content/find-replace content/find-replace

View File

@@ -1,8 +1,13 @@
;; content-on-sx — relative block reorder. ;; content-on-sx — block reorder + reparent.
;; ;;
;; Move a top-level block to just before / after another block by id — more ;; Relative reorder of top-level blocks (move-before/after/to-front/to-back by
;; ergonomic than the index-based doc-move. No-op if either id is missing. ;; id) plus TREE reparenting: move a block into a section (content/move-into) or
;; Immutable; composes the doc.sx list helpers. ;; promote a nested block back out to the top level (content/promote). Reparent
;; ops are tree-wide (the block may start anywhere) and cycle-safe — moving a
;; block into its own descendant is rejected (no-op), so a section can never
;; become its own ancestor. No-op if any id is missing. Immutable; composes the
;; doc.sx list + tree helpers (doc-find-deep / ct-find-id / ct-remove-id /
;; ct-replace-id / ct-insert-at).
;; ;;
;; Requires (loaded by harness): doc.sx. ;; Requires (loaded by harness): doc.sx.
@@ -67,3 +72,57 @@
(doc-with-blocks (doc-with-blocks
doc doc
(append (ct-remove-id (doc-blocks doc) id) (list blk))))))) (append (ct-remove-id (doc-blocks doc) id) (list blk)))))))
;; ── reparent (tree-wide) ──
;; move block `id` (from anywhere in the tree) to be a child of section
;; `section-id` at index `i`. No-op if either id is missing, if id = section-id,
;; or if section-id sits inside id's own subtree (would create a cycle).
(define
content/move-into
(fn
(doc id section-id i)
(let
((blk (doc-find-deep doc id)))
(if
(= blk nil)
doc
(if
(= (doc-find-deep doc section-id) nil)
doc
(if
(= id section-id)
doc
(if
(= (ct-find-id (list blk) section-id) nil)
(let
((without (ct-remove-id (doc-blocks doc) id)))
(doc-with-blocks
doc
(ct-replace-id
without
section-id
(fn
(sec)
(let
((ch (st-iv-get sec "children")))
(if
(list? ch)
(st-iv-set! sec "children" (ct-insert-at ch i blk))
sec))))))
doc)))))))
;; promote block `id` (wherever it sits) out to the end of the top level. If it
;; is already top-level this is a move-to-back. No-op if missing. A section keeps
;; its whole subtree.
(define
content/promote
(fn
(doc id)
(let
((blk (doc-find-deep doc id)))
(if
(= blk nil)
doc
(doc-with-blocks
doc
(append (ct-remove-id (doc-blocks doc) id) (list blk)))))))

118
lib/content/runs.sx Normal file
View File

@@ -0,0 +1,118 @@
;; content-on-sx — Phase 5: rich inline text (structured runs).
;;
;; A CtText's `text` ivar may be EITHER a plain string (backward compat) OR a
;; list of inline RUNS. A run is a 3-element list (text marks href):
;; text — a string
;; marks — a list of mark tokens, a subset of
;; :bold :italic :underline :strikethrough :code :subscript
;; :superscript :link (SX keywords evaluate to the strings the
;; Smalltalk renderer compares against; build them with keywords)
;; href — a string ("" when absent; the link target for a :link mark)
;;
;; Runs are a LIST, not a {:text :marks} dict, because rendering happens inside
;; the Smalltalk render methods (nested blocks dispatch asHTML/etc. via Smalltalk
;; message sends) and the Smalltalk-on-SX layer can iterate SX lists but cannot
;; read SX dict fields. Lists are Smalltalk-native, render under nesting, and
;; round-trip through data/wire for free.
;;
;; content-bootstrap-runs! OVERRIDES the render/markdown/text methods of CtText
;; and its subclasses (CtHeading/CtQuote rich; CtCode verbatim — runs render as
;; plain concatenated text) with run-aware versions that produce IDENTICAL output
;; for a plain-string body. Opt-in: call after the render/markdown/text
;; bootstraps; suites that don't call it are unaffected.
;;
;; Requires (loaded by harness): block.sx, render.sx, markdown.sx, text.sx.
;; ── SX-side run helpers ──
(define mk-run (fn (text marks href) (list text marks href)))
(define mk-run-plain (fn (text) (list text (list) "")))
(define run-text (fn (r) (nth r 0)))
(define run-marks (fn (r) (nth r 1)))
(define run-href (fn (r) (nth r 2)))
;; a CtText body is "rich" iff it is a runs list (vs a plain string)
(define runs? (fn (v) (list? v)))
;; build a CtText whose body is a list of runs
(define
mk-rich-text
(fn (id runs) (st-iv-set! (mk-text id "") "text" runs)))
(define
content-bootstrap-runs!
(fn
()
(begin
(ct-def-method!
"CtText"
"runHtml:"
"runHtml: run | frag marks href | frag := (run at: 1) htmlEscaped. marks := run at: 2. href := run at: 3. marks do: [:m | (m = 'bold') ifTrue: [frag := '<strong>' , frag , '</strong>']. (m = 'italic') ifTrue: [frag := '<em>' , frag , '</em>']. (m = 'underline') ifTrue: [frag := '<u>' , frag , '</u>']. (m = 'strikethrough') ifTrue: [frag := '<s>' , frag , '</s>']. (m = 'code') ifTrue: [frag := '<code>' , frag , '</code>']. (m = 'subscript') ifTrue: [frag := '<sub>' , frag , '</sub>']. (m = 'superscript') ifTrue: [frag := '<sup>' , frag , '</sup>']. (m = 'link') ifTrue: [frag := '<a href=\"' , href htmlEscaped , '\">' , frag , '</a>']]. ^ frag")
(ct-def-method!
"CtText"
"runSx:"
"runSx: run | frag marks href | frag := '\"' , (run at: 1) sxEscaped , '\"'. marks := run at: 2. href := run at: 3. marks do: [:m | (m = 'bold') ifTrue: [frag := '(strong ' , frag , ')']. (m = 'italic') ifTrue: [frag := '(em ' , frag , ')']. (m = 'underline') ifTrue: [frag := '(u ' , frag , ')']. (m = 'strikethrough') ifTrue: [frag := '(s ' , frag , ')']. (m = 'code') ifTrue: [frag := '(code ' , frag , ')']. (m = 'subscript') ifTrue: [frag := '(sub ' , frag , ')']. (m = 'superscript') ifTrue: [frag := '(sup ' , frag , ')']. (m = 'link') ifTrue: [frag := '(a :href \"' , href sxEscaped , '\" ' , frag , ')']]. ^ frag")
(ct-def-method!
"CtText"
"runMd:"
"runMd: run | frag marks href | frag := (run at: 1). marks := run at: 2. href := run at: 3. marks do: [:m | (m = 'bold') ifTrue: [frag := '**' , frag , '**']. (m = 'italic') ifTrue: [frag := '_' , frag , '_']. (m = 'strikethrough') ifTrue: [frag := '~~' , frag , '~~']. (m = 'code') ifTrue: [frag := '`' , frag , '`']. (m = 'underline') ifTrue: [frag := '<u>' , frag , '</u>']. (m = 'subscript') ifTrue: [frag := '<sub>' , frag , '</sub>']. (m = 'superscript') ifTrue: [frag := '<sup>' , frag , '</sup>']. (m = 'link') ifTrue: [frag := '[' , frag , '](' , href , ')']]. ^ frag")
(ct-def-method!
"CtText"
"inlineHtml"
"inlineHtml | out | (text class name = 'String') ifTrue: [^ text htmlEscaped]. out := ''. text do: [:run | out := out , (self runHtml: run)]. ^ out")
(ct-def-method!
"CtText"
"inlineSx"
"inlineSx | out | (text class name = 'String') ifTrue: [^ '\"' , text sxEscaped , '\"']. out := ''. text do: [:run | out := (out = '' ifTrue: [self runSx: run] ifFalse: [out , ' ' , (self runSx: run)])]. ^ out")
(ct-def-method!
"CtText"
"inlineMd"
"inlineMd | out | (text class name = 'String') ifTrue: [^ text]. out := ''. text do: [:run | out := out , (self runMd: run)]. ^ out")
(ct-def-method!
"CtText"
"inlineText"
"inlineText | out | (text class name = 'String') ifTrue: [^ text]. out := ''. text do: [:run | out := out , (run at: 1)]. ^ out")
(ct-def-method!
"CtText"
"asHTML"
"asHTML ^ '<p>' , self inlineHtml , '</p>'")
(ct-def-method! "CtText" "asSx" "asSx ^ '(p ' , self inlineSx , ')'")
(ct-def-method! "CtText" "asMarkdown:" "asMarkdown: nl ^ self inlineMd")
(ct-def-method! "CtText" "asText" "asText ^ self inlineText")
(ct-def-method!
"CtHeading"
"asHTML"
"asHTML | t | t := level printString. ^ '<h' , t , '>' , self inlineHtml , '</h' , t , '>'")
(ct-def-method!
"CtHeading"
"asSx"
"asSx | t | t := level printString. ^ '(h' , t , ' ' , self inlineSx , ')'")
(ct-def-method!
"CtHeading"
"asMarkdown:"
"asMarkdown: nl | h i | h := ''. i := 0. [i < level] whileTrue: [h := h , '#'. i := i + 1]. ^ h , ' ' , self inlineMd")
(ct-def-method! "CtHeading" "asText" "asText ^ self inlineText")
(ct-def-method!
"CtQuote"
"asHTML"
"asHTML ^ '<blockquote>' , self inlineHtml , '</blockquote>'")
(ct-def-method!
"CtQuote"
"asSx"
"asSx ^ '(blockquote ' , self inlineSx , ')'")
(ct-def-method!
"CtQuote"
"asMarkdown:"
"asMarkdown: nl ^ '> ' , self inlineMd")
(ct-def-method! "CtQuote" "asText" "asText ^ self inlineText")
(ct-def-method!
"CtCode"
"asHTML"
"asHTML ^ '<pre><code class=\"language-' , language htmlEscaped , '\">' , self inlineText htmlEscaped , '</code></pre>'")
(ct-def-method!
"CtCode"
"asSx"
"asSx ^ '(pre (code \"' , self inlineText sxEscaped , '\"))'")
(ct-def-method!
"CtCode"
"asMarkdown:"
"asMarkdown: nl ^ '```' , language , nl , self inlineText , nl , '```'")
(ct-def-method! "CtCode" "asText" "asText ^ self inlineText")
true)))

47
lib/content/sanitize.sx Normal file
View File

@@ -0,0 +1,47 @@
;; content-on-sx — make a document render-safe by dropping invalid blocks.
;;
;; The enforcement counterpart to validate: where content/validate REPORTS id /
;; field issues, content/sanitize REMOVES the offending blocks so the result can
;; be rendered/merged without faulting on malformed input (federated or imported
;; documents that failed validation). Tree-wide: descends into sections, pruning
;; invalid descendants; a section whose own shell is valid is kept (even if it
;; ends up empty — that is normalize's job, not sanitize's), but a section whose
;; own check fails (e.g. children is not a list) is dropped whole.
;;
;; Reuses validate's per-block predicate (content/-block-issues), so the set of
;; "what is invalid" stays single-sourced and can't drift from content/validate.
;; sanitize addresses per-block id/field validity only; it does NOT resolve
;; duplicate ids (a cross-block concern with no single right answer), so a
;; sanitized doc is render-safe but not necessarily content/valid? if the input
;; carried duplicate ids. Immutable; returns a new document.
;;
;; Requires (loaded by harness): block.sx, doc.sx, validate.sx
;; (content/-block-issues).
(define
san-section?
(fn (b) (and (st-instance? b) (= (get b :class) "CtSection"))))
;; a block is render-safe when it has no id/field issues (validate's own checks)
(define san-ok? (fn (b) (= (len (content/-block-issues b)) 0)))
;; drop invalid blocks at this level; recurse into surviving sections so invalid
;; descendants are pruned too.
(define
san-blocks
(fn
(blocks)
(map
(fn
(b)
(if
(san-section? b)
(let
((ch (st-iv-get b "children")))
(if (list? ch) (st-iv-set! b "children" (san-blocks ch)) b))
b))
(filter san-ok? blocks))))
(define
content/sanitize
(fn (doc) (doc-with-blocks doc (san-blocks (doc-blocks doc)))))

View File

@@ -8,11 +8,13 @@
"page": {"pass": 7, "fail": 0}, "page": {"pass": 7, "fail": 0},
"page-full": {"pass": 4, "fail": 0}, "page-full": {"pass": 4, "fail": 0},
"markdown": {"pass": 20, "fail": 0}, "markdown": {"pass": 20, "fail": 0},
"runs": {"pass": 36, "fail": 0},
"text": {"pass": 20, "fail": 0}, "text": {"pass": 20, "fail": 0},
"section": {"pass": 25, "fail": 0}, "section": {"pass": 25, "fail": 0},
"compose": {"pass": 17, "fail": 0}, "compose": {"pass": 17, "fail": 0},
"tree-edit": {"pass": 17, "fail": 0}, "tree-edit": {"pass": 17, "fail": 0},
"move": {"pass": 11, "fail": 0}, "move": {"pass": 24, "fail": 0},
"block-path": {"pass": 13, "fail": 0},
"clone": {"pass": 10, "fail": 0}, "clone": {"pass": 10, "fail": 0},
"query": {"pass": 20, "fail": 0}, "query": {"pass": 20, "fail": 0},
"toc": {"pass": 8, "fail": 0}, "toc": {"pass": 8, "fail": 0},
@@ -30,7 +32,8 @@
"media": {"pass": 15, "fail": 0}, "media": {"pass": 15, "fail": 0},
"data": {"pass": 25, "fail": 0}, "data": {"pass": 25, "fail": 0},
"wire": {"pass": 11, "fail": 0}, "wire": {"pass": 11, "fail": 0},
"validate": {"pass": 23, "fail": 0}, "validate": {"pass": 32, "fail": 0},
"sanitize": {"pass": 12, "fail": 0},
"store": {"pass": 46, "fail": 0}, "store": {"pass": 46, "fail": 0},
"snapshot": {"pass": 20, "fail": 0}, "snapshot": {"pass": 20, "fail": 0},
"crdt": {"pass": 34, "fail": 0}, "crdt": {"pass": 34, "fail": 0},
@@ -42,7 +45,7 @@
"md-doc": {"pass": 12, "fail": 0}, "md-doc": {"pass": 12, "fail": 0},
"fed": {"pass": 20, "fail": 0} "fed": {"pass": 20, "fail": 0}
}, },
"total_pass": 778, "total_pass": 861,
"total_fail": 0, "total_fail": 0,
"total": 778 "total": 861
} }

View File

@@ -12,11 +12,13 @@ _Generated by `lib/content/conformance.sh`_
| page | 7 | 0 | 7 | | page | 7 | 0 | 7 |
| page-full | 4 | 0 | 4 | | page-full | 4 | 0 | 4 |
| markdown | 20 | 0 | 20 | | markdown | 20 | 0 | 20 |
| runs | 36 | 0 | 36 |
| text | 20 | 0 | 20 | | text | 20 | 0 | 20 |
| section | 25 | 0 | 25 | | section | 25 | 0 | 25 |
| compose | 17 | 0 | 17 | | compose | 17 | 0 | 17 |
| tree-edit | 17 | 0 | 17 | | tree-edit | 17 | 0 | 17 |
| move | 11 | 0 | 11 | | move | 24 | 0 | 24 |
| block-path | 13 | 0 | 13 |
| clone | 10 | 0 | 10 | | clone | 10 | 0 | 10 |
| query | 20 | 0 | 20 | | query | 20 | 0 | 20 |
| toc | 8 | 0 | 8 | | toc | 8 | 0 | 8 |
@@ -34,7 +36,8 @@ _Generated by `lib/content/conformance.sh`_
| media | 15 | 0 | 15 | | media | 15 | 0 | 15 |
| data | 25 | 0 | 25 | | data | 25 | 0 | 25 |
| wire | 11 | 0 | 11 | | wire | 11 | 0 | 11 |
| validate | 23 | 0 | 23 | | validate | 32 | 0 | 32 |
| sanitize | 12 | 0 | 12 |
| store | 46 | 0 | 46 | | store | 46 | 0 | 46 |
| snapshot | 20 | 0 | 20 | | snapshot | 20 | 0 | 20 |
| crdt | 34 | 0 | 34 | | crdt | 34 | 0 | 34 |
@@ -45,4 +48,4 @@ _Generated by `lib/content/conformance.sh`_
| md-import | 38 | 0 | 38 | | md-import | 38 | 0 | 38 |
| md-doc | 12 | 0 | 12 | | md-doc | 12 | 0 | 12 |
| fed | 20 | 0 | 20 | | fed | 20 | 0 | 20 |
| **Total** | **778** | **0** | **778** | | **Total** | **861** | **0** | **861** |

View File

@@ -0,0 +1,59 @@
;; Extension — locate a block in the tree (ancestor section path).
(st-bootstrap-classes!)
(content/bootstrap!)
(content-bootstrap-section!)
;; doc: top-level "a", section "s" containing "x" and nested section "i"
;; containing "z".
(define
d
(doc-append
(doc-append (doc-empty "d") (mk-text "a" "A"))
(mk-section
"s"
(list (mk-text "x" "X") (mk-section "i" (list (mk-text "z" "Z")))))))
;; ── block-path ──
(content-test
"top-level block has empty path"
(content/block-path d "a")
(list))
(content-test "one-deep block path" (content/block-path d "x") (list "s"))
(content-test
"two-deep block path"
(content/block-path d "z")
(list "s" "i"))
(content-test "section's own path" (content/block-path d "i") (list "s"))
(content-test "missing id path nil" (content/block-path d "zzz") nil)
;; nil (absent) is distinct from () (present top-level)
(content-test
"absent vs top-level distinguishable"
(if (= (content/block-path d "a") nil) "nil" "list")
"list")
;; ── block-depth ──
(content-test "depth top-level" (content/block-depth d "a") 0)
(content-test "depth one" (content/block-depth d "x") 1)
(content-test "depth two" (content/block-depth d "z") 2)
(content-test "depth section" (content/block-depth d "i") 1)
(content-test "depth absent" (content/block-depth d "zzz") -1)
;; ── path tracks reparenting (composes with move.sx) ──
;; (rebuild expectation directly; move tested elsewhere)
(define
flat
(doc-append
(doc-append (doc-empty "d") (mk-section "sec" (list)))
(mk-text "p" "P")))
(content-test
"before: p at top level"
(content/block-depth flat "p")
0)
;; ── empty doc ──
(content-test
"empty doc path nil"
(content/block-path (doc-empty "e") "x")
nil)

View File

@@ -1,7 +1,8 @@
;; Extension — relative block reorder. ;; Extension — relative block reorder + tree reparent.
(st-bootstrap-classes!) (st-bootstrap-classes!)
(content/bootstrap!) (content/bootstrap!)
(content-bootstrap-section!)
(define (define
d d
@@ -61,3 +62,84 @@
"render after move" "render after move"
(asHTML (content/move-after d "a" "c")) (asHTML (content/move-after d "a" "c"))
"<p>B</p><p>C</p><p>A</p>") "<p>B</p><p>C</p><p>A</p>")
;; ── reparent: move a top-level block INTO a section ──
(define
nd
(doc-append
(doc-append (doc-empty "d") (mk-text "p" "P"))
(mk-section "s" (list (mk-text "x" "X")))))
(content-test
"move-into: block leaves top level"
(doc-ids (content/move-into nd "p" "s" 1))
(list "s"))
(content-test
"move-into: block lands in section at index"
(doc-tree-ids (content/move-into nd "p" "s" 1))
(list "s" "x" "p"))
(content-test
"move-into at front of section"
(doc-tree-ids (content/move-into nd "p" "s" 0))
(list "s" "p" "x"))
(content-test "move-into immutable" (doc-tree-ids nd) (list "p" "s" "x"))
;; ── reparent: move a NESTED block to a different section ──
(define
two
(doc-append
(doc-append (doc-empty "d") (mk-section "s1" (list (mk-text "n" "N"))))
(mk-section "s2" (list (mk-text "y" "Y")))))
(content-test
"move-into across sections"
(doc-tree-ids (content/move-into two "n" "s2" 1))
(list "s1" "s2" "y" "n"))
;; ── promote: nested block out to top level (appended last) ──
(content-test
"promote nested to top level"
(doc-tree-ids (content/promote two "n"))
(list "s1" "s2" "y" "n"))
(content-test
"promote leaves section empty shell"
(doc-ids (content/promote two "n"))
(list "s1" "s2" "n"))
(content-test
"promote a whole section keeps its subtree"
(doc-tree-ids
(content/promote
(doc-append
(doc-empty "d")
(mk-section "o" (list (mk-section "i" (list (mk-text "z" "Z"))))))
"i"))
(list "o" "i" "z"))
;; ── cycle guard: cannot move a section into its own descendant ──
(define
nest
(doc-append
(doc-empty "d")
(mk-section
"outer"
(list (mk-section "inner" (list (mk-text "t" "T")))))))
(content-test
"move section into its own child is a no-op"
(doc-tree-ids (content/move-into nest "outer" "inner" 0))
(list "outer" "inner" "t"))
(content-test
"move block into itself is a no-op"
(doc-tree-ids (content/move-into nest "inner" "inner" 0))
(list "outer" "inner" "t"))
;; ── reparent no-ops on missing ids ──
(content-test
"move-into missing block no-op"
(doc-tree-ids (content/move-into nd "zzz" "s" 0))
(list "p" "s" "x"))
(content-test
"move-into missing section no-op"
(doc-tree-ids (content/move-into nd "p" "zzz" 0))
(list "p" "s" "x"))
(content-test
"promote missing no-op"
(doc-tree-ids (content/promote nd "zzz"))
(list "p" "s" "x"))

227
lib/content/tests/runs.sx Normal file
View File

@@ -0,0 +1,227 @@
;; Phase 5 — rich inline text (structured runs). Acceptance suite.
(st-bootstrap-classes!)
(content/bootstrap!)
(content-bootstrap-markdown!)
(content-bootstrap-text!)
(content-bootstrap-section!)
(content-bootstrap-runs!)
;; one-run helper: a CtText with a single marked run
(define
one
(fn (marks href) (mk-rich-text "p" (list (mk-run "x" marks href)))))
;; ── (1) four render modes ──
;; a paragraph mixing plain + bold + a link
(define
rd
(doc-append
(doc-empty "d")
(mk-rich-text
"p"
(list
(mk-run "the " (list) "")
(mk-run "cat" (list :bold) "")
(mk-run " and " (list) "")
(mk-run "dog" (list :italic :link) "/d")
(mk-run " sat" (list) "")))))
(define p (doc-find rd "p"))
(content-test
"asHTML rich"
(asHTML rd)
"<p>the <strong>cat</strong> and <a href=\"/d\"><em>dog</em></a> sat</p>")
(content-test
"asMarkdown rich"
(asMarkdown rd)
"the **cat** and [_dog_](/d) sat")
(content-test
"asSx rich"
(asSx rd)
"(article (p \"the \" (strong \"cat\") \" and \" (a :href \"/d\" (em \"dog\")) \" sat\"))")
(content-test
"asText rich is plain (no markup)"
(asText rd)
"the cat and dog sat")
;; every mark renders in HTML
(content-test
"mark bold html"
(asHTML (one (list :bold) ""))
"<p><strong>x</strong></p>")
(content-test
"mark italic html"
(asHTML (one (list :italic) ""))
"<p><em>x</em></p>")
(content-test
"mark underline html"
(asHTML (one (list :underline) ""))
"<p><u>x</u></p>")
(content-test
"mark strike html"
(asHTML (one (list :strikethrough) ""))
"<p><s>x</s></p>")
(content-test
"mark code html"
(asHTML (one (list :code) ""))
"<p><code>x</code></p>")
(content-test
"mark sub html"
(asHTML (one (list :subscript) ""))
"<p><sub>x</sub></p>")
(content-test
"mark sup html"
(asHTML (one (list :superscript) ""))
"<p><sup>x</sup></p>")
(content-test
"mark link html"
(asHTML (one (list :link) "/u"))
"<p><a href=\"/u\">x</a></p>")
;; markdown marks
(content-test "mark bold md" (asMarkdown (one (list :bold) "")) "**x**")
(content-test "mark italic md" (asMarkdown (one (list :italic) "")) "_x_")
(content-test
"mark strike md"
(asMarkdown (one (list :strikethrough) ""))
"~~x~~")
(content-test "mark code md" (asMarkdown (one (list :code) "")) "`x`")
(content-test "mark link md" (asMarkdown (one (list :link) "/u")) "[x](/u)")
(content-test
"mark underline md fallback"
(asMarkdown (one (list :underline) ""))
"<u>x</u>")
;; nested marks (bold+italic) — deterministic nesting order
(content-test
"nested marks html"
(asHTML (one (list :bold :italic) ""))
"<p><em><strong>x</strong></em></p>")
;; escaping still happens inside runs
(content-test
"run text escaped html"
(asHTML (mk-rich-text "p" (list (mk-run "a & b <c>" (list :bold) ""))))
"<p><strong>a &amp; b &lt;c&gt;</strong></p>")
;; rich heading + quote + code
(content-test
"rich heading html"
(asHTML
(st-iv-set!
(mk-heading "h" 2 "")
"text"
(list (mk-run "Big " (list) "") (mk-run "bold" (list :bold) ""))))
"<h2>Big <strong>bold</strong></h2>")
(content-test
"rich quote html"
(asHTML
(st-iv-set!
(mk-quote "q" "" "")
"text"
(list (mk-run "wise" (list :italic) ""))))
"<blockquote><em>wise</em></blockquote>")
;; code is verbatim — runs concatenate as plain text, marks ignored
(content-test
"code runs plain html"
(asHTML
(st-iv-set!
(mk-code "c" "py" "")
"text"
(list (mk-run "a=" (list :bold) "") (mk-run "1" (list) ""))))
"<pre><code class=\"language-py\">a=1</code></pre>")
;; ── (2) backward compat: plain-string CtText unchanged ──
(content-test
"plain html"
(asHTML (mk-text "q" "hi & <b>"))
"<p>hi &amp; &lt;b&gt;</p>")
(content-test "plain sx" (asSx (mk-text "q" "hi")) "(p \"hi\")")
(content-test "plain md" (asMarkdown (mk-text "q" "hi")) "hi")
(content-test "plain text" (asText (mk-text "q" "hi")) "hi")
(content-test
"plain heading html"
(asHTML (mk-heading "h" 3 "T"))
"<h3>T</h3>")
;; ── (3) find-replace across runs (per-run, marks preserved) ──
(define
frd
(doc-append
(doc-empty "d")
(mk-rich-text
"p"
(list
(mk-run "the Foo" (list :bold) "")
(mk-run " and Foo here" (list) "")))))
(define frr (content/find-replace frd "Foo" "Bar"))
(content-test
"find-replace rich plain text"
(asText frr)
"the Bar and Bar here")
(content-test
"find-replace rich preserves marks"
(asHTML frr)
"<p><strong>the Bar</strong> and Bar here</p>")
(content-test
"find-replace rich run0 still bold"
(nth (nth (blk-get (doc-find frr "p") "text") 0) 1)
(list "bold"))
;; ── (4) search-text via asText, across run boundary ──
;; "cat sat" spans run1 ("the cat") and run2 (" sat")
(define
sd
(doc-append
(doc-empty "d")
(mk-rich-text
"p"
(list (mk-run "the cat" (list :bold) "") (mk-run " sat" (list) "")))))
(content-test
"search finds substring across runs"
(content/search-text-ids sd "cat sat")
(list "p"))
(content-test "search miss" (content/search-text-ids sd "zzz") (list))
;; ── (5) CRDT invariant — runs are an opaque block-level value ──
(define ra (list (mk-run "x" (list :bold) "")))
(define rb (list (mk-run "y" (list :italic) "")))
(define
s1
(crdt-insert
(crdt-empty)
"p"
"text"
(crdt-pos 5 "a")
{:text ra}
1
"a"))
(define s2 (crdt-update s1 "p" "text" rb 2 "b"))
(content-test
"crdt merge commutes with runs"
(get (crdt-merge s1 s2) :elements)
(get (crdt-merge s2 s1) :elements))
(content-test
"crdt merge idempotent with runs"
(get (crdt-merge s2 s2) :elements)
(get s2 :elements))
;; LWW: later ts (rb, ts 2) wins; runs survive as the field value
(content-test
"crdt LWW keeps latest runs"
(asHTML
(crdt-element->block (get (get (crdt-merge s1 s2) :elements) "p")))
"<p><em>y</em></p>")
;; ── (6) data + wire round-trip runs losslessly ──
(content-test
"data round-trip rich html"
(asHTML (content/from-data (content/to-data rd)))
(asHTML rd))
(content-test
"data round-trip rich text"
(asText (content/from-data (content/to-data rd)))
"the cat and dog sat")
(content-test
"wire round-trip rich html"
(asHTML (content/from-wire (content/to-wire rd)))
(asHTML rd))

View File

@@ -0,0 +1,128 @@
;; Extension — make a document render-safe by dropping invalid blocks.
;; Counterpart to validate; reuses its per-block checks. Tree-wide.
(st-bootstrap-classes!)
(content-bootstrap-blocks!)
(content-bootstrap-doc!)
(content-bootstrap-section!)
;; ── a valid document is returned unchanged (same ids, tree order) ──
(define
good
(doc-append
(doc-append (doc-empty "d") (mk-heading "h" 1 "Title"))
(mk-text "p" "Body")))
(content-test
"valid doc keeps all blocks"
(doc-ids (content/sanitize good))
(list "h" "p"))
(content-test
"valid doc still valid after sanitize"
(content/valid? (content/sanitize good))
true)
;; ── a block with a bad field is dropped ──
(content-test
"bad-field block dropped"
(doc-ids
(content/sanitize
(doc-append
(doc-append (doc-empty "d") (mk-text "ok" "fine"))
(mk-heading "bad" "notnum" "T"))))
(list "ok"))
;; ── unknown block type dropped ──
(define raw (st-iv-set! (st-make-instance "CtBlock") "id" "z"))
(content-test
"unknown-type block dropped"
(doc-ids
(content/sanitize
(doc-append (doc-append (doc-empty "d") (mk-text "ok" "x")) raw)))
(list "ok"))
;; ── blank-id block dropped ──
(content-test
"blank-id block dropped"
(doc-ids
(content/sanitize
(doc-append
(doc-append (doc-empty "d") (mk-text "ok" "x"))
(mk-text "" "y"))))
(list "ok"))
;; ── result is render-safe: no id/field issues remain ──
(content-test
"sanitized has no field/id issues"
(len
(filter
(fn (i) (if (= (get i :kind) "field") true (= (get i :kind) "id")))
(content/validate
(content/sanitize
(doc-append
(doc-append (doc-empty "d") (mk-text "ok" "x"))
(mk-heading "bad" "notnum" "T"))))))
0)
;; ── immutability: original document untouched ──
(define
withbad
(doc-append
(doc-append (doc-empty "d") (mk-text "ok" "x"))
(mk-heading "bad" "notnum" "T")))
(define _ (content/sanitize withbad))
(content-test "original unchanged" (doc-ids withbad) (list "ok" "bad"))
;; ── tree-wide: invalid nested child pruned, valid sibling + section kept ──
(define
nested
(doc-append
(doc-empty "d")
(mk-section
"s"
(list (mk-text "good" "keep") (mk-heading "badc" "notnum" "X")))))
(content-test
"invalid nested child pruned, section kept"
(doc-tree-ids (content/sanitize nested))
(list "s" "good"))
;; ── a section whose own shell is invalid (children not a list) is dropped ──
(define
badsec
(doc-append
(doc-append (doc-empty "d") (mk-text "ok" "x"))
(st-iv-set! (mk-section "s" (list)) "children" "nope")))
(content-test
"invalid section shell dropped whole"
(doc-tree-ids (content/sanitize badsec))
(list "ok"))
;; ── a valid section that loses all children is kept (empty) — sanitize is not
;; normalize; it removes invalid, not empty ──
(define
allbadchildren
(doc-append
(doc-empty "d")
(mk-section "s" (list (mk-heading "b1" "x" "X") (mk-text "" "y")))))
(content-test
"section kept though emptied of invalid children"
(doc-tree-ids (content/sanitize allbadchildren))
(list "s"))
;; ── deeply nested: invalid block two levels down is pruned ──
(define
deep
(doc-append
(doc-empty "d")
(mk-section
"o"
(list (mk-section "i" (list (mk-text "dok" "x") (mk-text "" "bad")))))))
(content-test
"deep invalid pruned"
(doc-tree-ids (content/sanitize deep))
(list "o" "i" "dok"))
;; ── empty document sanitizes to empty ──
(content-test
"empty doc stays empty"
(doc-ids (content/sanitize (doc-empty "e")))
(list))

View File

@@ -5,6 +5,7 @@
(content-bootstrap-blocks!) (content-bootstrap-blocks!)
(content-bootstrap-doc!) (content-bootstrap-doc!)
(content-bootstrap-section!) (content-bootstrap-section!)
(content-bootstrap-table!)
;; ── a fully valid document ── ;; ── a fully valid document ──
(define (define
@@ -164,3 +165,62 @@
(content/validate dup-tree))) (content/validate dup-tree)))
1) 1)
(content-test "tree dup not valid" (content/valid? dup-tree) false) (content-test "tree dup not valid" (content/valid? dup-tree) false)
;; ── collection blocks vetted ELEMENT-DEEP (items/cells must be strings) ──
;; A list whose items field is a list but holds a non-string would pass the old
;; "is a list" check yet crash asText/render — now caught.
(content-test
"list non-string item flagged"
(content/issue-kinds
(doc-append (doc-empty "d") (mk-list "l" true (list "a" 5))))
(list "field"))
(content-test
"list all-string items valid"
(content/valid?
(doc-append (doc-empty "d") (mk-list "l" false (list "a" "b" "c"))))
true)
(content-test
"list empty items valid"
(content/valid? (doc-append (doc-empty "d") (mk-list "l" true (list))))
true)
;; a malformed-list block reports exactly one element issue (not the is-a-list one)
(content-test
"list non-string item single issue"
(len
(content/validate
(doc-append
(doc-empty "d")
(mk-list "l" true (list 1 2)))))
1)
(content-test
"valid table ok"
(content/valid?
(doc-append
(doc-empty "d")
(mk-table "t" (list "H1" "H2") (list (list "a" "b") (list "c" "d")))))
true)
(content-test
"table empty rows valid"
(content/valid?
(doc-append (doc-empty "d") (mk-table "t" (list "H") (list))))
true)
(content-test
"table non-list row flagged"
(content/issue-kinds
(doc-append (doc-empty "d") (mk-table "t" (list "H") (list "notarow"))))
(list "field"))
(content-test
"table non-string cell flagged"
(content/issue-kinds
(doc-append
(doc-empty "d")
(mk-table "t" (list "H") (list (list "ok") (list 9)))))
(list "field"))
(content-test
"table non-string header flagged"
(content/issue-kinds
(doc-append
(doc-empty "d")
(mk-table "t" (list "H" 2) (list (list "a" "b")))))
(list "field"))

View File

@@ -6,6 +6,11 @@
;; Tree detection is inline (class + st-iv-get) so this file needs no section.sx. ;; Tree detection is inline (class + st-iv-get) so this file needs no section.sx.
;; Dispatch on block type is a validation-boundary concern, not core behaviour. ;; Dispatch on block type is a validation-boundary concern, not core behaviour.
;; ;;
;; Collection blocks are vetted element-deep: list items must all be strings and
;; table rows must all be lists of strings — exactly what render/asText/
;; find-replace/search assume — so malformed nested collections are caught at the
;; boundary instead of crashing the render layer downstream.
;;
;; Requires (loaded by harness): block.sx, doc.sx. ;; Requires (loaded by harness): block.sx, doc.sx.
(define ct-issue (fn (id kind detail) {:id id :detail detail :kind kind})) (define ct-issue (fn (id kind detail) {:id id :detail detail :kind kind}))
@@ -36,6 +41,28 @@
(define ct-uniq (fn (xs) (ct-uniq-loop xs (list)))) (define ct-uniq (fn (xs) (ct-uniq-loop xs (list))))
;; every element a string? / every row a list of strings? (for collection blocks)
(define
ct-all-str?
(fn
(xs)
(if
(= (len xs) 0)
true
(if (string? (first xs)) (ct-all-str? (rest xs)) false))))
(define
ct-all-rows?
(fn
(rows)
(if
(= (len rows) 0)
true
(if
(if (list? (first rows)) (ct-all-str? (first rows)) false)
(ct-all-rows? (rest rows))
false))))
;; ── tree flatten (descends into CtSection children; guards malformed children) ── ;; ── tree flatten (descends into CtSection children; guards malformed children) ──
(define (define
ct-section-block? ct-section-block?
@@ -136,30 +163,43 @@
"embed provider must be a string"))) "embed provider must be a string")))
((= t "divider") (list)) ((= t "divider") (list))
((= t "list") ((= t "list")
(append (let
(ct-field-issue ((items (blk-get b "items")))
id (append
(boolean? (blk-get b "ordered")) (ct-field-issue
"list ordered must be a boolean") id
(ct-field-issue (boolean? (blk-get b "ordered"))
id "list ordered must be a boolean")
(list? (blk-get b "items")) (append
"list items must be a list"))) (ct-field-issue id (list? items) "list items must be a list")
(ct-field-issue
id
(if (list? items) (ct-all-str? items) true)
"list items must all be strings")))))
((= t "section") ((= t "section")
(ct-field-issue (ct-field-issue
id id
(list? (blk-get b "children")) (list? (blk-get b "children"))
"section children must be a list")) "section children must be a list"))
((= t "table") ((= t "table")
(append (let
(ct-field-issue ((headers (blk-get b "headers")) (rows (blk-get b "rows")))
id (append
(list? (blk-get b "headers")) (append
"table headers must be a list") (ct-field-issue
(ct-field-issue id
id (list? headers)
(list? (blk-get b "rows")) "table headers must be a list")
"table rows must be a list"))) (ct-field-issue
id
(if (list? headers) (ct-all-str? headers) true)
"table headers must all be strings"))
(append
(ct-field-issue id (list? rows) "table rows must be a list")
(ct-field-issue
id
(if (list? rows) (ct-all-rows? rows) true)
"table rows must all be lists of strings")))))
((= t "callout") ((= t "callout")
(append (append
(ct-field-issue (ct-field-issue

View File

@@ -19,7 +19,7 @@ injected adapter, not core.
## Status (rolling) ## Status (rolling)
`bash lib/content/conformance.sh`**778/778** (Phases 14 COMPLETE + ~34 extensions, hardened: HTML/SX escaping, Markdown render + import/export incl. tables & frontmatter (full round-trip), CvRDT flat + nested-tree + durable replication, tree-aware validation, snapshot cache, doc metadata, plain-text render, nested block trees + deep editing + flatten + relative reorder, doc stats + summary + multi-doc index, table + callout + media blocks, HTML page wrapper + SEO page, doc composition + id-remap, portable data + wire serialization, block query + transforms + find/replace, TOC + anchored headings + outline, normalization) `bash lib/content/conformance.sh`**861/861** (Phases 14 COMPLETE + ~34 extensions, hardened: HTML/SX escaping, Markdown render + import/export incl. tables & frontmatter (full round-trip), CvRDT flat + nested-tree + durable replication, tree-aware validation, snapshot cache, doc metadata, plain-text render, nested block trees + deep editing + flatten + relative reorder, doc stats + summary + multi-doc index, table + callout + media blocks, HTML page wrapper + SEO page, doc composition + id-remap, portable data + wire serialization, block query + transforms + find/replace, TOC + anchored headings + outline, normalization)
## Ground rules ## Ground rules
@@ -106,13 +106,169 @@ lib/content/api.sx ── (content/edit) (content/render) (content/history) ─
- [x] document outline (`outline.sx`: content/outline, nested heading tree) - [x] document outline (`outline.sx`: content/outline, nested heading tree)
- [x] document flatten (`flatten.sx`: content/flatten, un-nest sections; inverse of wrap-section) - [x] document flatten (`flatten.sx`: content/flatten, un-nest sections; inverse of wrap-section)
- [x] relative reorder (`move.sx`: content/move-before/after/to-front/to-back by id) - [x] relative reorder (`move.sx`: content/move-before/after/to-front/to-back by id)
- [x] tree reparent (`move.sx`: content/move-into a section + content/promote out to top level; tree-wide, cycle-safe)
- [x] document normalization (`normalize.sx`: content/normalize, drop empty blocks/sections) - [x] document normalization (`normalize.sx`: content/normalize, drop empty blocks/sections)
- [x] document sanitization (`sanitize.sx`: content/sanitize, drop invalid blocks tree-wide; validate's enforcement partner)
- [x] global find/replace (`find-replace.sx`: content/find-replace across text-bearing blocks) - [x] global find/replace (`find-replace.sx`: content/find-replace across text-bearing blocks)
- [x] portable data serialization (`data.sx`: content/to-data + from-data, round-trips tree) - [x] portable data serialization (`data.sx`: content/to-data + from-data, round-trips tree)
- [x] wire serialization (`wire.sx`: content/to-wire + from-wire, SX-text on the wire) - [x] wire serialization (`wire.sx`: content/to-wire + from-wire, SX-text on the wire)
## Phase 5 — rich inline text (structured runs)
Drives the rose-ash blog migration: lexical post bodies carry inline formatting
(bold/italic/links) but `CtText` held one plain `text` string, so the canonical
lexical→blocks conversion was lossy. Variant **(b)**: a `CtText`'s `text` may be
EITHER a plain string (backward compat) OR a sequence of inline **runs**. Marks
cover the lexical bitmask (bold=1 italic=2 strike=4 underline=8 code=16 sub=32
sup=64) plus link nodes (which carry an href). Applies to `CtText` and its
subclasses `CtHeading`/`CtQuote` (rich), and `CtCode` (verbatim — runs render as
plain concatenated text, no marks).
**Run representation — a Smalltalk-renderable list, not a `{:text :marks}` dict.**
A run is `(text marks href)`: `text` a string, `marks` a list of mark tokens
(`:bold :italic :underline :strikethrough :code :subscript :superscript :link`
SX keywords evaluate to the strings the renderer matches), `href` a string (`""`
when absent; carries the link target). *Why a list and not the dict the brief
sketched:* rendering must happen inside the Smalltalk render methods (nested
blocks dispatch `asHTML`/etc. through Smalltalk message sends), and the
Smalltalk-on-SX layer can iterate SX lists (`do:`/`inject:into:`) but **cannot**
read SX dict fields (`Dictionary>>at:` is broken in lib/smalltalk, which is out
of scope). Lists are Smalltalk-native, render under nesting, and round-trip
through data/wire for free (they're just nested lists+strings). The blog-side
lexical→runs converter targets this `(text marks href)` shape.
Centralised in `runs.sx` (`content-bootstrap-runs!`) which OVERRIDES the
render/markdown/text methods of CtText/CtHeading/CtQuote/CtCode with run-aware
versions that fall through to identical output for plain strings — so it is
opt-in (the blog enables it) and the existing suites, which don't bootstrap it,
are untouched.
- [x] runs render in all four modes — asHTML `<strong>/<em>/<u>/<s>/<code>/<sub>/<sup>/<a href>`, asMarkdown `**`/`_`/`~~`/`` ` ``/`[..](..)` (u/sub/sup fall back to inline HTML), asSx emits nested run structure (`(p (strong "x") " y")`), asText returns the PLAIN concatenation (keeps search/stats/find-replace drift-proof)
- [x] backward compat — a plain-string CtText still renders identically; existing suites stay green
- [x] find-replace rewrites text across runs (per-run, marks preserved); runs join the text-bearing-field dispatch
- [x] search-text finds substrings via asText, including across run boundaries
- [x] CRDT invariant preserved — merge stays at BLOCK granularity (runs are the block's value): ops in any order / twice → identical document
- [x] data + wire serialization round-trip runs losslessly
### Future — Phase 6 (NOT in scope now)
Variant **(c)**: character/run-level concurrent inline CRDT (Peritext/Yjs-style)
so two authors can edit the same paragraph simultaneously — needed later for the
multi-author SX editor that replaces Ghost. Block-granularity (b) is sufficient
for the blog read-path migration. The lexical→runs converter itself lives on the
blog/migration side (mark-set reference: `blog/bp/blog/ghost/lexical_to_sx.py`),
not in lib/content.
## Known limitations
- **Markdown table cells containing `|` do not round-trip.** `asMarkdown` on a
table emits cell text raw (table.sx `CtTable>>asMarkdown:`), so a cell `x|y`
renders the row `| x|y | z |` — which `md/import` then splits into *three*
cells (`md-import.sx` `md/-cells` splits on every `|`). Repro: build
`(mk-table "t" (list "A" "B") (list (list "x|y" "z")))`, `asMarkdown`
re-`md/import` → cells become `("x" "y" "z")`. Same applies to a literal `|`
in a header. (HTML/SX/text/data/wire/CRDT round-trips are unaffected — only
the Markdown text boundary.)
*Fix sketch* (when sx-tree edit tooling is restored — see below): add
`String>>mdCellEscaped` (escape `|``\|`) in table.sx and use it for every
header/cell in `CtTable>>asMarkdown:`; in md-import.sx replace `md/-cells`'
naive `(split … "|")` with an escaped-aware splitter that breaks only on
unescaped `|` and unescapes `\|``|`. Both sides must change together
(export-only escaping makes self-round-trip worse, not better).
*Blocker:* in this worktree every sx-tree **edit** tool (`sx_replace_node`,
`sx_replace_by_pattern`, `sx_insert_near`, …) raises yojson `"Expected
string, got null"`; only `sx_write_file` works. md-import.sx is 449 lines, so
a safe surgical edit isn't currently possible — deferred rather than risk a
full manual rewrite of working import code.
## Progress log ## Progress log
- 2026-06-07 — **Phase 5 — rich inline text (structured runs) DONE.** A CtText's
`text` may now be a list of inline runs `(text marks href)` instead of a plain
string; CtHeading/CtQuote inherit rich rendering, CtCode renders runs as plain
verbatim text. New `runs.sx` (`content-bootstrap-runs!`) overrides the
render/markdown/text methods of CtText & subclasses with run-aware versions
that are byte-identical for plain-string bodies (opt-in; existing suites
untouched). All 4 modes: asHTML emits `<strong>/<em>/<u>/<s>/<code>/<sub>/<sup>`
+ `<a href>`, asMarkdown emits `**`/`_`/`~~`/`` ` ``/`[..](..)` (u/sub/sup →
inline HTML), asSx emits nested run structure `(p "a" (strong "b"))` (matches
the SX editor's wire format), asText returns the PLAIN concatenation — so
search-text/stats/find-replace stay drift-proof. find-replace (`fr-rep-text`)
rewrites per run with marks preserved; search-text finds across run boundaries
via asText; CRDT merge treats the runs list as one block-level LWW value
(commutes/idempotent, verified); data + wire round-trip runs losslessly.
**Design note:** runs are a Smalltalk-renderable LIST, not the brief's
`{:text :marks}` dict — the Smalltalk-on-SX render methods (which must run
under nested dispatch) can iterate SX lists but cannot read SX dict fields
(`Dictionary>>at:` is broken in lib/smalltalk, out of scope). Marks are built
from `:bold`-style keywords (which evaluate to the strings the renderer
matches). Phase 6 (char-level concurrent inline CRDT) recorded as future, not
built. +36 runs tests (44 suites). 861/861.
- 2026-06-07 — Feature: `content/block-path` + `content/block-depth`
(block-path.sx, new suite). The read-side companion to doc-find-deep (locate
the block) and move-into/promote (relocate it): returns the ancestor-section
id chain (root-first) for a block id — where it sits in the tree — or nil if
absent (distinct from the `()` path of a present top-level block). block-depth
is the path length (0 top-level, -1 absent). For breadcrumbs and scoping an
edit to a block's enclosing section; distinct from toc/outline (which work on
headings). Pure traversal. Also ran an adversarial probe this pass: confirmed
clone/remap-ids + prefix-ids are tree-wide, and all 12 block types + CtDoc have
asMarkdown: methods (no missing-render-method bug). +13 tests. 825/825 (43
suites).
- 2026-06-07 — Feature: tree reparent in move.sx. Until now insert/move were
positional and top-level only, so a block could never be moved *into* a section
or *out* of one — a real gap for editing nested documents. Added
`content/move-into doc id section-id i` (relocate a block, from anywhere in the
tree, to be a child of a section at index i) and `content/promote doc id`
(lift a nested block out to the end of the top level; a moved section keeps its
whole subtree). Both are pure tree transforms (consistent with the existing
move family — not new op-log ops) built on doc-find-deep / ct-find-id /
ct-remove-id / ct-replace-id. **Cycle-safe**: move-into no-ops when target is
the block itself or sits inside the block's own subtree, so a section can never
become its own ancestor. +13 move tests (into/promote/across-sections/empty-
shell/whole-section-subtree/cycle-guard/missing-id no-ops). 812/812.
- 2026-06-07 — Feature: `content/sanitize` — the enforcement counterpart to
`validate`. validate *reports* id/field issues; sanitize *removes* the
offending blocks (tree-wide) so federated/imported input that failed
validation can still be rendered/merged without faulting. Reuses validate's
own per-block predicate (`content/-block-issues`) so "what is invalid" stays
single-sourced and can't drift. Distinct from `normalize` (which drops *empty*
blocks): a section emptied of invalid children is kept (sanitize removes
invalid, not empty), but a section whose own shell is invalid (children not a
list) is dropped whole. Scope is per-block id/field validity — it does not
dedupe ids (cross-block, no single right answer). +12 tests (bad-field /
unknown-type / blank-id dropped, deep pruning, invalid-shell section dropped,
immutability, render-safe result). 799/799 (42 suites). (This was a genuine
remaining gap — validate had no enforcement partner — not filler; saturation
note below still holds for the roadmap proper.)
- 2026-06-07 — Audit (markdown round-trip): probed the Markdown text boundary
for round-trip fidelity. Found one real data-corruption bug — table cells
containing `|` don't survive `asMarkdown` → `md/import` (recorded under
**Known limitations** with repro + fix sketch). Could not land the fix this
pass: it must touch md-import.sx (449 lines) and every sx-tree *edit* tool is
currently broken in this worktree (yojson error; only `sx_write_file` works),
so a safe surgical edit isn't possible and a full manual rewrite of working
import code is too risky to be responsible. Deferred + documented rather than
half-fix (export-only escaping worsens self-round-trip). Engine remains
COMPLETE + audited at 787/787; with the roadmap exhausted, the tree-wide
audit done, and the one open finding tooling-blocked, the vertical is
**SATURATED** — pacing the loop down.
- 2026-06-07 — Hardening: validation now vets collection blocks ELEMENT-DEEP.
`validate` previously checked only that list `items` / table `headers`/`rows`
*are lists* — a list holding a non-string, or a table whose rows aren't lists
of strings, passed validation yet crashes asText/render/find-replace/search
(which all assume string items/cells). Added `ct-all-str?`/`ct-all-rows?` and
deepened the list/table branches (guarded so a non-list container reports only
the is-a-list issue, not a spurious element issue). Since validate's job is
guarding imports/federated input, this closes the boundary before the render
layer can fault. +9 validate tests (list non-string item, table non-list row /
non-string cell / non-string header, empties stay valid). 787/787.
- 2026-06-07 — Hardening (tree-wide audit): the public facade `content/find` / - 2026-06-07 — Hardening (tree-wide audit): the public facade `content/find` /
`content/has?` were top-level-only (`doc-find`/`doc-has?`), so you could `content/has?` were top-level-only (`doc-find`/`doc-has?`), so you could
`content/edit` an update/delete to a nested block by id (those ops are `content/edit` an update/delete to a nested block by id (those ops are