Compare commits

...

12 Commits

Author SHA1 Message Date
f68591456e content: Phase 5 — rich inline text via structured runs (861/861)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 40s
CtText.text may be a list of runs (text marks href); CtHeading/CtQuote rich,
CtCode verbatim. New runs.sx overrides render/markdown/text methods (byte-
identical for plain strings, opt-in). 4 modes: HTML tags / markdown / nested
SX / plain asText (drift-proof). find-replace per-run marks-preserving;
search across run boundaries; CRDT block-granularity LWW; data+wire round-trip.
Runs are a Smalltalk-renderable list (not a dict — substrate can't read dict
fields under nested render dispatch). +36 tests (44 suites). Phase 6 (char-
level inline CRDT) recorded as future.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 21:10:56 +00:00
160d0f2dd0 content: content/block-path + block-depth — locate a block in the tree (825/825)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 1m1s
Read-side companion to doc-find-deep + reparent: ancestor-section chain
(root-first) for a block id, nil if absent (distinct from () top-level path);
block-depth is the path length. For breadcrumbs / scoping. New suite +13.
Probe this pass found no bugs: clone/remap tree-wide, all block types have
asMarkdown:.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 19:45:29 +00:00
e9316b37c2 content: tree reparent — move-into section + promote (812/812)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 41s
insert/move were top-level only; a block could never move into/out of a
section. content/move-into (relocate to a section child at index, tree-wide)
+ content/promote (lift nested block to top level, subtree intact). Pure
tree transforms like the rest of move.sx; cycle-safe (rejects moving a block
into its own descendant). +13 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 18:28:36 +00:00
29954689bc content: content/sanitize — drop invalid blocks tree-wide (799/799)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 45s
Enforcement counterpart to validate: removes blocks failing validate's own
per-block id/field checks (reused via content/-block-issues, single-sourced)
so federated/imported input can render safely. Tree-wide; distinct from
normalize (empty vs invalid); keeps valid-shell sections, drops invalid ones.
New suite +12 tests (42 suites total).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 17:18:49 +00:00
f31c7a4002 content: document markdown table-pipe round-trip limitation + fix sketch
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 33s
Probed the Markdown boundary; found table cells containing | don't round-trip
(asMarkdown emits raw |, md/import splits on every |). Recorded under Known
limitations with repro + two-sided fix sketch. Fix blocked: md-import.sx is
449 lines and all sx-tree edit tools error in this worktree (only
sx_write_file works) — deferred rather than risk a full manual rewrite.
Engine SATURATED at 787/787.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 16:08:41 +00:00
c5d9e1480d content: validation vets list items + table cells element-deep (787/787)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 19s
validate only checked that list items / table rows-headers ARE lists; a
non-string item or non-list/non-string-cell row passed yet crashes asText/
render/find-replace/search. Added ct-all-str?/ct-all-rows? + deepened list/
table branches (guarded against double-reporting). +9 validate tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 15:29:54 +00:00
c9a8f05244 content: tree-wide content/find + has? (778/778)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 22s
Facade read-by-id was top-level only while content/edit's update/delete are
tree-wide — could not read back a nested block content/edit just modified.
Added generic ct-find-id (doc.sx) + doc-find-deep/doc-has-deep?; content/find
+ has? now descend into sections. content/find-top/has-top? keep top-level
lookup. Audit: remaining doc-find/ct-index-of callers are positional
insert/move (top-level by design). +6 api tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 14:49:15 +00:00
9051f52f53 content: tree-wide revision diff (772/772)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 1m9s
content/diff + diff-versions enumerated ids top-level only (doc-ids/
doc-find), so diffs of documents with sections missed every nested add/
remove/change. Now via doc-tree-ids + doc-deep-find; sections excluded from
:changed (no own content), still reported in :added/:removed. Flat-doc
diffs unchanged. +9 store tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 13:39:08 +00:00
4d889716a3 content: in-document prose search via asText (763/763)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 36s
content/search-text + search-text-ids find every block whose (asText b)
contains a term — spanning all text-bearing fields by reusing the canonical
asText projection, so it can't drift from stats/find-replace. Section
wrappers excluded. +7 query tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 12:52:34 +00:00
2f626173d9 content: find-replace rewrites all text-bearing fields (756/756)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 53s
fr-rewrite dispatches per block type so image alt, list items, and table
headers/cells are renamed alongside text/heading/code/quote/callout —
matching exactly the set asText/stats/word-count fold into prose. Prior
find-replace skipped them, so a rename stayed visible in counts/exports.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 12:05:11 +00:00
92c0c853a9 content: find-replace covers callout text + 2 tests (752/752)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 53s
fr-has-text? now treats callout as text-bearing, matching asText/stats/
summary. content/find-replace previously skipped callout bodies silently.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 11:10:25 +00:00
94b889c911 content: by-id ops (update/delete) act tree-wide — fixes op-log no-op on nested blocks + 4 tests (750/750)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 14m45s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 10:25:54 +00:00
23 changed files with 1532 additions and 81 deletions

View File

@@ -25,8 +25,13 @@
(define content/append doc-append)
(define content/blocks doc-blocks)
(define content/count doc-count)
(define content/find doc-find)
(define content/has? doc-has?)
;; find / has? are TREE-WIDE by id (descend into sections) — so the facade reads
;; back any block content/edit can update or delete. content/find-top / has-top?
;; keep the top-level-only lookup for callers that mean the ordered sequence.
(define content/find doc-find-deep)
(define content/has? doc-has-deep?)
(define content/find-top doc-find)
(define content/has-top? doc-has?)
(define content/ids doc-ids)
(define content/types doc-types)

45
lib/content/block-path.sx Normal file
View File

@@ -0,0 +1,45 @@
;; content-on-sx — locate a block in the tree (ancestor section path).
;;
;; The read-side companion to doc-find-deep (which returns the block) and the
;; move/reparent ops (which relocate it): content/block-path returns the list of
;; ancestor section ids, root-first, leading to a block id — i.e. where the
;; block sits in the tree. A top-level block has an empty path; a block one
;; section deep has a one-element path; a missing id returns nil (distinct from
;; the empty-list path of a present top-level block). content/block-depth is the
;; path length (0 = top level, -1 = absent). Useful for breadcrumbs and for
;; scoping an edit to a block's enclosing section. Pure traversal; descends into
;; any block carrying a children list, like the rest of the tree helpers.
;;
;; Requires (loaded by harness): block.sx, doc.sx.
(define
bp-in-blocks
(fn
(blocks id trail)
(if
(= (len blocks) 0)
nil
(let
((b (first blocks)))
(if
(= (blk-id b) id)
trail
(let
((ch (st-iv-get b "children")))
(let
((found (if (list? ch) (bp-in-blocks ch id (append trail (list (blk-id b)))) nil)))
(if (= found nil) (bp-in-blocks (rest blocks) id trail) found))))))))
;; ancestor section ids (root-first) for `id`, or nil if the block is absent.
(define
content/block-path
(fn (doc id) (bp-in-blocks (doc-blocks doc) id (list))))
;; depth of `id`: 0 at top level, n nested n sections deep, -1 if absent.
(define
content/block-depth
(fn
(doc id)
(let
((p (content/block-path doc id)))
(if (= p nil) -1 (len p)))))

View File

@@ -15,7 +15,7 @@ if [ ! -x "$SX_SERVER" ]; then
fi
fi
SUITES=(block doc render api meta page page-full markdown text section compose tree-edit move clone query toc anchor outline flatten transform normalize find-replace stats summary index table callout media data wire validate store snapshot crdt crdt-tree crdt-blocks crdt-store sync md-import md-doc fed)
SUITES=(block doc render api meta page page-full markdown runs text section compose tree-edit move block-path clone query toc anchor outline flatten transform normalize find-replace stats summary index table callout media data wire validate sanitize store snapshot crdt crdt-tree crdt-blocks crdt-store sync md-import md-doc fed)
OUT_JSON="lib/content/scoreboard.json"
OUT_MD="lib/content/scoreboard.md"
@@ -48,6 +48,7 @@ run_suite() {
(load "lib/content/compose.sx")
(load "lib/content/tree-edit.sx")
(load "lib/content/move.sx")
(load "lib/content/block-path.sx")
(load "lib/content/clone.sx")
(load "lib/content/query.sx")
(load "lib/content/toc.sx")
@@ -68,7 +69,9 @@ run_suite() {
(load "lib/content/page.sx")
(load "lib/content/page-full.sx")
(load "lib/content/markdown.sx")
(load "lib/content/runs.sx")
(load "lib/content/validate.sx")
(load "lib/content/sanitize.sx")
(load "lib/content/store.sx")
(load "lib/content/snapshot.sx")
(load "lib/content/crdt.sx")

View File

@@ -5,14 +5,19 @@
;; and returns a NEW document — the input is never mutated, so any version is the
;; head of an op stream (replay-friendly for persist + CRDT merge).
;;
;; CtDoc also carries optional metadata (title/slug/tags) — see meta.sx for the
;; ergonomic API; they default nil and do not affect block operations.
;; By-id ops (update/delete) and by-id lookup (doc-find-deep/doc-has-deep?) are
;; TREE-WIDE: they descend into any block carrying a `children` list (i.e.
;; sections), since ids are unique across the tree. This keeps the persist
;; op-log, content/edit and content/find correct for nested documents.
;; insert/move are positional and act at the top level.
;;
;; CtDoc also carries optional metadata (title/slug/tags) — see meta.sx.
;;
;; Op shapes (data, not objects — they are the persist event payload):
;; {:op "insert" :block <blk> :after <id|nil>} ; after nil = prepend
;; {:op "update" :id <id> :field <name> :value <v>}
;; {:op "move" :id <id> :index <n>}
;; {:op "delete" :id <id>}
;; {:op "insert" :block <blk> :after <id|nil>} ; after nil = prepend (top level)
;; {:op "update" :id <id> :field <name> :value <v>} ; tree-wide by id
;; {:op "move" :id <id> :index <n>} ; top level
;; {:op "delete" :id <id>} ; tree-wide by id
(define
content-bootstrap-doc!
@@ -76,17 +81,58 @@
(first blocks)
(ct-insert-at (rest blocks) (- i 1) x))))))
;; tree-wide remove by id: drop matches at this level, recurse into children
;; (blocks carrying a `children` list, i.e. sections).
(define
ct-remove-id
(fn
(blocks id)
(filter (fn (b) (if (= (blk-id b) id) false true)) blocks)))
(map
(fn
(b)
(let
((ch (st-iv-get b "children")))
(if (list? ch) (st-iv-set! b "children" (ct-remove-id ch id)) b)))
(filter (fn (b) (if (= (blk-id b) id) false true)) blocks))))
;; tree-wide replace by id: apply f to the match wherever it sits in the tree.
(define
ct-replace-id
(fn
(blocks id f)
(map (fn (b) (if (= (blk-id b) id) (f b) b)) blocks)))
(map
(fn
(b)
(if
(= (blk-id b) id)
(f b)
(let
((ch (st-iv-get b "children")))
(if
(list? ch)
(st-iv-set! b "children" (ct-replace-id ch id f))
b))))
blocks)))
;; tree-wide find by id: first block matching id anywhere in the tree, or nil.
;; Descends into any `children` list, mirroring ct-replace-id/ct-remove-id.
(define
ct-find-id
(fn
(blocks id)
(if
(= (len blocks) 0)
nil
(let
((b (first blocks)))
(if
(= (blk-id b) id)
b
(let
((ch (st-iv-get b "children")))
(let
((nested (if (list? ch) (ct-find-id ch id) nil)))
(if (= nested nil) (ct-find-id (rest blocks) id) nested))))))))
;; ── query ──
(define doc-index-of (fn (doc id) (ct-index-of (doc-blocks doc) id)))
@@ -103,6 +149,14 @@
doc-has?
(fn (doc id) (if (= (doc-index-of doc id) -1) false true)))
;; tree-wide lookup by id — reads a nested block by the same id content/edit can
;; update/delete (no section.sx dependency; uses the generic children descent).
(define doc-find-deep (fn (doc id) (ct-find-id (doc-blocks doc) id)))
(define
doc-has-deep?
(fn (doc id) (if (= (doc-find-deep doc id) nil) false true)))
;; ── structural edits (each returns a new document) ──
(define doc-with-blocks (fn (doc blocks) (st-iv-set! doc "blocks" blocks)))

View File

@@ -1,10 +1,22 @@
;; content-on-sx — global find/replace across text-bearing blocks.
;; content-on-sx — global find/replace across every text-bearing field.
;;
;; Replaces every occurrence of `from` with `to` in the text field of text /
;; heading / code / quote blocks, tree-wide (via the transform layer). For
;; renaming a term throughout a document. Immutable; case-sensitive.
;; Replaces every occurrence of `from` with `to` in the text-bearing fields of
;; a document, tree-wide (via the transform layer):
;; - the `text` of text / heading / code / quote / callout blocks
;; - the `alt` of image blocks
;; - each item of list blocks
;; - every header and cell of table blocks
;; This is exactly the set asText / stats / summary draw prose from, so a rename
;; via content/find-replace and a word count over asText stay consistent.
;; Immutable; case-sensitive.
;;
;; Requires (loaded by harness): block.sx, transform.sx (content/map-blocks).
;; A text field may be a plain string OR a list of rich-text runs (Phase 5,
;; run = (text marks href)). fr-rep-text rewrites per run, preserving each run's
;; marks/href; a match that physically straddles two runs is not joined (the
;; replacement would have no single mark set) — each run is rewritten in place.
;;
;; Requires (loaded by harness): block.sx, transform.sx (content/map-blocks),
;; table.sx (CtTable ivars).
(define
fr-in?
@@ -15,17 +27,71 @@
((= (first xs) x) true)
(else (fr-in? x (rest xs))))))
(define fr-rep (fn (s from to) (replace (str s) from to)))
;; rewrite a text-bearing field that is either a plain string or a runs list
(define
fr-rep-text
(fn
(v from to)
(if
(list? v)
(map
(fn
(r)
(list
(fr-rep (nth r 0) from to)
(nth r 1)
(nth r 2)))
v)
(fr-rep v from to))))
;; Blocks whose prose content find/replace rewrites (matches asText's set).
(define
fr-has-text?
(fn (b) (fr-in? (blk-type b) (list "text" "heading" "code" "quote"))))
(fn
(b)
(fr-in?
(blk-type b)
(list "text" "heading" "code" "quote" "callout" "image" "list" "table"))))
;; Per-type field rewrite. Each branch returns a new (copy-on-write) block.
(define
fr-rewrite
(fn
(b from to)
(let
((t (blk-type b)))
(cond
((= t "image")
(blk-set b "alt" (fr-rep (blk-get b "alt") from to)))
((= t "list")
(let
((items (blk-get b "items")))
(if
(list? items)
(blk-set b "items" (map (fn (it) (fr-rep it from to)) items))
b)))
((= t "table")
(let
((hs (blk-get b "headers")) (rs (blk-get b "rows")))
(let
((b1 (if (list? hs) (blk-set b "headers" (map (fn (h) (fr-rep h from to)) hs)) b)))
(if
(list? rs)
(blk-set
b1
"rows"
(map
(fn
(r)
(if (list? r) (map (fn (c) (fr-rep c from to)) r) r))
rs))
b1))))
(else (blk-set b "text" (fr-rep-text (blk-get b "text") from to)))))))
(define
content/find-replace
(fn
(doc from to)
(content/map-blocks
doc
fr-has-text?
(fn
(b)
(blk-set b "text" (replace (str (blk-get b "text")) from to))))))
(content/map-blocks doc fr-has-text? (fn (b) (fr-rewrite b from to)))))

View File

@@ -1,8 +1,13 @@
;; content-on-sx — relative block reorder.
;; content-on-sx — block reorder + reparent.
;;
;; Move a top-level block to just before / after another block by id — more
;; ergonomic than the index-based doc-move. No-op if either id is missing.
;; Immutable; composes the doc.sx list helpers.
;; Relative reorder of top-level blocks (move-before/after/to-front/to-back by
;; id) plus TREE reparenting: move a block into a section (content/move-into) or
;; promote a nested block back out to the top level (content/promote). Reparent
;; ops are tree-wide (the block may start anywhere) and cycle-safe — moving a
;; block into its own descendant is rejected (no-op), so a section can never
;; become its own ancestor. No-op if any id is missing. Immutable; composes the
;; doc.sx list + tree helpers (doc-find-deep / ct-find-id / ct-remove-id /
;; ct-replace-id / ct-insert-at).
;;
;; Requires (loaded by harness): doc.sx.
@@ -67,3 +72,57 @@
(doc-with-blocks
doc
(append (ct-remove-id (doc-blocks doc) id) (list blk)))))))
;; ── reparent (tree-wide) ──
;; move block `id` (from anywhere in the tree) to be a child of section
;; `section-id` at index `i`. No-op if either id is missing, if id = section-id,
;; or if section-id sits inside id's own subtree (would create a cycle).
(define
content/move-into
(fn
(doc id section-id i)
(let
((blk (doc-find-deep doc id)))
(if
(= blk nil)
doc
(if
(= (doc-find-deep doc section-id) nil)
doc
(if
(= id section-id)
doc
(if
(= (ct-find-id (list blk) section-id) nil)
(let
((without (ct-remove-id (doc-blocks doc) id)))
(doc-with-blocks
doc
(ct-replace-id
without
section-id
(fn
(sec)
(let
((ch (st-iv-get sec "children")))
(if
(list? ch)
(st-iv-set! sec "children" (ct-insert-at ch i blk))
sec))))))
doc)))))))
;; promote block `id` (wherever it sits) out to the end of the top level. If it
;; is already top-level this is a move-to-back. No-op if missing. A section keeps
;; its whole subtree.
(define
content/promote
(fn
(doc id)
(let
((blk (doc-find-deep doc id)))
(if
(= blk nil)
doc
(doc-with-blocks
doc
(append (ct-remove-id (doc-blocks doc) id) (list blk)))))))

View File

@@ -1,10 +1,10 @@
;; content-on-sx — block query + table of contents.
;;
;; Collect blocks across the whole tree (descending into sections) by predicate
;; or type, and derive a table of contents from headings. Tree detection is
;; inline (class + st-iv-get) so this needs no section.sx.
;; or type, search them by prose, and derive a table of contents from headings.
;; Tree detection is inline (class + st-iv-get) so this needs no section.sx.
;;
;; Requires (loaded by harness): block.sx, doc.sx.
;; Requires (loaded by harness): block.sx, doc.sx, text.sx (asText for search).
(define
qry-section?
@@ -45,6 +45,30 @@
content/select-ids
(fn (doc pred) (map (fn (b) (blk-id b)) (content/select doc pred))))
;; Blocks (tree-wide, excluding section containers) whose own prose contains
;; `term`. "Prose" is (asText b), so search covers exactly what every block
;; exposes as text — text/heading/code/quote/callout text, image alt, list
;; items, table headers+cells — with no separate field list to drift from
;; asText / find-replace / stats. Case-sensitive substring match.
(define
content/search-text
(fn
(doc term)
(content/select
doc
(fn
(b)
(and
(not (qry-section? b))
(>= (index-of (asText b) term) 0))))))
;; Same search, returning matching block ids in document order.
(define
content/search-text-ids
(fn
(doc term)
(map (fn (b) (blk-id b)) (content/search-text doc term))))
;; table of contents: {:id :level :text} for every heading, in document order.
(define
content/headings

118
lib/content/runs.sx Normal file
View File

@@ -0,0 +1,118 @@
;; content-on-sx — Phase 5: rich inline text (structured runs).
;;
;; A CtText's `text` ivar may be EITHER a plain string (backward compat) OR a
;; list of inline RUNS. A run is a 3-element list (text marks href):
;; text — a string
;; marks — a list of mark tokens, a subset of
;; :bold :italic :underline :strikethrough :code :subscript
;; :superscript :link (SX keywords evaluate to the strings the
;; Smalltalk renderer compares against; build them with keywords)
;; href — a string ("" when absent; the link target for a :link mark)
;;
;; Runs are a LIST, not a {:text :marks} dict, because rendering happens inside
;; the Smalltalk render methods (nested blocks dispatch asHTML/etc. via Smalltalk
;; message sends) and the Smalltalk-on-SX layer can iterate SX lists but cannot
;; read SX dict fields. Lists are Smalltalk-native, render under nesting, and
;; round-trip through data/wire for free.
;;
;; content-bootstrap-runs! OVERRIDES the render/markdown/text methods of CtText
;; and its subclasses (CtHeading/CtQuote rich; CtCode verbatim — runs render as
;; plain concatenated text) with run-aware versions that produce IDENTICAL output
;; for a plain-string body. Opt-in: call after the render/markdown/text
;; bootstraps; suites that don't call it are unaffected.
;;
;; Requires (loaded by harness): block.sx, render.sx, markdown.sx, text.sx.
;; ── SX-side run helpers ──
(define mk-run (fn (text marks href) (list text marks href)))
(define mk-run-plain (fn (text) (list text (list) "")))
(define run-text (fn (r) (nth r 0)))
(define run-marks (fn (r) (nth r 1)))
(define run-href (fn (r) (nth r 2)))
;; a CtText body is "rich" iff it is a runs list (vs a plain string)
(define runs? (fn (v) (list? v)))
;; build a CtText whose body is a list of runs
(define
mk-rich-text
(fn (id runs) (st-iv-set! (mk-text id "") "text" runs)))
(define
content-bootstrap-runs!
(fn
()
(begin
(ct-def-method!
"CtText"
"runHtml:"
"runHtml: run | frag marks href | frag := (run at: 1) htmlEscaped. marks := run at: 2. href := run at: 3. marks do: [:m | (m = 'bold') ifTrue: [frag := '<strong>' , frag , '</strong>']. (m = 'italic') ifTrue: [frag := '<em>' , frag , '</em>']. (m = 'underline') ifTrue: [frag := '<u>' , frag , '</u>']. (m = 'strikethrough') ifTrue: [frag := '<s>' , frag , '</s>']. (m = 'code') ifTrue: [frag := '<code>' , frag , '</code>']. (m = 'subscript') ifTrue: [frag := '<sub>' , frag , '</sub>']. (m = 'superscript') ifTrue: [frag := '<sup>' , frag , '</sup>']. (m = 'link') ifTrue: [frag := '<a href=\"' , href htmlEscaped , '\">' , frag , '</a>']]. ^ frag")
(ct-def-method!
"CtText"
"runSx:"
"runSx: run | frag marks href | frag := '\"' , (run at: 1) sxEscaped , '\"'. marks := run at: 2. href := run at: 3. marks do: [:m | (m = 'bold') ifTrue: [frag := '(strong ' , frag , ')']. (m = 'italic') ifTrue: [frag := '(em ' , frag , ')']. (m = 'underline') ifTrue: [frag := '(u ' , frag , ')']. (m = 'strikethrough') ifTrue: [frag := '(s ' , frag , ')']. (m = 'code') ifTrue: [frag := '(code ' , frag , ')']. (m = 'subscript') ifTrue: [frag := '(sub ' , frag , ')']. (m = 'superscript') ifTrue: [frag := '(sup ' , frag , ')']. (m = 'link') ifTrue: [frag := '(a :href \"' , href sxEscaped , '\" ' , frag , ')']]. ^ frag")
(ct-def-method!
"CtText"
"runMd:"
"runMd: run | frag marks href | frag := (run at: 1). marks := run at: 2. href := run at: 3. marks do: [:m | (m = 'bold') ifTrue: [frag := '**' , frag , '**']. (m = 'italic') ifTrue: [frag := '_' , frag , '_']. (m = 'strikethrough') ifTrue: [frag := '~~' , frag , '~~']. (m = 'code') ifTrue: [frag := '`' , frag , '`']. (m = 'underline') ifTrue: [frag := '<u>' , frag , '</u>']. (m = 'subscript') ifTrue: [frag := '<sub>' , frag , '</sub>']. (m = 'superscript') ifTrue: [frag := '<sup>' , frag , '</sup>']. (m = 'link') ifTrue: [frag := '[' , frag , '](' , href , ')']]. ^ frag")
(ct-def-method!
"CtText"
"inlineHtml"
"inlineHtml | out | (text class name = 'String') ifTrue: [^ text htmlEscaped]. out := ''. text do: [:run | out := out , (self runHtml: run)]. ^ out")
(ct-def-method!
"CtText"
"inlineSx"
"inlineSx | out | (text class name = 'String') ifTrue: [^ '\"' , text sxEscaped , '\"']. out := ''. text do: [:run | out := (out = '' ifTrue: [self runSx: run] ifFalse: [out , ' ' , (self runSx: run)])]. ^ out")
(ct-def-method!
"CtText"
"inlineMd"
"inlineMd | out | (text class name = 'String') ifTrue: [^ text]. out := ''. text do: [:run | out := out , (self runMd: run)]. ^ out")
(ct-def-method!
"CtText"
"inlineText"
"inlineText | out | (text class name = 'String') ifTrue: [^ text]. out := ''. text do: [:run | out := out , (run at: 1)]. ^ out")
(ct-def-method!
"CtText"
"asHTML"
"asHTML ^ '<p>' , self inlineHtml , '</p>'")
(ct-def-method! "CtText" "asSx" "asSx ^ '(p ' , self inlineSx , ')'")
(ct-def-method! "CtText" "asMarkdown:" "asMarkdown: nl ^ self inlineMd")
(ct-def-method! "CtText" "asText" "asText ^ self inlineText")
(ct-def-method!
"CtHeading"
"asHTML"
"asHTML | t | t := level printString. ^ '<h' , t , '>' , self inlineHtml , '</h' , t , '>'")
(ct-def-method!
"CtHeading"
"asSx"
"asSx | t | t := level printString. ^ '(h' , t , ' ' , self inlineSx , ')'")
(ct-def-method!
"CtHeading"
"asMarkdown:"
"asMarkdown: nl | h i | h := ''. i := 0. [i < level] whileTrue: [h := h , '#'. i := i + 1]. ^ h , ' ' , self inlineMd")
(ct-def-method! "CtHeading" "asText" "asText ^ self inlineText")
(ct-def-method!
"CtQuote"
"asHTML"
"asHTML ^ '<blockquote>' , self inlineHtml , '</blockquote>'")
(ct-def-method!
"CtQuote"
"asSx"
"asSx ^ '(blockquote ' , self inlineSx , ')'")
(ct-def-method!
"CtQuote"
"asMarkdown:"
"asMarkdown: nl ^ '> ' , self inlineMd")
(ct-def-method! "CtQuote" "asText" "asText ^ self inlineText")
(ct-def-method!
"CtCode"
"asHTML"
"asHTML ^ '<pre><code class=\"language-' , language htmlEscaped , '\">' , self inlineText htmlEscaped , '</code></pre>'")
(ct-def-method!
"CtCode"
"asSx"
"asSx ^ '(pre (code \"' , self inlineText sxEscaped , '\"))'")
(ct-def-method!
"CtCode"
"asMarkdown:"
"asMarkdown: nl ^ '```' , language , nl , self inlineText , nl , '```'")
(ct-def-method! "CtCode" "asText" "asText ^ self inlineText")
true)))

47
lib/content/sanitize.sx Normal file
View File

@@ -0,0 +1,47 @@
;; content-on-sx — make a document render-safe by dropping invalid blocks.
;;
;; The enforcement counterpart to validate: where content/validate REPORTS id /
;; field issues, content/sanitize REMOVES the offending blocks so the result can
;; be rendered/merged without faulting on malformed input (federated or imported
;; documents that failed validation). Tree-wide: descends into sections, pruning
;; invalid descendants; a section whose own shell is valid is kept (even if it
;; ends up empty — that is normalize's job, not sanitize's), but a section whose
;; own check fails (e.g. children is not a list) is dropped whole.
;;
;; Reuses validate's per-block predicate (content/-block-issues), so the set of
;; "what is invalid" stays single-sourced and can't drift from content/validate.
;; sanitize addresses per-block id/field validity only; it does NOT resolve
;; duplicate ids (a cross-block concern with no single right answer), so a
;; sanitized doc is render-safe but not necessarily content/valid? if the input
;; carried duplicate ids. Immutable; returns a new document.
;;
;; Requires (loaded by harness): block.sx, doc.sx, validate.sx
;; (content/-block-issues).
(define
san-section?
(fn (b) (and (st-instance? b) (= (get b :class) "CtSection"))))
;; a block is render-safe when it has no id/field issues (validate's own checks)
(define san-ok? (fn (b) (= (len (content/-block-issues b)) 0)))
;; drop invalid blocks at this level; recurse into surviving sections so invalid
;; descendants are pruned too.
(define
san-blocks
(fn
(blocks)
(map
(fn
(b)
(if
(san-section? b)
(let
((ch (st-iv-get b "children")))
(if (list? ch) (st-iv-set! b "children" (san-blocks ch)) b))
b))
(filter san-ok? blocks))))
(define
content/sanitize
(fn (doc) (doc-with-blocks doc (san-blocks (doc-blocks doc)))))

View File

@@ -3,25 +3,27 @@
"block": {"pass": 38, "fail": 0},
"doc": {"pass": 40, "fail": 0},
"render": {"pass": 42, "fail": 0},
"api": {"pass": 26, "fail": 0},
"api": {"pass": 32, "fail": 0},
"meta": {"pass": 27, "fail": 0},
"page": {"pass": 7, "fail": 0},
"page-full": {"pass": 4, "fail": 0},
"markdown": {"pass": 20, "fail": 0},
"runs": {"pass": 36, "fail": 0},
"text": {"pass": 20, "fail": 0},
"section": {"pass": 25, "fail": 0},
"compose": {"pass": 17, "fail": 0},
"tree-edit": {"pass": 17, "fail": 0},
"move": {"pass": 11, "fail": 0},
"move": {"pass": 24, "fail": 0},
"block-path": {"pass": 13, "fail": 0},
"clone": {"pass": 10, "fail": 0},
"query": {"pass": 13, "fail": 0},
"query": {"pass": 20, "fail": 0},
"toc": {"pass": 8, "fail": 0},
"anchor": {"pass": 6, "fail": 0},
"outline": {"pass": 14, "fail": 0},
"flatten": {"pass": 10, "fail": 0},
"transform": {"pass": 12, "fail": 0},
"normalize": {"pass": 11, "fail": 0},
"find-replace": {"pass": 10, "fail": 0},
"find-replace": {"pass": 16, "fail": 0},
"stats": {"pass": 17, "fail": 0},
"summary": {"pass": 14, "fail": 0},
"index": {"pass": 13, "fail": 0},
@@ -30,8 +32,9 @@
"media": {"pass": 15, "fail": 0},
"data": {"pass": 25, "fail": 0},
"wire": {"pass": 11, "fail": 0},
"validate": {"pass": 23, "fail": 0},
"store": {"pass": 33, "fail": 0},
"validate": {"pass": 32, "fail": 0},
"sanitize": {"pass": 12, "fail": 0},
"store": {"pass": 46, "fail": 0},
"snapshot": {"pass": 20, "fail": 0},
"crdt": {"pass": 34, "fail": 0},
"crdt-tree": {"pass": 21, "fail": 0},
@@ -42,7 +45,7 @@
"md-doc": {"pass": 12, "fail": 0},
"fed": {"pass": 20, "fail": 0}
},
"total_pass": 746,
"total_pass": 861,
"total_fail": 0,
"total": 746
"total": 861
}

View File

@@ -7,25 +7,27 @@ _Generated by `lib/content/conformance.sh`_
| block | 38 | 0 | 38 |
| doc | 40 | 0 | 40 |
| render | 42 | 0 | 42 |
| api | 26 | 0 | 26 |
| api | 32 | 0 | 32 |
| meta | 27 | 0 | 27 |
| page | 7 | 0 | 7 |
| page-full | 4 | 0 | 4 |
| markdown | 20 | 0 | 20 |
| runs | 36 | 0 | 36 |
| text | 20 | 0 | 20 |
| section | 25 | 0 | 25 |
| compose | 17 | 0 | 17 |
| tree-edit | 17 | 0 | 17 |
| move | 11 | 0 | 11 |
| move | 24 | 0 | 24 |
| block-path | 13 | 0 | 13 |
| clone | 10 | 0 | 10 |
| query | 13 | 0 | 13 |
| query | 20 | 0 | 20 |
| toc | 8 | 0 | 8 |
| anchor | 6 | 0 | 6 |
| outline | 14 | 0 | 14 |
| flatten | 10 | 0 | 10 |
| transform | 12 | 0 | 12 |
| normalize | 11 | 0 | 11 |
| find-replace | 10 | 0 | 10 |
| find-replace | 16 | 0 | 16 |
| stats | 17 | 0 | 17 |
| summary | 14 | 0 | 14 |
| index | 13 | 0 | 13 |
@@ -34,8 +36,9 @@ _Generated by `lib/content/conformance.sh`_
| media | 15 | 0 | 15 |
| data | 25 | 0 | 25 |
| wire | 11 | 0 | 11 |
| validate | 23 | 0 | 23 |
| store | 33 | 0 | 33 |
| validate | 32 | 0 | 32 |
| sanitize | 12 | 0 | 12 |
| store | 46 | 0 | 46 |
| snapshot | 20 | 0 | 20 |
| crdt | 34 | 0 | 34 |
| crdt-tree | 21 | 0 | 21 |
@@ -45,4 +48,4 @@ _Generated by `lib/content/conformance.sh`_
| md-import | 38 | 0 | 38 |
| md-doc | 12 | 0 | 12 |
| fed | 20 | 0 | 20 |
| **Total** | **746** | **0** | **746** |
| **Total** | **861** | **0** | **861** |

View File

@@ -5,9 +5,10 @@
;; replay of its op stream up to a sequence number; the materialised doc is a
;; cache, never primary state.
;;
;; Requires (loaded by the harness): block.sx, doc.sx, and persist
;; (event/backend/log/kv/api). The persist backend `b` is opened by the caller
;; via (persist/open) and injected — content knows nothing about which backend.
;; Requires (loaded by the harness): block.sx, doc.sx, section.sx (doc-deep-find
;; + doc-tree-ids, for the tree-wide diff), plus persist (event/backend/log/kv/
;; api). The persist backend `b` is opened by the caller via (persist/open) and
;; injected — content knows nothing about which backend.
(define content/-stream (fn (doc-id) (str "content:" doc-id)))
@@ -69,11 +70,18 @@
(fn (b doc-id) (map (fn (ev) {:type (persist/event-type ev) :at (persist/event-at ev) :seq (persist/event-seq ev)}) (content/log b doc-id))))
;; ── diff between two materialised document versions ──
;; Returns {:added (ids) :removed (ids) :changed (ids)} where changed = ids
;; present in both whose block content differs.
(define
content/-missing?
(fn (doc id) (= (ct-index-of (doc-blocks doc) id) -1)))
;; Tree-wide: ids are enumerated across the whole block tree (descending into
;; sections), so nested-block adds/removes/changes are detected, not just
;; top-level ones. Returns {:added :removed :changed} (lists of ids):
;; :added — ids present (anywhere) in `new` but not in `old`
;; :removed — ids present (anywhere) in `old` but not in `new`
;; :changed — content blocks present in both whose block value differs
;; Section containers never appear in :changed (they hold no own content — a
;; child change surfaces as that child's own entry); a whole section appearing
;; or disappearing shows up in :added / :removed by its id.
(define content/-all-ids (fn (doc) (doc-tree-ids doc)))
(define content/-missing? (fn (doc id) (= (doc-deep-find doc id) nil)))
(define
content/-changed
@@ -83,15 +91,16 @@
(fn
(id)
(let
((bo (doc-find old id)) (bn (doc-find new id)))
((bo (doc-deep-find old id)) (bn (doc-deep-find new id)))
(cond
((= bo nil) false)
((= bn nil) false)
((= (blk-type bo) "section") false)
((= bo bn) false)
(else true))))
(doc-ids old))))
(content/-all-ids old))))
(define content/diff (fn (old new) {:changed (content/-changed old new) :removed (filter (fn (id) (content/-missing? new id)) (doc-ids old)) :added (filter (fn (id) (content/-missing? old id)) (doc-ids new))}))
(define content/diff (fn (old new) {:changed (content/-changed old new) :removed (filter (fn (id) (content/-missing? new id)) (content/-all-ids old)) :added (filter (fn (id) (content/-missing? old id)) (content/-all-ids new))}))
;; convenience: diff two persisted versions by seq.
(define

View File

@@ -97,3 +97,37 @@
"render original unchanged"
(content/render d1 "html")
"<h1>Hi</h1><p>World</p>")
;; ── facade find/has? are TREE-WIDE (reach into sections); find-top/has-top?
;; keep the top-level-only lookup. This makes the read-by-id surface consistent
;; with content/edit, whose update/delete are already tree-wide. ──
(content-bootstrap-section!)
(define
nd
(content/append
(content/empty "nested")
(mk-section
"sec"
(list (content/block "text" "inner" (list (list "text" "deep")))))))
(content-test
"find nested (deep)"
(blk-id (content/find nd "inner"))
"inner")
(content-test "has? nested (deep)" (content/has? nd "inner") true)
(content-test "find-top misses nested" (content/find-top nd "inner") nil)
(content-test "has-top? misses nested" (content/has-top? nd "inner") false)
(content-test
"find-top sees top-level"
(blk-id (content/find-top nd "sec"))
"sec")
;; a nested block updated by id via content/edit is now readable by id via
;; content/find (was impossible when find was top-level-only).
(content-test
"edit-then-find nested round-trip"
(str
(blk-send
(content/find
(content/edit nd (content/update "inner" "text" "edited"))
"inner")
"text"))
"edited")

View File

@@ -0,0 +1,59 @@
;; Extension — locate a block in the tree (ancestor section path).
(st-bootstrap-classes!)
(content/bootstrap!)
(content-bootstrap-section!)
;; doc: top-level "a", section "s" containing "x" and nested section "i"
;; containing "z".
(define
d
(doc-append
(doc-append (doc-empty "d") (mk-text "a" "A"))
(mk-section
"s"
(list (mk-text "x" "X") (mk-section "i" (list (mk-text "z" "Z")))))))
;; ── block-path ──
(content-test
"top-level block has empty path"
(content/block-path d "a")
(list))
(content-test "one-deep block path" (content/block-path d "x") (list "s"))
(content-test
"two-deep block path"
(content/block-path d "z")
(list "s" "i"))
(content-test "section's own path" (content/block-path d "i") (list "s"))
(content-test "missing id path nil" (content/block-path d "zzz") nil)
;; nil (absent) is distinct from () (present top-level)
(content-test
"absent vs top-level distinguishable"
(if (= (content/block-path d "a") nil) "nil" "list")
"list")
;; ── block-depth ──
(content-test "depth top-level" (content/block-depth d "a") 0)
(content-test "depth one" (content/block-depth d "x") 1)
(content-test "depth two" (content/block-depth d "z") 2)
(content-test "depth section" (content/block-depth d "i") 1)
(content-test "depth absent" (content/block-depth d "zzz") -1)
;; ── path tracks reparenting (composes with move.sx) ──
;; (rebuild expectation directly; move tested elsewhere)
(define
flat
(doc-append
(doc-append (doc-empty "d") (mk-section "sec" (list)))
(mk-text "p" "P")))
(content-test
"before: p at top level"
(content/block-depth flat "p")
0)
;; ── empty doc ──
(content-test
"empty doc path nil"
(content/block-path (doc-empty "e") "x")
nil)

View File

@@ -1,8 +1,10 @@
;; Extension — global find/replace across text-bearing blocks.
;; Extension — global find/replace across every text-bearing field.
(st-bootstrap-classes!)
(content/bootstrap!)
(content-bootstrap-section!)
(content-bootstrap-callout!)
(content-bootstrap-table!)
(define
d
@@ -30,11 +32,12 @@
(str (blk-send (doc-deep-find r "n") "text"))
"nested Bar")
;; ── does NOT touch image alt/src (not a text field) ──
;; ── image alt IS a text field (asText ^ alt), so it is rewritten ──
(content-test
"image alt untouched"
"image alt replaced"
(str (blk-send (doc-deep-find r "img") "alt"))
"Foo alt")
"Bar alt")
;; ── but src is a URL, not prose, so it stays put ──
(content-test
"image src untouched"
(str (blk-send (doc-deep-find r "img") "src"))
@@ -76,6 +79,68 @@
(str (blk-send (doc-find r2 "q") "text"))
"new saying")
;; ── callout text is covered (consistency with asText/stats/summary) ──
(content-test
"replace callout text"
(str
(blk-send
(doc-find
(content/find-replace
(doc-append (doc-empty "d") (mk-callout "co" "note" "Foo here"))
"Foo"
"Bar")
"co")
"text"))
"Bar here")
(content-test
"callout kind untouched by text replace"
(str
(blk-send
(doc-find
(content/find-replace
(doc-append (doc-empty "d") (mk-callout "co" "note" "x"))
"note"
"X")
"co")
"kind"))
"note")
;; ── list items are rewritten (asText folds items) ──
(define
rl
(content/find-replace
(doc-append
(doc-empty "d")
(mk-list "l" false (list "Foo one" "two Foo")))
"Foo"
"Bar"))
(content-test
"replace first list item"
(str (first (blk-send (doc-find rl "l") "items")))
"Bar one")
(content-test
"replace second list item"
(str (first (rest (blk-send (doc-find rl "l") "items"))))
"two Bar")
;; ── table headers + cells are rewritten (asText folds rows) ──
(define
rt
(content/find-replace
(doc-append
(doc-empty "d")
(mk-table "t" (list "Foo head") (list (list "a Foo" "b"))))
"Foo"
"Bar"))
(content-test
"replace table header"
(str (first (table-headers (doc-find rt "t"))))
"Bar head")
(content-test
"replace table cell"
(str (first (first (table-rows (doc-find rt "t")))))
"a Bar")
;; ── no match → unchanged render ──
(content-test
"no match"

View File

@@ -1,7 +1,8 @@
;; Extension — relative block reorder.
;; Extension — relative block reorder + tree reparent.
(st-bootstrap-classes!)
(content/bootstrap!)
(content-bootstrap-section!)
(define
d
@@ -61,3 +62,84 @@
"render after move"
(asHTML (content/move-after d "a" "c"))
"<p>B</p><p>C</p><p>A</p>")
;; ── reparent: move a top-level block INTO a section ──
(define
nd
(doc-append
(doc-append (doc-empty "d") (mk-text "p" "P"))
(mk-section "s" (list (mk-text "x" "X")))))
(content-test
"move-into: block leaves top level"
(doc-ids (content/move-into nd "p" "s" 1))
(list "s"))
(content-test
"move-into: block lands in section at index"
(doc-tree-ids (content/move-into nd "p" "s" 1))
(list "s" "x" "p"))
(content-test
"move-into at front of section"
(doc-tree-ids (content/move-into nd "p" "s" 0))
(list "s" "p" "x"))
(content-test "move-into immutable" (doc-tree-ids nd) (list "p" "s" "x"))
;; ── reparent: move a NESTED block to a different section ──
(define
two
(doc-append
(doc-append (doc-empty "d") (mk-section "s1" (list (mk-text "n" "N"))))
(mk-section "s2" (list (mk-text "y" "Y")))))
(content-test
"move-into across sections"
(doc-tree-ids (content/move-into two "n" "s2" 1))
(list "s1" "s2" "y" "n"))
;; ── promote: nested block out to top level (appended last) ──
(content-test
"promote nested to top level"
(doc-tree-ids (content/promote two "n"))
(list "s1" "s2" "y" "n"))
(content-test
"promote leaves section empty shell"
(doc-ids (content/promote two "n"))
(list "s1" "s2" "n"))
(content-test
"promote a whole section keeps its subtree"
(doc-tree-ids
(content/promote
(doc-append
(doc-empty "d")
(mk-section "o" (list (mk-section "i" (list (mk-text "z" "Z"))))))
"i"))
(list "o" "i" "z"))
;; ── cycle guard: cannot move a section into its own descendant ──
(define
nest
(doc-append
(doc-empty "d")
(mk-section
"outer"
(list (mk-section "inner" (list (mk-text "t" "T")))))))
(content-test
"move section into its own child is a no-op"
(doc-tree-ids (content/move-into nest "outer" "inner" 0))
(list "outer" "inner" "t"))
(content-test
"move block into itself is a no-op"
(doc-tree-ids (content/move-into nest "inner" "inner" 0))
(list "outer" "inner" "t"))
;; ── reparent no-ops on missing ids ──
(content-test
"move-into missing block no-op"
(doc-tree-ids (content/move-into nd "zzz" "s" 0))
(list "p" "s" "x"))
(content-test
"move-into missing section no-op"
(doc-tree-ids (content/move-into nd "p" "zzz" 0))
(list "p" "s" "x"))
(content-test
"promote missing no-op"
(doc-tree-ids (content/promote nd "zzz"))
(list "p" "s" "x"))

View File

@@ -1,8 +1,11 @@
;; Extension — block query + table of contents.
;; Extension — block query + table of contents + prose search.
(st-bootstrap-classes!)
(content/bootstrap!)
(content-bootstrap-text!)
(content-bootstrap-section!)
(content-bootstrap-table!)
(content-bootstrap-callout!)
(define
d
@@ -87,3 +90,49 @@
"deep toc level"
(get (first (content/headings deep)) :level)
3)
;; ── prose search (content/search-text) ──
;; "cat" appears in text, image alt, a list item, a table cell, and a callout
;; — every text-bearing field — so search must find all five via asText.
(define
sd
(doc-append
(doc-append
(doc-append
(doc-append
(doc-append
(doc-empty "sd")
(mk-heading "sh" 1 "Welcome aboard"))
(mk-text "st" "the cat sat"))
(mk-image "si" "/x.png" "a cat photo"))
(mk-list "sl" false (list "first cat" "second dog")))
(mk-section
"sec"
(list
(mk-table "stb" (list "Animal") (list (list "cat") (list "fish")))
(mk-callout "sc" "note" "beware of cat")))))
(content-test
"search across every text-bearing field"
(content/search-text-ids sd "cat")
(list "st" "si" "sl" "stb" "sc"))
(content-test "search count" (len (content/search-text sd "cat")) 5)
(content-test
"search heading text"
(content/search-text-ids sd "Welcome")
(list "sh"))
(content-test
"search list item only"
(content/search-text-ids sd "dog")
(list "sl"))
(content-test "search no match" (content/search-text-ids sd "zzz") (list))
;; section containers are excluded — a term living only inside a section's
;; children returns the child, never the section wrapper.
(content-test
"search excludes section wrapper"
(content/search-text-ids sd "fish")
(list "stb"))
(content-test
"search returns block objects"
(blk-id (first (content/search-text sd "Welcome")))
"sh")

227
lib/content/tests/runs.sx Normal file
View File

@@ -0,0 +1,227 @@
;; Phase 5 — rich inline text (structured runs). Acceptance suite.
(st-bootstrap-classes!)
(content/bootstrap!)
(content-bootstrap-markdown!)
(content-bootstrap-text!)
(content-bootstrap-section!)
(content-bootstrap-runs!)
;; one-run helper: a CtText with a single marked run
(define
one
(fn (marks href) (mk-rich-text "p" (list (mk-run "x" marks href)))))
;; ── (1) four render modes ──
;; a paragraph mixing plain + bold + a link
(define
rd
(doc-append
(doc-empty "d")
(mk-rich-text
"p"
(list
(mk-run "the " (list) "")
(mk-run "cat" (list :bold) "")
(mk-run " and " (list) "")
(mk-run "dog" (list :italic :link) "/d")
(mk-run " sat" (list) "")))))
(define p (doc-find rd "p"))
(content-test
"asHTML rich"
(asHTML rd)
"<p>the <strong>cat</strong> and <a href=\"/d\"><em>dog</em></a> sat</p>")
(content-test
"asMarkdown rich"
(asMarkdown rd)
"the **cat** and [_dog_](/d) sat")
(content-test
"asSx rich"
(asSx rd)
"(article (p \"the \" (strong \"cat\") \" and \" (a :href \"/d\" (em \"dog\")) \" sat\"))")
(content-test
"asText rich is plain (no markup)"
(asText rd)
"the cat and dog sat")
;; every mark renders in HTML
(content-test
"mark bold html"
(asHTML (one (list :bold) ""))
"<p><strong>x</strong></p>")
(content-test
"mark italic html"
(asHTML (one (list :italic) ""))
"<p><em>x</em></p>")
(content-test
"mark underline html"
(asHTML (one (list :underline) ""))
"<p><u>x</u></p>")
(content-test
"mark strike html"
(asHTML (one (list :strikethrough) ""))
"<p><s>x</s></p>")
(content-test
"mark code html"
(asHTML (one (list :code) ""))
"<p><code>x</code></p>")
(content-test
"mark sub html"
(asHTML (one (list :subscript) ""))
"<p><sub>x</sub></p>")
(content-test
"mark sup html"
(asHTML (one (list :superscript) ""))
"<p><sup>x</sup></p>")
(content-test
"mark link html"
(asHTML (one (list :link) "/u"))
"<p><a href=\"/u\">x</a></p>")
;; markdown marks
(content-test "mark bold md" (asMarkdown (one (list :bold) "")) "**x**")
(content-test "mark italic md" (asMarkdown (one (list :italic) "")) "_x_")
(content-test
"mark strike md"
(asMarkdown (one (list :strikethrough) ""))
"~~x~~")
(content-test "mark code md" (asMarkdown (one (list :code) "")) "`x`")
(content-test "mark link md" (asMarkdown (one (list :link) "/u")) "[x](/u)")
(content-test
"mark underline md fallback"
(asMarkdown (one (list :underline) ""))
"<u>x</u>")
;; nested marks (bold+italic) — deterministic nesting order
(content-test
"nested marks html"
(asHTML (one (list :bold :italic) ""))
"<p><em><strong>x</strong></em></p>")
;; escaping still happens inside runs
(content-test
"run text escaped html"
(asHTML (mk-rich-text "p" (list (mk-run "a & b <c>" (list :bold) ""))))
"<p><strong>a &amp; b &lt;c&gt;</strong></p>")
;; rich heading + quote + code
(content-test
"rich heading html"
(asHTML
(st-iv-set!
(mk-heading "h" 2 "")
"text"
(list (mk-run "Big " (list) "") (mk-run "bold" (list :bold) ""))))
"<h2>Big <strong>bold</strong></h2>")
(content-test
"rich quote html"
(asHTML
(st-iv-set!
(mk-quote "q" "" "")
"text"
(list (mk-run "wise" (list :italic) ""))))
"<blockquote><em>wise</em></blockquote>")
;; code is verbatim — runs concatenate as plain text, marks ignored
(content-test
"code runs plain html"
(asHTML
(st-iv-set!
(mk-code "c" "py" "")
"text"
(list (mk-run "a=" (list :bold) "") (mk-run "1" (list) ""))))
"<pre><code class=\"language-py\">a=1</code></pre>")
;; ── (2) backward compat: plain-string CtText unchanged ──
(content-test
"plain html"
(asHTML (mk-text "q" "hi & <b>"))
"<p>hi &amp; &lt;b&gt;</p>")
(content-test "plain sx" (asSx (mk-text "q" "hi")) "(p \"hi\")")
(content-test "plain md" (asMarkdown (mk-text "q" "hi")) "hi")
(content-test "plain text" (asText (mk-text "q" "hi")) "hi")
(content-test
"plain heading html"
(asHTML (mk-heading "h" 3 "T"))
"<h3>T</h3>")
;; ── (3) find-replace across runs (per-run, marks preserved) ──
(define
frd
(doc-append
(doc-empty "d")
(mk-rich-text
"p"
(list
(mk-run "the Foo" (list :bold) "")
(mk-run " and Foo here" (list) "")))))
(define frr (content/find-replace frd "Foo" "Bar"))
(content-test
"find-replace rich plain text"
(asText frr)
"the Bar and Bar here")
(content-test
"find-replace rich preserves marks"
(asHTML frr)
"<p><strong>the Bar</strong> and Bar here</p>")
(content-test
"find-replace rich run0 still bold"
(nth (nth (blk-get (doc-find frr "p") "text") 0) 1)
(list "bold"))
;; ── (4) search-text via asText, across run boundary ──
;; "cat sat" spans run1 ("the cat") and run2 (" sat")
(define
sd
(doc-append
(doc-empty "d")
(mk-rich-text
"p"
(list (mk-run "the cat" (list :bold) "") (mk-run " sat" (list) "")))))
(content-test
"search finds substring across runs"
(content/search-text-ids sd "cat sat")
(list "p"))
(content-test "search miss" (content/search-text-ids sd "zzz") (list))
;; ── (5) CRDT invariant — runs are an opaque block-level value ──
(define ra (list (mk-run "x" (list :bold) "")))
(define rb (list (mk-run "y" (list :italic) "")))
(define
s1
(crdt-insert
(crdt-empty)
"p"
"text"
(crdt-pos 5 "a")
{:text ra}
1
"a"))
(define s2 (crdt-update s1 "p" "text" rb 2 "b"))
(content-test
"crdt merge commutes with runs"
(get (crdt-merge s1 s2) :elements)
(get (crdt-merge s2 s1) :elements))
(content-test
"crdt merge idempotent with runs"
(get (crdt-merge s2 s2) :elements)
(get s2 :elements))
;; LWW: later ts (rb, ts 2) wins; runs survive as the field value
(content-test
"crdt LWW keeps latest runs"
(asHTML
(crdt-element->block (get (get (crdt-merge s1 s2) :elements) "p")))
"<p><em>y</em></p>")
;; ── (6) data + wire round-trip runs losslessly ──
(content-test
"data round-trip rich html"
(asHTML (content/from-data (content/to-data rd)))
(asHTML rd))
(content-test
"data round-trip rich text"
(asText (content/from-data (content/to-data rd)))
"the cat and dog sat")
(content-test
"wire round-trip rich html"
(asHTML (content/from-wire (content/to-wire rd)))
(asHTML rd))

View File

@@ -0,0 +1,128 @@
;; Extension — make a document render-safe by dropping invalid blocks.
;; Counterpart to validate; reuses its per-block checks. Tree-wide.
(st-bootstrap-classes!)
(content-bootstrap-blocks!)
(content-bootstrap-doc!)
(content-bootstrap-section!)
;; ── a valid document is returned unchanged (same ids, tree order) ──
(define
good
(doc-append
(doc-append (doc-empty "d") (mk-heading "h" 1 "Title"))
(mk-text "p" "Body")))
(content-test
"valid doc keeps all blocks"
(doc-ids (content/sanitize good))
(list "h" "p"))
(content-test
"valid doc still valid after sanitize"
(content/valid? (content/sanitize good))
true)
;; ── a block with a bad field is dropped ──
(content-test
"bad-field block dropped"
(doc-ids
(content/sanitize
(doc-append
(doc-append (doc-empty "d") (mk-text "ok" "fine"))
(mk-heading "bad" "notnum" "T"))))
(list "ok"))
;; ── unknown block type dropped ──
(define raw (st-iv-set! (st-make-instance "CtBlock") "id" "z"))
(content-test
"unknown-type block dropped"
(doc-ids
(content/sanitize
(doc-append (doc-append (doc-empty "d") (mk-text "ok" "x")) raw)))
(list "ok"))
;; ── blank-id block dropped ──
(content-test
"blank-id block dropped"
(doc-ids
(content/sanitize
(doc-append
(doc-append (doc-empty "d") (mk-text "ok" "x"))
(mk-text "" "y"))))
(list "ok"))
;; ── result is render-safe: no id/field issues remain ──
(content-test
"sanitized has no field/id issues"
(len
(filter
(fn (i) (if (= (get i :kind) "field") true (= (get i :kind) "id")))
(content/validate
(content/sanitize
(doc-append
(doc-append (doc-empty "d") (mk-text "ok" "x"))
(mk-heading "bad" "notnum" "T"))))))
0)
;; ── immutability: original document untouched ──
(define
withbad
(doc-append
(doc-append (doc-empty "d") (mk-text "ok" "x"))
(mk-heading "bad" "notnum" "T")))
(define _ (content/sanitize withbad))
(content-test "original unchanged" (doc-ids withbad) (list "ok" "bad"))
;; ── tree-wide: invalid nested child pruned, valid sibling + section kept ──
(define
nested
(doc-append
(doc-empty "d")
(mk-section
"s"
(list (mk-text "good" "keep") (mk-heading "badc" "notnum" "X")))))
(content-test
"invalid nested child pruned, section kept"
(doc-tree-ids (content/sanitize nested))
(list "s" "good"))
;; ── a section whose own shell is invalid (children not a list) is dropped ──
(define
badsec
(doc-append
(doc-append (doc-empty "d") (mk-text "ok" "x"))
(st-iv-set! (mk-section "s" (list)) "children" "nope")))
(content-test
"invalid section shell dropped whole"
(doc-tree-ids (content/sanitize badsec))
(list "ok"))
;; ── a valid section that loses all children is kept (empty) — sanitize is not
;; normalize; it removes invalid, not empty ──
(define
allbadchildren
(doc-append
(doc-empty "d")
(mk-section "s" (list (mk-heading "b1" "x" "X") (mk-text "" "y")))))
(content-test
"section kept though emptied of invalid children"
(doc-tree-ids (content/sanitize allbadchildren))
(list "s"))
;; ── deeply nested: invalid block two levels down is pruned ──
(define
deep
(doc-append
(doc-empty "d")
(mk-section
"o"
(list (mk-section "i" (list (mk-text "dok" "x") (mk-text "" "bad")))))))
(content-test
"deep invalid pruned"
(doc-tree-ids (content/sanitize deep))
(list "o" "i" "dok"))
;; ── empty document sanitizes to empty ──
(content-test
"empty doc stays empty"
(doc-ids (content/sanitize (doc-empty "e")))
(list))

View File

@@ -151,3 +151,58 @@
"op-log media type"
(blk-type (doc-find (content/head B3 "rich") "v"))
"media")
;; ── op-log update/delete reach NESTED blocks (tree-wide by id) ──
(content-bootstrap-section!)
(define B4 (persist/open))
(content/commit!
B4
"nest"
(op-insert (mk-section "sec" (list (mk-text "n" "orig"))) nil)
1)
(content/commit! B4 "nest" (op-update "n" "text" "edited") 2)
(content-test
"op-log nested update"
(str (blk-send (doc-deep-find (content/head B4 "nest") "n") "text"))
"edited")
(content-test
"op-log nested update tree intact"
(doc-tree-ids (content/head B4 "nest"))
(list "sec" "n"))
(content/commit! B4 "nest" (op-delete "n") 3)
(content-test
"op-log nested delete"
(doc-tree-ids (content/head B4 "nest"))
(list "sec"))
(content-test
"op-log nested delete via content/at seq2"
(doc-tree-ids (content/at B4 "nest" 2))
(list "sec" "n"))
;; ── diff is TREE-WIDE: nested-block add/change/remove are detected, and
;; section containers never appear in :changed (a top-level-only diff would miss
;; "n" entirely and instead flag the section). ──
(define dn01 (content/diff-versions B4 "nest" 0 1))
(content-test
"diff nested added (section + child)"
(get dn01 :added)
(list "sec" "n"))
(content-test "diff nested added removed empty" (get dn01 :removed) (list))
(content-test "diff nested added changed empty" (get dn01 :changed) (list))
(define dn12 (content/diff-versions B4 "nest" 1 2))
(content-test
"diff nested changed child only"
(get dn12 :changed)
(list "n"))
(content-test "diff nested changed no add" (get dn12 :added) (list))
(content-test "diff nested changed no remove" (get dn12 :removed) (list))
(define dn23 (content/diff-versions B4 "nest" 2 3))
(content-test "diff nested removed child" (get dn23 :removed) (list "n"))
(content-test "diff nested removed no change" (get dn23 :changed) (list))
(content-test
"diff nested no-op"
(get (content/diff-versions B4 "nest" 1 1) :changed)
(list))

View File

@@ -5,6 +5,7 @@
(content-bootstrap-blocks!)
(content-bootstrap-doc!)
(content-bootstrap-section!)
(content-bootstrap-table!)
;; ── a fully valid document ──
(define
@@ -164,3 +165,62 @@
(content/validate dup-tree)))
1)
(content-test "tree dup not valid" (content/valid? dup-tree) false)
;; ── collection blocks vetted ELEMENT-DEEP (items/cells must be strings) ──
;; A list whose items field is a list but holds a non-string would pass the old
;; "is a list" check yet crash asText/render — now caught.
(content-test
"list non-string item flagged"
(content/issue-kinds
(doc-append (doc-empty "d") (mk-list "l" true (list "a" 5))))
(list "field"))
(content-test
"list all-string items valid"
(content/valid?
(doc-append (doc-empty "d") (mk-list "l" false (list "a" "b" "c"))))
true)
(content-test
"list empty items valid"
(content/valid? (doc-append (doc-empty "d") (mk-list "l" true (list))))
true)
;; a malformed-list block reports exactly one element issue (not the is-a-list one)
(content-test
"list non-string item single issue"
(len
(content/validate
(doc-append
(doc-empty "d")
(mk-list "l" true (list 1 2)))))
1)
(content-test
"valid table ok"
(content/valid?
(doc-append
(doc-empty "d")
(mk-table "t" (list "H1" "H2") (list (list "a" "b") (list "c" "d")))))
true)
(content-test
"table empty rows valid"
(content/valid?
(doc-append (doc-empty "d") (mk-table "t" (list "H") (list))))
true)
(content-test
"table non-list row flagged"
(content/issue-kinds
(doc-append (doc-empty "d") (mk-table "t" (list "H") (list "notarow"))))
(list "field"))
(content-test
"table non-string cell flagged"
(content/issue-kinds
(doc-append
(doc-empty "d")
(mk-table "t" (list "H") (list (list "ok") (list 9)))))
(list "field"))
(content-test
"table non-string header flagged"
(content/issue-kinds
(doc-append
(doc-empty "d")
(mk-table "t" (list "H" 2) (list (list "a" "b")))))
(list "field"))

View File

@@ -6,6 +6,11 @@
;; Tree detection is inline (class + st-iv-get) so this file needs no section.sx.
;; Dispatch on block type is a validation-boundary concern, not core behaviour.
;;
;; Collection blocks are vetted element-deep: list items must all be strings and
;; table rows must all be lists of strings — exactly what render/asText/
;; find-replace/search assume — so malformed nested collections are caught at the
;; boundary instead of crashing the render layer downstream.
;;
;; Requires (loaded by harness): block.sx, doc.sx.
(define ct-issue (fn (id kind detail) {:id id :detail detail :kind kind}))
@@ -36,6 +41,28 @@
(define ct-uniq (fn (xs) (ct-uniq-loop xs (list))))
;; every element a string? / every row a list of strings? (for collection blocks)
(define
ct-all-str?
(fn
(xs)
(if
(= (len xs) 0)
true
(if (string? (first xs)) (ct-all-str? (rest xs)) false))))
(define
ct-all-rows?
(fn
(rows)
(if
(= (len rows) 0)
true
(if
(if (list? (first rows)) (ct-all-str? (first rows)) false)
(ct-all-rows? (rest rows))
false))))
;; ── tree flatten (descends into CtSection children; guards malformed children) ──
(define
ct-section-block?
@@ -136,30 +163,43 @@
"embed provider must be a string")))
((= t "divider") (list))
((= t "list")
(append
(ct-field-issue
id
(boolean? (blk-get b "ordered"))
"list ordered must be a boolean")
(ct-field-issue
id
(list? (blk-get b "items"))
"list items must be a list")))
(let
((items (blk-get b "items")))
(append
(ct-field-issue
id
(boolean? (blk-get b "ordered"))
"list ordered must be a boolean")
(append
(ct-field-issue id (list? items) "list items must be a list")
(ct-field-issue
id
(if (list? items) (ct-all-str? items) true)
"list items must all be strings")))))
((= t "section")
(ct-field-issue
id
(list? (blk-get b "children"))
"section children must be a list"))
((= t "table")
(append
(ct-field-issue
id
(list? (blk-get b "headers"))
"table headers must be a list")
(ct-field-issue
id
(list? (blk-get b "rows"))
"table rows must be a list")))
(let
((headers (blk-get b "headers")) (rows (blk-get b "rows")))
(append
(append
(ct-field-issue
id
(list? headers)
"table headers must be a list")
(ct-field-issue
id
(if (list? headers) (ct-all-str? headers) true)
"table headers must all be strings"))
(append
(ct-field-issue id (list? rows) "table rows must be a list")
(ct-field-issue
id
(if (list? rows) (ct-all-rows? rows) true)
"table rows must all be lists of strings")))))
((= t "callout")
(append
(ct-field-issue

View File

@@ -19,7 +19,7 @@ injected adapter, not core.
## Status (rolling)
`bash lib/content/conformance.sh`**746/746** (Phases 14 COMPLETE + ~34 extensions, hardened: HTML/SX escaping, Markdown render + import/export incl. tables & frontmatter (full round-trip), CvRDT flat + nested-tree + durable replication, tree-aware validation, snapshot cache, doc metadata, plain-text render, nested block trees + deep editing + flatten + relative reorder, doc stats + summary + multi-doc index, table + callout + media blocks, HTML page wrapper + SEO page, doc composition + id-remap, portable data + wire serialization, block query + transforms + find/replace, TOC + anchored headings + outline, normalization)
`bash lib/content/conformance.sh`**861/861** (Phases 14 COMPLETE + ~34 extensions, hardened: HTML/SX escaping, Markdown render + import/export incl. tables & frontmatter (full round-trip), CvRDT flat + nested-tree + durable replication, tree-aware validation, snapshot cache, doc metadata, plain-text render, nested block trees + deep editing + flatten + relative reorder, doc stats + summary + multi-doc index, table + callout + media blocks, HTML page wrapper + SEO page, doc composition + id-remap, portable data + wire serialization, block query + transforms + find/replace, TOC + anchored headings + outline, normalization)
## Ground rules
@@ -106,13 +106,229 @@ lib/content/api.sx ── (content/edit) (content/render) (content/history) ─
- [x] document outline (`outline.sx`: content/outline, nested heading tree)
- [x] document flatten (`flatten.sx`: content/flatten, un-nest sections; inverse of wrap-section)
- [x] relative reorder (`move.sx`: content/move-before/after/to-front/to-back by id)
- [x] tree reparent (`move.sx`: content/move-into a section + content/promote out to top level; tree-wide, cycle-safe)
- [x] document normalization (`normalize.sx`: content/normalize, drop empty blocks/sections)
- [x] document sanitization (`sanitize.sx`: content/sanitize, drop invalid blocks tree-wide; validate's enforcement partner)
- [x] global find/replace (`find-replace.sx`: content/find-replace across text-bearing blocks)
- [x] portable data serialization (`data.sx`: content/to-data + from-data, round-trips tree)
- [x] wire serialization (`wire.sx`: content/to-wire + from-wire, SX-text on the wire)
## Phase 5 — rich inline text (structured runs)
Drives the rose-ash blog migration: lexical post bodies carry inline formatting
(bold/italic/links) but `CtText` held one plain `text` string, so the canonical
lexical→blocks conversion was lossy. Variant **(b)**: a `CtText`'s `text` may be
EITHER a plain string (backward compat) OR a sequence of inline **runs**. Marks
cover the lexical bitmask (bold=1 italic=2 strike=4 underline=8 code=16 sub=32
sup=64) plus link nodes (which carry an href). Applies to `CtText` and its
subclasses `CtHeading`/`CtQuote` (rich), and `CtCode` (verbatim — runs render as
plain concatenated text, no marks).
**Run representation — a Smalltalk-renderable list, not a `{:text :marks}` dict.**
A run is `(text marks href)`: `text` a string, `marks` a list of mark tokens
(`:bold :italic :underline :strikethrough :code :subscript :superscript :link`
SX keywords evaluate to the strings the renderer matches), `href` a string (`""`
when absent; carries the link target). *Why a list and not the dict the brief
sketched:* rendering must happen inside the Smalltalk render methods (nested
blocks dispatch `asHTML`/etc. through Smalltalk message sends), and the
Smalltalk-on-SX layer can iterate SX lists (`do:`/`inject:into:`) but **cannot**
read SX dict fields (`Dictionary>>at:` is broken in lib/smalltalk, which is out
of scope). Lists are Smalltalk-native, render under nesting, and round-trip
through data/wire for free (they're just nested lists+strings). The blog-side
lexical→runs converter targets this `(text marks href)` shape.
Centralised in `runs.sx` (`content-bootstrap-runs!`) which OVERRIDES the
render/markdown/text methods of CtText/CtHeading/CtQuote/CtCode with run-aware
versions that fall through to identical output for plain strings — so it is
opt-in (the blog enables it) and the existing suites, which don't bootstrap it,
are untouched.
- [x] runs render in all four modes — asHTML `<strong>/<em>/<u>/<s>/<code>/<sub>/<sup>/<a href>`, asMarkdown `**`/`_`/`~~`/`` ` ``/`[..](..)` (u/sub/sup fall back to inline HTML), asSx emits nested run structure (`(p (strong "x") " y")`), asText returns the PLAIN concatenation (keeps search/stats/find-replace drift-proof)
- [x] backward compat — a plain-string CtText still renders identically; existing suites stay green
- [x] find-replace rewrites text across runs (per-run, marks preserved); runs join the text-bearing-field dispatch
- [x] search-text finds substrings via asText, including across run boundaries
- [x] CRDT invariant preserved — merge stays at BLOCK granularity (runs are the block's value): ops in any order / twice → identical document
- [x] data + wire serialization round-trip runs losslessly
### Future — Phase 6 (NOT in scope now)
Variant **(c)**: character/run-level concurrent inline CRDT (Peritext/Yjs-style)
so two authors can edit the same paragraph simultaneously — needed later for the
multi-author SX editor that replaces Ghost. Block-granularity (b) is sufficient
for the blog read-path migration. The lexical→runs converter itself lives on the
blog/migration side (mark-set reference: `blog/bp/blog/ghost/lexical_to_sx.py`),
not in lib/content.
## Known limitations
- **Markdown table cells containing `|` do not round-trip.** `asMarkdown` on a
table emits cell text raw (table.sx `CtTable>>asMarkdown:`), so a cell `x|y`
renders the row `| x|y | z |` — which `md/import` then splits into *three*
cells (`md-import.sx` `md/-cells` splits on every `|`). Repro: build
`(mk-table "t" (list "A" "B") (list (list "x|y" "z")))`, `asMarkdown`
re-`md/import` → cells become `("x" "y" "z")`. Same applies to a literal `|`
in a header. (HTML/SX/text/data/wire/CRDT round-trips are unaffected — only
the Markdown text boundary.)
*Fix sketch* (when sx-tree edit tooling is restored — see below): add
`String>>mdCellEscaped` (escape `|``\|`) in table.sx and use it for every
header/cell in `CtTable>>asMarkdown:`; in md-import.sx replace `md/-cells`'
naive `(split … "|")` with an escaped-aware splitter that breaks only on
unescaped `|` and unescapes `\|``|`. Both sides must change together
(export-only escaping makes self-round-trip worse, not better).
*Blocker:* in this worktree every sx-tree **edit** tool (`sx_replace_node`,
`sx_replace_by_pattern`, `sx_insert_near`, …) raises yojson `"Expected
string, got null"`; only `sx_write_file` works. md-import.sx is 449 lines, so
a safe surgical edit isn't currently possible — deferred rather than risk a
full manual rewrite of working import code.
## Progress log
- 2026-06-07 — **Phase 5 — rich inline text (structured runs) DONE.** A CtText's
`text` may now be a list of inline runs `(text marks href)` instead of a plain
string; CtHeading/CtQuote inherit rich rendering, CtCode renders runs as plain
verbatim text. New `runs.sx` (`content-bootstrap-runs!`) overrides the
render/markdown/text methods of CtText & subclasses with run-aware versions
that are byte-identical for plain-string bodies (opt-in; existing suites
untouched). All 4 modes: asHTML emits `<strong>/<em>/<u>/<s>/<code>/<sub>/<sup>`
+ `<a href>`, asMarkdown emits `**`/`_`/`~~`/`` ` ``/`[..](..)` (u/sub/sup →
inline HTML), asSx emits nested run structure `(p "a" (strong "b"))` (matches
the SX editor's wire format), asText returns the PLAIN concatenation — so
search-text/stats/find-replace stay drift-proof. find-replace (`fr-rep-text`)
rewrites per run with marks preserved; search-text finds across run boundaries
via asText; CRDT merge treats the runs list as one block-level LWW value
(commutes/idempotent, verified); data + wire round-trip runs losslessly.
**Design note:** runs are a Smalltalk-renderable LIST, not the brief's
`{:text :marks}` dict — the Smalltalk-on-SX render methods (which must run
under nested dispatch) can iterate SX lists but cannot read SX dict fields
(`Dictionary>>at:` is broken in lib/smalltalk, out of scope). Marks are built
from `:bold`-style keywords (which evaluate to the strings the renderer
matches). Phase 6 (char-level concurrent inline CRDT) recorded as future, not
built. +36 runs tests (44 suites). 861/861.
- 2026-06-07 — Feature: `content/block-path` + `content/block-depth`
(block-path.sx, new suite). The read-side companion to doc-find-deep (locate
the block) and move-into/promote (relocate it): returns the ancestor-section
id chain (root-first) for a block id — where it sits in the tree — or nil if
absent (distinct from the `()` path of a present top-level block). block-depth
is the path length (0 top-level, -1 absent). For breadcrumbs and scoping an
edit to a block's enclosing section; distinct from toc/outline (which work on
headings). Pure traversal. Also ran an adversarial probe this pass: confirmed
clone/remap-ids + prefix-ids are tree-wide, and all 12 block types + CtDoc have
asMarkdown: methods (no missing-render-method bug). +13 tests. 825/825 (43
suites).
- 2026-06-07 — Feature: tree reparent in move.sx. Until now insert/move were
positional and top-level only, so a block could never be moved *into* a section
or *out* of one — a real gap for editing nested documents. Added
`content/move-into doc id section-id i` (relocate a block, from anywhere in the
tree, to be a child of a section at index i) and `content/promote doc id`
(lift a nested block out to the end of the top level; a moved section keeps its
whole subtree). Both are pure tree transforms (consistent with the existing
move family — not new op-log ops) built on doc-find-deep / ct-find-id /
ct-remove-id / ct-replace-id. **Cycle-safe**: move-into no-ops when target is
the block itself or sits inside the block's own subtree, so a section can never
become its own ancestor. +13 move tests (into/promote/across-sections/empty-
shell/whole-section-subtree/cycle-guard/missing-id no-ops). 812/812.
- 2026-06-07 — Feature: `content/sanitize` — the enforcement counterpart to
`validate`. validate *reports* id/field issues; sanitize *removes* the
offending blocks (tree-wide) so federated/imported input that failed
validation can still be rendered/merged without faulting. Reuses validate's
own per-block predicate (`content/-block-issues`) so "what is invalid" stays
single-sourced and can't drift. Distinct from `normalize` (which drops *empty*
blocks): a section emptied of invalid children is kept (sanitize removes
invalid, not empty), but a section whose own shell is invalid (children not a
list) is dropped whole. Scope is per-block id/field validity — it does not
dedupe ids (cross-block, no single right answer). +12 tests (bad-field /
unknown-type / blank-id dropped, deep pruning, invalid-shell section dropped,
immutability, render-safe result). 799/799 (42 suites). (This was a genuine
remaining gap — validate had no enforcement partner — not filler; saturation
note below still holds for the roadmap proper.)
- 2026-06-07 — Audit (markdown round-trip): probed the Markdown text boundary
for round-trip fidelity. Found one real data-corruption bug — table cells
containing `|` don't survive `asMarkdown` → `md/import` (recorded under
**Known limitations** with repro + fix sketch). Could not land the fix this
pass: it must touch md-import.sx (449 lines) and every sx-tree *edit* tool is
currently broken in this worktree (yojson error; only `sx_write_file` works),
so a safe surgical edit isn't possible and a full manual rewrite of working
import code is too risky to be responsible. Deferred + documented rather than
half-fix (export-only escaping worsens self-round-trip). Engine remains
COMPLETE + audited at 787/787; with the roadmap exhausted, the tree-wide
audit done, and the one open finding tooling-blocked, the vertical is
**SATURATED** — pacing the loop down.
- 2026-06-07 — Hardening: validation now vets collection blocks ELEMENT-DEEP.
`validate` previously checked only that list `items` / table `headers`/`rows`
*are lists* — a list holding a non-string, or a table whose rows aren't lists
of strings, passed validation yet crashes asText/render/find-replace/search
(which all assume string items/cells). Added `ct-all-str?`/`ct-all-rows?` and
deepened the list/table branches (guarded so a non-list container reports only
the is-a-list issue, not a spurious element issue). Since validate's job is
guarding imports/federated input, this closes the boundary before the render
layer can fault. +9 validate tests (list non-string item, table non-list row /
non-string cell / non-string header, empties stay valid). 787/787.
- 2026-06-07 — Hardening (tree-wide audit): the public facade `content/find` /
`content/has?` were top-level-only (`doc-find`/`doc-has?`), so you could
`content/edit` an update/delete to a nested block by id (those ops are
tree-wide) but couldn't read that same block back by id through the facade — a
concrete read/write asymmetry. Added a generic `ct-find-id` to doc.sx (descends
into any `children` list, mirroring ct-replace-id/ct-remove-id, no section.sx
dependency) plus `doc-find-deep`/`doc-has-deep?`; `content/find`/`content/has?`
now point at them. Kept `content/find-top`/`content/has-top?` for the
top-level-only lookup. Audited all `doc-find`/`doc-ids`/`ct-index-of` callers:
the remaining ones are insert/move (positional, top-level by design) — no other
seams. +6 api tests (nested deep find/has, top variants miss nested,
edit-then-find round-trip). 778/778.
- 2026-06-07 — Hardening: `content/diff` (and `content/diff-versions`) are now
TREE-WIDE. They enumerated ids via `doc-ids`/`doc-find` (top-level only), so a
diff between two versions of a document containing sections silently missed
every nested-block add/remove/change — the same class of seam as the by-id
op-log bug. Now ids come from `doc-tree-ids` and lookups from `doc-deep-find`,
so nested changes surface precisely. Section containers are excluded from
`:changed` (they hold no own content; a child change reports as that child),
while whole-section add/remove still shows in `:added`/`:removed`. Flat-doc
diffs are unchanged (deep == top-level with no sections). +9 store tests
(nested add = section+child, nested change = child only, nested remove,
no-op). 772/772.
- 2026-06-07 — Feature: in-document prose search. `content/search-text` (and
`content/search-text-ids`) return every content block, tree-wide, whose
`(asText b)` contains a term — so search spans text/heading/code/quote/callout
text, image alt, list items and table cells **by construction**: it reuses the
one canonical "prose of a block" projection (asText) rather than re-listing
fields, so it can't drift from stats/find-replace. Section containers are
excluded (a term living only in a section's children returns the child, not the
wrapper). +7 query tests (cross-field match, count, single-field, no-match,
section exclusion, object return). 763/763.
- 2026-06-07 — Consistency: `find-replace` now rewrites **every** text-bearing
field, not just `text`. New `fr-rewrite` dispatches per block type — `alt` of
image blocks, each item of list blocks, and every header/cell of table blocks
now get rewritten alongside text/heading/code/quote/callout. This closes a real
seam: `asText`/stats/word-count already fold image alt, list items, and table
cells into a document's prose, so a `content/find-replace` rename that skipped
them was inconsistent (a renamed term would still show up in word counts and
exports). Flipped the two `image alt untouched` tests to `image alt replaced`;
+4 tests (list items ×2, table header + cell). find-replace 16/16, 756/756.
- 2026-06-07 — Consistency: `find-replace` now covers `callout` text. `fr-has-text?`
(find-replace.sx) added `callout` to its text-bearing block kinds, matching
`asText`/stats/summary which already treat callout bodies as prose. Previously a
`content/find-replace` over a doc containing callouts silently skipped them. +2
find-replace tests (replace callout text; callout kind untouched by text replace).
752/752 (41 suites).
- 2026-06-07 — Hardening: fixed a real layer seam (surfaced in the architecture
review) — by-id ops (update/delete) now act TREE-WIDE. `ct-replace-id` /
`ct-remove-id` (doc.sx) descend into any block carrying a `children` list, so
the persist op-log and `content/edit` correctly reach blocks nested in
sections (previously a silent no-op). `doc-move` stays top-level (guarded by
doc-find); insert/move remain positional. Inline section detection (no
section.sx dep). +4 store regression tests (nested update/delete via op-log +
replay-to-seq). Full gate over foundational doc.sx: 750/750.
- 2026-06-07 — Hardening: audit confirmed the persist op-log (store.sx) carries
every block type through commit → replay (op-insert carries the block
instance; updates apply by id). Locked with +4 store tests (callout/media