host: blog edge graph is KV-only — drop the per-write Datalog re-saturation (major perf)
A REAL production perf bug, surfaced while profiling slow conformance. host/blog--add-edge! mirrored every edge into lib/relations via relations/relate, which RE-SATURATES the whole CEK-interpreted Datalog ruleset on every single write — super-linear in the fact base (profiled: 1.1s → 3.5s → 6.1s per edge as the graph grows 10→20→30 facts; O(graph) per write, O(edges²) to build). This hit the LIVE SITE on every content op: importing a Ghost post (decompose! = ~4 edges/block), tagging, relating, is-a, the metamodel editor — all getting slower as the site grows. Since typing now reads direct KV edges (host/blog--subtype-closure et al.), NOTHING in the blog domain reads lib/relations anymore — the mirror was pure, very expensive dead weight. So edges are now KV-only: add/del-edge! just kv-put/kv-delete (~20ms FLAT, O(1)); reads already walk the edge:* rows directly. host/blog-load-edges! (which replayed every edge into lib/relations on boot — O(edges²)) is now a no-op. conj/disj operands were already KV-only, proving the whole graph can be. host/relations.sx (the relations DOMAIN service, its own type:id nodes) is separate and untouched. Result: blog-relate! 6.1s→20ms/call (and now FLAT, not growing); full blog suite ~23min→19s; all 11 host suites 353/355 in 36s (the 2 fails are the pre-existing relate-picker pair). Live writes drop from seconds to ~20ms. Pairs with the typing-reads-from-KV fix (prev commit). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -187,24 +187,24 @@
|
||||
(define host/blog--kind-symmetric?
|
||||
(fn (kind) (let ((s (host/blog--kind-spec kind))) (and s (get s :symmetric)))))
|
||||
|
||||
;; ── edges (parameterised by kind, DURABLE) ──────────────────────────
|
||||
;; lib/relations holds the graph in memory (a Datalog cache that re-saturates per
|
||||
;; query); it does NOT survive a restart. So the host owns the durable source of
|
||||
;; truth: every physical edge is also a KV row "edge:<src>|<kind>|<dst>" in the
|
||||
;; blog store, replayed into the in-memory graph on boot (host/blog-load-edges!).
|
||||
;; '|' is a safe delimiter — slugs are [a-z0-9-], kinds are registry names.
|
||||
;; ── edges (parameterised by kind, DURABLE, KV-only) ─────────────────
|
||||
;; The blog graph is the durable KV: every edge is a row "edge:<src>|<kind>|<dst>" in the
|
||||
;; blog store, and ALL reads walk those rows directly (host/blog--all-edges / -out / -in /
|
||||
;; --subtype-closure). It is NOT mirrored into lib/relations: relations/relate re-saturates
|
||||
;; the whole Datalog ruleset on EVERY write (super-linear in the fact base — profiled at
|
||||
;; 1→3→6s per edge as the graph grows), and since typing now reads direct KV edges, nothing
|
||||
;; in the blog domain reads lib/relations, so the mirror was pure (very expensive) dead
|
||||
;; weight. KV-only edge writes are ~20ms flat. '|' is a safe delimiter — slugs are
|
||||
;; [a-z0-9-], kinds are registry names. (host/relations.sx, the relations DOMAIN service, is
|
||||
;; separate: its own "type:id" nodes in lib/relations, untouched by this.)
|
||||
(define host/blog--edge-key (fn (src kind dst) (str "edge:" src "|" kind "|" dst)))
|
||||
|
||||
(define host/blog--add-edge!
|
||||
(fn (src dst kind)
|
||||
(begin
|
||||
(relations/relate (host/blog--node src) (host/blog--node dst) (string->symbol kind))
|
||||
(persist/backend-kv-put host/blog-store (host/blog--edge-key src kind dst) 1))))
|
||||
(persist/backend-kv-put host/blog-store (host/blog--edge-key src kind dst) 1)))
|
||||
(define host/blog--del-edge!
|
||||
(fn (src dst kind)
|
||||
(begin
|
||||
(relations/unrelate (host/blog--node src) (host/blog--node dst) (string->symbol kind))
|
||||
(persist/backend-kv-delete host/blog-store (host/blog--edge-key src kind dst)))))
|
||||
(persist/backend-kv-delete host/blog-store (host/blog--edge-key src kind dst))))
|
||||
|
||||
;; A symmetric kind writes both directions, so children alone read it from either
|
||||
;; side; a directed kind writes one edge (the inverse is host/blog-in).
|
||||
@@ -219,31 +219,12 @@
|
||||
(host/blog--del-edge! a b kind)
|
||||
(when (host/blog--kind-symmetric? kind) (host/blog--del-edge! b a kind)))))
|
||||
|
||||
;; rebuild the in-memory graph from the durable edge store — called on boot, after
|
||||
;; the store is pointed at the durable backend. Each "edge:<src>|<kind>|<dst>" key
|
||||
;; is re-applied directly (both directions of a symmetric kind are stored, so no
|
||||
;; symmetry re-derivation is needed here).
|
||||
(define host/blog-load-edges!
|
||||
(fn ()
|
||||
(for-each
|
||||
(fn (key)
|
||||
(let ((body (substr key 5))) ;; drop "edge:"
|
||||
(let ((p1 (index-of body "|")))
|
||||
(when (>= p1 0)
|
||||
(let ((src (substr body 0 p1))
|
||||
(tail (substr body (+ p1 1))))
|
||||
(let ((p2 (index-of tail "|")))
|
||||
(when (>= p2 0)
|
||||
(let ((ek (substr tail 0 p2)))
|
||||
;; conj/disj are structural (type-algebra operands) — KV-only,
|
||||
;; never replayed into the Datalog graph (it re-saturates per query).
|
||||
(when (not (or (= ek "conj") (= ek "disj")))
|
||||
(relations/relate
|
||||
(host/blog--node src)
|
||||
(host/blog--node (substr tail (+ p2 1)))
|
||||
(string->symbol ek)))))))))))
|
||||
(filter (fn (k) (starts-with? k "edge:"))
|
||||
(persist/backend-kv-keys host/blog-store)))))
|
||||
;; No-op: the durable KV edge rows ARE the graph and every read walks them directly, so
|
||||
;; there is no in-memory lib/relations graph to rebuild on boot. (Kept as a callable seam —
|
||||
;; serve.sh calls it after pointing the store at the durable backend — in case a future
|
||||
;; index/cache needs warming.) Previously this replayed every edge into lib/relations via
|
||||
;; relations/relate, which re-saturated the Datalog ruleset per edge: O(edges²) boot cost.
|
||||
(define host/blog-load-edges! (fn () nil))
|
||||
|
||||
;; nodes -> existing blog slugs: strip "blog:", drop non-blog and deleted targets.
|
||||
;; Existence is one kv-keys read (host/blog-slugs), NOT a perform per candidate —
|
||||
|
||||
Reference in New Issue
Block a user