rose-ash/plans/business-logic-fed-flows.md

Business logic as composition — a content-addressed DAG over pluggable substrates

Vision (elevated 2026-07-02): business logic IS art-dag. An object's behavior is a content-addressed DAG (lib/artdag), declared on its type alongside content grammar + allowed relations. Everything else is a pluggable ADAPTER — the same fold/adapter principle as render-vs-execute-vs-deps, applied to execution/communication/deployment:

• Behavior = an artdag DAG — the invariant, content-addressed (artdag/dag, analyze/plan/ optimize/schedule). Business logic, art media pipelines, workflows — all the same abstraction.
• Execution = an injected RUNNER (artdag/run dag RUNNER cache; artdag/op-table-runner). Substrates are runners on a capability ladder, same DAG throughout — a node DECLARES the capabilities it needs, a runner ADVERTISES what it supports, and the match is checked at bind time (fail fast, not a mystery at run time). artdag/analyze computes a DAG's required-capability set → its MINIMUM runner. So "simple in SX / durable in Erlang / distributed in celery-sx" is a DERIVED property of the DAG, not a human judgment; a trivial rule is a one-node DAG needing only {effect} and runs synchronously with zero ceremony. Business logic and media pipelines are the SAME structure (a content-addressed op-DAG); they differ only in the capabilities their nodes require, hence the runner. The ladder (capabilities each runner adds):
  • op-table / execute-fold runner — {effect, branch, each} — synchronous, local, in-request. Covers P0.
  • Erlang runner — + suspend/resume/wait (durable), deterministic replay (flow-on-erlang).
  • celery-sx runner — + parallel, retry, offload — distributed/durable task executor, "Celery the way it should have been" on erlang-on-sx, ZERO packages. It's a LEAN GLUE of parts we already have, not a reimplementation: broker = lib/persist KV (durable enqueue/claim/ack/visibility-timeout) · worker pool = the er-scheduler / Erlang processes · result backend = content-addressed results (artdag keys by content-id → dedup/memoization FREE — Celery bolts this on badly) · retries/ replay = flow-on-erlang · scheduling/fan-out/chords = artdag/schedule (minikanren CLP(FD)) + the DAG's topo batches · the plug point = artdag/op-table-runner. The genuinely-new code is small (~a few hundred lines): a durable queue + a worker loop (pull node → runner → write content-addressed result) + retry/backoff. BUILD WHEN A DAG DEMANDS IT — heavy compute, long-running/retryable tasks, or fan-out across machines — NOT for the synchronous P0.
  • real-Celery over artdag/L1 — + gpu/heavy-compute — the existing Python media pipeline (JAX/IPFS) as a runner.
• Communication = an injected TRANSPORT (artdag/federation, transport injected). Substrates: fed-sx (ActivityPub/next/), internal HMAC HTTP (services), IPFS (content-addressed). Because content-ids are global, a result computed on one instance is reusable on another by id.
• Deployment = PLACEMENT — a subdomain service, a fed-sx peer, an L1 worker: just where a runner runs. Not the essence.
• State change → triggers a DAG (over a transport) → executed by a runner → effects (data) a driver dispatches. fed-sx + Erlang is ONE adapter set (durable/federated), not THE architecture.

So: the TYPE carries content-grammar + allowed-relations + a behavior DAG (+ triggers); the object's state changes emit activities; the platform picks runner/transport/placement per context.

Design (decided 2026-07-02; corrected after review):

• Activity log = every OBSERVABLE object-level state change — the event source. NOT just CID deltas: relations write edge:* rows, NOT the record, so a relation change does NOT shift the CID. Two ActivityPub-faithful event classes: content/status change → a CID-carrying Create/Update; relation change → an Add/Remove referencing the edge.
• Verbs are TRANSITIONS, not raw deltas. on-publish = draft→published (fire-once), not every CID delta of a published post. Emitter picks the verb (Create/Update/Add/Remove/Delete). Global fire-once is the emitter's job (+ a durable inbox for federation); the seam only dedups in-call.
• Triggers = declared subscriptions — a type declares named triggers; flows fire only on matching ones. Log complete, execution precise.
• Flows split by CAPABILITY (DERIVED, not a human "complexity" call): every flow is a content-addressed op-DAG. A node declares the capabilities it needs (a wait needs suspend, a fan-out needs parallel, a heavy op needs offload); a runner advertises what it supports; artdag/analyze computes the DAG's required set → its MINIMUM runner; bind-time checks required ⊆ runner (fail fast). So the sync/durable/distributed choice falls out of the DAG — a {effect}-only DAG runs synchronously (no ceremony); a DAG with a wait node auto-requires the Erlang runner. Business logic and media pipelines are the SAME structure — they differ only in node capabilities.
• Effects are DATA; a DRIVER dispatches them (perform no IO — a blocking call deadlocks the er-scheduler). The host is the driver for P0.
• The type carries its whole contract: fields+grammar (content) · allowed relations (external) · behavior bindings (triggers → DAG). All composition, all editable in the type-def editor.
• CANONICAL ACTIVITY = the seam shape {:verb :actor :object <cid> :object-type :delta :prev :ts :id}; each runner adapter MARSHALS it to its substrate (e.g. the Erlang runner → next/'s [{type,…},{object,[…]}] proplist via host/blog--activity->erl). DONE (P0.4): host/blog--publish- activity emits the canonical shape; host/blog--activity->erl is the RA marshaller (staged).

Reference (one durable-runner substrate, verified): next/tests/triggers_e2e.sh = 10/10 — next/'s trigger_registry + flow_dispatch + blog_publish_digest (suspend/resume/guard/dedup). This is what the ERLANG RUNNER ADAPTER wraps (Phase RA), not what P0 wires directly.

The ADAPTER SEAM (design first — the contract every substrate plugs into)

The invariant is an activity (state-change event) + a behavior DAG. Everything between is a swappable adapter — each a dict-of-functions (SX-native, like the fold domains). Six contracts:

1. Activity (content-addressed event): {:verb "create"|"update"|"add"|"remove"|"delete" :actor :object <cid> :object-type :delta :prev <cid> :ts}.
2. Behavior binding (declared on a TYPE): {:on {:verb :object-type :guard} :dag <ref>}. The TYPE carries content-grammar + allowed-relations + these bindings. NO :runner hint — the runner is DERIVED: the DAG's required-capability set (artdag/analyze over its nodes; a node declares :needs e.g. #{suspend}) selects the minimum runner that advertises them.
3. Trigger-registry adapter — {:register! (fn spec dag hint) :match (fn activity -> [binding])}. Impls: a local SX matcher / next/'s Erlang trigger_registry.
4. Runner adapter — {:capabilities <set> :run (fn dag env -> {:status "done"|"suspended"| "failed" :effects :resume :error})}. :capabilities is what it ADVERTISES (op-table {effect, branch, each}; Erlang + suspend; celery-sx + parallel, retry, offload); the binder checks dag-required ⊆ runner-caps and fails fast otherwise. env = {:activity :actor :ctx :effects :binding} (:effects = injected external-read interfaces, deterministic for replay). :results is runner-INTERNAL. Durability = the runner's, not the DAG's. A durable runner that SUSPENDS is wired at construction to the transport's INBOUND channel and injects its out-of-band completion there (pump drains it).
5. Transport adapter — {:emit (fn activity) :deliver (fn -> [activity])}. :emit = outbound (log/publish), :deliver = inbound (peers + async runner completions). Impls: in-process / fed-sx (next/) / HTTP / IPFS. Content-ids global → results move by id.
6. Effect driver — {:dispatch (fn effect -> [activity])}. Perform the effect-as-data; may emit NEW activities (loop closure).

Engine (behavior/make-engine {:triggers :runner :transport :driver :effects? :ctx-of?}) → behavior/process(engine, activity) / behavior/pump(engine):

step(activity):  if activity.id already :seen → skip (dedup)      ; in-call cycle guard
  emit(transport, activity)                                       ; log
  for b in match(triggers, activity):
     r = run(runner, b.dag, {:activity :actor :ctx :effects :binding})
     case r.status:
       "done"      → for eff in r.effects: for a in dispatch(driver, eff): step(a)   ; loop closes
       "suspended" → record (+ r.resume); a durable runner completes out-of-band → inbound → pump
       "failed"    → record (+ r.error) for flow-level retry / dead-letter
pump():  for a in transport.deliver(): step(a)                    ; inbound drain, shared :seen

Every stage injected → swap runner (sync→Erlang→celery-sx), transport (in-proc→fed-sx), registry, driver — DAG + engine unchanged. Global idempotency = emitter fire-once + a durable inbox.

Build order: (a) DONE — seam as SX contracts + reference engine, tested substrate-agnostic: :status branch (done/suspended/failed), injected env (:ctx + :effects), dedup by activity :id, behavior/pump for inbound, async-completion loop (suspend → out-of-band inject → pump). behavior 10/10. (b) P0 supplies the REAL SYNCHRONOUS adapters; durable/federated runners+transports are later adapter phases (RA/TA below).

P0 — publish workflow on the seam (SYNCHRONOUS, all-SX)

Prove: live host publish → the seam engine (in-process transport · local-SX trigger registry · sync op-table runner over an SX publish-DAG · host driver) → the effect surfaces. NO Erlang, NO fed-sx yet — those are adapter phases (RA/TA). Every piece swaps later; the DAG + engine don't.

• P0.1 — publish-activity contract (SX side). host/blog--publish-activity + post-category. blog 200/200. NOTE: emits the next/-Erlang shape today; P0.4 reconciles to the canonical seam shape.
• P0.2 — the publish-DAG + execute-fold runner + the CAPABILITY check. DONE 2026-07-02. HYPOTHESIS-TEST FINDING: the synchronous business flow expresses NATURALLY as an EXECUTE-FOLD composition (host/execute.sx: seq/effect/alt — the branch on category IS alt, exactly what it's for), NOT an artdag DAG — artdag is pure DATAFLOW with no control flow. So "business logic = art- dag" is confirmed at the ABSTRACTION (both content-addressed op-DAGs) and REFINED at the vocabulary: the SYNCHRONOUS control-flow runner is the execute-fold (caps {effect,branch,each}); artdag is the DATAFLOW sibling (a different runner). Two instances of one thing, run very differently — as predicted. Built: lib/host/flows.sx (host/flow--{node-cap, required-caps, subset?, exec-runner, bind}); host/blog--publish-dag + publish-ctx. Verified: publish-DAG required-caps derived = {effect,branch} → binds to exec-runner; runs → newsletter→[validate,digest]/urgent→[validate, notify]/other→[validate,skip]; a wait node → required {suspend} → binds FAIL-FAST against the exec-runner (would need Erlang, RA). flows 7/7, blog 203/203, conformance 591/591. IMPLICATION for RA/TA: the Erlang runner isn't a "different flow language" — it's the SAME op-DAG with +{suspend} nodes; RA is the runner that advertises suspend + wraps flow_dispatch. CAVEAT (don't calcify this finding): execute-fold-vs-artdag is a CURRENT capability SNAPSHOT, NOT a permanent boundary. artdag MAY GROW control-flow node-kinds (a runner advertising +{effect, branch, each}), and business logic then MIGRATES to artdag to inherit content-addressed memoization / optimize (fuse/dedup/dce) / schedule / FEDERATION (a flow result reused across peers by content-id — the federation vision, free). The capability model makes that migration seamless (same DAGs, richer runner; the execute-fold is just the pragmatic sync runner NOW). See phase AX.
• P0.3 — wire the seam on the live host. DONE + LIVE-VERIFIED 2026-07-02. host/blog-- {transport, triggers (on-publish: create+article → publish-DAG), driver (records each effect), publish-engine, fire-publish!, maybe-publish!}. Both write handlers (form-submit POST /new, edit-submit POST /:slug/edit) call maybe-publish!(slug, prev-status, new-status) — a non-published → published TRANSITION fires the flow (fire-once), in the handler BODY. /flows renders the flow log. LIVE PROOF: logged in + POST /new on blog.rose-ash.com → /flows shows validate + notify (category defaulted to urgent). behavior→exec-runner→driver all real. blog 207/207, conformance 595/595. GAP: the flow log is IN-MEMORY (clears on restart) — "durable record" is P0.3b (persist the log to the blog store + boot-load, string-keyed to dodge the keyword/persist split). Also: the live test post p0.3-seam-live-test persists (no delete route) — harmless, clean up if wanted.
• P0.4 — canonical activity + reconcile. DONE + LIVE-VERIFIED 2026-07-02. host/blog-- publish-activity now emits the CANONICAL seam shape {:verb :actor :object :object-type :slug :category :delta :id} — :object is a content-addressed REFERENCE (the CID, was an inlined dict), :id the dedup identity, :slug+:category the domain fields the DAG reads. Consumers reconciled: the trigger matches :verb+:object-type; publish-ctx reads top-level :category+:slug. Added the runner MARSHALLER host/blog--activity->erl (canonical → next/'s proplist, for RA — defined+tested, unused until RA). (:ts/:prev omitted — no clock primitive in the host; deferred.) LIVE: published on blog.rose-ash.com → /flows fired validate+notify with the canonical activity. blog 209/209, conformance 597/597.

P0 COMPLETE — the synchronous publish workflow runs end-to-end on the LIVE host through the substrate-agnostic seam, durably, in the canonical activity shape, with the Erlang-runner marshaller staged. Every piece is a swappable adapter: RA (Erlang runner) + TA (fed-sx transport) plug in next without touching the DAG or the wiring.

P0 REVIEW (2026-07-02) — findings + carried-forward debt

• FIXED (was a live bug): edit-submit ran maybe-publish! BEFORE set-field-values!, so an edit that set a category AND published branched on the STALE category. Reordered (fields first); regression test added. blog 210/210.
• DEBT #1 (blocks P2): activity identity. Dedup uses :id = the object CID. Relation Add/Remove events don't change the CID → multiple relation activities would share :id → false dedup. P2 needs a real activity id (verb+object+seq/edge), NOT the bare CID.
• DEBT #2 (P1): the capability bind isn't wired into the live engine. host/blog--publish-engine calls exec-runner directly; host/flow--bind is tested but decorative in the live path. P1's engine must DERIVE the runner via bind (required ⊆ advertised), not hardcode it.
• DEBT #3 (RA): flow execution is synchronous in the request path. Fine for 2 cheap effects, but a durable/suspend runner CANNOT block a request — RA needs dispatch moved OFF the request path (emit in-request → a background loop calls behavior/pump). The seam supports it (pump); the wiring doesn't exist. This is structural, not just the marshaller.
• DEBT #4 (minor): the "urgent" category default notifies everyone for an uncategorised post — demo-convenient, semantically wrong. Default to skip/none once the demo value is past.
• Acknowledged/known: unbounded flow log (cap/rotate), registry :register! is a stub (P1 makes it real), :actor "site" placeholder (TA needs the actor model), marshaller unverified-until-RA.

Sequencing note (hidden prerequisites)

• P1's "runner derived via caps" is near-vacuous until a SECOND runner exists — with only exec-runner, everything derives to it. The derivation only MEANS something once RA lands.
• RA is where the real risk lives (SX→erlang-on-sx dispatch + suspend→resume→pump + the er-scheduler context bug) AND P1/P2 quietly depend on what RA forces (a 2nd runner, the async boundary). RECOMMENDATION: a narrow RA SPIKE next — prove one dispatch + one suspend/resume/ pump cycle in isolation — de-risks the whole durable/federated half before building P1/P2 on it.

P1 — types DECLARE behavior (generalize)

• The type carries :behavior = [{:on {:verb :object-type :guard} :dag }] — edited in the type-def editor beside grammar + relations. NO runner hint (the runner is DERIVED from the DAG's required capabilities).
• host/blog--engine-for(object) builds the seam engine from the object's type bindings + registers the triggers, selecting each DAG's MINIMUM runner via artdag/analyze (required-caps) — a {effect}-only DAG runs synchronously; a wait node pulls in the Erlang runner automatically.
• The type-def editor can SHOW each behavior's required capabilities + which runner it resolves to (the derived sync/durable/distributed classification, visible not hand-set).

P2 — state-change → activity emission (ALL events, not just publish)

• Wire the host write path: put!/set-comp!/edit-submit → Create/Update; relate!/unrelate!/tag → Add/Remove. Emit canonical activities into the transport log. Define the delta summary.

RA — the ERLANG (durable) RUNNER adapter ← the old "fed-sx spike", now an adapter

• A seam runner wrapping next/'s flow_dispatch + flow_store: marshal the canonical activity → next/'s proplist, dispatch to a named flow (blog_publish_digest), map the result back to {:status done|suspended :effects :resume}. On suspend, wire the resumed completion → the transport inbound (pump). Baseline: next/tests/triggers_e2e.sh 10/10. RISK: er-scheduler context (project_fed_prims_http_listen_scheduler); keep flows effect-as-data (no blocking). SPIKE the SX→erlang-on-sx call in isolation first.

TA — the FED-SX TRANSPORT adapter ← federation proper

• A seam transport over next/ delivery: :emit → outbox → peers; :deliver → inbox. A remote peer's engine fires its own triggers. Everything works over fed-sx. RISK: next/ delivery M2 blockers (er-scheduler context). Needs the ACTOR MODEL real (:actor was a P0 placeholder — peer_actors / follower_graph / per-author identity).

AX — artdag GROWS control-flow (business logic MIGRATES to artdag) [DEMAND-DRIVEN]

Today artdag is pure dataflow and the execute-fold is the synchronous control-flow runner. That's a capability snapshot. When a business flow WANTS the artdag engine's benefits — content-addressed memoization (recompute only on input-CID change), optimize (fuse/dedup/dce), schedule, and above all FEDERATION (a flow result reused across peers by content-id) — grow artdag's node vocabulary so its runner advertises +{effect, branch, each}, and the SAME behavior DAGs migrate onto it (the capability model makes this seamless; the seam + DAGs don't change). Two real design pieces: (1) DYNAMIC control in a static DAG — a branch PRUNES a path (conditional nodes make downstream nodes live/dead; content-addressing holds on the taken path); (2) EFFECT nodes vs memoization — pure nodes memoize, {effect} nodes are marked non-cacheable (must run / be idempotent). Build when a flow's cost, reuse, or cross-peer sharing makes the execute-fold's re-run-everything insufficient — not before. The execute-fold stays the lean default for cheap synchronous flows.

RX — celery-sx runner (DEMAND-DRIVEN, not scheduled)

Build the distributed/durable runner adapter the moment a real DAG needs heavy compute / long-running-retryable tasks / cross-machine fan-out (the artdag/JAX media case, or federated flows that can't run in-request). New code is small — glue persist (durable queue: enqueue/claim/ ack/visibility-timeout) + er-scheduler (worker loop: pull node → op-table-runner → content- addressed result) + artdag/schedule (fan-out) + retry/backoff. Slots in at artdag/op-table-runner alongside the synchronous + Erlang runners. Zero packages. Do NOT pre-build; the op-table runner covers everything until a DAG's cost/latency/placement forces the substrate.

P4 — close the loop

• Flow effects mutate objects back durably (a flow's DescribeEffects → host writes / new activities), so business logic can change state, which federates, which triggers more flows.

Progress log (newest first)

• 2026-07-02 — P0.4 DONE + LIVE-VERIFIED → P0 COMPLETE. Canonical seam activity shape ({:verb :object=cid :object-type :slug :category :delta :id}); consumers reconciled (trigger match, publish-ctx); host/blog--activity->erl marshaller staged for RA. Published live → /flows fired validate+notify with the canonical activity. blog 209/209, conformance 597/597. The synchronous publish workflow is end-to-end on the live host through the substrate-agnostic seam, durable, in the canonical shape. NEXT: P1 (types declare :behavior, engine built per type, runner derived via caps) — or RA (Erlang durable runner, the marshaller is ready).
• 2026-07-02 — P0.3b DONE + LIVE-VERIFIED. The flow log is now DURABLE: the driver persists string-keyed effect records to the blog store (dodging the keyword/persist top-level split); host/blog-load-flowlog! rebuilds it on boot (serve.sh). Proof: published on blog.rose-ash.com, RESTARTED the container, /flows still showed validate+notify (reloaded from the store). blog 208/208, conformance 596/596. Whole-list rewrite per effect — cap/rotate later.
• 2026-07-02 — P0.3 DONE + LIVE-VERIFIED. The seam wired into the live publish path: on-publish registry + in-process transport + host driver + the execute-fold runner, fired by the draft→ published transition in both write handlers. Published a real post on blog.rose-ash.com → /flows surfaced validate + notify, driven by the actual behavior engine. blog 207/207, conformance 595/595. NEXT: P0.4 (canonical activity shape) or P0.3b (durable flow log); then P1 (types declare behavior — build the engine per type from its :behavior bindings, runner derived via caps).
• 2026-07-02 — DON'T-CALCIFY note (user: "artdag may in the future contain business logic"). The execute-fold-vs-artdag split from P0.2 is a capability SNAPSHOT, not a boundary. Added phase AX: artdag grows +{effect,branch,each} node-kinds and business logic migrates onto it to inherit content-addressed memoization / optimize / FEDERATION (flow result reused across peers by CID — the federation vision, free). Design work named: dynamic control (branch prunes) in a static DAG; effect nodes non-cacheable vs pure nodes memoized. Demand-driven; execute-fold stays the lean default. P0.2 finding + flows.sx header annotated so the finding doesn't harden.
• 2026-07-02 — P0.2 DONE + the hypothesis CONFIRMED (and refined). The synchronous publish workflow is NATURAL as an execute-fold composition (seq/effect/alt), NOT artdag dataflow (no branch there). So business-logic = art-dag holds at the abstraction (content-addressed op-DAG) but the SYNCHRONOUS runner is the execute-fold, artdag the dataflow sibling — two instances, run differently, exactly the framing. lib/host/flows.sx (capability layer + exec-runner + bind) + host/blog--publish-dag. Runner DERIVED via required-caps ⊆ advertised; wait→fail-fast. flows 7/7, blog 203/203, 591/591.
• 2026-07-02 — folded in CAPABILITY-TYPED nodes / CAPABILITY-ADVERTISING runners. A node declares :needs (wait→suspend, fan-out→parallel, heavy→offload); a runner advertises :capabilities (op-table {effect,branch,each}; Erlang +suspend; celery-sx +parallel,retry,offload); artdag/analyze computes a DAG's min-runner; the binder checks required ⊆ runner-caps (fail fast). So the sync/ durable/distributed split is DERIVED from the DAG, not a human call — a {effect}-only DAG runs with zero ceremony; a wait node auto-requires Erlang. Removed the :runner hint from the binding. P0.2 gains the hypothesis test: does the publish workflow express naturally as a DAG, and does flipping a node to wait fail-fast against the op-table runner? Clarifies "business logic = art-dag" — same op-DAG structure, differing only in node capabilities, hence runner. (Insight: they're two instances of one thing, suitable to run very differently.)
• 2026-07-02 — whole-plan coherence review. The reframe (artdag+seam) had left the middle stale: P0.2/P0.3 still described the Erlang-bridge-first path; P0.1's activity didn't match the seam contract; the seam section predated the enrichment. Fixed: P0 rewritten around the seam + SX op-table runner (all-SX, no Erlang/fed-sx); the Erlang/fed-sx path DEMOTED to explicit adapter phases RA (durable runner) + TA (fed-sx transport); canonical activity shape + P0.4 reconcile; seam contract refreshed to behavior.sx (status/dedup/pump/async, behavior 10/10); stray bits cleared.
• 2026-07-02 — seam DONE + reviewed twice. lib/host/behavior.sx (engine + 4 adapters); enriched substrate-agnostic (status/env/dedup/pump); 2nd review corrected the async-completion contract (construction-wired inbound + pump, not env :emit) + proved it. behavior 10/10, conformance 580.
• 2026-07-02 — P0.1 done. host/blog--publish-activity + host/blog--post-category; the publish contract in SX, 200/200. Verified next/ triggers e2e baseline 10/10. Roadmap anchored. NEXT: P0.2 the dispatch bridge (in-process: serve.sh loads next/ kernel + registers the on-publish trigger; host emits the activity via the erlang-on-sx bridge to pipeline:apply_triggers).