From 8c48cac46fd533c5082ceb1d2d7671b1f892f20c Mon Sep 17 00:00:00 2001 From: giles Date: Thu, 2 Jul 2026 14:09:24 +0000 Subject: [PATCH] plan: capability-typed nodes + capability-advertising runners (derived runner) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Folds in the sharpest refinement: business logic and art-dag are the SAME op-DAG structure, differing only in the CAPABILITIES their nodes require — so the runner is DERIVED, not chosen. A node declares :needs (wait→suspend, fan-out→parallel, heavy→offload); a runner advertises :capabilities (op-table {effect,branch,each}; Erlang +suspend; celery-sx +parallel,retry,offload); artdag/analyze computes a DAG's required set → its minimum runner; the binder checks required ⊆ runner-caps (fail fast). The sync/durable/distributed split falls out of the DAG (a {effect}-only DAG runs with zero ceremony; a wait node auto-requires Erlang) — turning 'simple in SX / complex in Erlang' from a judgment call into a derivable property. Removed the :runner hint from the type binding; P0.2 gains the hypothesis test (natural-as-a-DAG? + flip-to-wait fails fast); runner contract gains :capabilities; type-def editor can show the derived classification. Doc-only. Co-Authored-By: Claude Opus 4.8 --- plans/business-logic-fed-flows.md | 80 ++++++++++++++++++++++--------- 1 file changed, 57 insertions(+), 23 deletions(-) diff --git a/plans/business-logic-fed-flows.md b/plans/business-logic-fed-flows.md index 33a2a47f..8c1e7295 100644 --- a/plans/business-logic-fed-flows.md +++ b/plans/business-logic-fed-flows.md @@ -8,11 +8,17 @@ render-vs-execute-vs-deps, applied to execution/communication/deployment: - **Behavior = an artdag DAG** — the invariant, content-addressed (`artdag/dag`, analyze/plan/ optimize/schedule). Business logic, art media pipelines, workflows — all the same abstraction. - **Execution = an injected RUNNER** (`artdag/run dag RUNNER cache`; `artdag/op-table-runner`). - Substrates are just runners — a ladder by capability, same DAG throughout (**durability is a - runner capability, not a DAG feature**): - - **op-table / execute-fold runner** — synchronous, local, in-request. Covers P0. - - **Erlang runner** — durable: suspend/resume/`wait`, deterministic replay (flow-on-erlang). - - **celery-sx runner** — distributed/durable task executor, "Celery the way it should have + Substrates are runners on a **capability ladder**, same DAG throughout — a node DECLARES the + capabilities it needs, a runner ADVERTISES what it supports, and the match is checked at bind + time (fail fast, not a mystery at run time). artdag/analyze computes a DAG's required-capability + set → its MINIMUM runner. So "simple in SX / durable in Erlang / distributed in celery-sx" is a + DERIVED property of the DAG, not a human judgment; a trivial rule is a one-node DAG needing only + `{effect}` and runs synchronously with zero ceremony. Business logic and media pipelines are the + SAME structure (a content-addressed op-DAG); they differ only in the capabilities their nodes + require, hence the runner. The ladder (capabilities each runner adds): + - **op-table / execute-fold runner** — `{effect, branch, each}` — synchronous, local, in-request. Covers P0. + - **Erlang runner** — `+ suspend/resume/wait` (durable), deterministic replay (flow-on-erlang). + - **celery-sx runner** — `+ parallel, retry, offload` — distributed/durable task executor, "Celery the way it should have been" on erlang-on-sx, ZERO packages. It's a LEAN GLUE of parts we already have, not a reimplementation: broker = lib/persist KV (durable enqueue/claim/ack/visibility-timeout) · worker pool = the er-scheduler / Erlang processes · result backend = content-addressed results @@ -22,7 +28,7 @@ render-vs-execute-vs-deps, applied to execution/communication/deployment: small (~a few hundred lines): a durable queue + a worker loop (pull node → runner → write content-addressed result) + retry/backoff. **BUILD WHEN A DAG DEMANDS IT** — heavy compute, long-running/retryable tasks, or fan-out across machines — NOT for the synchronous P0. - - **real-Celery over artdag/L1** — the existing Python media pipeline (JAX/IPFS) as a runner. + - **real-Celery over artdag/L1** — `+ gpu/heavy-compute` — the existing Python media pipeline (JAX/IPFS) as a runner. - **Communication = an injected TRANSPORT** (`artdag/federation`, transport injected). Substrates: fed-sx (ActivityPub/next/), internal HMAC HTTP (services), IPFS (content-addressed). Because content-ids are global, a result computed on one instance is reusable on another by id. @@ -44,9 +50,13 @@ object's state changes emit activities; the platform picks runner/transport/plac fire-once is the emitter's job (+ a durable inbox for federation); the seam only dedups in-call. - **Triggers = declared subscriptions** — a type declares named triggers; flows fire only on matching ones. Log complete, execution precise. -- **Flows split by DURABILITY not "complexity":** SYNCHRONOUS logic = an SX artdag DAG run by the - op-table runner (eager, one-pass, no suspend). Anything needing timer/suspend/human-gate = a - DURABLE runner (Erlang flow_dispatch, celery-sx). Same DAG; the RUNNER supplies durability. +- **Flows split by CAPABILITY (DERIVED, not a human "complexity" call):** every flow is a + content-addressed op-DAG. A node declares the capabilities it needs (a `wait` needs `suspend`, a + fan-out needs `parallel`, a heavy op needs `offload`); a runner advertises what it supports; + artdag/analyze computes the DAG's required set → its MINIMUM runner; bind-time checks required ⊆ + runner (fail fast). So the sync/durable/distributed choice falls out of the DAG — a `{effect}`-only + DAG runs synchronously (no ceremony); a DAG with a `wait` node auto-requires the Erlang runner. + Business logic and media pipelines are the SAME structure — they differ only in node capabilities. - **Effects are DATA; a DRIVER dispatches them** (perform no IO — a blocking call deadlocks the er-scheduler). The host is the driver for P0. - **The type carries its whole contract:** fields+grammar (content) · allowed relations (external) @@ -67,15 +77,20 @@ a swappable adapter — each a dict-of-functions (SX-native, like the fold domai 1. **Activity** (content-addressed event): `{:verb "create"|"update"|"add"|"remove"|"delete" :actor :object :object-type :delta :prev :ts}`. -2. **Behavior binding** (declared on a TYPE): `{:on {:verb :object-type :guard} :dag :runner - }`. The type carries content-grammar + allowed-relations + these bindings. +2. **Behavior binding** (declared on a TYPE): `{:on {:verb :object-type :guard} :dag }`. The + TYPE carries content-grammar + allowed-relations + these bindings. NO :runner hint — the runner + is DERIVED: the DAG's required-capability set (artdag/analyze over its nodes; a node declares + `:needs` e.g. `#{suspend}`) selects the minimum runner that advertises them. 3. **Trigger-registry adapter** — `{:register! (fn spec dag hint) :match (fn activity -> [binding])}`. Impls: a local SX matcher / next/'s Erlang trigger_registry. -4. **Runner adapter** — `{:run (fn dag env -> {:status "done"|"suspended"|"failed" :effects :resume - :error})}`; env = `{:activity :actor :ctx :effects :binding}` (:effects = injected external-read - interfaces, deterministic for replay). `:results` is runner-INTERNAL (artdag's memoized outputs). - Durability = the runner's, not the DAG's. A durable runner that SUSPENDS is wired at construction - to the transport's INBOUND channel and injects its out-of-band completion there (pump drains it). +4. **Runner adapter** — `{:capabilities :run (fn dag env -> {:status "done"|"suspended"| + "failed" :effects :resume :error})}`. :capabilities is what it ADVERTISES (op-table `{effect, + branch, each}`; Erlang `+ suspend`; celery-sx `+ parallel, retry, offload`); the binder checks + dag-required ⊆ runner-caps and fails fast otherwise. env = `{:activity :actor :ctx :effects + :binding}` (:effects = injected external-read interfaces, deterministic for replay). `:results` + is runner-INTERNAL. Durability = the runner's, not the DAG's. A durable runner that SUSPENDS is + wired at construction to the transport's INBOUND channel and injects its out-of-band completion + there (pump drains it). 5. **Transport adapter** — `{:emit (fn activity) :deliver (fn -> [activity])}`. `:emit` = outbound (log/publish), `:deliver` = inbound (peers + async runner completions). Impls: in-process / fed-sx (next/) / HTTP / IPFS. Content-ids global → results move by id. @@ -111,10 +126,15 @@ fed-sx yet — those are adapter phases (RA/TA). Every piece swaps later; the DA - [x] **P0.1 — publish-activity contract (SX side).** host/blog--publish-activity + post-category. blog 200/200. NOTE: emits the next/-Erlang shape today; P0.4 reconciles to the canonical seam shape. -- [ ] **P0.2 — the publish-DAG + op-table runner.** Author the publish workflow as an SX artdag DAG - (validate → publish → notify/digest), + a runner (artdag/op-table-runner over an op-table of the - verbs, OR a thin op-table). Test: run the DAG with a publish env → the expected effect-as-data. - Synchronous — no wait/suspend (that's the durable runner, RA). +- [ ] **P0.2 — the publish-DAG + op-table runner + the CAPABILITY check.** Author the publish + workflow as an SX artdag DAG (validate → publish → notify/digest) whose nodes need only + `{effect, branch}`, + the op-table runner (advertises `{effect, branch, each}`). ACCEPTANCE + (the hypothesis test): (a) the workflow expresses NATURALLY as a DAG — if it's forced, that's the + signal the node vocabulary / representation needs rethinking BEFORE RA/TA; (b) artdag/analyze + computes the DAG's required-capability set = `{effect, branch}`; (c) the binder confirms required + ⊆ op-table caps → runs; (d) run it → the expected effect-as-data. Then flip one node to `wait` + and confirm the bind FAILS FAST against the op-table runner (would require the Erlang runner, RA) + — proving durability is derived, not chosen. - [ ] **P0.3 — wire the seam on the live host.** A local-SX trigger registry (on-publish → publish-DAG), an in-process transport (a KV-backed log), the host as effect driver. In edit-submit detect the draft→published TRANSITION (prev≠published & new=published — fire-once), build the @@ -124,10 +144,14 @@ fed-sx yet — those are adapter phases (RA/TA). Every piece swaps later; the DA {:verb :actor :object :object-type :delta :ts :id}; keep the category/slug fields the DAG reads. ## P1 — types DECLARE behavior (generalize) -- [ ] The type carries :behavior = [{:on {:verb :object-type :guard} :dag :runner }] — - edited in the type-def editor beside grammar + relations. +- [ ] The type carries :behavior = [{:on {:verb :object-type :guard} :dag }] — edited in the + type-def editor beside grammar + relations. NO runner hint (the runner is DERIVED from the DAG's + required capabilities). - [ ] host/blog--engine-for(object) builds the seam engine from the object's type bindings + - registers the triggers. Simple DAGs authored as SX composition; complex ones name a durable runner. + registers the triggers, selecting each DAG's MINIMUM runner via artdag/analyze (required-caps) — + a `{effect}`-only DAG runs synchronously; a `wait` node pulls in the Erlang runner automatically. +- [ ] The type-def editor can SHOW each behavior's required capabilities + which runner it resolves + to (the derived sync/durable/distributed classification, visible not hand-set). ## P2 — state-change → activity emission (ALL events, not just publish) - [ ] Wire the host write path: put!/set-comp!/edit-submit → Create/Update; relate!/unrelate!/tag → @@ -161,6 +185,16 @@ covers everything until a DAG's cost/latency/placement forces the substrate. activities), so business logic can change state, which federates, which triggers more flows. ## Progress log (newest first) +- 2026-07-02 — folded in CAPABILITY-TYPED nodes / CAPABILITY-ADVERTISING runners. A node declares + `:needs` (wait→suspend, fan-out→parallel, heavy→offload); a runner advertises `:capabilities` + (op-table {effect,branch,each}; Erlang +suspend; celery-sx +parallel,retry,offload); artdag/analyze + computes a DAG's min-runner; the binder checks required ⊆ runner-caps (fail fast). So the sync/ + durable/distributed split is DERIVED from the DAG, not a human call — a {effect}-only DAG runs with + zero ceremony; a wait node auto-requires Erlang. Removed the :runner hint from the binding. P0.2 + gains the hypothesis test: does the publish workflow express naturally as a DAG, and does flipping + a node to `wait` fail-fast against the op-table runner? Clarifies "business logic = art-dag" — same + op-DAG structure, differing only in node capabilities, hence runner. (Insight: they're two + instances of one thing, suitable to run very differently.) - 2026-07-02 — whole-plan coherence review. The reframe (artdag+seam) had left the middle stale: P0.2/P0.3 still described the Erlang-bridge-first path; P0.1's activity didn't match the seam contract; the seam section predated the enrichment. Fixed: P0 rewritten around the seam + SX