plan: capability-typed nodes + capability-advertising runners (derived runner)
Folds in the sharpest refinement: business logic and art-dag are the SAME op-DAG structure,
differing only in the CAPABILITIES their nodes require — so the runner is DERIVED, not chosen.
A node declares :needs (wait→suspend, fan-out→parallel, heavy→offload); a runner advertises
:capabilities (op-table {effect,branch,each}; Erlang +suspend; celery-sx +parallel,retry,offload);
artdag/analyze computes a DAG's required set → its minimum runner; the binder checks required ⊆
runner-caps (fail fast). The sync/durable/distributed split falls out of the DAG (a {effect}-only
DAG runs with zero ceremony; a wait node auto-requires Erlang) — turning 'simple in SX / complex
in Erlang' from a judgment call into a derivable property. Removed the :runner hint from the type
binding; P0.2 gains the hypothesis test (natural-as-a-DAG? + flip-to-wait fails fast); runner
contract gains :capabilities; type-def editor can show the derived classification.
Doc-only.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -8,11 +8,17 @@ render-vs-execute-vs-deps, applied to execution/communication/deployment:
|
||||
- **Behavior = an artdag DAG** — the invariant, content-addressed (`artdag/dag`, analyze/plan/
|
||||
optimize/schedule). Business logic, art media pipelines, workflows — all the same abstraction.
|
||||
- **Execution = an injected RUNNER** (`artdag/run dag RUNNER cache`; `artdag/op-table-runner`).
|
||||
Substrates are just runners — a ladder by capability, same DAG throughout (**durability is a
|
||||
runner capability, not a DAG feature**):
|
||||
- **op-table / execute-fold runner** — synchronous, local, in-request. Covers P0.
|
||||
- **Erlang runner** — durable: suspend/resume/`wait`, deterministic replay (flow-on-erlang).
|
||||
- **celery-sx runner** — distributed/durable task executor, "Celery the way it should have
|
||||
Substrates are runners on a **capability ladder**, same DAG throughout — a node DECLARES the
|
||||
capabilities it needs, a runner ADVERTISES what it supports, and the match is checked at bind
|
||||
time (fail fast, not a mystery at run time). artdag/analyze computes a DAG's required-capability
|
||||
set → its MINIMUM runner. So "simple in SX / durable in Erlang / distributed in celery-sx" is a
|
||||
DERIVED property of the DAG, not a human judgment; a trivial rule is a one-node DAG needing only
|
||||
`{effect}` and runs synchronously with zero ceremony. Business logic and media pipelines are the
|
||||
SAME structure (a content-addressed op-DAG); they differ only in the capabilities their nodes
|
||||
require, hence the runner. The ladder (capabilities each runner adds):
|
||||
- **op-table / execute-fold runner** — `{effect, branch, each}` — synchronous, local, in-request. Covers P0.
|
||||
- **Erlang runner** — `+ suspend/resume/wait` (durable), deterministic replay (flow-on-erlang).
|
||||
- **celery-sx runner** — `+ parallel, retry, offload` — distributed/durable task executor, "Celery the way it should have
|
||||
been" on erlang-on-sx, ZERO packages. It's a LEAN GLUE of parts we already have, not a
|
||||
reimplementation: broker = lib/persist KV (durable enqueue/claim/ack/visibility-timeout) ·
|
||||
worker pool = the er-scheduler / Erlang processes · result backend = content-addressed results
|
||||
@@ -22,7 +28,7 @@ render-vs-execute-vs-deps, applied to execution/communication/deployment:
|
||||
small (~a few hundred lines): a durable queue + a worker loop (pull node → runner → write
|
||||
content-addressed result) + retry/backoff. **BUILD WHEN A DAG DEMANDS IT** — heavy compute,
|
||||
long-running/retryable tasks, or fan-out across machines — NOT for the synchronous P0.
|
||||
- **real-Celery over artdag/L1** — the existing Python media pipeline (JAX/IPFS) as a runner.
|
||||
- **real-Celery over artdag/L1** — `+ gpu/heavy-compute` — the existing Python media pipeline (JAX/IPFS) as a runner.
|
||||
- **Communication = an injected TRANSPORT** (`artdag/federation`, transport injected). Substrates:
|
||||
fed-sx (ActivityPub/next/), internal HMAC HTTP (services), IPFS (content-addressed). Because
|
||||
content-ids are global, a result computed on one instance is reusable on another by id.
|
||||
@@ -44,9 +50,13 @@ object's state changes emit activities; the platform picks runner/transport/plac
|
||||
fire-once is the emitter's job (+ a durable inbox for federation); the seam only dedups in-call.
|
||||
- **Triggers = declared subscriptions** — a type declares named triggers; flows fire only on
|
||||
matching ones. Log complete, execution precise.
|
||||
- **Flows split by DURABILITY not "complexity":** SYNCHRONOUS logic = an SX artdag DAG run by the
|
||||
op-table runner (eager, one-pass, no suspend). Anything needing timer/suspend/human-gate = a
|
||||
DURABLE runner (Erlang flow_dispatch, celery-sx). Same DAG; the RUNNER supplies durability.
|
||||
- **Flows split by CAPABILITY (DERIVED, not a human "complexity" call):** every flow is a
|
||||
content-addressed op-DAG. A node declares the capabilities it needs (a `wait` needs `suspend`, a
|
||||
fan-out needs `parallel`, a heavy op needs `offload`); a runner advertises what it supports;
|
||||
artdag/analyze computes the DAG's required set → its MINIMUM runner; bind-time checks required ⊆
|
||||
runner (fail fast). So the sync/durable/distributed choice falls out of the DAG — a `{effect}`-only
|
||||
DAG runs synchronously (no ceremony); a DAG with a `wait` node auto-requires the Erlang runner.
|
||||
Business logic and media pipelines are the SAME structure — they differ only in node capabilities.
|
||||
- **Effects are DATA; a DRIVER dispatches them** (perform no IO — a blocking call deadlocks the
|
||||
er-scheduler). The host is the driver for P0.
|
||||
- **The type carries its whole contract:** fields+grammar (content) · allowed relations (external)
|
||||
@@ -67,15 +77,20 @@ a swappable adapter — each a dict-of-functions (SX-native, like the fold domai
|
||||
|
||||
1. **Activity** (content-addressed event): `{:verb "create"|"update"|"add"|"remove"|"delete"
|
||||
:actor :object <cid> :object-type :delta :prev <cid> :ts}`.
|
||||
2. **Behavior binding** (declared on a TYPE): `{:on {:verb :object-type :guard} :dag <ref> :runner
|
||||
<hint>}`. The type carries content-grammar + allowed-relations + these bindings.
|
||||
2. **Behavior binding** (declared on a TYPE): `{:on {:verb :object-type :guard} :dag <ref>}`. The
|
||||
TYPE carries content-grammar + allowed-relations + these bindings. NO :runner hint — the runner
|
||||
is DERIVED: the DAG's required-capability set (artdag/analyze over its nodes; a node declares
|
||||
`:needs` e.g. `#{suspend}`) selects the minimum runner that advertises them.
|
||||
3. **Trigger-registry adapter** — `{:register! (fn spec dag hint) :match (fn activity -> [binding])}`.
|
||||
Impls: a local SX matcher / next/'s Erlang trigger_registry.
|
||||
4. **Runner adapter** — `{:run (fn dag env -> {:status "done"|"suspended"|"failed" :effects :resume
|
||||
:error})}`; env = `{:activity :actor :ctx :effects :binding}` (:effects = injected external-read
|
||||
interfaces, deterministic for replay). `:results` is runner-INTERNAL (artdag's memoized outputs).
|
||||
Durability = the runner's, not the DAG's. A durable runner that SUSPENDS is wired at construction
|
||||
to the transport's INBOUND channel and injects its out-of-band completion there (pump drains it).
|
||||
4. **Runner adapter** — `{:capabilities <set> :run (fn dag env -> {:status "done"|"suspended"|
|
||||
"failed" :effects :resume :error})}`. :capabilities is what it ADVERTISES (op-table `{effect,
|
||||
branch, each}`; Erlang `+ suspend`; celery-sx `+ parallel, retry, offload`); the binder checks
|
||||
dag-required ⊆ runner-caps and fails fast otherwise. env = `{:activity :actor :ctx :effects
|
||||
:binding}` (:effects = injected external-read interfaces, deterministic for replay). `:results`
|
||||
is runner-INTERNAL. Durability = the runner's, not the DAG's. A durable runner that SUSPENDS is
|
||||
wired at construction to the transport's INBOUND channel and injects its out-of-band completion
|
||||
there (pump drains it).
|
||||
5. **Transport adapter** — `{:emit (fn activity) :deliver (fn -> [activity])}`. `:emit` = outbound
|
||||
(log/publish), `:deliver` = inbound (peers + async runner completions). Impls: in-process /
|
||||
fed-sx (next/) / HTTP / IPFS. Content-ids global → results move by id.
|
||||
@@ -111,10 +126,15 @@ fed-sx yet — those are adapter phases (RA/TA). Every piece swaps later; the DA
|
||||
|
||||
- [x] **P0.1 — publish-activity contract (SX side).** host/blog--publish-activity + post-category.
|
||||
blog 200/200. NOTE: emits the next/-Erlang shape today; P0.4 reconciles to the canonical seam shape.
|
||||
- [ ] **P0.2 — the publish-DAG + op-table runner.** Author the publish workflow as an SX artdag DAG
|
||||
(validate → publish → notify/digest), + a runner (artdag/op-table-runner over an op-table of the
|
||||
verbs, OR a thin op-table). Test: run the DAG with a publish env → the expected effect-as-data.
|
||||
Synchronous — no wait/suspend (that's the durable runner, RA).
|
||||
- [ ] **P0.2 — the publish-DAG + op-table runner + the CAPABILITY check.** Author the publish
|
||||
workflow as an SX artdag DAG (validate → publish → notify/digest) whose nodes need only
|
||||
`{effect, branch}`, + the op-table runner (advertises `{effect, branch, each}`). ACCEPTANCE
|
||||
(the hypothesis test): (a) the workflow expresses NATURALLY as a DAG — if it's forced, that's the
|
||||
signal the node vocabulary / representation needs rethinking BEFORE RA/TA; (b) artdag/analyze
|
||||
computes the DAG's required-capability set = `{effect, branch}`; (c) the binder confirms required
|
||||
⊆ op-table caps → runs; (d) run it → the expected effect-as-data. Then flip one node to `wait`
|
||||
and confirm the bind FAILS FAST against the op-table runner (would require the Erlang runner, RA)
|
||||
— proving durability is derived, not chosen.
|
||||
- [ ] **P0.3 — wire the seam on the live host.** A local-SX trigger registry (on-publish →
|
||||
publish-DAG), an in-process transport (a KV-backed log), the host as effect driver. In edit-submit
|
||||
detect the draft→published TRANSITION (prev≠published & new=published — fire-once), build the
|
||||
@@ -124,10 +144,14 @@ fed-sx yet — those are adapter phases (RA/TA). Every piece swaps later; the DA
|
||||
{:verb :actor :object <cid> :object-type :delta :ts :id}; keep the category/slug fields the DAG reads.
|
||||
|
||||
## P1 — types DECLARE behavior (generalize)
|
||||
- [ ] The type carries :behavior = [{:on {:verb :object-type :guard} :dag <ref> :runner <hint>}] —
|
||||
edited in the type-def editor beside grammar + relations.
|
||||
- [ ] The type carries :behavior = [{:on {:verb :object-type :guard} :dag <ref>}] — edited in the
|
||||
type-def editor beside grammar + relations. NO runner hint (the runner is DERIVED from the DAG's
|
||||
required capabilities).
|
||||
- [ ] host/blog--engine-for(object) builds the seam engine from the object's type bindings +
|
||||
registers the triggers. Simple DAGs authored as SX composition; complex ones name a durable runner.
|
||||
registers the triggers, selecting each DAG's MINIMUM runner via artdag/analyze (required-caps) —
|
||||
a `{effect}`-only DAG runs synchronously; a `wait` node pulls in the Erlang runner automatically.
|
||||
- [ ] The type-def editor can SHOW each behavior's required capabilities + which runner it resolves
|
||||
to (the derived sync/durable/distributed classification, visible not hand-set).
|
||||
|
||||
## P2 — state-change → activity emission (ALL events, not just publish)
|
||||
- [ ] Wire the host write path: put!/set-comp!/edit-submit → Create/Update; relate!/unrelate!/tag →
|
||||
@@ -161,6 +185,16 @@ covers everything until a DAG's cost/latency/placement forces the substrate.
|
||||
activities), so business logic can change state, which federates, which triggers more flows.
|
||||
|
||||
## Progress log (newest first)
|
||||
- 2026-07-02 — folded in CAPABILITY-TYPED nodes / CAPABILITY-ADVERTISING runners. A node declares
|
||||
`:needs` (wait→suspend, fan-out→parallel, heavy→offload); a runner advertises `:capabilities`
|
||||
(op-table {effect,branch,each}; Erlang +suspend; celery-sx +parallel,retry,offload); artdag/analyze
|
||||
computes a DAG's min-runner; the binder checks required ⊆ runner-caps (fail fast). So the sync/
|
||||
durable/distributed split is DERIVED from the DAG, not a human call — a {effect}-only DAG runs with
|
||||
zero ceremony; a wait node auto-requires Erlang. Removed the :runner hint from the binding. P0.2
|
||||
gains the hypothesis test: does the publish workflow express naturally as a DAG, and does flipping
|
||||
a node to `wait` fail-fast against the op-table runner? Clarifies "business logic = art-dag" — same
|
||||
op-DAG structure, differing only in node capabilities, hence runner. (Insight: they're two
|
||||
instances of one thing, suitable to run very differently.)
|
||||
- 2026-07-02 — whole-plan coherence review. The reframe (artdag+seam) had left the middle stale:
|
||||
P0.2/P0.3 still described the Erlang-bridge-first path; P0.1's activity didn't match the seam
|
||||
contract; the seam section predated the enrichment. Fixed: P0 rewritten around the seam + SX
|
||||
|
||||
Reference in New Issue
Block a user