host: correct the seam's async-completion contract + prove it (2nd review)

Second review of the (core) seam caught a subtle one — and that my first 'fix' was itself wrong.
The async completion of a SUSPENDED durable flow happens AFTER the synchronous process call has
returned, so an :emit captured in the run env would be stale. The correct seam is construction-
wiring: a durable runner is wired to the transport's INBOUND channel at construction and injects
its completion activity there, out-of-band; a later behavior/pump drains it → effects flow. So the
engine code was already right (pump is the async re-entry seam); only the contract comment was
wrong — corrected. New test proves the loop: process(wait) suspends (no effect), then pump drains
the out-of-band completion → the flow's digest effect flows. Also clarified: dedup is per-
invocation (global idempotency = emitter fire-once + durable inbox); retry is flow-level; the
engine-facing runner result is {:status :effects :resume :error} (:results is runner-internal).

behavior 10/10 (+ async-completion). No engine change — comment + test only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-02 13:55:32 +00:00
parent e4fc66bfeb
commit 6ed523623b
2 changed files with 41 additions and 8 deletions

View File

@@ -11,15 +11,24 @@
;; The engine bundles the four (+ optional :effects injected external-read interfaces and a ;; The engine bundles the four (+ optional :effects injected external-read interfaces and a
;; :ctx-of (fn activity -> ctx)). behavior/process folds an activity through the pipeline — ;; :ctx-of (fn activity -> ctx)). behavior/process folds an activity through the pipeline —
;; emit → match triggers → run each DAG → branch on :status → (done) dispatch each effect-as-data ;; emit → match triggers → run each DAG → branch on :status → (done) dispatch each effect-as-data
;; → recurse on new activities. Substrate-agnostic: ;; → recurse on new activities. The engine-facing runner result is {:status :effects :resume
;; :error} (anything else — e.g. artdag's memoized node :results — is runner-internal).
;; Substrate-agnostic:
;; - :status "done" → dispatch effects (synchronous op-table runner). ;; - :status "done" → dispatch effects (synchronous op-table runner).
;; - :status "suspended" → record it (+ :resume token); NO effects yet — a durable (Erlang) ;; - :status "suspended" → record it (+ :resume token); NO effects yet. A durable (Erlang) runner
;; runner holds the flow, and its later completion re-enters as a NEW activity via behavior/pump. ;; HOLDS the flow OFF-ENGINE; a timer/event resumes it OUT-OF-BAND — AFTER this process call has
;; - :status "failed" → record it (+ :error) for retry/dead-letter (celery-sx). ;; returned. On completion the runner injects a completion ACTIVITY into the transport's INBOUND
;; DEDUP: activities are content-addressed; each :id processes once (a cycle is caught by identity, ;; channel (the runner is wired to it at CONSTRUCTION — the sync run env is long gone by then),
;; not just the depth guard). INBOUND: behavior/pump drains transport.deliver (peer activities + ;; and a later behavior/pump drains it → effects flow. The effect-as-data model is intact: the
;; async runner completions). The env carries the injected :effects + per-activity :ctx so real ;; completion is just another inbound activity. Without that wiring a suspended flow is a dead end.
;; DAGs (e.g. an injected fetch_followers, deterministic for replay) can run. ;; - :status "failed" → record it (+ :error) for retry/dead-letter (celery-sx). Retry is
;; FLOW-level (re-run the runner for that binding), NOT activity re-entry (dedup would block it).
;; DEDUP: a cycle within ONE process/pump is caught by activity :id (identity, not just the depth
;; guard). This is PER-INVOCATION, not global — global idempotency is the EMITTER's job (fire-once
;; on the transition) + a DURABLE inbox for federation (cross-pump re-delivery isn't caught here).
;; The env carries injected :effects (external reads, deterministic for replay) + per-activity :ctx.
;; The ASYNC RE-ENTRY seam is the transport's inbound channel + pump — NOT the run env (which is
;; synchronous). Runners that complete out-of-band are wired to that channel at construction.
(define behavior/make-engine (fn (adapters) adapters)) ;; {:triggers :runner :transport :driver :effects? :ctx-of?} (define behavior/make-engine (fn (adapters) adapters)) ;; {:triggers :runner :transport :driver :effects? :ctx-of?}
(define behavior/-triggers (fn (e) (get e :triggers))) (define behavior/-triggers (fn (e) (get e :triggers)))

View File

@@ -110,6 +110,30 @@
(list (len (get tr :emitted)) (len (get tr :ran)) (len (get tr :effects)))) (list (len (get tr :emitted)) (len (get tr :ran)) (len (get tr :effects))))
(list 2 2 2)) (list 2 2 2))
;; ── async completion: a durable runner SUSPENDS, then (wired to an inbox at construction) injects
;; a completion activity OUT-OF-BAND; a later pump drains it → the flow's effect flows. Proves the
;; suspend→resume→complete loop closes via the transport's INBOUND channel, not the sync run env. ──
(define ba-inbox (list)) ;; the durable runner's out-of-band inbox
(define ba-transport {:emit (fn (a) nil) ;; outbound log — SEPARATE from the inbox
:deliver (fn () (let ((batch ba-inbox)) (begin (set! ba-inbox (list)) batch)))})
(define ba-runner
{:run (fn (dag env)
(if (= dag "wait-dag")
(begin (set! ba-inbox (concat ba-inbox (list {:verb "resumed" :actor "a"}))) ;; timer fires (simulated)
{:status "suspended" :resume "morning"})
{:status "done" :effects (list {:kind "digest"})}))})
(define ba-triggers {:register! (fn (s d h) nil)
:match (fn (a) (cond ((= (get a :verb) "wait") (list {:dag "wait-dag"}))
((= (get a :verb) "resumed") (list {:dag "resume-dag"}))
(else (list))))})
(define ba-engine (behavior/make-engine {:triggers ba-triggers :runner ba-runner :transport ba-transport :driver be-driver}))
(host-be-test "async completion — suspend, then a later pump drains the out-of-band completion → effect"
(let ((t1 (behavior/process ba-engine {:verb "wait" :actor "a"})))
(let ((t2 (behavior/pump ba-engine)))
(list (len (get t1 :suspended)) (len (get t1 :effects))
(len (get t2 :emitted)) (get (first (get t2 :effects)) :kind))))
(list 1 0 1 "digest"))
(define host-be-tests-run! (define host-be-tests-run!
(fn () (fn ()
{:total (+ host-be-pass host-be-fail) {:total (+ host-be-pass host-be-fail)