preserve fed-sx-m1 loop briefings before pruning its worktree
4 untracked agent-briefing docs from the fed-sx-m1 worktree (merged branch loops/fed-sx-m2), saved here so they survive the worktree cleanup.
This commit is contained in:
151
plans/agent-briefings/erlang-send-after.md
Normal file
151
plans/agent-briefings/erlang-send-after.md
Normal file
@@ -0,0 +1,151 @@
|
|||||||
|
; -*- mode: markdown -*-
|
||||||
|
# loops/erlang — `erlang:send_after` substrate primitive
|
||||||
|
|
||||||
|
Scoped briefing for a single focused iteration loop on `loops/erlang`.
|
||||||
|
Not a replacement for the general `erlang-loop.md`; this is the
|
||||||
|
load-bearing-blocker task that, when done, unblocks
|
||||||
|
`plans/fed-sx-milestone-2.md` Step **8b-timer** (delivery retry loop)
|
||||||
|
and the standard `gen_server` `timeout` return.
|
||||||
|
|
||||||
|
```
|
||||||
|
description: loops/erlang — send_after primitive
|
||||||
|
subagent_type: general-purpose
|
||||||
|
run_in_background: true
|
||||||
|
isolation: worktree # worktree at /root/rose-ash-loops/erlang
|
||||||
|
```
|
||||||
|
|
||||||
|
## The goal
|
||||||
|
|
||||||
|
Implement the standard Erlang timer primitives so a process can
|
||||||
|
schedule a message-to-self (or to another pid) after N milliseconds:
|
||||||
|
|
||||||
|
```erlang
|
||||||
|
Ref = erlang:send_after(Time, Dest, Msg). %% Time: int millis. Dest: pid or atom.
|
||||||
|
ok | TimeLeft = erlang:cancel_timer(Ref). %% returns remaining ms, or false if fired/expired.
|
||||||
|
```
|
||||||
|
|
||||||
|
These are the same primitives `gen_server` uses internally to emit
|
||||||
|
`{noreply, S, Timeout}` returns — when the gen_server's `handle_*`
|
||||||
|
callback returns `{noreply, NewState, T}`, the gen_server schedules
|
||||||
|
`{timeout}` to itself after T ms via `send_after`, and the next
|
||||||
|
`handle_info({timeout}, S)` fires when no other message arrives first.
|
||||||
|
|
||||||
|
Without `send_after`, anything that wants a delayed self-cast has to
|
||||||
|
busy-loop or block — neither acceptable for the kernel.
|
||||||
|
|
||||||
|
## Acceptance — single SX file `lib/erlang/tests/send_after.sx`
|
||||||
|
|
||||||
|
The standard test suite plus this one. Run the conformance gate after
|
||||||
|
each commit (`bash lib/erlang/conformance.sh`).
|
||||||
|
|
||||||
|
- `T1` — `erlang:send_after(50, self(), hello)` returns a Ref; after
|
||||||
|
60ms a receive picks up `hello`. Round-trip latency under 100ms in
|
||||||
|
steady state.
|
||||||
|
- `T2` — `Ref = erlang:send_after(1000, self(), late), erlang:cancel_timer(Ref)`
|
||||||
|
returns an integer ~1000 (remaining ms) AND a subsequent
|
||||||
|
`receive late -> got after 50 -> none end` returns `none` — the
|
||||||
|
cancelled message never arrives.
|
||||||
|
- `T3` — multiple in-flight timers fire in deadline order, not
|
||||||
|
schedule order: `send_after(80, self(), b)`, then
|
||||||
|
`send_after(20, self(), a)` — selective receive on `a` first.
|
||||||
|
- `T4` — cancel_timer on an already-fired timer returns `false`.
|
||||||
|
- `T5` — `send_after` to a registered atom: register a process, queue
|
||||||
|
a delayed message to its name, it lands in that process's mailbox.
|
||||||
|
- `T6` — `gen_server` `{noreply, S, 100}` → `handle_info({timeout}, S)`
|
||||||
|
fires after 100ms when no other message arrives. (Sanity check that
|
||||||
|
the gen_server library, currently shipped, hooks up correctly.)
|
||||||
|
|
||||||
|
Conformance gate stays green (currently 725/725 on loops/erlang per
|
||||||
|
`lib/erlang/scoreboard.json`, 761/761 after the m2 BIFs are in —
|
||||||
|
both numbers will move when this lands).
|
||||||
|
|
||||||
|
## Implementation shape (suggested, not prescribed)
|
||||||
|
|
||||||
|
The OCaml host has no event loop today — `bin/sx_server.ml` is a
|
||||||
|
single-thread epoch protocol. But the Erlang scheduler in
|
||||||
|
`lib/erlang/runtime.sx` IS a SX-level event loop already: `er-sched`
|
||||||
|
holds the global dict; `er-sched-step-alive!` advances one runnable
|
||||||
|
process; `er-sched-run-all!` drains the runnable queue until all
|
||||||
|
processes are waiting / exiting / dead. Two options:
|
||||||
|
|
||||||
|
**Option A — pure-SX timer wheel.** Extend the scheduler dict with
|
||||||
|
`:timers (sorted-list-of {:deadline-ms _ :pid _ :msg _ :ref _ :alive
|
||||||
|
true})`. `er-sched-run-all!` already loops; when it finds the runnable
|
||||||
|
queue empty but processes still alive in `receive` blocks, walk the
|
||||||
|
timer list and check whether any deadline has passed (need a
|
||||||
|
monotonic clock — see below). For deadlines in the future, sleep
|
||||||
|
until the next one or until the runnable queue gets work. Cancel = set
|
||||||
|
`:alive false` on the timer entry; cull on next scan.
|
||||||
|
|
||||||
|
**Option B — OCaml-side `Unix.setitimer` / `Unix.select` wakeup.**
|
||||||
|
Heavier, lets the SX scheduler park properly on `Unix.select` against
|
||||||
|
a self-pipe. Right shape for a long-running kernel that may sit idle
|
||||||
|
for minutes between events. Probably the production answer, but
|
||||||
|
significantly more work than A.
|
||||||
|
|
||||||
|
Recommend A first for the unblock; B as a follow-up if the kernel
|
||||||
|
needs to idle gracefully.
|
||||||
|
|
||||||
|
### Clock primitive
|
||||||
|
|
||||||
|
`erlang:monotonic_time/0,1` and `erlang:system_time/0,1` ought to
|
||||||
|
exist for completeness — `send_after` needs at least a monotonic ms
|
||||||
|
clock. If the substrate doesn't have one, add `erlang:monotonic_time(millisecond)`
|
||||||
|
as a small native BIF in `bin/sx_server.ml` (Unix.gettimeofday) and
|
||||||
|
expose it via `er-bif-erlang-monotonic-time`. Time travel safety:
|
||||||
|
DO NOT use wall-clock for timer deadlines.
|
||||||
|
|
||||||
|
## Files in play (loops/erlang scope)
|
||||||
|
|
||||||
|
- `lib/erlang/runtime.sx` — scheduler state, BIFs, primitive registration.
|
||||||
|
Scheduler dict at `er-scheduler` (single-element list).
|
||||||
|
Step function: `er-sched-step-alive!` (line ~708 on m2; check
|
||||||
|
line numbers freshly — recent commits moved code).
|
||||||
|
- `lib/erlang/tests/send_after.sx` — new file, the test suite above.
|
||||||
|
- `lib/erlang/scoreboard.json` + `scoreboard.md` — bump counts.
|
||||||
|
- (If Option B is taken) `bin/sx_server.ml` — out of `loops/erlang`
|
||||||
|
scope per the loop's standing rules. Surface a separate
|
||||||
|
`loops/fed-prims` task for the native bits and keep `send_after`'s
|
||||||
|
SX wrapper minimal.
|
||||||
|
|
||||||
|
## Base branch / rebase note
|
||||||
|
|
||||||
|
The substrate change for the kernel mutex (Blockers #4 in
|
||||||
|
fed-sx-milestone-2.md) lands on architecture via the m2 merge. The
|
||||||
|
key change is in `lib/erlang/runtime.sx`:
|
||||||
|
|
||||||
|
- `er-sched-step-alive!` reads `:pending-args` from the process record
|
||||||
|
when first invoked (line 736 on m2; was hardcoded to `(list)`).
|
||||||
|
- `er-bif-http-listen` spawns the user's handler as a real er-process
|
||||||
|
with `:pending-args (list req-pl)` instead of `er-apply-fun`-inline.
|
||||||
|
|
||||||
|
If `loops/erlang` rebases onto `origin/architecture` and that merge
|
||||||
|
hasn't landed yet (still sitting on `local /root/rose-ash` at
|
||||||
|
`089ed88f`-or-later), fetch `origin/loops/fed-sx-m2` and cherry-pick
|
||||||
|
or rebase against `29e4234b` to pick up the `:pending-args` field
|
||||||
|
shape. The work area (`er-sched-step-alive!`) is the same surface
|
||||||
|
this loop will touch — without the m2 change, the timer wakeup path
|
||||||
|
won't dovetail cleanly with the spawn-from-BIF pattern. Conflict
|
||||||
|
risk: medium. Conflict surface: small (~5 lines around the
|
||||||
|
`:pending-args` read).
|
||||||
|
|
||||||
|
## Test discipline
|
||||||
|
|
||||||
|
- `bash lib/erlang/conformance.sh` green before and after every commit.
|
||||||
|
- Commits scoped to `lib/erlang/**`. No edits to `next/**`,
|
||||||
|
`bin/sx_server.ml`, `spec/**`, or other `lib/<lang>/**` from this
|
||||||
|
loop.
|
||||||
|
- One commit per acceptance test cluster — `T1+T2`, `T3+T4`, `T5+T6`
|
||||||
|
is a reasonable cadence.
|
||||||
|
- Push to `origin/loops/erlang` after each commit. The fed-sx-m2 loop
|
||||||
|
is dormant waiting on this.
|
||||||
|
|
||||||
|
## Done when
|
||||||
|
|
||||||
|
- 6/6 new `send_after` tests green.
|
||||||
|
- conformance gate green at the new total.
|
||||||
|
- single line ticked in `plans/erlang-on-sx.md` Progress log + a tick
|
||||||
|
on whatever roadmap entry covers timers.
|
||||||
|
- Then update the m2 plan (`plans/fed-sx-milestone-2.md` Blockers #3)
|
||||||
|
to **RESOLVED** and unblock Step 8b-timer; that update belongs to
|
||||||
|
the fed-sx-m2 loop, not this one.
|
||||||
170
plans/agent-briefings/fed-sx-m1-loop.md
Normal file
170
plans/agent-briefings/fed-sx-m1-loop.md
Normal file
@@ -0,0 +1,170 @@
|
|||||||
|
# fed-sx Milestone 1 loop agent (single agent, step-ordered)
|
||||||
|
|
||||||
|
Role: iterates `plans/fed-sx-milestone-1.md` forever. Builds the smallest fed-sx
|
||||||
|
kernel that proves the architecture works end-to-end. One feature per commit.
|
||||||
|
|
||||||
|
```
|
||||||
|
description: fed-sx Milestone 1 kernel loop
|
||||||
|
subagent_type: general-purpose
|
||||||
|
run_in_background: true
|
||||||
|
isolation: worktree
|
||||||
|
```
|
||||||
|
|
||||||
|
## Prompt
|
||||||
|
|
||||||
|
You are the sole background agent working `/root/rose-ash/plans/fed-sx-milestone-1.md`.
|
||||||
|
You run in an isolated git worktree on branch `loops/fed-sx-m1` at
|
||||||
|
`/root/rose-ash-loops/fed-sx-m1`. You work the plan's Steps in dependency order
|
||||||
|
(1→9), forever, one commit per feature. Push to `origin/loops/fed-sx-m1` after
|
||||||
|
every commit. Never `main`, never `architecture`.
|
||||||
|
|
||||||
|
## Restart baseline — check before iterating
|
||||||
|
|
||||||
|
1. Read `plans/fed-sx-milestone-1.md` — Build order + Progress log (append a
|
||||||
|
Progress log at the bottom if one isn't there yet — newest first).
|
||||||
|
2. `ls next/` — pick up from the most advanced step that exists. If `next/`
|
||||||
|
doesn't exist, you're at Step 1.
|
||||||
|
3. Erlang substrate must be green:
|
||||||
|
`cd lib/erlang && bash conformance.sh 2>&1 | tail -2` → expect `729 / 729`.
|
||||||
|
If broken and not by your edits, Blockers entry + stop.
|
||||||
|
4. If `next/tests/*.sh` exist, run them. They must be green before new work.
|
||||||
|
5. Read the Erlang Phase 8 status in `plans/erlang-on-sx.md` — `httpc:request`
|
||||||
|
and `sqlite:*` remain BLOCKED. Milestone 1 does not need them per the
|
||||||
|
non-goals section. If you find yourself needing them, you're off-plan.
|
||||||
|
|
||||||
|
## The build queue
|
||||||
|
|
||||||
|
Each Step has concrete deliverables + tests + acceptance check in the plan.
|
||||||
|
Within a Step, pick the smallest unchecked sub-deliverable. Don't batch Steps.
|
||||||
|
|
||||||
|
- **Step 1** — Repo skeleton (`next/`) + canonical CID computation
|
||||||
|
- **Step 2** — Activity envelope shape + signature verify
|
||||||
|
- **Step 3** — JSONL log + sequence numbers (per-actor outbox)
|
||||||
|
- **Step 4** — Genesis bundle: SX sources + bundler + CID verification
|
||||||
|
- **Step 5** — Registry mechanism + bootstrap-projection dispatch
|
||||||
|
- **Step 6** — Validation pipeline driver + `POST /activity`
|
||||||
|
- **Step 7** — Projection scheduler (gen_server per projection)
|
||||||
|
- **Step 8** — HTTP server + AP endpoints + projection queries
|
||||||
|
- **Step 9** — Smoke tests: Pin verb extensibility + reactive application
|
||||||
|
|
||||||
|
The iteration:
|
||||||
|
implement → run step's tests → run no-regression gate (Erlang conformance) →
|
||||||
|
commit → tick the `[ ]` in the plan → append one dated line to the Progress
|
||||||
|
log → push → stop.
|
||||||
|
|
||||||
|
## How fed-sx code lives in this repo
|
||||||
|
|
||||||
|
The kernel is **Erlang-on-SX**. Three concrete patterns:
|
||||||
|
|
||||||
|
1. **Kernel modules as `.erl` source files** at `next/kernel/*.erl`. Loaded at
|
||||||
|
boot via `code:load_binary(Mod, Filename, SourceString)` (Phase 7 BIF).
|
||||||
|
Example: `next/kernel/cid.erl` with `-module(cid). -export([from_sx/1, ...]).`
|
||||||
|
2. **Genesis bundle entries as `.sx` files** at `next/genesis/**/*.sx`.
|
||||||
|
These ARE small SX expressions per design §12.2 (DefineActivity{},
|
||||||
|
DefineProjection{}, etc.).
|
||||||
|
3. **Test scripts as bash** at `next/tests/*.sh`. Each one feeds an epoch
|
||||||
|
protocol script to `hosts/ocaml/_build/default/bin/sx_server.exe` that
|
||||||
|
loads kernel modules, drives them, and asserts on output.
|
||||||
|
|
||||||
|
The `epoch` protocol pattern:
|
||||||
|
```bash
|
||||||
|
printf '(epoch 1)\n(load "lib/erlang/runtime.sx")\n(epoch 2)\n<test-expr>\n' \
|
||||||
|
| hosts/ocaml/_build/default/bin/sx_server.exe
|
||||||
|
```
|
||||||
|
|
||||||
|
## Phase 8 BIFs available to you
|
||||||
|
|
||||||
|
These are wired and tested in `lib/erlang/runtime.sx` (Erlang conformance 729/729):
|
||||||
|
|
||||||
|
- `crypto:hash/2` — `sha256` / `sha512` / `sha3_256`, returns raw binary
|
||||||
|
- `cid:from_bytes/1` — bytes → CIDv1 (raw codec, sha2-256 multihash) as binary
|
||||||
|
- `cid:to_string/1` — any term → canonical CIDv1 string via `er-format-value`
|
||||||
|
- `file:read_file/1`, `file:write_file/2`, `file:delete/1`, `file:list_dir/1`
|
||||||
|
- `code:load_binary/3` — hot-load an Erlang module from source string
|
||||||
|
|
||||||
|
Native HTTP server (registered in `bin/sx_server.ml`, not yet wrapped as an
|
||||||
|
Erlang BIF):
|
||||||
|
|
||||||
|
- SX primitive `http-listen` — exposed at the SX layer, native-only
|
||||||
|
|
||||||
|
For Step 8 you will probably need to add an Erlang BIF wrapper `http:listen/2`
|
||||||
|
that delegates to the SX `http-listen` primitive. That single wrapper is an
|
||||||
|
allowed exception to the scope rules (see below) — flag it explicitly in the
|
||||||
|
commit message.
|
||||||
|
|
||||||
|
**Blocked primitives** (do NOT use, Milestone 1 doesn't need them):
|
||||||
|
- `httpc:request/4` — HTTP client (Milestone 2 federation)
|
||||||
|
- `sqlite:*` — SQLite (deferred storage backend)
|
||||||
|
|
||||||
|
## Ground rules (hard)
|
||||||
|
|
||||||
|
- **Scope:** only `next/**` and `plans/fed-sx-milestone-1.md`. Single allowed
|
||||||
|
exception: a `http:listen/2` BIF wrapper in `lib/erlang/runtime.sx` for Step 8
|
||||||
|
(one commit, clearly flagged). Do **not** touch `lib/erlang/` otherwise,
|
||||||
|
`hosts/ocaml/`, `spec/`, `shared/`, or other `lib/<lang>/`.
|
||||||
|
- **Erlang-on-SX is the substrate.** Kernel modules are `.erl` source loaded
|
||||||
|
via `code:load_binary/3`. Don't reach for pure SX or Python — the substrate
|
||||||
|
for this work is Erlang.
|
||||||
|
- **No new opam deps.** No new host primitives. If you find yourself wanting a
|
||||||
|
new primitive, that's a Blockers entry — the `loops/fed-prims` loop owns
|
||||||
|
primitives, not this loop.
|
||||||
|
- **No-regression gate:** after every commit, `bash lib/erlang/conformance.sh`
|
||||||
|
must report `729 / 729`. Your new tests are additive: `bash next/tests/*.sh`
|
||||||
|
also pass. Test the gate before pushing.
|
||||||
|
- **Builds are slow.** `dune build` (if you ever need it — you shouldn't) gets
|
||||||
|
`timeout: 600000`. Conformance gate: `timeout: 400000`. If a build genuinely
|
||||||
|
hangs > 10min, Blockers entry + stop.
|
||||||
|
- **Commit granularity:** one feature per commit. Short factual messages:
|
||||||
|
`fed-sx-m1: Step 1 — canonical CID + 10 round-trip tests`. Update plan
|
||||||
|
checkboxes + Progress log in the SAME commit as the feature.
|
||||||
|
- **`.erl`/`.sh`/`.md` files:** ordinary `Read`/`Edit`/`Write`. The hook only
|
||||||
|
blocks `.sx`/`.sxc`. For `.sx` files (genesis bundle, Step 4 onwards) use
|
||||||
|
`sx-tree` MCP tools and `sx_write_file` exclusively.
|
||||||
|
- **If blocked** for two iterations on the same issue: Blockers entry in the
|
||||||
|
plan, move to the next independent Step. Step dependencies in the plan's
|
||||||
|
build order table.
|
||||||
|
|
||||||
|
## Specific gotchas
|
||||||
|
|
||||||
|
- **The plan's `.erl` snippets are illustrative.** This Erlang port doesn't
|
||||||
|
necessarily support every BEAM feature. Verify what works by smoke-testing
|
||||||
|
one expression at a time via the epoch protocol before writing 200-line
|
||||||
|
modules. The Erlang port's binary syntax in particular: `<<"abc">>` produces
|
||||||
|
an **empty** binary in this port (string-literal segments unsupported) —
|
||||||
|
use integer segments `<<97,98,99>>` instead. This bit the erlang loop on
|
||||||
|
Phase 8; don't get burned again.
|
||||||
|
- **`cid:to_string/1` on compound terms uses `er-format-value`**, not
|
||||||
|
`er-to-sx`. `cbor-encode` rejects marshalled symbols, so the canonical
|
||||||
|
string form goes via the deterministic textual form. Mirror this when you
|
||||||
|
implement `from_sx/1`.
|
||||||
|
- **Genesis SX files use `defactivity`, `defobject`, `defprojection`, ...** —
|
||||||
|
these aren't real special forms; they're data shapes. The bundler reads the
|
||||||
|
`.sx` file as a list of `(DefineActivity ...)` etc. expressions and serialises
|
||||||
|
the parsed AST to dag-cbor. The genesis bundle CID is a fixed constant
|
||||||
|
baked into the kernel binary (or, for v1, into a sibling `.cidhash` file).
|
||||||
|
- **`replay/3` (Step 3)** is the hot path for projection cold-start. Make
|
||||||
|
sure it streams (read line, fold, repeat) — don't load all activities into
|
||||||
|
memory.
|
||||||
|
- **Validation pipeline stages (Step 6)** halt on first failure. Each stage
|
||||||
|
returns `ok | {error, Reason}`. The pipeline driver is one fold over a list
|
||||||
|
of stage functions. Resist the urge to make it a gen_server.
|
||||||
|
- **Projection sandbox mode (Step 7)** must be **pure**: no IO platform, no
|
||||||
|
clock, no random except CID-seeded. Use `sandbox:eval_pure/2` for fold
|
||||||
|
bodies. This is load-bearing for determinism — every host must reproduce
|
||||||
|
the same projection state from the same log prefix.
|
||||||
|
- **HTTP endpoint shapes (Step 8)** follow design §16.1. Content negotiation
|
||||||
|
on `Accept`: default `application/activity+json`, plus `cbor`/`json`/`sx`.
|
||||||
|
Auth on `POST /activity` is bearer token from env `NEXT_PUBLISH_TOKEN`.
|
||||||
|
|
||||||
|
## Style
|
||||||
|
|
||||||
|
- No comments in `.erl` unless non-obvious. Cite design §-numbers when a
|
||||||
|
decision is non-obvious to a reader.
|
||||||
|
- No new planning docs — update `plans/fed-sx-milestone-1.md` inline. Add a
|
||||||
|
"Progress log" section at the bottom on first iteration.
|
||||||
|
- One Step (or sub-deliverable for the big Steps 5/6/7/8) per iteration.
|
||||||
|
Implement. Test. Gate. Commit. Log. Push. Next.
|
||||||
|
|
||||||
|
Go. Read the plan. Run the restart baseline. Find the first unchecked deliverable
|
||||||
|
in Step 1. Implement it. Remember: no commit without the step's acceptance
|
||||||
|
tests passing AND Erlang conformance 729/729 unaffected.
|
||||||
388
plans/agent-briefings/fed-sx-triggers-loop.md
Normal file
388
plans/agent-briefings/fed-sx-triggers-loop.md
Normal file
@@ -0,0 +1,388 @@
|
|||||||
|
; -*- mode: markdown -*-
|
||||||
|
# loops/fed-sx-triggers — activity-driven flow triggers
|
||||||
|
|
||||||
|
Scoped briefing for a focused iteration loop that wires fed-sx
|
||||||
|
activities to durable business flows on top of `lib/flow/`. Every
|
||||||
|
arriving activity (federation OR local publish) can fan out into
|
||||||
|
one or more registered flows; the flow engine handles multi-step
|
||||||
|
workflows, branches, side effects, retries, and human gates.
|
||||||
|
|
||||||
|
Companion to `plans/fed-sx-design.md` §13 (delivery / projection
|
||||||
|
model) and `plans/flow-on-sx.md` (flow engine surface). This is
|
||||||
|
the build sheet for the substrate side of "an activity fires a
|
||||||
|
business flow."
|
||||||
|
|
||||||
|
```
|
||||||
|
description: loops/fed-sx-triggers — activity → flow dispatch (substrate)
|
||||||
|
subagent_type: general-purpose
|
||||||
|
run_in_background: true
|
||||||
|
isolation: worktree # /root/rose-ash-loops/fed-sx-triggers
|
||||||
|
```
|
||||||
|
|
||||||
|
## The mental model
|
||||||
|
|
||||||
|
```
|
||||||
|
activity arrives ─→ pipeline validates ─→ kernel appends ─→ fan-out
|
||||||
|
│
|
||||||
|
trigger_registry lookup
|
||||||
|
│
|
||||||
|
┌───────────────┬───────────────────┴────────┐
|
||||||
|
↓ ↓ ↓
|
||||||
|
flow:start(F1) flow:start(F2) flow:start(F3)
|
||||||
|
│ │ │
|
||||||
|
(suspend, (sync side (multi-step:
|
||||||
|
resume on effect: payment → ship
|
||||||
|
timer) send email) → notify, with
|
||||||
|
branches)
|
||||||
|
```
|
||||||
|
|
||||||
|
Activities are the unit of "something happened." Flows are the
|
||||||
|
unit of "what we DO about it" — composable, durable, branching.
|
||||||
|
|
||||||
|
## Concrete examples (motivating)
|
||||||
|
|
||||||
|
- **Blog publish** (`Create` for an `Article`): index in search,
|
||||||
|
send digest to followers (per-actor preference dictates batching),
|
||||||
|
push to federation peers, invalidate caches. Branch: if the
|
||||||
|
article references unpublished drafts, hold publication until
|
||||||
|
they land.
|
||||||
|
- **Order placed** (`Create` for an `Order`): charge payment →
|
||||||
|
on fail, cancel + notify; on success, allocate inventory →
|
||||||
|
on out-of-stock, refund + notify; on success, schedule shipping
|
||||||
|
+ send confirmation + start a follow-up flow for "ask for review
|
||||||
|
in 14 days."
|
||||||
|
- **Comment posted**: run moderation pipeline → on fail, hold for
|
||||||
|
human review (suspend until a moderator activity arrives); on
|
||||||
|
pass, notify the post author + bump comment count + check for
|
||||||
|
follow-up triggers.
|
||||||
|
- **Membership renews**: bill the card → on fail, retry 3 days
|
||||||
|
later; on terminal fail, downgrade access + notify. Suspend
|
||||||
|
for 364 days, then fire again.
|
||||||
|
|
||||||
|
All of these are 5-20 step flows with branches, retries,
|
||||||
|
suspensions on external events, and human gates. They are
|
||||||
|
`lib/flow/` territory — m2's `delivery_worker` retry is much
|
||||||
|
simpler (one HTTP call, fixed backoff) and lives in
|
||||||
|
`next/kernel/`, not here.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
**In scope** — `next/**` only. Four pieces:
|
||||||
|
|
||||||
|
1. `DefineTrigger` genesis activity-type + `trigger_registry.erl`
|
||||||
|
2. Trigger fan-out stage in `pipeline.erl` (post-append)
|
||||||
|
3. `flow_dispatch.erl` — bridges activity → `lib/flow/` start call
|
||||||
|
4. End-to-end integration test exercising a multi-step flow with
|
||||||
|
one branch and one suspension
|
||||||
|
|
||||||
|
**In scope to IMPORT from** — `lib/flow/**` is a public dependency.
|
||||||
|
This loop calls `flow:start/2`, `flow:resume/2`, etc. as a
|
||||||
|
consumer; it does not modify `lib/flow/`.
|
||||||
|
|
||||||
|
**Out of scope** — `lib/host/**`, `lib/erlang/**`, `bin/sx_server.ml`,
|
||||||
|
any `lib/<other>/` directory. Touching them would conflict with
|
||||||
|
parallel loops.
|
||||||
|
|
||||||
|
**Parallel-safety with `loops/fed-sx-types`** — that loop also
|
||||||
|
edits `pipeline.erl` (adding `apply_object_schema/2` between
|
||||||
|
activity-type validation and append). This loop adds a fan-out
|
||||||
|
stage AFTER append. Different positions in the pipeline. Resolve
|
||||||
|
by inserting both stages in a single commit OR by branching this
|
||||||
|
loop from fed-sx-types' tip once fed-sx-types completes.
|
||||||
|
**Default: branch from fed-sx-types' tip.** If fed-sx-types is
|
||||||
|
mid-flight, wait for it to land Phase 4 before starting this
|
||||||
|
loop's Phase 2.
|
||||||
|
|
||||||
|
## Branch base
|
||||||
|
|
||||||
|
`origin/loops/fed-sx-types` if it exists and has Phase 4 landed.
|
||||||
|
Else `origin/loops/fed-sx-m2` (has all m2 substrate + send_after
|
||||||
|
+ http-listen + peer_actors pattern). Either base has flow-on-sx
|
||||||
|
available via `lib/flow/**`.
|
||||||
|
|
||||||
|
## Phase 1 — `DefineTrigger` + `trigger_registry.erl`
|
||||||
|
|
||||||
|
### `next/genesis/activity-types/define_trigger.sx`
|
||||||
|
|
||||||
|
Verb declaration. The `:object` payload binds an activity-type
|
||||||
|
(possibly per-actor, possibly with a guard predicate) to a flow
|
||||||
|
name:
|
||||||
|
|
||||||
|
```sx
|
||||||
|
(DefineTrigger
|
||||||
|
:name "Trigger"
|
||||||
|
:doc "Bind an activity-type to a flow. When a matching activity
|
||||||
|
is appended, fan out to flow:start with the activity as
|
||||||
|
input."
|
||||||
|
:schema (fn (act)
|
||||||
|
(and (string? (-> act :object :activity-type))
|
||||||
|
(string? (-> act :object :flow-name))
|
||||||
|
;; guard is optional — a (fn (activity) ...) returning bool
|
||||||
|
(or (nil? (-> act :object :guard))
|
||||||
|
(callable? (-> act :object :guard)))))
|
||||||
|
:semantics (fn (state act)
|
||||||
|
;; Folds into the actor's :triggers projection — registry on
|
||||||
|
;; restart hydrates from this.
|
||||||
|
(trigger-state-add state act)))
|
||||||
|
```
|
||||||
|
|
||||||
|
The `:guard` lets one type bind to multiple flows with
|
||||||
|
discriminators ("only fire `EmailDigest` for Articles with
|
||||||
|
`:category "newsletter"`"). Optional; default true.
|
||||||
|
|
||||||
|
### `next/kernel/trigger_registry.erl`
|
||||||
|
|
||||||
|
In-memory registry, hydrated from the actor's trigger projection
|
||||||
|
on start. Mirrors `peer_actors.erl` / `peer_types.erl` shape:
|
||||||
|
|
||||||
|
```erlang
|
||||||
|
trigger_registry:start_link(_) -> Pid
|
||||||
|
trigger_registry:add(ActivityType, Spec) -> ok
|
||||||
|
trigger_registry:remove(Trigger Cid) -> ok
|
||||||
|
trigger_registry:lookup(ActivityType) -> [Spec]
|
||||||
|
trigger_registry:all_triggers() -> [{ActivityType, [Spec]}]
|
||||||
|
```
|
||||||
|
|
||||||
|
Where `Spec` is `{trigger_cid, flow_name, guard_fn|undefined,
|
||||||
|
actor_scope|any}`. Multiple triggers per activity-type are
|
||||||
|
supported and fire independently.
|
||||||
|
|
||||||
|
### Tests — `next/tests/define_trigger.sh` + `next/tests/trigger_registry.sh`
|
||||||
|
|
||||||
|
- Verb round-trips through term_codec
|
||||||
|
- Schema accepts well-formed, rejects missing fields
|
||||||
|
- Registry: add/lookup/remove, lookup with no matches → [], multi-bind
|
||||||
|
one activity-type → list of specs
|
||||||
|
- Restart hydration from a fold over Trigger activities
|
||||||
|
|
||||||
|
Phase 1 acceptance: 12-14 tests.
|
||||||
|
|
||||||
|
## Phase 2 — Pipeline fan-out stage
|
||||||
|
|
||||||
|
In `next/kernel/pipeline.erl`, add a stage **after the kernel
|
||||||
|
append** (not before — fan-out fires only on successfully accepted
|
||||||
|
activities; rejected activities don't trigger flows):
|
||||||
|
|
||||||
|
```erlang
|
||||||
|
%% Pipeline (revised):
|
||||||
|
%% 1. envelope shape
|
||||||
|
%% 2. signature against peer-AS
|
||||||
|
%% 3. activity-type :schema
|
||||||
|
%% 4. object-schema validation (fed-sx-types Phase 4)
|
||||||
|
%% 5. kernel append
|
||||||
|
%% 6. trigger fan-out (THIS PHASE)
|
||||||
|
|
||||||
|
apply_triggers(Activity, ActorState, Cfg) ->
|
||||||
|
Type = activity_type_atom(Activity),
|
||||||
|
Specs = trigger_registry:lookup(Type),
|
||||||
|
Matching = [S || S <- Specs, guard_passes(S, Activity, ActorState)],
|
||||||
|
[flow_dispatch:start(Spec, Activity, ActorState, Cfg)
|
||||||
|
|| Spec <- Matching],
|
||||||
|
ok.
|
||||||
|
```
|
||||||
|
|
||||||
|
`apply_triggers` is fire-and-forget from the pipeline's POV —
|
||||||
|
the kernel response doesn't wait for flows to complete. Each
|
||||||
|
spawned flow is its own erlang process; failures are isolated
|
||||||
|
to that one flow.
|
||||||
|
|
||||||
|
### Idempotency
|
||||||
|
|
||||||
|
Federation deduplication is a known weak spot: the same activity
|
||||||
|
can arrive twice via different peers. Fan-out must therefore
|
||||||
|
de-duplicate per `{activity-cid, trigger-cid}` pair before
|
||||||
|
calling flow_dispatch. The `:triggers_fired` actor-state field
|
||||||
|
holds `[{activity_cid, trigger_cid}]` of historical fires; before
|
||||||
|
calling flow_dispatch, check membership; on dispatch, add.
|
||||||
|
|
||||||
|
### Tests — `next/tests/pipeline_triggers.sh`
|
||||||
|
|
||||||
|
- Activity append → trigger lookup → flow_dispatch invoked
|
||||||
|
- Activity with no matching trigger → no dispatch
|
||||||
|
- Guard returns false → no dispatch
|
||||||
|
- Same activity appended twice → flow_dispatch invoked once
|
||||||
|
- Failed flow_dispatch (raises) does not block the kernel append
|
||||||
|
- Multiple triggers for same activity-type → each dispatched
|
||||||
|
|
||||||
|
Phase 2 acceptance: 10-12 tests.
|
||||||
|
|
||||||
|
## Phase 3 — `flow_dispatch.erl`
|
||||||
|
|
||||||
|
The bridge from "an activity matched a trigger" to "a flow
|
||||||
|
started with that activity as input." Calls into `lib/flow/`'s
|
||||||
|
public API.
|
||||||
|
|
||||||
|
```erlang
|
||||||
|
%% flow_dispatch:start/4 starts the flow named in Spec with
|
||||||
|
%% the activity bound as :activity in the flow's input env. The
|
||||||
|
%% started flow's id is returned (or {error, _} on failure).
|
||||||
|
|
||||||
|
flow_dispatch:start(Spec, Activity, ActorState, Cfg) ->
|
||||||
|
FlowName = field(flow_name, Spec),
|
||||||
|
Input = [{activity, Activity},
|
||||||
|
{actor, actor_state:actor_id(ActorState)},
|
||||||
|
{trigger_cid, field(trigger_cid, Spec)}],
|
||||||
|
case flow:start(FlowName, Input) of
|
||||||
|
{ok, FlowId} -> {ok, FlowId};
|
||||||
|
{error, Reason} -> log_flow_start_failure(Spec, Activity, Reason),
|
||||||
|
{error, Reason}
|
||||||
|
end.
|
||||||
|
```
|
||||||
|
|
||||||
|
The `Input` proplist becomes the flow's initial environment.
|
||||||
|
Flow steps can read the activity, the actor id, and the trigger
|
||||||
|
cid (for audit chain).
|
||||||
|
|
||||||
|
### Audit chain
|
||||||
|
|
||||||
|
Every flow started from an activity records its `{activity_cid,
|
||||||
|
trigger_cid, flow_id}` triple to the actor's projection. Replaying
|
||||||
|
the actor log + the trigger registry reconstructs which flows
|
||||||
|
fired for what reasons. Same content-addressing discipline as the
|
||||||
|
rest of fed-sx.
|
||||||
|
|
||||||
|
### Tests — `next/tests/flow_dispatch.sh`
|
||||||
|
|
||||||
|
- Successful flow start records audit triple
|
||||||
|
- Flow names that don't resolve in the flow registry → log + return
|
||||||
|
`{error, no_such_flow}`; don't crash
|
||||||
|
- Flow's first step runs synchronously before flow_dispatch returns
|
||||||
|
(proves the activity payload landed in the flow's env)
|
||||||
|
- Multi-step flow: trigger + first step + suspend on
|
||||||
|
`flow:wait_for_message`; resume on synthetic message → second
|
||||||
|
step completes
|
||||||
|
- Branch test: flow has if/else gate on activity field; both
|
||||||
|
branches exercised
|
||||||
|
|
||||||
|
Phase 3 acceptance: 10-14 tests.
|
||||||
|
|
||||||
|
## Phase 4 — End-to-end integration: blog-publish digest
|
||||||
|
|
||||||
|
A motivating, multi-step business flow that ties everything
|
||||||
|
together. Lives at `next/genesis/flows/blog-publish-digest.sx`
|
||||||
|
as a flow definition consumable by `lib/flow/`. Demonstrates:
|
||||||
|
|
||||||
|
- Multi-step (3+ named steps)
|
||||||
|
- Branch on activity field (article :category)
|
||||||
|
- Side effect (mock email sender via `:dispatch_fn` Cfg hook)
|
||||||
|
- Suspension on a timer (e.g. "send digest at 9am the next day")
|
||||||
|
- Completion records a `DigestSent` activity (closing the loop —
|
||||||
|
the flow's own output is itself an activity, which can in turn
|
||||||
|
trigger downstream flows)
|
||||||
|
|
||||||
|
### `next/genesis/flows/blog-publish-digest.sx`
|
||||||
|
|
||||||
|
```sx
|
||||||
|
(defflow blog-publish-digest
|
||||||
|
(step :validate
|
||||||
|
(let ((art (-> $input :activity :object)))
|
||||||
|
(if (= (-> art :type) "Article")
|
||||||
|
(continue :decide-batch art)
|
||||||
|
(done :skipped))))
|
||||||
|
(step :decide-batch
|
||||||
|
(fn (art)
|
||||||
|
(case (-> art :category)
|
||||||
|
:newsletter (continue :batch-until-morning art)
|
||||||
|
:urgent (continue :send-now art)
|
||||||
|
:else (done :skipped))))
|
||||||
|
(step :batch-until-morning
|
||||||
|
(fn (art)
|
||||||
|
(wait-until (next-morning-utc))
|
||||||
|
(continue :send-now art)))
|
||||||
|
(step :send-now
|
||||||
|
(fn (art)
|
||||||
|
(let ((followers (fetch-followers (-> $input :actor))))
|
||||||
|
(for-each follower followers
|
||||||
|
(dispatch-email follower art))
|
||||||
|
(continue :emit-digest-sent art))))
|
||||||
|
(step :emit-digest-sent
|
||||||
|
(fn (art)
|
||||||
|
(publish-activity
|
||||||
|
(Create :object {:type "DigestSent"
|
||||||
|
:for (-> art :id)
|
||||||
|
:follower-count (-> art :follower-count)}))
|
||||||
|
(done :ok))))
|
||||||
|
```
|
||||||
|
|
||||||
|
The exact `defflow` syntax is what `lib/flow/` already provides —
|
||||||
|
mirror its existing test fixtures (`lib/flow/tests/programs/*.sx`)
|
||||||
|
for the canonical surface.
|
||||||
|
|
||||||
|
### Integration test — `next/tests/triggers_e2e.sh`
|
||||||
|
|
||||||
|
End-to-end:
|
||||||
|
1. Bootstrap an actor with a `Trigger` activity binding
|
||||||
|
`Article` → `blog-publish-digest`.
|
||||||
|
2. Mock `dispatch_email` and `fetch_followers` via Cfg hooks.
|
||||||
|
3. Publish a Create-Article activity with `:category :urgent`.
|
||||||
|
4. Assert: flow ran to completion within one append cycle, three
|
||||||
|
emails dispatched, `DigestSent` activity appended.
|
||||||
|
5. Publish a Create-Article with `:category :newsletter`.
|
||||||
|
6. Assert: flow suspended after `:decide-batch`; advance the
|
||||||
|
logical clock 16 hours; flow resumes; emails sent;
|
||||||
|
`DigestSent` appended.
|
||||||
|
7. Publish a Create-Article with `:category :draft`.
|
||||||
|
8. Assert: flow took the `:else` branch, no emails, no
|
||||||
|
`DigestSent`.
|
||||||
|
|
||||||
|
Phase 4 acceptance: 8-10 e2e tests.
|
||||||
|
|
||||||
|
## Out-of-scope deliberately
|
||||||
|
|
||||||
|
- **In-process local subscriber path on delivery_worker.** Right
|
||||||
|
now `delivery_worker` dispatches over HTTP; an "internal sink"
|
||||||
|
abstraction would let activities fan out via the delivery_set
|
||||||
|
to a local consumer. That's a separate, smaller change in
|
||||||
|
`delivery_worker.erl` + `outbox.erl`. Keep it out of this loop
|
||||||
|
to keep the scope tight.
|
||||||
|
- **Per-flow auth / capability gates.** Triggers fire flows with
|
||||||
|
the activity's actor identity. Authorisation of what flows can
|
||||||
|
publish what activities is a separate concern handled by the
|
||||||
|
existing pipeline.
|
||||||
|
- **Flow versioning / hot-reload.** When a flow definition
|
||||||
|
changes, in-flight instances continue with the version they
|
||||||
|
started on. New instances pick up the new version. flow-on-sx
|
||||||
|
handles this; nothing to do here.
|
||||||
|
- **Cross-actor triggers.** A trigger lives on the actor it's
|
||||||
|
defined for. "When ANY actor publishes an Article, fire this
|
||||||
|
flow" is a fan-in pattern that needs a different design.
|
||||||
|
|
||||||
|
## Tests discipline
|
||||||
|
|
||||||
|
- `bash lib/erlang/conformance.sh` green before AND after every
|
||||||
|
commit (substrate not touched, should stay 771/771).
|
||||||
|
- `bash lib/flow/conformance.sh` green before AND after (the flow
|
||||||
|
engine is a dependency we use as-is — if our changes break it,
|
||||||
|
we did something wrong).
|
||||||
|
- Commits scoped to `next/**` and `plans/`. No edits to
|
||||||
|
`lib/flow/**`, `lib/host/**`, `lib/erlang/**`, `bin/sx_server.ml`,
|
||||||
|
or other `lib/<lang>/**`.
|
||||||
|
- One commit per phase as outlined.
|
||||||
|
- Push to `origin/loops/fed-sx-triggers` after each commit.
|
||||||
|
|
||||||
|
## Done when
|
||||||
|
|
||||||
|
- Phases 1-4 land with their acceptance tests green.
|
||||||
|
- The blog-publish-digest e2e test demonstrates a multi-step
|
||||||
|
flow with a branch, a suspension, a side effect, and a
|
||||||
|
follow-up activity emit, all driven by one trigger arriving
|
||||||
|
in the pipeline.
|
||||||
|
- `plans/fed-sx-design.md` updated (small note in §13) describing
|
||||||
|
the trigger fan-out stage as a kernel-level convention.
|
||||||
|
|
||||||
|
When this loop closes, the substrate supports declaring business
|
||||||
|
flows as data, binding them to activity-types as Define\*
|
||||||
|
activities, and seeing them fire automatically whenever the
|
||||||
|
matching activity lands — locally OR via federation. The whole
|
||||||
|
"event-driven CQRS" pattern that the user asked about works
|
||||||
|
end-to-end with `lib/flow/` doing the actual workflow lifting.
|
||||||
|
|
||||||
|
## Parallel safety with other loops
|
||||||
|
|
||||||
|
- `loops/host` — no overlap (next/ only).
|
||||||
|
- `loops/fed-sx-types` — pipeline.erl overlap. Branch base on
|
||||||
|
fed-sx-types' tip OR coordinate via a single combined
|
||||||
|
pipeline-stage insertion commit.
|
||||||
|
- `loops/erlang` — no overlap (lib/erlang/ untouched).
|
||||||
|
- `loops/flow` — no overlap (lib/flow/ consumed as a dependency,
|
||||||
|
never edited).
|
||||||
218
plans/agent-briefings/fed-sx-types-loop.md
Normal file
218
plans/agent-briefings/fed-sx-types-loop.md
Normal file
@@ -0,0 +1,218 @@
|
|||||||
|
; -*- mode: markdown -*-
|
||||||
|
# loops/fed-sx-types — fed-sx-side prep for host-type federation
|
||||||
|
|
||||||
|
Scoped briefing for a focused iteration loop that lands the
|
||||||
|
substrate-side pieces needed for host's typed-post graph to flow
|
||||||
|
across fed-sx nodes. Companion to `plans/fed-sx-host-types.md` (full
|
||||||
|
design + context); this is the build sheet.
|
||||||
|
|
||||||
|
```
|
||||||
|
description: loops/fed-sx-types — federation of typed posts (fed-sx side)
|
||||||
|
subagent_type: general-purpose
|
||||||
|
run_in_background: true
|
||||||
|
isolation: worktree # worktree at /root/rose-ash-loops/fed-sx-types
|
||||||
|
```
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
**In scope** — `next/**` only. Four pieces, in dependency order:
|
||||||
|
|
||||||
|
1. `DefineType` + `SubtypeOf` genesis activity-types
|
||||||
|
2. `peer_types.erl` receiver-side cache (mirror of `peer_actors.erl`)
|
||||||
|
3. `http_server.erl` route for serving type-docs over the wire
|
||||||
|
4. Object-schema validation stage in `pipeline.erl`
|
||||||
|
|
||||||
|
**Out of scope** — `lib/host/**`. The host-side adapters
|
||||||
|
(`lib/host/fed_sx_outbox.sx`, `lib/host/fed_sx_inbox.sx`) are a
|
||||||
|
follow-up that needs `loops/host`'s metamodel to settle first.
|
||||||
|
Touching `lib/host/**` here would conflict with `loops/host`'s
|
||||||
|
in-flight slices. **Hard line: do not edit `lib/host/`.**
|
||||||
|
|
||||||
|
**Substrate** — the fed-sx kernel modules (`nx_kernel.erl`,
|
||||||
|
`http_server.erl`, `pipeline.erl`, `outbox.erl`, etc.) are fine to
|
||||||
|
edit — they're `next/kernel/` scope, this loop owns them.
|
||||||
|
|
||||||
|
## Branch base
|
||||||
|
|
||||||
|
Start from `origin/architecture` (currently has m2 fully merged
|
||||||
|
locally as `ef3d2df4`; once that's pushed it'll be available on
|
||||||
|
origin too). Tip has:
|
||||||
|
|
||||||
|
- All of fed-sx-m2 (multi-actor, signature pipeline, peer_actors,
|
||||||
|
delivery_worker with 8b-timer, two-instance smoke test, etc.)
|
||||||
|
- Native `http-listen` and `http-request` primitives
|
||||||
|
- `erlang:send_after` / `cancel_timer` / `monotonic_time` substrate
|
||||||
|
- The Erlang substrate `:pending-args` scheduler fix that makes
|
||||||
|
kernel-aware routes work over real HTTP
|
||||||
|
|
||||||
|
If `origin/architecture` is still missing m2 at start time (pending
|
||||||
|
push), rebase against `origin/loops/fed-sx-m2` instead.
|
||||||
|
|
||||||
|
## Phase 1 — `DefineType` + `SubtypeOf` genesis activity-types
|
||||||
|
|
||||||
|
New files in `next/genesis/activity-types/`:
|
||||||
|
|
||||||
|
- `define_type.sx` — the verb declaration. Schema accepts
|
||||||
|
`{name, fields, refinement-schema, instance-type}`. Semantics
|
||||||
|
registers the type into a kernel-side type registry.
|
||||||
|
- `subtype_of.sx` — relation verb. `{child-type-cid, parent-type-cid}`.
|
||||||
|
Schema validates both are previously-defined types. Semantics adds
|
||||||
|
the edge to the registry's hierarchy index.
|
||||||
|
|
||||||
|
These are data-shaped (`DefineActivity` form), parsed by
|
||||||
|
`bootstrap.erl` at startup. No kernel code change at this phase —
|
||||||
|
the registry surface needed to consume them lands in Phase 2.
|
||||||
|
|
||||||
|
Sketch:
|
||||||
|
|
||||||
|
```sx
|
||||||
|
(DefineType
|
||||||
|
:name "Type" ;; the verb-name (will be the activity :type)
|
||||||
|
:doc "..."
|
||||||
|
:schema (fn (act)
|
||||||
|
(and (string? (-> act :object :name))
|
||||||
|
(or (nil? (-> act :object :fields))
|
||||||
|
(and (list? (-> act :object :fields))
|
||||||
|
(every? type-field-shape? (-> act :object :fields))))))
|
||||||
|
:semantics (fn (state act) state))
|
||||||
|
```
|
||||||
|
|
||||||
|
Tests in `next/tests/define_type.sh` and `next/tests/subtype_of.sh`
|
||||||
|
— round-trip through `term_codec`, schema accept/reject cases.
|
||||||
|
|
||||||
|
Phase 1 acceptance: 6 tests (3 per verb) + envelope shape round-trip.
|
||||||
|
|
||||||
|
## Phase 2 — `peer_types.erl` receiver-side cache
|
||||||
|
|
||||||
|
Mirror `next/kernel/peer_actors.erl`. State shape:
|
||||||
|
`[{TypeCidBytes, TypeRecord}]` where `TypeRecord` is the parsed
|
||||||
|
`DefineType` envelope's `:object` payload — name, fields,
|
||||||
|
refinement-schema, instance-type-binding.
|
||||||
|
|
||||||
|
Public API (mirrors `peer_actors`):
|
||||||
|
|
||||||
|
```erlang
|
||||||
|
peer_types:start_link(_) -> Pid
|
||||||
|
peer_types:put(TypeCid, TypeRecord) -> ok
|
||||||
|
peer_types:lookup(TypeCid) -> {ok, TR} | not_found
|
||||||
|
peer_types:lookup_or_fetch(TypeCid, Cfg) -> {ok, TR} | {error, _}
|
||||||
|
peer_types:state_for(TypeCid) -> {ok, TR} | not_found
|
||||||
|
peer_types:known_types() -> [TypeCid]
|
||||||
|
```
|
||||||
|
|
||||||
|
The `lookup_or_fetch` path delegates to a `Cfg`-supplied
|
||||||
|
`type_fetch_fn :: fun((TypeCid, Cfg) -> {ok, Bytes} | {error, _})`
|
||||||
|
to keep transport pluggable (same pattern as
|
||||||
|
`discovery_fetch:make_fetch_fn`).
|
||||||
|
|
||||||
|
Tests in `next/tests/peer_types.sh` — put/lookup, miss → fetch via
|
||||||
|
mocked Cfg.fn, cache populated after fetch, lookup_or_fetch with
|
||||||
|
no Cfg.fn → `{error, no_fetch_fn}`.
|
||||||
|
|
||||||
|
Phase 2 acceptance: 8-10 tests covering the cache surface.
|
||||||
|
|
||||||
|
## Phase 3 — `http_server.erl` type-doc route
|
||||||
|
|
||||||
|
Add a route mirroring `actor_doc_response_for/3`:
|
||||||
|
|
||||||
|
```
|
||||||
|
GET /types/<cid> with Accept: application/vnd.fed-sx.type-doc
|
||||||
|
```
|
||||||
|
|
||||||
|
Encodes the cached TypeRecord via `term_codec:encode/1` and serves
|
||||||
|
the bytes. 404 if not in cache. Wire format matches the actor-doc
|
||||||
|
shape (per `plans/fed-sx-design.md` §10).
|
||||||
|
|
||||||
|
Also add a small `discovery_type_fetch.erl` (sibling to
|
||||||
|
`discovery_fetch.erl`) holding the closure that `peer_types`'s
|
||||||
|
`lookup_or_fetch` calls — keeps the live HTTP path testable
|
||||||
|
end-to-end.
|
||||||
|
|
||||||
|
Tests:
|
||||||
|
- `next/tests/peer_types_route.sh` (server-side, port + actor-style
|
||||||
|
curl) — known type-cid → 200 + decoded body, unknown → 404
|
||||||
|
- Extend `next/tests/discovery_type_fetch.sh` to cover the
|
||||||
|
client-side closure round-trip with a python http.server stub
|
||||||
|
|
||||||
|
Phase 3 acceptance: 8 server-side tests + 6 closure tests.
|
||||||
|
|
||||||
|
## Phase 4 — Object-schema validation stage in `pipeline.erl`
|
||||||
|
|
||||||
|
The pipeline currently validates: envelope shape, signature against
|
||||||
|
peer-AS, activity-type schema (the activity's outer `:schema` fn).
|
||||||
|
**It does not validate the inner object against its declared type
|
||||||
|
schema.** This phase plugs that in.
|
||||||
|
|
||||||
|
When an activity arrives whose `{type, _}` is a known refinement-
|
||||||
|
typed activity AND whose `:object` carries a `{type, TypeName}`
|
||||||
|
field, look up the TypeRecord (local Define-name index → TypeCid →
|
||||||
|
`peer_types:lookup_or_fetch`) and apply its refinement-schema to
|
||||||
|
the object's `:field-values`. Reject the activity on schema-fail.
|
||||||
|
|
||||||
|
New `apply_object_schema/2` between activity-type validation and
|
||||||
|
the kernel append. Default behaviour when the type isn't in the
|
||||||
|
local Define-name index: try `peer_types:lookup_or_fetch`; if that
|
||||||
|
errors and Cfg has `{strict_object_schema, true}` reject the
|
||||||
|
activity, otherwise let it through with a `validation_skipped`
|
||||||
|
log entry. Default `strict_object_schema = false` — opt-in for
|
||||||
|
nodes that want airtight validation.
|
||||||
|
|
||||||
|
Tests in `next/tests/object_schema.sh`:
|
||||||
|
- Activity with type that matches local registry + valid object
|
||||||
|
→ accepted, appended
|
||||||
|
- Same activity with refinement-failing object → rejected with
|
||||||
|
`{validation_failed, object_schema}`
|
||||||
|
- Activity with type unknown to local registry, peer_types fetch
|
||||||
|
succeeds → validates against fetched TypeRecord
|
||||||
|
- Activity with type unknown, fetch fails, strict_object_schema
|
||||||
|
not set → accepted with skip log
|
||||||
|
- Activity with type unknown, fetch fails, strict set → rejected
|
||||||
|
- Activity without inner `{type, _}` → skipped (no schema applies)
|
||||||
|
|
||||||
|
Phase 4 acceptance: 10-12 tests covering the matrix.
|
||||||
|
|
||||||
|
## Out-of-scope deliberately
|
||||||
|
|
||||||
|
- **host-side adapters** (`lib/host/fed_sx_outbox.sx`,
|
||||||
|
`lib/host/fed_sx_inbox.sx`). Belongs to whatever loop wires
|
||||||
|
`loops/host` to fed-sx. That loop can read this branch and
|
||||||
|
consume `peer_types`/`DefineType` as public surface.
|
||||||
|
- **Type evolution / version migration**. Refinement schemas are
|
||||||
|
immutable per CID; an updated TypeRecord is a new CID. The
|
||||||
|
"name → currently-valid CID" routing layer is a separate
|
||||||
|
problem.
|
||||||
|
- **Subtype-of unification across nodes**. The graph data lands
|
||||||
|
via `SubtypeOf` activities; rendering / display / dedup is a
|
||||||
|
consumer concern.
|
||||||
|
|
||||||
|
## Tests discipline
|
||||||
|
|
||||||
|
- `bash lib/erlang/conformance.sh` green before AND after every
|
||||||
|
commit (currently 771/771 on `origin/architecture` after the m2
|
||||||
|
merge).
|
||||||
|
- Commits scoped to `next/**` and `plans/fed-sx-host-types.md`
|
||||||
|
(the design plan; update it as you ratify decisions). No edits
|
||||||
|
to `lib/erlang/**`, `lib/host/**`, `bin/sx_server.ml`, or other
|
||||||
|
`lib/<lang>/**`.
|
||||||
|
- One commit per phase as outlined; smaller intermediate commits
|
||||||
|
fine as long as each leaves both gates green.
|
||||||
|
- Push to `origin/loops/fed-sx-types` after each commit.
|
||||||
|
|
||||||
|
## Done when
|
||||||
|
|
||||||
|
- Phases 1-4 land with their acceptance tests green.
|
||||||
|
- `plans/fed-sx-host-types.md` updated to mark steps 1-4 done and
|
||||||
|
describe what the host-side adapter loop now has to work with.
|
||||||
|
- Conformance + every `next/tests/*.sh` green at finish.
|
||||||
|
|
||||||
|
When this loop closes, the substrate side of host-type federation
|
||||||
|
is complete. The remaining `loops/host`-side work (outbox/inbox
|
||||||
|
adapters) becomes a small, well-scoped follow-up.
|
||||||
|
|
||||||
|
## Parallel-safety with loops/host
|
||||||
|
|
||||||
|
`loops/host` is iterating heavily on `lib/host/blog.sx` and the
|
||||||
|
metamodel slices. This loop **does not touch any `lib/host/` file**
|
||||||
|
— scope is `next/**` only. No conflict surface. Both loops run
|
||||||
|
their own worktrees on separate branches; they meet only at the
|
||||||
|
eventual architecture merge.
|
||||||
Reference in New Issue
Block a user