; -*- mode: markdown -*- # loops/host — fed-sx adapter slice (host side of host-type federation) Scoped briefing for the follow-up that wires `loops/host`'s SX/dream front door to the fed-sx kernel substrate landed by `loops/fed-sx-types`. Companion to `plans/fed-sx-host-types.md` (the substrate design + public surface). This is the build sheet for the host-side adapters the substrate loop deliberately deferred. ``` description: loops/host — fed-sx adapter (publish/serve/ingest typed posts) subagent_type: general-purpose run_in_background: true isolation: worktree # worktree at /root/rose-ash-loops/host ``` ## Why this is small now The substrate is done and tested (`origin/loops/fed-sx-types`, 4 phases). And the host already has *most of a type system*: `lib/host/blog.sx` models a **type as a post** with a content-address `:cid`, a `:schema` (`{:required [...]}`), `:fields`, `:template`, and a **`subtype-of` graph over lib/relations**. So this loop is not building a type model — it is **projecting the host's existing one onto fed-sx** and ingesting peers' types back. The pieces line up almost 1:1: | host (lib/host/blog.sx) | fed-sx (next/kernel) | |--------------------------------------------|------------------------------------------| | a type-post + `:schema {:required [...]}` | `DefineType` activity + refinement `{required, [...]}` | | `subtype-of` edge (lib/relations) | `SubtypeOf` activity | | `host/blog-by-cid` / `host/blog-type-defs` | `peer_types` cache + `GET /types/` | | `host/blog-type-issues` (local validate) | `pipeline:apply_object_schema/2` (inbound) | The host's `{:required [...]}` schema maps **directly** onto the term_codec-safe `{required, [Field,...]}` refinement form the substrate already validates — so schema translation is nearly trivial. Derive both validators from the *same* schema data to avoid drift. ## Scope **In scope** — `lib/host/**` only (the mirror of fed-sx-types' `next/**` only). New: `lib/host/fed_sx_outbox.sx`, `lib/host/fed_sx_inbox.sx` (and a small shared `lib/host/fed_sx.sx` bridge + a `host/types-routes`), plus `serve.sh` bring-up wiring and `lib/host/tests/`. **Out of scope** — `next/**`. The substrate is frozen public surface; do **not** edit `next/kernel/**` or the genesis verbs from here. If a gap is found, file it against fed-sx-types, don't patch across the boundary. **Hard line: do not edit `next/`.** ## Branch base Start from `loops/host`, then bring in the substrate (clean, additive — disjoint paths, no conflicts): ```bash cd /root/rose-ash-loops/host git fetch origin git merge origin/loops/fed-sx-types # adds next/kernel + genesis + plan doc ``` After the merge the worktree has both `lib/host/**` (host) and `next/kernel/**` (fed-sx substrate) on one branch. ## Phase 0 — settle the runtime boundary (DECIDE FIRST, blocks all else) `lib/host` is **pure same-runtime SX** (one `sx_server.exe`: stdlib → R7RS → APL → Datalog → ACL → Relations → Feed → Persist → Dream → Host, per `serve.sh`). The fed-sx kernel is **Erlang-on-SX** on the er-scheduler (`erlang-load-module` + gen_servers: `peer_types`, `nx_kernel`). The host's dream handlers run on the native `http-listen` accept loop — **outside** the er-scheduler. Calling a kernel `gen_server:call` synchronously from a native-thread handler hits the known scheduler-context deadlock (see `plans/fed-sx-design.md` §, and the fed-prims http-listen note: a handler on `Thread.create` outside er-sched can't complete a `gen_server:call → receive`). Two architectures; **the loop's first deliverable is choosing one with a tiny spike**: - **Option A — in-process Erlang bridge.** Host's sx_server also `erlang-load-module`s the kernel and calls it directly. Pro: one process, no serialization. Con: the deadlock above — kernel calls must be marshalled onto the er-scheduler or restricted to pure (non-gen_server) functions. Fragile; not recommended. - **Option B — HTTP boundary (RECOMMENDED).** Run the fed-sx kernel with its own `http_server`/`http-listen` loop (it already has the whole route surface, and the m2 two-instance smoke test proves HTTP federation between fed-sx nodes). The host talks to its local fed-sx node **over localhost HTTP using the wire it already speaks** (term_codec / activity+json / type-doc). This is literally what federation is — the host is just another peer to its own node. An `httpc`/localhost call from a native-thread host handler does **not** touch the er-scheduler, so the deadlock never arises; the kernel's own listen handler runs the gen_server calls within er-sched context. Works whether the kernel is a sidecar process or spawned on an er-scheduler process inside the host's sx_server (two ports, one process). Pro: clean, reuses the fully-tested surface, no deadlock. Con: serialization + lifecycle coordination. **Recommendation: Option B.** Spike: host handler → `httpc` POST to the local kernel's `/activity` → 200/cid back, with no hang. Lock the decision before Phase 1. ## Phase 1 — outbox: project host types → DefineType / SubtypeOf `lib/host/fed_sx_outbox.sx`. When a host type-post is created/updated (`host/blog-put!` path), project it and publish to the local fed-sx node: - type-post → `DefineType` activity: `:object` = `{name: slug, fields: (host/blog-fields-of slug), refinement-schema: (host/blog-schema-of slug), instance-type: }`. The host `{:required [...]}` becomes the substrate `{required, [...]}` form verbatim. - each `subtype-of` edge (`relations/parents` over `"subtype-of"`) → a `SubtypeOf` activity `{child-type-cid, parent-type-cid}`. - publish via Phase 0's transport (POST `/activity` to the local node, authed with the node's publish token). **Key open decision — the type CID.** The host computes `:cid` via `host/blog--cid-of` (double-hash over the canonical record); fed-sx keys `peer_types` by a `TypeCid`. Either: (a) **adopt the host `:cid` as the fed-sx TypeCid** — one identity, no reconciliation, but peers can't content-verify it from the wire bytes; or (b) **let the kernel content-address the TypeRecord** — verifiable, but the host must keep a `slug → fed-sx-cid` map (and `SubtypeOf` edges must reference fed-sx CIDs, not host CIDs). Pick one and document it in `plans/fed-sx-host-types.md`. (a) is simpler and probably right for v1; revisit when cross-node verification matters. ## Phase 2 — inbox: ingest peers' types + validate typed objects `lib/host/fed_sx_inbox.sx`. Inbound from the local node's inbox: - inbound `DefineType` → `peer_types:put` (cache it). Decide whether to also **materialize** it as a host post (`host/blog-put!` + `host/blog--set-schema!`) or keep federation-only types out of the local blog (recommended for v1: cache-only, materialize on demand). - inbound `SubtypeOf` → record the edge (peer_types hierarchy and/or `host/blog-relate! child parent "subtype-of"` if materialized). - inbound typed `Create` (a post that `is-a` some refinement type) → the kernel inbound pipeline runs `pipeline:apply_object_schema/2` (configured with a `type_index` + `{peer_types, peer_types}` + `type_fetch_fn`), so a typed object is validated against its declared type **before** the host sees it. Choose `strict_object_schema` per-node (default false = open-world). **Avoid double-validation drift:** the host already has `host/blog-type-issues`. Let the **kernel validate federation inbound** and the **host validate local writes**, both deriving from the same `{:required [...]}` schema data — don't fork the rules. ## Phase 3 — serve + bring-up wiring - **Serve `GET /types/`** on the host front door. Either proxy to the kernel's `/types/` (Option B keeps one source of truth), or serve directly from `host/blog-by-cid` + the projected TypeRecord. Hook as `(dream-get "/types/:cid" host/types-by-cid)` and add `host/types-routes` to the `host/serve` list (per `router.sx` / `serve.sh` pattern). - **Bring-up** in `serve.sh`: start the fed-sx node (Phase 0 transport), start `peer_types`, configure `type_fetch_fn = discovery_type_fetch:make_fetch_fn()` + a `type_url` resolver, and on startup project existing host types (Phase 1) so the node is type-aware from boot. Gate writes behind the existing `host/require-auth` / `host/require-permission` middleware, same as the relations write routes. ## Phase 4 — end-to-end round-trip test Two nodes (host A + host B, or host + a sidecar fed-sx node): A defines a refinement type → B fetches the type-doc via `GET /types/` → B ingests an inbound typed object and `apply_object_schema` accepts the valid one / rejects a refinement-failing one. Mirror the m2 two-instance smoke test style. Plus per-phase suites in `lib/host/tests/` (the host runs its own `conformance.sh`). ## Tests discipline - The host's `lib/host/conformance.sh` green before AND after every commit. `lib/host` is **LIVE at blog.rose-ash.com** — pushing `loops/host` reloads dev, so treat pushes as deliberate. - Commits scoped to `lib/host/**` (+ `plans/fed-sx-host-types.md` as decisions ratify). Do **not** edit `next/**`. - One commit per phase; smaller intermediate commits fine if each leaves the gate green. The Phase-0 spike can be its own commit. ## Done when - A host type round-trips: defined locally → published as `DefineType`/`SubtypeOf` → fetchable at `GET /types/` → a peer validates an inbound typed object against it. - `peer_types` is populated from inbound `DefineType`, and the inbound pipeline rejects refinement-failing typed objects (strict node). - The runtime-boundary decision and the type-CID decision are recorded in `plans/fed-sx-host-types.md`. - Host `conformance.sh` + the new fed-sx adapter suites green. ## Parallel-safety with loops/fed-sx-types That loop owns `next/**` and is feature-complete; this loop owns `lib/host/**`. Disjoint surfaces — they meet only at the merge that brings the substrate in (Branch base, above). If this loop needs a substrate change, file it against fed-sx-types rather than editing `next/` here.