coop/rose-ash
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
afb9ce5e909ed9da0c55bb65849e0e2ba049572a
rose-ash/lib/host/serve.sh
giles afb9ce5e90 TA-live: real A→B federation over HTTP + a durable outbox (LIVE-VERIFIED) 
Step 3 — federation, live-verified with TWO real host instances.

- host/ta.sx: host/ta--post/make-http-wire/federate (POST a serialized activity to a peer's /inbox
  over real HTTP). host/blog.sx: POST /inbox (host/blog-inbox → receive! → process locally, does NOT
  re-federate — no loops).
- DURABLE OUTBOX (fed-sx reliability, after the user asked 'if B is down does it still work?'):
  emit! processes locally (always succeeds), QUEUES per-peer to a persisted outbox, delivers
  best-effort. A peer being DOWN no longer fails the publish — delivery is GUARDED (SX guard catches
  the http-request connection error), failed items stay queued and retry on next emit / on boot /
  manual /flows?flush=1. /flows shows the outbox depth.
- serve.sh: SX_PEERS → peers; boot load+flush of the outbox. docker-compose: a 2nd host sx_host_b
  (peer B, own store, no peers).

LIVE PROOF: (1) a peer POSTs create/article to blog.rose-ash.com/inbox → A fires validate+notify.
(2) publish on A → federates to B → B fires ITS behaviors on A's activity (B's /flows + /activities).
(3) RESILIENCE: publish with B DOWN → A returns 303 (was 500) + queues; start B + flush → B receives
the backlog + fires. blog 218/218 (+TA receive test), full host conformance green.

A = blog.rose-ash.com (public/Caddy); B = sx_host_b (internal docker DNS only, no public domain).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 19:47:07 +00:00

283 lines
13 KiB
Bash
Executable File
Raw Blame History

#!/usr/bin/env bash
# host-on-sx live server launcher. Loads the kernel stdlib, the subsystem
# libraries, and the host modules into one sx_server process, then calls
# (host/serve PORT ...) which binds the native http-listen server to the
# Dream-shaped host app. Runs in the FOREGROUND (http-listen blocks), so this
# doubles as a container entrypoint and a local launcher.
#
# Usage:
# bash lib/host/serve.sh # serve on $HOST_PORT (default 8910)
# HOST_PORT=8920 bash lib/host/serve.sh # pick a port
#
# The module list is kept identical to lib/host/conformance.sh so what serves is
# exactly what the suites verify.
set -uo pipefail
# Project root: SX_PROJECT_DIR in containers (set to /app by the compose stack),
# else the git toplevel for local runs.
cd "${SX_PROJECT_DIR:-$(git rev-parse --show-toplevel 2>/dev/null || echo .)}"
SX_SERVER="${SX_SERVER:-hosts/ocaml/_build/default/bin/sx_server.exe}"
if [ ! -x "$SX_SERVER" ]; then
SX_SERVER="/root/rose-ash/hosts/ocaml/_build/default/bin/sx_server.exe"
fi
if [ ! -x "$SX_SERVER" ]; then
echo "ERROR: sx_server.exe not found." >&2
exit 1
fi
PORT="${HOST_PORT:-8910}"
# Self-warm the SERVING JIT. The FIRST HTTP request after a restart pays a one-time
# ~2.5s cost: the shared request path (session middleware + router + instrumented
# handlers + blog render) JIT-compiles under the http-listen resolver, a context a
# boot-time render can't reach. So once the server is answering, we GET the hot
# pages ONCE over real HTTP to absorb that compile before a visitor does (~2.5s ->
# ~78ms warm). The container image has no curl/wget, so we speak HTTP over bash's
# /dev/tcp. Runs detached (survives the exec below); exits after warming.
_warm_serving_jit() {
local path
for _ in $(seq 1 180); do
case "$( { exec 3<>/dev/tcp/127.0.0.1/"$PORT" && printf 'GET /health HTTP/1.0\r\nHost: x\r\n\r\n' >&3 && head -1 <&3; exec 3>&- 3<&-; } 2>/dev/null )" in
*200*) break ;;
esac
sleep 0.5
done
for path in / /welcome /nt-live-encore /otel; do
{ exec 3<>/dev/tcp/127.0.0.1/"$PORT" && printf 'GET %s HTTP/1.0\r\nHost: x\r\n\r\n' "$path" >&3 && cat <&3 >/dev/null; exec 3>&- 3<&-; } 2>/dev/null
done
}
_warm_serving_jit &
# Modules: every load line from conformance.sh's MODULES list, minus the ledger
# (not needed to serve). server.sx supplies host/serve.
MODULES=(
"spec/stdlib.sx"
"lib/r7rs.sx"
"lib/apl/runtime.sx"
"lib/datalog/tokenizer.sx"
"lib/datalog/parser.sx"
"lib/datalog/unify.sx"
"lib/datalog/db.sx"
"lib/datalog/builtins.sx"
"lib/datalog/aggregates.sx"
"lib/datalog/strata.sx"
"lib/datalog/eval.sx"
"lib/datalog/api.sx"
"lib/datalog/magic.sx"
"lib/acl/schema.sx"
"lib/acl/facts.sx"
"lib/acl/engine.sx"
"lib/acl/explain.sx"
"lib/acl/audit.sx"
"lib/acl/federation.sx"
"lib/acl/api.sx"
"lib/relations/schema.sx"
"lib/relations/engine.sx"
"lib/relations/api.sx"
"lib/relations/explain.sx"
"lib/relations/federation.sx"
"lib/relations/tree.sx"
"lib/feed/normalize.sx"
"lib/feed/stream.sx"
"lib/feed/api.sx"
"lib/persist/event.sx"
"lib/persist/backend.sx"
"lib/persist/log.sx"
"lib/persist/kv.sx"
"lib/persist/api.sx"
"lib/persist/durable.sx"
"spec/render.sx"
"web/adapter-html.sx"
"lib/dream/types.sx"
"lib/dream/json.sx"
"lib/dream/auth.sx"
"lib/dream/error.sx"
"lib/dream/form.sx"
"lib/dream/session.sx"
"lib/dream/router.sx"
"lib/host/handler.sx"
"lib/host/middleware.sx"
"lib/host/session.sx"
"lib/host/auth.sx"
"lib/host/sxtp.sx"
"lib/host/router.sx"
"lib/host/static.sx"
"lib/host/sx/relate-picker.sx"
"lib/host/sx/kg-cards.sx"
"lib/host/feed.sx"
"lib/host/relations.sx"
"lib/host/compose.sx"
"lib/host/execute.sx"
"lib/host/behavior.sx"
"lib/host/flows.sx"
"lib/host/ra.sx"
"lib/host/ta.sx"
"lib/host/htmlsx.sx"
"lib/host/blog.sx"
"lib/host/server.sx"
"lib/host/otel.sx"
)
# Admin login credentials + session signing secret. Override via the container
# env; the in-source defaults are dev-only. The blog write routes are now GUARDED
# (session login or Bearer), so these gate publishing on blog.rose-ash.com.
ADMIN_USER="${SX_ADMIN_USER:-admin}"
ADMIN_PASS="${SX_ADMIN_PASSWORD:-letmein}"
SESSION_SECRET="${SX_SESSION_SECRET:-rose-ash-host-dev-secret-change-me}"
EPOCH=1
{
for M in "${MODULES[@]}"; do
echo "(epoch $EPOCH)"; echo "(load \"$M\")"; EPOCH=$((EPOCH+1))
done
# 100% serving JIT — NO host exclude. The serving-JIT perform-in-HO-callback
# miscompile (map/rest/drop wrong args → blank pages, empty picker) is fixed by
# two composing pieces: sx-vm-extensions 81177d0e resolves a callback's IO
# inline (instead of unwinding the native HO loop) WHEN a synchronous resolver
# is installed, and sx_server.ml's http-listen now installs that resolver (it
# mirrors cek_run_with_io exactly). So the whole request path — host app +
# Dream + Datalog — runs under JIT with no exclude. Verified: ephemeral durable
# server, 100% JIT, zero fallbacks, real content, picker lists candidates.
# Point the blog at the DURABLE file backend (persists under $SX_PERSIST_DIR),
# then idempotently seed a welcome post (sx_content = SX element markup, the
# editor's content model). Re-seeding is a no-op if the slug already exists.
echo "(epoch $EPOCH)"
echo "(eval \"(host/blog-use-store! (persist/durable-backend))\")"
EPOCH=$((EPOCH+1))
# Rebuild the relations graph from the durable edge store. lib/relations holds
# the graph in memory only, so without this, related/tags/types vanish on every
# restart even though the posts persist.
echo "(epoch $EPOCH)"
echo "(eval \"(host/blog-load-edges!)\")"
EPOCH=$((EPOCH+1))
# P0.3b: rebuild the in-memory publish flow log from the durable store, so /flows
# survives a restart (the driver persists each effect record under "flowlog").
echo "(epoch $EPOCH)"
echo "(eval \"(host/blog-load-flowlog!)\")"
EPOCH=$((EPOCH+1))
# P2: rebuild the activity log (the event source) from the durable store, so /activities survives.
echo "(epoch $EPOCH)"
echo "(eval \"(host/blog-load-activitylog!)\")"
EPOCH=$((EPOCH+1))
# Sessions on the DURABLE store, LAZILY: only a logged-in session (one that
# writes a field) persists, so a login survives a restart while anonymous /
# crawler traffic leaves no rows. host/session-init! bumps the per-boot epoch
# that keeps sids unique across restarts. Then the signing secret + admin
# credentials, and grant admin "edit" on "blog" so a logged-in session passes
# the ACL gate on the write routes.
echo "(epoch $EPOCH)"
echo "(eval \"(host/session-use-store! (persist/durable-backend))\")"
EPOCH=$((EPOCH+1))
echo "(epoch $EPOCH)"
echo "(eval \"(host/session-init!)\")"
EPOCH=$((EPOCH+1))
echo "(epoch $EPOCH)"
echo "(eval \"(host/session-set-secret! \\\"$SESSION_SECRET\\\")\")"
EPOCH=$((EPOCH+1))
echo "(epoch $EPOCH)"
echo "(eval \"(host/auth-set-admin! \\\"$ADMIN_USER\\\" \\\"$ADMIN_PASS\\\")\")"
EPOCH=$((EPOCH+1))
echo "(epoch $EPOCH)"
echo "(eval \"(acl/load! (list (acl-grant \\\"$ADMIN_USER\\\" \\\"edit\\\" \\\"blog\\\")))\")"
EPOCH=$((EPOCH+1))
# Idempotently seed a welcome post (sx_content = SX element markup, the editor's
# content model). Re-seeding is a no-op if the slug already exists.
echo "(epoch $EPOCH)"
echo "(eval \"(host/blog-seed! \\\"welcome\\\" \\\"Welcome to the SX host\\\" \\\"(article (h1 \\\\\\\"Welcome to the SX host\\\\\\\") (p \\\\\\\"Rendered by lib/host via render-to-html, from the durable SX store.\\\\\\\"))\\\" \\\"published\\\")\")"
EPOCH=$((EPOCH+1))
# Seed the root type-posts (type, tag) — types ARE posts. Idempotent.
echo "(epoch $EPOCH)"
echo "(eval \"(host/blog-seed-types!)\")"
EPOCH=$((EPOCH+1))
# RA-LIVE: point at the durable-execution KERNEL, add it to the runner fleet, register the durable
# "blog-digest" DAG (needs {suspend} → routes to the kernel), and declare a DURABLE behavior on the
# article: an UPDATE (a published edit) runs the kernel flow (newsletter suspends until resumed,
# urgent completes now). create→publish stays SYNCHRONOUS (exec-fold). Then reload the registry +
# the pending log. host/blog--kernel-base is a mutable define set here.
echo "(epoch $EPOCH)"
echo "(eval \"(set! host/blog--kernel-base \\\"http://sx_kernel:8930\\\")\")"
EPOCH=$((EPOCH+1))
echo "(epoch $EPOCH)"
echo "(eval \"(host/blog--add-runner! (host/ra--make-kernel-runner host/blog--kernel-base))\")"
EPOCH=$((EPOCH+1))
echo "(epoch $EPOCH)"
echo "(eval \"(host/blog--register-dag! \\\"blog-digest\\\" {:erl-flow \\\"blog_digest\\\" :needs (list \\\"effect\\\" \\\"branch\\\" \\\"suspend\\\")})\")"
EPOCH=$((EPOCH+1))
echo "(epoch $EPOCH)"
echo "(eval \"(host/blog--set-type-behavior! \\\"article\\\" (list {\\\"verb\\\" \\\"create\\\" \\\"type\\\" \\\"article\\\" \\\"dag\\\" \\\"publish\\\"} {\\\"verb\\\" \\\"update\\\" \\\"type\\\" \\\"article\\\" \\\"dag\\\" \\\"blog-digest\\\"}))\")"
EPOCH=$((EPOCH+1))
# Give the article type a "category" field so the edit form can set newsletter/urgent — the
# branch the durable kernel flow keys on (newsletter suspends until resumed).
echo "(epoch $EPOCH)"
echo "(eval \"(host/blog--set-fields! \\\"article\\\" (list {:name \\\"category\\\" :type \\\"text\\\"}))\")"
EPOCH=$((EPOCH+1))
# P1: gather the types' declared :behavior bindings into the registry the trigger match
# consults (so publishing an article fires its declared on-publish DAG, runner derived).
echo "(epoch $EPOCH)"
echo "(eval \"(host/blog--load-behaviors!)\")"
EPOCH=$((EPOCH+1))
echo "(epoch $EPOCH)"
echo "(eval \"(host/blog-load-pendinglog!)\")"
EPOCH=$((EPOCH+1))
# TA-live: federate emitted activities to peer /inbox endpoints (comma-separated SX_PEERS, e.g.
# "http://sx_host_b:8000"). Empty by default (no federation). A peer that receives does NOT
# re-federate, so an acyclic peer graph doesn't loop.
PEERS_SX="(list"
IFS=',' read -ra _PEER_ARR <<< "${SX_PEERS:-}"
for _p in "${_PEER_ARR[@]:-}"; do [ -n "$_p" ] && PEERS_SX="$PEERS_SX \\\"$_p\\\""; done
PEERS_SX="$PEERS_SX)"
echo "(epoch $EPOCH)"
echo "(eval \"(host/blog--set-peers! $PEERS_SX)\")"
EPOCH=$((EPOCH+1))
# TA-live: rebuild the durable outbox + RETRY any deliveries that were pending from before a
# restart (a peer that was down gets its backlog once it + we are back up).
echo "(epoch $EPOCH)"
echo "(eval \"(host/blog-load-outbox!)\")"
EPOCH=$((EPOCH+1))
echo "(epoch $EPOCH)"
echo "(eval \"(host/blog--flush-outbox!)\")"
EPOCH=$((EPOCH+1))
# Seed a live demo of the composition fold (plans/composition-objects.md): /compose-demo
# is one composition object rendered by host/comp-render — renders differently by context.
echo "(epoch $EPOCH)"
echo "(eval \"(host/blog-seed-compose-demo!)\")"
EPOCH=$((EPOCH+1))
# Seed the EXECUTE-fold demo (composition step 7): /workflow-demo runs ONE composition
# object through host/exec-run — the same algebra as render, folded to an effect log.
echo "(epoch $EPOCH)"
echo "(eval \"(host/blog-seed-workflow-demo!)\")"
EPOCH=$((EPOCH+1))
# Seed a REAL imported blog post (rose-ash.com/nt-live-encore) decomposed into the :body
# composition — so the import survives store wipes, reseeded on boot like the demos.
echo "(epoch $EPOCH)"
echo "(eval \"(host/blog-seed-nt-live-encore!)\")"
EPOCH=$((EPOCH+1))
# Seed the layer-2 demo: a Landing type with TWO composition fields (:body + :aside) + a
# populated instance — so the two-field composition editor + render show side by side.
echo "(epoch $EPOCH)"
echo "(eval \"(host/blog-seed-landing-demo!)\")"
EPOCH=$((EPOCH+1))
# Load relation metadata (symmetry/labels) from the relation-posts into the
# in-memory cache, so render paths read it without a (VmSuspending) durable read.
echo "(epoch $EPOCH)"
echo "(eval \"(host/blog-load-rel-kinds!)\")"
EPOCH=$((EPOCH+1))
# Index the web-stack .sxbc by content hash so /sx/h/{hash} can serve them
# immutably and the shell can emit the data-sx-manifest (content-addressed
# client module cache). Done once at boot.
echo "(epoch $EPOCH)"
echo "(eval \"(host/static-build-sxh-index!)\")"
EPOCH=$((EPOCH+1))
echo "(epoch $EPOCH)"
# Anonymous reads (feed timeline + relations container reads + blog post detail)
# plus the GUARDED blog write routes: POST /new (editor form ingest), POST/PUT/
# DELETE /posts behind host/require-user (session login OR Bearer) + ACL. make-app
# auto-mounts /login + /logout and wraps everything in the signed-session
# middleware, so a browser logs in then publishes. The bearer resolver is a stub
# (no API tokens configured) — browser session is the live auth path for now.
# blog-routes LAST — its GET /:slug catch-all must not shadow the rest. otel/routes must
# come BEFORE blog-routes, or GET /otel is swallowed by /:slug (only /otel/stream survives).
echo "(eval \"(host/serve $PORT (list host/static-routes host/feed-routes host/relations-routes otel/routes (host/blog-write-routes (fn (tok) nil)) host/blog-routes))\")"
} | exec "$SX_SERVER"
Reference in New Issue View Git Blame Copy Permalink