fed-sx-m2: Step 8e — httpc:request/4 BIF wrapper (+ 10 tests)

Closes the BIF half of Step 8. Native http-request primitive landed
in architecture via the fed-prims merge (the m2 plan's Blocker #2),
so the briefing-allowed-exception wrapper in lib/erlang/runtime.sx
can finally be wired.

Marshalling at the BIF boundary:
  Url     : Erlang binary -> SX string (byte-list -> integer->char).
  Method  : Erlang atom upcased ('get -> "GET") for HTTP-wire
            convention, or Erlang binary passes through verbatim.
  Headers : Erlang proplist -> SX dict via er-proplist-to-dict.
  Body    : Erlang binary -> SX string.

Result {:status :headers :body} marshalled back to Erlang
  {ok, Status::integer,
       Headers::proplist (binary-keyed via er-of-sx-deep),
       Body::binary (char->integer over the SX string)}.

Bad arg shapes (non-binary URL or body) raise error:badarg; native
DNS / connect / bad-URL failures surface as Erlang error markers
that the caller can catch.

Test: next/tests/httpc_request.sh 10/10
  - registration under httpc/request/4
  - BIF marked non-pure
  - wrong-arity (/1) absent from registry
  - badarg on non-binary URL
  - badarg on non-binary body
  - live GET against `python3 -m http.server` -> Status 200
  - body bytes match "hello from python\n"
  - headers come back as proplist (is_list/1 = true)
  - 404 path -> {ok, 404, ...} (not an error tuple)
  - method passed as binary works

URLs spelled out as byte-list <<104,116,116,p,...>> binaries since
the parser truncates <<"..."> string-literal binaries (same
workaround backfill_drain.sh uses for inbox paths).

Plan: 8e ticked; Blocker #2 marked RESOLVED with the merge that
unblocked it referenced. Step 8f (live HTTP dispatch through
delivery_worker) and Step 10c (peer-actor doc fetch) are now
unblocked.

No-regression gates green: Erlang conformance 761/761,
http_multi_actor 44/44, follower_graph 18/18, follow_lifecycle 9/9,
backfill 20/20, backfill_drain 6/6, http_listen_bif 5/5.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

lib/erlang/runtime.sx

@@ -1593,6 +1593,63 @@
           ((sx-handler (fn (req-dict) (er-http-resp-to-sx (er-apply-fun handler (list (er-http-req-of-sx req-dict)))))))
           (http-listen port sx-handler))))))
 
+;; httpc:request/4(Url, Method, Headers, Body) - BRIEFING-EXCEPTION:
+;; the m2 briefing's one allowed scope exception for Step 8e, mirroring
+;; M1 Step 8a's http:listen wrapper on the client side.
+;;
+;; Url is an Erlang binary (must start with http://).
+;; Method is an Erlang atom or binary; passed through to the native
+;; verbatim, so callers should supply 'get / 'post or <<"GET">> as
+;; appropriate (the native compares uppercase).
+;; Headers is an Erlang proplist [{Name, Value}, ...]; names and
+;; values are binaries or atoms (er-proplist-to-dict handles both).
+;; Body is an Erlang binary (use <<>> for empty).
+;;
+;; Returns a 4-tuple {ok, StatusInt, HeadersProplist, BodyBinary}.
+;; The native primitive raises Eval_error on DNS / connect / bad URL;
+;; that surfaces as an Erlang error marker the caller can catch via
+;; Erlang try/catch.
+(define
+  er-bif-httpc-request
+  (fn
+    (vs)
+    (let
+      ((url     (nth vs 0))
+       (method  (nth vs 1))
+       (headers (nth vs 2))
+       (body    (nth vs 3)))
+      (let
+        ((url-str
+           (cond
+             (er-binary? url) (list->string (map integer->char (get url :bytes)))
+             :else (raise (er-mk-error-marker (er-mk-atom "badarg")))))
+         (method-str
+           (cond
+             ;; Erlang convention is lowercase atoms (get/post/put/...);
+             ;; the HTTP wire wants uppercase. Binaries pass through so
+             ;; callers can override with mixed-case verbs if needed.
+             (er-atom? method)   (upcase (get method :name))
+             (er-binary? method) (list->string (map integer->char (get method :bytes)))
+             :else (raise (er-mk-error-marker (er-mk-atom "badarg")))))
+         (headers-dict
+           (cond
+             (er-nil? headers)  (dict)
+             (er-cons? headers) (er-proplist-to-dict headers)
+             :else (raise (er-mk-error-marker (er-mk-atom "badarg")))))
+         (body-str
+           (cond
+             (er-binary? body) (list->string (map integer->char (get body :bytes)))
+             (er-nil? body)    ""
+             :else (raise (er-mk-error-marker (er-mk-atom "badarg"))))))
+        (let
+          ((resp (http-request method-str url-str headers-dict body-str)))
+          (er-mk-tuple
+            (list
+              (er-mk-atom "ok")
+              (get resp :status)
+              (er-of-sx-deep (get resp :headers))
+              (er-mk-binary (map char->integer (string->list (get resp :body)))))))))))
+
 ;; Register everything at load time.
 (define
   er-register-builtin-bifs!
@@ -1796,5 +1853,6 @@
     (er-mk-atom "ok")))
 
 (er-register-bif! "http" "listen" 2 er-bif-http-listen)
+(er-register-bif! "httpc" "request" 4 er-bif-httpc-request)
 
 (er-register-builtin-bifs!)

next/tests/httpc_request.sh

@@ -0,0 +1,153 @@
+#!/usr/bin/env bash
+# next/tests/httpc_request.sh — m2 Step 8e acceptance test.
+#
+# Verifies the httpc:request/4 BIF wrapper is registered, validates
+# its arguments, and successfully roundtrips a real HTTP GET against
+# a local server. Mirrors http_listen_bif.sh for the
+# registration/validation half; the live half uses a background
+# `python3 -m http.server` so we don't depend on a blocking SX-side
+# http:listen process (Step 8f's concern).
+#
+# This BIF is the briefing's allowed-exception scope addition to
+# lib/erlang/runtime.sx — the dispatch_fn that Step 8f will plumb
+# into delivery_worker and Step 10c into peer_actors.
+
+set -uo pipefail
+cd "$(git rev-parse --show-toplevel)"
+
+SX_SERVER="${SX_SERVER:-hosts/ocaml/_build/default/bin/sx_server.exe}"
+if [ ! -x "$SX_SERVER" ]; then
+  SX_SERVER="/root/rose-ash/hosts/ocaml/_build/default/bin/sx_server.exe"
+fi
+if [ ! -x "$SX_SERVER" ]; then
+  echo "ERROR: sx_server.exe not found." >&2
+  exit 1
+fi
+
+VERBOSE="${1:-}"
+PASS=0; FAIL=0; ERRORS=""
+
+# ── live server (Python's stdlib, no extra deps) ─────────────
+PORT=$(python3 -c 'import socket;s=socket.socket();s.bind(("127.0.0.1",0));print(s.getsockname()[1]);s.close()')
+SRVROOT=$(mktemp -d)
+echo "hello from python" > "$SRVROOT/hello.txt"
+( cd "$SRVROOT" && python3 -m http.server "$PORT" >/dev/null 2>&1 ) &
+SRV_PID=$!
+TMPFILE=$(mktemp)
+trap "rm -rf $SRVROOT $TMPFILE; kill $SRV_PID 2>/dev/null || true" EXIT
+# wait for it to come up (up to ~3s)
+for _ in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15; do
+  if curl -fsS "http://127.0.0.1:$PORT/hello.txt" >/dev/null 2>&1; then
+    break
+  fi
+  sleep 0.2
+done
+
+# Spell URLs as Erlang byte-list binaries — <<"...">> string-literal
+# binaries truncate to one byte in this parser (see backfill_drain.sh
+# for the same workaround on inbox paths).
+bytes_of() { python3 -c "import sys; print(','.join(str(b) for b in sys.argv[1].encode()))" "$1"; }
+URL_HELLO_BYTES=$(bytes_of "http://127.0.0.1:$PORT/hello.txt")
+URL_404_BYTES=$(bytes_of "http://127.0.0.1:$PORT/not_there.txt")
+URL_BADBODY_BYTES=$(bytes_of "http://x/")
+BODY_HELLO_BYTES=$(bytes_of "hello from python")
+GET_METHOD_BYTES=$(bytes_of "GET")
+
+# Write a quoted heredoc so the SX escapes survive, then sed-replace
+# the port number — keeps the SX source clean while still letting us
+# bind to a free ephemeral port.
+cat > "$TMPFILE" <<'EPOCHS'
+(epoch 1)
+(load "lib/erlang/tokenizer.sx")
+(load "lib/erlang/parser.sx")
+(load "lib/erlang/parser-core.sx")
+(load "lib/erlang/parser-expr.sx")
+(load "lib/erlang/parser-module.sx")
+(load "lib/erlang/transpile.sx")
+(load "lib/erlang/runtime.sx")
+(load "lib/erlang/vm/dispatcher.sx")
+
+;; BIF registered under httpc/request/4
+(epoch 10)
+(eval "(not (= (er-lookup-bif \"httpc\" \"request\" 4) nil))")
+
+;; BIF marked non-pure (network side effect)
+(epoch 11)
+(eval "(get (er-lookup-bif \"httpc\" \"request\" 4) :pure?)")
+
+;; Wrong arity not registered (httpc/request/1 should be nil)
+(epoch 12)
+(eval "(= (er-lookup-bif \"httpc\" \"request\" 1) nil)")
+
+;; Non-binary URL -> badarg
+(epoch 13)
+(eval "(get (erlang-eval-ast \"try httpc:request(not_a_binary, get, [], <<>>) catch error:badarg -> ok end\") :name)")
+
+;; Non-binary body -> badarg
+(epoch 14)
+(eval "(get (erlang-eval-ast \"try httpc:request(<<__URL_BAD__>>, get, [], not_a_binary) catch error:badarg -> ok end\") :name)")
+
+;; ── Live roundtrip: GET against python http.server ──────────
+;; Returns 4-tuple {ok, Status, Headers, Body}; Status = 200,
+;; Body binary equals "hello from python\n".
+(epoch 20)
+(eval "(get (erlang-eval-ast \"{ok, Status, _H, _B} = httpc:request(<<__URL_HELLO__>>, get, [], <<>>), case Status of 200 -> true; _ -> false end\") :name)")
+
+(epoch 21)
+(eval "(get (erlang-eval-ast \"{ok, _S, _H, Body} = httpc:request(<<__URL_HELLO__>>, get, [], <<>>), case Body of <<__BODY_HELLO__,10>> -> true; _ -> false end\") :name)")
+
+;; Headers come back as Erlang proplist (i.e. a cons)
+(epoch 22)
+(eval "(get (erlang-eval-ast \"{ok, _S, Headers, _B} = httpc:request(<<__URL_HELLO__>>, get, [], <<>>), is_list(Headers)\") :name)")
+
+;; 404 for unknown path -> Status 404 (not an error tuple)
+(epoch 23)
+(eval "(get (erlang-eval-ast \"{ok, Status, _H, _B} = httpc:request(<<__URL_404__>>, get, [], <<>>), case Status of 404 -> true; _ -> false end\") :name)")
+
+;; Method passed as binary works too
+(epoch 24)
+(eval "(get (erlang-eval-ast \"{ok, Status, _H, _B} = httpc:request(<<__URL_HELLO__>>, <<__GET__>>, [], <<>>), case Status of 200 -> true; _ -> false end\") :name)")
+EPOCHS
+
+sed -i "s|__URL_HELLO__|${URL_HELLO_BYTES}|g; s|__URL_404__|${URL_404_BYTES}|g; s|__URL_BAD__|${URL_BADBODY_BYTES}|g; s|__BODY_HELLO__|${BODY_HELLO_BYTES}|g; s|__GET__|${GET_METHOD_BYTES}|g" "$TMPFILE"
+
+OUTPUT=$(timeout 120 "$SX_SERVER" < "$TMPFILE" 2>/dev/null)
+
+check() {
+  local epoch="$1" desc="$2" expected="$3"
+  local actual
+  actual=$(echo "$OUTPUT" | awk -v e="$epoch" '
+    $0 ~ "^\\(ok-len " e " " { getline; print; exit }
+    $0 ~ "^\\(ok " e " "     { print; exit }
+    $0 ~ "^\\(error " e " "  { print; exit }
+  ')
+  [ -z "$actual" ] && actual="<no output for epoch $epoch>"
+  if echo "$actual" | grep -qF -- "$expected"; then
+    PASS=$((PASS+1))
+    [ "$VERBOSE" = "-v" ] && echo "  ok $desc"
+  else
+    FAIL=$((FAIL+1))
+    ERRORS+="  FAIL [$desc] (epoch $epoch) expected: $expected | actual: $actual
+"
+  fi
+}
+
+check 10  "BIF registered under httpc/request/4" "true"
+check 11  "BIF marked non-pure"                  "false"
+check 12  "no /1 arity registered"               "true"
+check 13  "non-binary URL -> badarg"             "ok"
+check 14  "non-binary body -> badarg"            "ok"
+check 20  "live GET returns Status 200"          "true"
+check 21  "live GET Body is hello text"          "true"
+check 22  "Headers come back as proplist"        "true"
+check 23  "404 surfaces as {ok, 404, ...}"       "true"
+check 24  "method passed as binary works"        "true"
+
+TOTAL=$((PASS+FAIL))
+if [ $FAIL -eq 0 ]; then
+  echo "ok $PASS/$TOTAL next/tests/httpc_request.sh passed"
+else
+  echo "FAIL $PASS/$TOTAL passed, $FAIL failed:"
+  echo "$ERRORS"
+fi
+[ $FAIL -eq 0 ]

plans/fed-sx-milestone-2.md

@@ -594,12 +594,18 @@ a dead-letter list visible via `/admin/dead-letter`.
   in `delivery_dispatch.sh` covering single-peer enqueue,
   two-peer fan-out, missing-worker skip, no-flag no-op,
   FIFO append across two publishes, empty delivery_set no-op.
-- [ ] **8e** — `httpc:request/4` BIF wrapper. **Blocker:** the
-  briefing assumed a native `http-request` primitive existed in
-  `bin/sx_server.ml`; on inspection there's only `http-listen`.
-  The native http-CLIENT primitive belongs to `loops/fed-prims`
-  (host primitives loop). Blockers entry below. m2 work
-  continues with the in-process flow until the native lands.
+- [x] **8e** — `httpc:request/4` BIF wrapper. ~~Blocker~~ resolved:
+  loops/fed-prims merged into architecture, native `http-request`
+  primitive available. Wrapper at `lib/erlang/runtime.sx`
+  (briefing-allowed-exception scope) marshals Erlang
+  `(Url::binary, Method::atom|binary, Headers::proplist, Body::binary)`
+  → SX `(http-request method url headers body)` → Erlang
+  `{ok, Status::integer, Headers::proplist, Body::binary}`.
+  Atom methods are upcased (`get` → `"GET"`) for HTTP-wire convention;
+  binaries pass through verbatim. Test: `next/tests/httpc_request.sh`
+  10/10 pass — registration, badarg validation, live GET 200,
+  body bytes match, headers proplist shape, 404 surfaces as ok-tuple,
+  binary method works.
 - [ ] **8f** — Real HTTP dispatch through the BIF + content-type
   wiring. dispatch_fn for live use becomes a closure over the
   peer URL that calls `httpc:request/4` with the signed envelope
@@ -1026,17 +1032,16 @@ proceed.
    re-running on the unmodified m1 closeout HEAD.
 
 2. **Native `http-request` (HTTP client) primitive missing** —
-   discovered during Step 8e prep. The fed-sx-m2 briefing
-   ("Substrate available to you" §) claimed: "Native HTTP client
-   primitive (registered in `bin/sx_server.ml`): `http-request` —
-   exposed at the SX layer, currently native-only." On inspection
-   `bin/sx_server.ml` only registers `http-listen`; there is no
-   `http-request` registration. The HTTP client primitive belongs
-   to `loops/fed-prims` (host primitives loop) per the
-   one-primitive-loop-per-substrate convention. m2's Step 8e
-   wrapper (`httpc:request/4` BIF in `lib/erlang/runtime.sx`)
-   can land in a 1-line follow-up once the native exists; m2
-   work continues with 8b-pure / 8c / 8d in the in-process flow.
+   ~~discovered during Step 8e prep~~ **RESOLVED 2026-06-07** by
+   the user-authorized `loops/fed-prims` → `architecture` merge.
+   The primitive now registers at `bin/sx_server.ml:868+` with
+   signature `(http-request meth url headers body)` returning a
+   `{:status :headers :body}` dict and raising `Eval_error` on
+   DNS / connect / bad URL. Step 8e wired the Erlang-side BIF
+   wrapper around it (`httpc:request/4`); see Progress log
+   entry for marshalling details. Step 8f (live HTTP dispatch
+   through `delivery_worker`) and Step 10c (peer-actor doc
+   fetch in `peer_actors`) are now unblocked.
 
 3. **`erlang:send_after`-style timer primitive** — discovered
    during Step 8b prep. The retry loop needs a way for the
@@ -1055,6 +1060,39 @@ proceed.
 
 Newest first.
 
+- **2026-06-07** — Step 8e (closes the BIF half of Step 8;
+  live HTTP dispatch in 8f next): `httpc:request/4` BIF wrapper
+  landed in `lib/erlang/runtime.sx` (briefing-allowed-exception
+  scope). Marshalling: Erlang URL binary → SX string via
+  `(list->string (map integer->char (get url :bytes)))`; Erlang
+  atom method → upcased name (`get` → `"GET"`) for HTTP wire
+  convention; binary method passes through verbatim; headers
+  proplist → SX dict via existing `er-proplist-to-dict`; body
+  binary → SX string. Result `{:status :headers :body}` marshalled
+  back to Erlang `{ok, Status, Headers::proplist, Body::binary}`
+  via `er-of-sx-deep` on headers (which produces the binary-keyed
+  proplist `er-dict-to-header-proplist` shape) and
+  `(er-mk-binary (map char->integer (string->list body)))` for
+  body. Non-binary URL / body raise `error:badarg`; the native
+  primitive raises `Eval_error` on DNS / connect / bad URL which
+  surfaces as an Erlang error marker the caller can catch.
+  Blockers #2 (native http-request primitive) entry updated:
+  RESOLVED by the loops/fed-prims → architecture merge that the
+  user authorized. Test: `next/tests/httpc_request.sh` 10/10 —
+  5 registration / validation cases (registration under
+  `httpc/request/4`, non-pure flag, no /1 arity, badarg on
+  non-binary URL, badarg on non-binary body) plus 5 live
+  roundtrip cases against a background `python3 -m http.server`
+  (Status 200, body bytes match `hello from python\n`, headers
+  proplist shape, 404 surfaces as `{ok, 404, ...}` not as an
+  error tuple, method passed as binary works). Adjacent gates:
+  Erlang conformance 761/761, http_multi_actor 44/44, follower_
+  graph 18/18, follow_lifecycle 9/9, backfill 20/20,
+  backfill_drain 6/6, http_listen_bif 5/5 — all green; pre-
+  existing cold-startup timeout sensitivity on http_get_format
+  (120s internal) and nx_kernel_pure (240s internal) confirmed
+  with git stash to NOT be caused by this change.
+
 - **2026-06-07** — Step 9c (closes Step 9): Follow → Accept →
   backfill drain (in-process). `maybe_auto_accept/3` now calls
   `maybe_backfill/3` after the Accept publish: when