rose-ash/sx/sx/essays/hypermedia-age-of-ai.sx

;; ---------------------------------------------------------------------------
;; Hypermedia in the Age of AI
;; A response to Nick Blow's article on JSON hypermedia and LLM agents.
;; ---------------------------------------------------------------------------

(defcomp ~essays/hypermedia-age-of-ai/essay-hypermedia-age-of-ai ()
  (~docs/page :title "Hypermedia in the Age of AI"
    (p :class "text-stone-500 text-sm italic mb-8"
      "Neither JSON nor HTML is hypermedia. There is only the hypermedium — a self-defining representation — and s-expressions are an instance of it.")

    (~docs/section :title "I. The argument" :id "the-argument"
      (p :class "text-stone-600"
        (a :href "https://nickblow.tech/posts/hypermedia-in-the-age-of-ai" :class "text-violet-600 hover:underline" "Nick Blow argues")
        " that JSON hypermedia can serve AI agents better than HTML or RPC. Carson Gross contends that sprinkling link objects into JSON does not make an API truly RESTful, because REST demands generic clients capable of interpreting hypermedia controls. Blow agrees this has merit but thinks the position too restrictive. HTML is not the universal hypermedia format — LLMs choke on it. The real prize is " (em "progressive discovery") ": a client that learns what it can do by following links, not by reading documentation upfront.")
      (p :class "text-stone-600"
        "He contrasts this with MCP, the Model Context Protocol now dominant for LLM tool use. MCP is RPC in a trench coat: the server declares all its tools upfront, the model receives the full catalogue in its system prompt, and it calls functions by name. This works, but it does not scale. MCP forces " (em "total disclosure") " where hypermedia would offer " (em "progressive revelation") ".")
      (p :class "text-stone-600"
        "Blow proposes JSON-flavoured hypermedia as the fix — " (code "vnd.siren+json") ", custom content types, link relations in response payloads. Resources become state machines. Hyperlinks are state transitions. The agent explores the graph.")
      (p :class "text-stone-600"
        "This is the right diagnosis. But the prescription misidentifies the disease. The problem is not that we need a better wire format for links. The problem is that neither JSON nor HTML is actually hypermedia. And no amount of convention layered on top will make them so, because what makes something hypermedia is not what it carries but whether it can " (em "define itself") "."))

    (~docs/section :title "II. The self-definition criterion" :id "self-definition"
      (p :class "text-stone-600"
        "What does it mean for a format to be hypermedia? The conventional answer focuses on " (em "affordances") ": a hypermedia format includes controls that tell the client what it can do next. Links, forms, actions. This is the definition that drives the JSON hypermedia proposals — add links to JSON, and JSON becomes hypermedia.")
      (p :class "text-stone-600"
        "But this definition is too shallow. It describes a property of the content without asking where the " (em "meaning") " of the content comes from. A Siren response includes an " (code "actions") " array — but what makes that array mean \"available actions\" rather than just \"a list of objects with href fields\"? The Siren specification, written in English prose, hosted on a separate document, maintained by a separate community. The meaning is " (em "external") " to the format.")
      (p :class "text-stone-600"
        "A true hypermedium does not need an external document to explain what it means. It defines its own interpretation. The format carries not just content and controls, but the rules by which content and controls are understood. The map " (em "is") " the territory — not because the map is accurate, but because the map can redraw itself.")
      (p :class "text-stone-600"
        "This is a strict criterion. It eliminates almost everything. And it should, because almost nothing on the web is actually hypermedia in this sense. Most of what we call hypermedia is data decorated with navigational metadata, interpreted by an engine built from a separate specification in a separate language."))

    (~docs/section :title "III. JSON cannot define itself" :id "json-cannot"
      (p :class "text-stone-600"
        "You cannot write the JSON specification in JSON.")
      (p :class "text-stone-600"
        "This is not a trivia question. It reveals something fundamental about the format. JSON is a data serialization: maps, arrays, strings, numbers, booleans, null. It has no evaluation semantics. It cannot express a grammar. It cannot express a parser. It cannot express the rules by which a JSON document should be interpreted — because it cannot express " (em "rules") " at all. JSON describes " (em "what") " without any capacity to describe " (em "how") " or " (em "why") ".")
      (p :class "text-stone-600"
        "The JSON specification is " (a :href "https://www.rfc-editor.org/rfc/rfc8259" :class "text-violet-600 hover:underline" "RFC 8259") " — English prose. This is not an accident or a stylistic choice. JSON " (em "cannot") " define itself because it has no mechanisms for definition. It is inert data. Every scrap of meaning attached to a JSON document — this key means a link, this object means an action, this array means available transitions — must come from outside the document. From a spec. From a schema. From shared understanding between producer and consumer.")
      (p :class "text-stone-600"
        "This is why JSON hypermedia formats never achieve escape velocity. HAL, Siren, Hydra, JSON:API, UBER, Collection+JSON — each adds a linking convention on top of JSON, and each generates the same amount of adoption: enough for a conference talk, not enough for an ecosystem. The problem is not that the conventions are poorly designed. The problem is that a convention layered on inert data is " (em "always") " a separate document the client must already understand. You have not eliminated the manual. You have moved it from the system prompt to a media type specification."))

    (~docs/section :title "IV. HTML carries but does not define itself" :id "html-carries"
      (p :class "text-stone-600"
        "HTML is closer to the mark. You " (em "can") " write the HTML specification in HTML — and indeed, the " (a :href "https://html.spec.whatwg.org/" :class "text-violet-600 hover:underline" "WHATWG specification") " is an HTML document. This already puts HTML in a different category from JSON. The format can at least " (em "carry") " its own definition.")
      (p :class "text-stone-600"
        "But carrying is not defining. The HTML spec rendered in a browser is text and diagrams — documentation that happens to be displayed in the format it documents. The browser interpreting that document was built from a " (em "separate implementation") " — millions of lines of C++ in Chromium, Gecko, WebKit — that was written by reading the spec and translating it into executable code. The spec does not " (em "execute") ". It does not " (em "interpret") ". It does not " (em "define itself") " in any operative sense.")
      (p :class "text-stone-600"
        "Put it this way: if you gave the HTML specification (as an HTML document) to a system that had never seen HTML before, could that system learn to render HTML by reading it? No. The document is English prose in HTML tags. The tags are meaningless without a pre-existing renderer. The spec " (em "presupposes") " the very thing it specifies. It is circular, but not " (em "productively") " circular — not metacircular in the way that would let the system bootstrap itself.")
      (p :class "text-stone-600"
        "This matters because it means HTML's meaning is ultimately external too. More deeply embedded than JSON's — the browser internalizes the spec so thoroughly that HTML " (em "feels") " self-interpreting — but still dependent on a vast external system to give it life. The " (code "<form>") " tag is self-describing only to a client that already knows what forms are. To everything else, it is angle brackets."))

    (~docs/section :title "V. SX defines itself" :id "sx-defines-itself"
      (p :class "text-stone-600"
        "The SX specification is written in SX. This sounds like the same trick as HTML — the spec in its own format — but it is categorically different.")
      (p :class "text-stone-600"
        (code "eval.sx") " defines the SX evaluator as s-expressions. Not as documentation " (em "about") " the evaluator. As the evaluator " (em "itself") ". A bootstrapper reads " (code "eval.sx") " and produces a working evaluator — in JavaScript, in Python, in any target language. " (code "parser.sx") " defines the SX parser as s-expressions. A bootstrapper reads it and produces a working parser. " (code "render.sx") " defines the renderer. " (code "primitives.sx") " defines the primitive operations.")
      (~docs/code :code (highlight ";; From eval.sx — the evaluator defining itself:\n\n(define eval-expr\n  (fn (expr env mode)\n    (cond\n      ((number? expr) expr)\n      ((string? expr) expr)\n      ((boolean? expr) expr)\n      ((nil? expr) expr)\n      ((symbol? expr) (resolve-symbol expr env))\n      ((keyword? expr) (keyword-name expr))\n      ((dict? expr) (eval-dict expr env mode))\n      ((list? expr)\n        (let ((head (first expr)))\n          (if (symbol? head)\n            (let ((name (symbol-name head)))\n              (if (special-form? name)\n                (eval-special-form name (rest expr) env mode)\n                (eval-call expr env mode)))\n            (eval-call expr env mode)))))))" "lisp"))
      (p :class "text-stone-600"
        "This is not documentation. It is a " (em "program that defines the rules of its own interpretation") ". The evaluator that processes SX expressions is itself an SX expression. The spec does not merely " (em "describe") " how SX works — it " (em "is") " how SX works. Give this file to a bootstrapper, and out comes a functioning evaluator. The specification is executable. The definition " (em "is") " the implementation.")
      (p :class "text-stone-600"
        "This is what makes SX a genuine hypermedium. The meaning of an SX document is not external. It is not in a separate RFC. It is not in a browser engine compiled from a prose specification. The meaning is " (em "in the same language as the content") " — and it is the kind of meaning that executes. You do not need prior knowledge of SX to interpret SX, because SX carries the knowledge required to interpret it. You need only a bootstrapper — a minimal bridge to a host language — and the spec bootstraps the rest.")
      (p :class "text-stone-600"
        "The " (a :href "/sx/(etc.(essay.self-defining-medium))" :class "text-violet-600 hover:underline" "true hypermedium must define itself with itself") ". JSON cannot even attempt this. HTML can carry the words but not the meaning. SX carries the meaning because the meaning is code, and SX is code all the way down."))

    (~docs/section :title "VI. What self-definition gives you" :id "consequences"
      (p :class "text-stone-600"
        "Every practical advantage of SX over JSON and HTML for hypermedia flows from this single property.")
      (p :class "text-stone-600"
        (strong "Progressive discovery") " works because controls are not metadata interpreted by convention — they are expressions evaluated by the same evaluator that processes content. The server renders conditional controls using the language itself:")
      (~docs/code :code (highlight "(when can-cancel\n  (button :sx-post \"/orders/4281/cancel\"\n          :sx-confirm \"Cancel this order?\"\n          \"Cancel order\"))" "lisp"))
      (p :class "text-stone-600"
        "The " (code "when") " is not a convention layered on data. It is a special form defined in " (code "eval.sx") ". The server evaluates it. The client sees only the controls that survive evaluation. The state machine is authored in the language, not in metadata " (em "about") " the language.")
      (p :class "text-stone-600"
        (strong "Components") " work because the language has function abstraction — " (code "defcomp") " — defined in its own spec. A component is a reusable hypermedia control that accepts parameters and renders children. JSON cannot have components because JSON cannot define functions. HTML cannot have components natively — Web Components are defined in JavaScript, a " (em "separate") " language. SX components are defined in SX, evaluated by an evaluator defined in SX.")
      (p :class "text-stone-600"
        (strong "Evaluable URLs") " work because the URL is an s-expression — " (code "(etc.(essay.hypermedia-age-of-ai))") " — evaluated by the same evaluator. Following a link " (em "is") " calling a function. The addressing scheme is not a convention imposed from outside. It is the " (a :href "/sx/(applications.(sx-urls))" :class "text-violet-600 hover:underline" "language applied to navigation") ".")
      (p :class "text-stone-600"
        (strong "AI legibility") " works because the " (a :href "/sx/(etc.(essay.sx-and-ai))" :class "text-violet-600 hover:underline" "spec fits in a context window") ". An AI agent can load " (code "eval.sx") ", " (code "parser.sx") ", " (code "render.sx") ", and " (code "primitives.sx") " — roughly 3,000 lines — and hold the " (em "complete definition of the language") " alongside the content it is reading. It does not need training data about SX. It does not need documentation. It has the actual executable specification. The language it reads and the language that defines how to read it are the same language.")
      (p :class "text-stone-600"
        "None of these are features bolted onto a data format. They are consequences of a format that defines itself."))

    (~docs/section :title "VII. What MCP gets wrong" :id "mcp"
      (p :class "text-stone-600"
        "MCP's design reflects a fundamental confusion: it treats capability as a catalogue rather than an affordance. The server lists every tool. The model reads the list. The model calls a tool by name. The response is data. The model must maintain its own mental model of the server's state to know which tools are appropriate next.")
      (p :class "text-stone-600"
        "This is the exact inversion of the hypermedia principle. In a hypermedia system, the server tells you what you can do " (em "in each response") ". You do not need a mental model of the server because the server renders its own state into the controls it offers you.")
      (p :class "text-stone-600"
        "But the deeper problem with MCP is not the catalogue model. It is that MCP tools are defined " (em "outside") " the protocol — in Python functions, TypeScript classes, whatever the server implementer chose. The tool definitions are opaque to the client. The model sees names and descriptions, not the actual logic. It is trusting documentation written by a human about code written by a human. Every layer of indirection is a place where meaning can leak.")
      (p :class "text-stone-600"
        "In an SX hypermedia system, the controls in the response " (em "are") " the capability. Not a description of capability. Not a pointer to capability. The control itself — its structure, its attributes, its evaluable URL — carries everything needed to act on it. And the rules for interpreting the control are carried by the same language the control is written in. There is no indirection. There is no separate layer where meaning must be maintained. The response means what it says, and it says what it means, because it carries its own interpreter."))

    (~docs/section :title "VIII. There is only the hypermedium" :id "the-hypermedium"
      (p :class "text-stone-600"
        "The hypermedia discourse frames the question as a competition between formats: HTML vs JSON, server-rendered vs client-rendered, REST vs RPC. Blow adds a new axis — human clients vs AI agents — and asks which format serves both. These are all the wrong questions. They assume that hypermedia is a property that a format can have in greater or lesser degree, and the task is to find the format with the most of it.")
      (p :class "text-stone-600"
        "Hypermedia is not a spectrum. It is a threshold. Either a format can define its own interpretation, or it cannot. If it cannot, it depends on external meaning — an RFC, a browser engine, a media type specification — and that dependency makes it something less than a " (em "medium") ". A medium is self-sustaining. It does not require a separate system to explain what it is. Sheet music is a medium because a musician can read it and produce sound. A JSON object with " (code "\"type\": \"sheet-music\"") " is not a medium. It is data that requires a separate program, written in a separate language, consulting a separate specification, to become anything at all.")
      (p :class "text-stone-600"
        "By this criterion, HTML is not hypermedia. It is the closest the web has come — close enough that the browser's deep internalization of the spec creates the " (em "illusion") " of self-interpretation. But the illusion breaks the moment you step outside the browser. Give HTML to a new client — an AI agent, a screen reader, a search crawler — and that client must reimagine the spec from training data, heuristics, and hope. The meaning was never in the HTML. It was in the browser.")
      (p :class "text-stone-600"
        "JSON is not hypermedia and never will be, no matter how many link relations you attach to it. The meaning is always elsewhere.")
      (p :class "text-stone-600"
        "There is only the hypermedium: a representation that defines its own interpretation. A representation where the spec is written in the language, and the spec " (em "is") " the language — executable, bootstrappable, self-sustaining. S-expressions are not the only possible instance of this. Any homoiconic, metacircular language could satisfy the criterion. But s-expressions are the simplest. The minimal syntax — " (code "(head args...)") " — is the minimal overhead between a format and self-definition. McCarthy arrived at this in 1958. It has taken sixty-eight years for the rest of computing to need it.")
      (p :class "text-stone-600"
        "The age of AI makes the need urgent. When your clients are no longer just browsers with built-in specs but arbitrary agents that must learn interpretation on the fly, a format that carries its own interpreter is not a luxury. It is the only thing that works. Not because it is clever. Because everything else depends on meaning that lives somewhere else — and " (em "somewhere else") " is the one place an ad-hoc agent cannot be guaranteed to look.")
      (p :class "text-stone-600"
        "SX represents the hypermedium. Not a new format in the format wars. Not a better JSON. Not a simpler HTML. The thing itself — a representation that " (em "means what it says") " because it carries the rules for saying it."))))