Commit Graph

72 Commits

Author SHA1 Message Date
gilesb
678d0e0ea3 Fix NoneType subscript error in record_run
- Add get_asset_by_name_tx for transaction-aware asset lookup
- Use transaction connection instead of separate connection
- Prevents race condition where asset might not be visible

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 15:07:42 +00:00
gilesb
59484936fd Fix db.get_asset_by_name -> db.get_asset
The function is named get_asset(), not get_asset_by_name().

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 12:27:05 +00:00
gilesb
c1cbf0b4ad Phase 2: Multiple storage configs per type with new UI structure
- Database: Add description field, remove unique constraint to allow
  multiple configs of same provider type
- UI: Main page shows provider types as cards with counts
- UI: Per-type page (/storage/type/{type}) for managing configs
- API: Add get_user_storage_by_type() for filtered queries
- Form: Add description field for distinguishing configs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 00:53:28 +00:00
gilesb
2326658518 Fix storage test and delete endpoints to support cookie auth
Both /storage/{id}/test and DELETE /storage/{id} were using Bearer
token auth only. Now they also check cookie auth for browser sessions.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 00:27:05 +00:00
gilesb
770c36479f Add support for more decentralized storage providers
Added 4 new storage providers:
- NFT.Storage (free for NFT data)
- Infura IPFS (5GB free)
- Filebase (5GB free, S3-compatible IPFS)
- Storj (25GB free, decentralized cloud)

Updated UI with 7 total storage options in a 4-column grid,
each with distinct colored borders for visibility.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 00:25:09 +00:00
gilesb
8bef9afb1f Add form-based storage endpoint for browser submissions
The POST /storage endpoint required Bearer token auth and JSON body,
which didn't work with browser form submissions using cookies. Added
new /storage/add endpoint that accepts form data and cookie auth.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 00:19:36 +00:00
gilesb
fb5c46330d Fix storage page authentication to support cookie-based sessions
The /storage route was only checking Bearer token authentication,
causing logged-in browser users to be redirected to login. Now also
checks cookie authentication like other HTML pages.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 00:10:23 +00:00
gilesb
70cde17fef Fix storage provider buttons visibility with colored borders
The storage option buttons were nearly invisible due to low contrast
between bg-dark-600 buttons and bg-dark-700 background. Added distinct
colored borders (blue/green/purple) and darker backgrounds to make
the Pinata, web3.storage, and Local Storage options clearly visible.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 00:04:56 +00:00
gilesb
cf94600d63 Add borders to storage provider buttons for visibility
Buttons had same background color as container, making them nearly
invisible. Added border-dark-500 and hover:border-blue-500.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 23:44:17 +00:00
gilesb
efd3a2dd16 Fix /storage endpoint to default to HTML
Same content negotiation fix as other endpoints - default to HTML
for browsers, only return JSON if explicitly requested.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 23:40:58 +00:00
gilesb
1e3d1bb65e Add user-attachable storage system
Phase 1 of distributed storage implementation:

Database:
- user_storage table for storage providers (Pinata, web3.storage, local)
- storage_pins table to track what's stored where
- source_url/source_type columns on assets for reconstruction

Storage Providers:
- Abstract StorageProvider base class
- PinataProvider for Pinata IPFS pinning
- Web3StorageProvider for web3.storage
- LocalStorageProvider for filesystem storage
- Factory function create_provider()

API Endpoints:
- GET/POST /storage - list/add storage providers
- GET/PATCH/DELETE /storage/{id} - manage individual providers
- POST /storage/{id}/test - test connectivity

UI:
- /storage page with provider cards
- Add provider form (Pinata, web3.storage, local)
- Test/remove buttons per provider
- Usage stats (capacity, donated, used, pins)

50% donation model: half of user capacity is available for
system use to store shared content across the network.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 23:19:10 +00:00
gilesb
64749af3fc Call /auth/revoke-user on L1s during logout
L2 now calls /auth/revoke-user (revokes by username) instead of
/auth/revoke (revokes by token), because L1 has scoped tokens that
differ from L2's own token.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 22:22:22 +00:00
gilesb
5ebfdac887 Add scoped tokens, L2-side revocation, and security docs
Security improvements:
- Tokens now include optional l1_server claim for scoping
- /auth/verify checks token scope matches requesting L1
- L2 maintains revoked_tokens table - even if L1 ignores revoke, token fails
- Logout revokes token in L2 db before notifying L1s
- /renderers/attach creates scoped tokens (not embedded in HTML)
- Add get_token_claims() to auth.py

Database:
- Add revoked_tokens table with token_hash, username, expires_at
- Add db functions: revoke_token, is_token_revoked, cleanup_expired_revocations

Documentation:
- Document security features in README

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 18:21:13 +00:00
gilesb
4351c97ce0 Revoke tokens on attached L1s when user logs out
On logout:
1. Call /auth/revoke on each attached L1 renderer
2. Remove all attachments from user_renderers table
3. Clear L2 cookie

This ensures logging out of L2 also logs user out of all L1s.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 18:08:05 +00:00
gilesb
813f6a1d44 Record renderer attachment when L1 verifies token
When L1 successfully calls /auth/verify, the user's attachment to
that L1 server is recorded in the user_renderers table.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 17:51:15 +00:00
gilesb
e9df81db40 Require L1 server authorization for token verification
L1 servers must now identify themselves when calling /auth/verify.
Only servers listed in L1_SERVERS can verify tokens.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 17:43:08 +00:00
gilesb
e5d1c93034 Remove cross-subdomain cookie sharing, use lax SameSite
L1 servers now verify tokens by calling L2's /auth/verify endpoint,
so L2 no longer needs to share cookies across subdomains. Each L1
sets its own first-party cookie via its /auth endpoint.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 17:40:14 +00:00
gilesb
28843ea185 Add Renderers page for L1 server management
- Add user_renderers table to track which L1 servers users are attached to
- Add L1_SERVERS config to define available renderers
- Add /renderers page showing attachment status for each L1 server
- Add attach functionality (redirects to L1 /auth with token)
- Add detach functionality with HTMX updates
- Add Renderers link to nav bar

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 17:18:41 +00:00
gilesb
990ac44108 Support return_to redirect with token for iOS Safari
Login page accepts return_to URL. After login, redirects to
return_to with auth_token in URL so target site can set its
own first-party cookie (works around iOS Safari ITP).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 17:06:27 +00:00
gilesb
f2397e0a73 Fix cross-subdomain cookie for iOS Safari
Change SameSite from Lax to None to allow cookie to be sent
when navigating between L1 and L2 subdomains. iOS Safari's
Intelligent Tracking Prevention may block Lax cookies.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 16:54:29 +00:00
gilesb
d72b5e0e50 Fix download links to use /raw endpoint
Download links now point to /cache/{hash}/raw which returns proper
Content-Type headers and filename with extension.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 12:47:32 +00:00
gilesb
f9b7f784e5 Return existing asset instead of error when asset already exists
When publishing to L2, if the asset already exists, return it with
existing: true flag instead of raising a 400 error. This makes
re-publishing idempotent and handles race conditions gracefully.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 12:35:01 +00:00
gilesb
b810a65f5b Fix embedded media to use /raw endpoint
After L1 content negotiation fix, /cache/{hash} returns HTML for
browsers. Embedded <img> tags need /raw to get actual image data.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 12:24:18 +00:00
gilesb
00820b09cf Use content_hash as asset names for all run assets
- RecordRunRequest.output_name now optional (deprecated)
- Input assets named by their content_hash
- Output assets named by their content_hash
- Simplified registered_inputs - just content_hash and ipfs_cid
- All assets now referenced by content_hash, not friendly names

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 12:06:43 +00:00
gilesb
c53f67fb19 Fix content negotiation - default to HTML, not JSON
All public endpoints (/assets/{name}, /activities/{id}, /objects/{hash})
now default to HTML for browsers. JSON/ActivityStreams is only returned
when explicitly requested via Accept header. Fixes browser link clicks
showing JSON instead of HTML pages.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 11:59:37 +00:00
gilesb
a0d2328b01 Register recipes with proper naming and tags
When auto-registering input assets from a run, detect recipes
and give them appropriate names (recipe-{hash}) and tags
(auto-registered, input, recipe) instead of generic input names.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 11:51:26 +00:00
gilesb
4c177a64c2 Show recipe source on L2 asset detail page
Recipe assets now display their YAML source code in a formatted
code block instead of just a download link.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 11:33:44 +00:00
gilesb
3d08b586ec Add CLI client download
- Include artdag-client.tar.gz package
- Add /download/client route to serve the package
- Add "Download Client" link in nav bar

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 11:16:57 +00:00
gilesb
3ed4fe89ed Add content-addressable runs and activities
- compute_run_id() for deterministic run identification
- /assets/by-run-id/{run_id} endpoint for L1 recovery
- Store run_id in provenance when recording runs
- Activities now use content hash as ID instead of UUID
- URLs: /activities/{content_hash} instead of /activities/1

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 11:04:59 +00:00
gilesb
0f01d8e12c Add 404 page and Web UI section to README
- Add styled 404 page for HTML requests (returns JSON for API calls)
- Add Web UI section to README documenting available routes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 10:42:44 +00:00
gilesb
0a0dd278fe Add Tailwind typography plugin for prose styling
The README on the home page uses prose classes which require the
typography plugin to render headers and other markdown elements.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 10:30:32 +00:00
gilesb
29b838fdda Fix uvicorn workers - use import string
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 03:41:29 +00:00
gilesb
32435fe291 Add timing to L1 request for debugging
Log how long requests.get() actually takes to identify network delays.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 03:38:27 +00:00
gilesb
3756f5e630 Add uvicorn workers for better concurrency
Run with 4 workers to handle concurrent requests without blocking.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 03:36:16 +00:00
gilesb
43a9a1cd64 Store full provenance on IPFS with provenance_cid
Provenance records are now stored on IPFS before the DB transaction.
The provenance CID is included in both:
- The provenance stored in the asset record
- The ActivityPub activity object

This enables:
- Immutable, content-addressed provenance
- Bitcoin timestamping of provenance for cross-L2 dispute resolution
- Verifiable chain of custody: inputs → recipe → output

Provenance structure on IPFS:
{
  "inputs": [...],
  "output": {"content_hash": "...", "ipfs_cid": "..."},
  "recipe": {...},
  "recipe_cid": "Qm...",
  "provenance_cid": "Qm...",  // self-reference for verification
  ...
}

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 02:31:03 +00:00
gilesb
91d8093a1b Improve L1 fetch error handling with better diagnostics
- Handle 404 explicitly with clear "not found" message
- Log actual response body when JSON parsing fails
- Include status code and body preview in error messages

This helps diagnose issues like empty responses or HTML error pages from L1.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 02:00:27 +00:00
gilesb
449ed0c100 Add comprehensive logging to publishing endpoints
Added logging throughout record_run, publish_cache, and _register_asset_impl:
- Log start of each operation with key parameters
- Log L1 fetch operations and their results
- Log IPFS pin operations
- Log DB transaction start and completion
- Log errors with context
- Log successful completion

This makes it easier to diagnose timeout and other issues.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 01:54:17 +00:00
gilesb
9ef45e295b Make all IPFS and L1 operations non-blocking
Wrap synchronous requests.get() and ipfs_client calls in
asyncio.to_thread() to prevent blocking the FastAPI event loop.
This fixes web UI slowness during publishing operations.

- record_run: L1 fetches and IPFS operations now async
- _register_asset_impl: IPFS pin now async
- publish_cache: IPFS pin now async
- get_anchor_tree: IPFS get_bytes now async
- verify_activity_anchor: IPFS get_bytes now async

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 01:42:15 +00:00
gilesb
7ead2026ef Add Bitcoin anchoring via OpenTimestamps
Implements merkle tree anchoring for ActivityPub activities with
provable Bitcoin timestamps. Activities are batched into merkle trees,
submitted to OpenTimestamps (free), and proofs stored on IPFS.

Key features:
- Merkle tree construction with membership proofs
- OpenTimestamps integration for Bitcoin anchoring
- JSONL backup file on persistent volume (survives DB wipes)
- API endpoints for creating/verifying anchors

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 01:22:39 +00:00
gilesb
2267096271 Add format_date helper to handle datetime objects in templates
The db now returns datetime objects instead of strings in some cases.
Added format_date() helper function that handles both datetime and string
values, and replaced all [:10] date slicing with calls to this helper.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 01:20:32 +00:00
gilesb
647c564c47 Implement atomic publishing with IPFS and DB transactions
All publishing operations now use three-phase atomic approach:
1. Phase 1: Preparation - validate inputs, gather IPFS CIDs
2. Phase 2: IPFS operations - pin all content before any DB changes
3. Phase 3: DB transaction - all-or-nothing database commits

Changes:

ipfs_client.py:
- Add IPFSError exception class
- Add add_bytes() to store content on IPFS
- Add add_json() to store JSON documents on IPFS
- Add pin_or_raise() for synchronous pinning with error handling

db.py:
- Add transaction() context manager for atomic DB operations
- Add create_asset_tx() for transactional asset creation
- Add create_activity_tx() for transactional activity creation
- Add get_asset_by_hash_tx() for lookup within transactions
- Add asset_exists_by_name_tx() for existence check within transactions

server.py:
- Rewrite record_run:
  - Check L2 first for inputs, fall back to L1
  - Store recipe JSON on IPFS with CID in provenance
  - Auto-register input assets if not already on L2
  - All operations atomic
- Rewrite publish_cache:
  - IPFS CID now required
  - Synchronous pinning before DB commit
  - Transaction for asset + activity
- Rewrite _register_asset_impl:
  - IPFS CID now required
  - Synchronous pinning before DB commit
  - Transaction for asset + activity

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 00:59:12 +00:00
gilesb
a0ed1ae5ae Make IPFS pinning non-blocking (fire-and-forget)
IPFS pinning can take a long time if content needs to be fetched
from the network. Changed all pin operations to run in background
threads so they don't block the HTTP response.

This fixes the 30s timeout issue when publishing assets.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 00:26:32 +00:00
gilesb
a7dfdc8a39 Fix asset_type detection when recording runs from L1
Fetch media_type and ipfs_cid from L1's cache endpoint instead of
hardcoding asset_type to "video". This fixes images being incorrectly
displayed as videos.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 00:22:58 +00:00
gilesb
e69b72cbe1 Add ipfs_cid support to asset update endpoint
Allow updating asset's IPFS CID via PATCH endpoint.
Pins content on IPFS when CID is provided.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 23:21:20 +00:00
gilesb
ba7dc3b89c Fix NoneType error when asset provenance is null
Use 'or {}' pattern to handle both missing keys and explicit None
values for origin, provenance, and metadata fields.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 23:02:15 +00:00
gilesb
fd239d99f9 Fix f-string syntax error in IPFS display
Build ipfs_html separately to avoid nested f-string issues.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 19:18:41 +00:00
gilesb
8ae354d503 Add IPFS info display to L2 asset detail page
- Add IPFS_GATEWAY_URL config
- Show IPFS CID on asset detail page
- Add gateway links (local, ipfs.io, dweb.link)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 18:57:00 +00:00
gilesb
96631e1e4c Add IPFS node to L2 for federated content storage
- Add IPFS container to docker-compose
- Add ipfshttpclient dependency
- Add ipfs_client.py module for IPFS operations
- Add ipfs_cid field to Asset model and database schema
- Pin content on L2 IPFS when assets are published/registered

L2 now stores content on its own IPFS node, enabling
federation - content remains available even if L1 goes down.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 18:21:33 +00:00
gilesb
82cb1cf711 Fix logout to clear both legacy and shared domain cookies
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 17:38:14 +00:00
gilesb
1d463352a7 Add configurable cookie domain for cross-subdomain auth sharing
- Add COOKIE_DOMAIN env var (e.g., ".rose-ash.com")
- Auto-derive from ARTDAG_DOMAIN if not set (strips first subdomain)
- Set domain on auth cookies for sharing across L1/L2 subdomains
- Add secure=True for cross-subdomain cookies

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 17:26:42 +00:00