TopoloCommerce Resilience

What It Is

TopoloCommerce resilience is layered:

• cached local client state for the current production foundation
• venue-local authority through Venue Edge in a later phase
• constrained 5G rescue uplink for backlog drain and recovery, not as the primary operating model

The hard rule remains: a venue should continue operating as close to normal as possible even when cloud connectivity is degraded.

Architecture

The current implementation is phase-1 resilience plus the first cloud-side Venue Edge control-plane contract. It is still not full venue-edge offline authority.

Today:

• apps/guest-web caches venue lists, venue snapshots, guest sessions, and baskets in browser storage
• apps/guest-web queues guest orders, service requests, and bookings in a local outbox when the cloud path is unavailable
• apps/guest-web attaches stable clientMutationId values to queued guest writes so replay stays idempotent after connectivity returns
• apps/guest-web replays that outbox automatically when connectivity returns
• apps/ops-web caches org context and venue snapshots in browser storage
• apps/ops-web queues ticket transitions, ticket assignments, team-section creation, team-member creation, and team-member updates in a local outbox when the cloud path is unavailable
• apps/ops-web attaches stable clientMutationId values to queued staff writes so replay does not duplicate queue or team changes
• apps/ops-web replays that outbox automatically when connectivity returns
• apps/api records successful guest and staff mutations in commerce_mutation_journal keyed by route plus clientMutationId
• apps/api records append-only venue activity in commerce_venue_event_journal so cloud-side venue history exists before Venue Edge is introduced
• apps/api now persists enrolled Venue Edge nodes in commerce_venue_edge_nodes
• apps/api now exposes cloud-side Venue Edge routes for node enrollment, bootstrap snapshots, cursor-based journal pull, and cursor acknowledgement
• apps/api falls back to D1-derived live queue snapshots when the Durable Object queue cache is unavailable and does not fail persisted mutations just because queue fanout could not be refreshed immediately
• both browser surfaces expose an explicit resilience banner so operators and guests can see when they are in cached local mode, degraded mode, or replay mode
• apps/ops-web now exposes a module-aware resilience route so staff can inspect the current event journal, recovery posture, and enrolled Venue Edge sync state for a venue

Runtime Surfaces

This foundation is implemented in:

• PlatformApplications/TopoloCommerce/apps/guest-web/src/App.jsx
• PlatformApplications/TopoloCommerce/apps/ops-web/src/App.jsx

Current runtime behavior:

• apps/guest-web keeps cached venue snapshots, guest-session fallbacks, basket persistence, and a replay queue for guest actions
• apps/ops-web keeps cached org and venue snapshots plus a replay queue for staff actions
• apps/api remains the cloud source of truth and replay target in the current phase
• apps/api now de-duplicates replayed mutations for the current browser outbox contract instead of treating each retry as a fresh write
• apps/api now maintains a cloud-side venue event journal that future Venue Edge reconciliation can replay against
• apps/api now also provides a cloud-side Venue Edge handshake so a future local runtime can enroll, bootstrap, pull journal pages, and acknowledge cursors against Commerce

API Reference

The current resilience foundation still relies on the existing guest and staff write routes for browser outbox replay, but it now also includes a dedicated cloud-side Venue Edge API family.

Current replay behavior:

• replay-safe routes accept an optional clientMutationId in the request body
• when the same route and clientMutationId are replayed, the API returns the original successful response instead of creating a second record
• the current replay-safe store is commerce_mutation_journal in D1
• the current live queue cache remains a derived Durable Object projection, not the canonical write store

Current replay targets:

• /api/orders
• /api/service-requests
• /api/bookings
• /api/venues/:venueId/ops/tickets/:ticketId
• /api/venues/:venueId/team/sections
• /api/venues/:venueId/team/members
• /api/venues/:venueId/team/members/:memberId

Current resilience read route:

• /api/venues/:venueId/resilience/state

Current Venue Edge routes:

• /api/venues/:venueId/edge/nodes
• /api/venues/:venueId/edge/bootstrap
• /api/venues/:venueId/edge/events
• /api/venues/:venueId/edge/ack

Current Venue Edge contract:

• staff enroll a node from ops and receive a one-time enrollment token
• an enrolled node bootstraps a venue snapshot plus the current journal cursor
• an enrolled node pages the append-only venue journal after a cursor
• an enrolled node acknowledges the highest fully-synced cursor back to Commerce
• edge auth currently uses x-topolo-edge-token or Authorization: Edge <token>

Auth and Permissions

Guest outbox replay continues to use guest-facing public write routes.

Ops outbox replay continues to use the existing staff auth contract:

• @topolo/auth-client in the browser
• @topolo/auth-middleware in the API
• the same Commerce role bundles already required for normal queue and team actions

Data Ownership

The current resilience foundation adds browser-local cached copies of:

• guest venue lists
• guest venue snapshots
• guest baskets and session fallbacks
• ops org context
• ops venue snapshots
• guest local outbox entries
• ops local outbox entries

TopoloCommerce remains the cloud owner of the canonical venue and queue state once replay succeeds.

TopoloCommerce also owns the current replay journal used to de-duplicate browser outbox retries:

• commerce_mutation_journal

TopoloCommerce now also owns the current cloud-side venue event journal:

• commerce_venue_event_journal

TopoloCommerce now also owns the current cloud-side Venue Edge node registry:

• commerce_venue_edge_nodes

Deployments

The current phase does not change deployment topology. The resilience foundation ships inside the existing guest and ops browser surfaces:

• https://guest.topolo.app
• https://commerce.topolo.app

Future local Venue Edge runtime and 5G rescue uplink work will require additional deployment and MDM planning. The current cloud-side control-plane contract ships inside the existing Commerce Worker and ops app.

Failure Modes

This phase does not yet make TopoloCommerce fully outage-resilient at the venue level.

Not yet implemented:

• Venue Edge as a local venue authority
• venue-local SQLite event log
• venue-local websocket bus across multiple devices
• venue-local printer bridges
• venue-local payment reconciliation service
• controlled 5G rescue uplink activation policy
• wearable notification delivery and acceptance flow

In the current phase, cached local browser operation protects the user experience on the current device and preserves queued actions for later replay. It does not yet guarantee full venue-wide multi-device operation during a total cloud outage.

The cloud-side event journal plus edge-node registry are a bridge to the next phase, not the end state. They improve auditability and future reconciliation readiness, but they are not a substitute for venue-local authority.

Planned Next Phase

The next resilience phase should introduce Venue Edge with:

• venue-local event authority
• local websocket updates across guest, ops, and staff devices
• append-only event journal with idempotent replay to cloud
• connectivity manager with explicit state transitions:
  • online
  • degraded
  • offline_local
  • rescue_uplink
  • reconciling
  • recovered

5G rescue uplink should be policy-driven and dormant by default. It should activate only after outage thresholds are crossed and should prioritize event replay over general venue traffic.

Debugging

• inspect apps/guest-web/src/App.jsx when guest requests disappear after a connectivity blip or are expected to replay
• inspect apps/ops-web/src/App.jsx when operator actions are expected to queue locally or replay
• inspect apps/api/src/index.js when replayed guest or staff writes should resolve idempotently but appear to create duplicates
• inspect apps/api/src/index.js when resilience state or venue event history looks incomplete
• inspect apps/api/src/index.js plus commerce_venue_edge_nodes when an enrolled edge node is not bootstrapping, syncing, or advancing its acknowledged cursor
• inspect browser storage for the current topolo_commerce_* keys when validating local snapshot and outbox behavior
• inspect commerce_mutation_journal in D1 when checking whether a replayed write should have returned a prior result
• inspect commerce_venue_event_journal in D1 when validating cloud-side venue event history and future reconciliation readiness
• do not describe the current browser-storage foundation or the new cloud-side handshake as the final Venue Edge; the local venue authority still does not exist yet

Change Log / Verification

• Added the canonical TopoloCommerce resilience document on 2026-04-10
• Verified on 2026-04-10 that guest-web and ops-web now persist cached snapshots and local outbox state so core venue interactions survive temporary cloud interruption and replay automatically when connectivity returns
• Verified on 2026-04-10 that replayed browser outbox writes now use stable clientMutationId values and D1-backed mutation journaling so duplicate guest orders, service requests, bookings, and staff actions are not created during retry
• Verified on 2026-04-10 that Commerce now appends venue activity to commerce_venue_event_journal and exposes a staff-facing resilience view backed by /api/venues/:venueId/resilience/state
• Verified on 2026-04-10 that Commerce now stores enrolled Venue Edge nodes in commerce_venue_edge_nodes and exposes node enrollment, bootstrap, journal pull, and cursor acknowledgement routes for the next local-authority phase