# Superb backend

The API, reader and contribution flow share one semantic database. The native Node
server uses SQLite; the Cloudflare worker uses the equivalent D1 schema. Passages,
review records and account data stay off chain. Solana is the canonical state
machine for protocol consensus: it stores rule and evaluator identity, contrast
commitments, ordered submission and eligible roots, consensus results,
attestations, challenges, finalization and semantic release lineage. Reward
allocations, treasury reservations and claim receipts are a separate track.

When `CONSENSUS_RECEIPT_SIGNING_KEY` (base64 PKCS#8 Ed25519) and
`CONSENSUS_RECEIPT_PUBLISHER` (matching Solana public key) are configured with
`SOLANA_CLUSTER` and `LITERATE_PROGRAM_ID`, accepted task answers get signed,
context-bound sequence receipts in `submission_log`. The returned receipt can
later prove an omission against a published checkpoint. On a database with
older judgments lacking receipts, those contrasts are blocked from protocol
export; create fresh contrasts instead of silently upgrading old records.
The receipt key is a dedicated online key. The program rejects a proposal if
it is the Solana administrator key and records both identities separately.

`node scripts/export-consensus.mjs CONTRAST_ID CONFIG.json OUTPUT.json` reads
one fully receipted contrast and prepares an independently replayable artifact
and compact Solana proposal. `CONFIG.json` supplies network/program/publisher,
registered rule/evaluator IDs, evidence hashes and retrieval locators. The
script refuses gaps, invalid signatures and missing receipts. The separate
`solana/consensus-local.mjs` operator client submits program instructions after
the updated IDL is built. The core transactions have passed a fresh local
validator run; the export-to-operator-command flow still needs a fully signed
real contrast. A prepared file is not published consensus.

## Run

From the repository root with Node 24 and Python 3.12+:

```sh
npm ci
npm run data:build
npm run corpus:ingest
npm run corpus:tasks
npm run build
npm start
```

`SUPERB_DB` selects the SQLite file (default `data/local/superb.sqlite`). `PORT`
defaults to 4173; `HOST` defaults to loopback. For a server behind a TLS proxy, set
`SITE_ORIGIN` to the exact public HTTPS origin. Persist and back up the SQLite
volume. Use SQLite's backup operation, not a live copy that omits WAL records.
Do not serve `data/local/` as static files.

The importer resumes from cached books, records source hashes and permissions,
and samples up to 300 ambiguous-word usages per book. The current author-based
split keeps each author in contribution, validation or holdout. This is a first
partition, not a verified complete genealogy of translations and derivative works.
Review related editions before evaluation. Public-domain clearance uses the
catalogue's US rights statements; territorial redistribution requires review.

## Contribution and release operations

```sh
node scripts/corpus.mjs status
node scripts/corpus.mjs review-sample
node scripts/corpus.mjs review reviewed-labels.json
node scripts/corpus.mjs induce
node scripts/corpus.mjs release 1.0.0
node scripts/corpus.mjs epoch epoch-input.json
```

Review import accepts `actor`, `contexts: [{id, senseId}]` and
`contrasts: [{id, answer, calibration?}]`. Labels must come from an independent
reviewer. The tool refuses repeated labels, self-review of a submitted contrast,
and calibration labels added after exposure. Operator access is a trust boundary;
this code cannot establish that an operator actually performed independent review.
Never label the real corpus with synthetic test fixtures or model output.

A release needs reviewed contribution anchors and at least 30 labels from five
source groups in each of validation and holdout. The anchor model must have
nonzero correct predictions, no lower total accuracy and no lower coverage than
the lexical baseline. Abstentions count as incorrect for total accuracy. This is
an initial regression gate, not proof of domain-wide quality. Keep holdout review
private and use a fresh sealed set after tuning against a reported result.

Induction groups reviewed `same_meaning` edges and rejects components contradicted
by a reviewed negative edge. Other contrast kinds do not merge senses. Components
are proposals; this version does not invent or automatically approve definitions.
Accepted source definitions remain intact. There is no trained embedding model or
validated learned sense induction claim in this release.

Each release records canonical manifest, inventory/source/artifact hashes and
actual evaluation counts. Public APIs expose the accepted artifact; source
withdrawal disables affected releases and future epoch preparation/publication.
Historical on-chain commitments and paid claims cannot be erased.

Epoch input is `{ "releaseHash": "...", "budgetBaseUnits": "1000000000",
"pointCap": "100" }`. Points come only from matching answers to independently
reviewed contrasts included in that release. Calibration earns reputation, not
rewards. Missing wallets and allocations rounded to zero carry forward. Prepared
allocations reserve judgment IDs atomically, preventing double allocation.
`solana/settle.mjs publish` also requires the matching release ID to be
protocol-finalized on Solana. The old arbitrary-hash reward instruction rejects
every call in the current program build.

```sh
node scripts/corpus.mjs withdraw source-id "Reason for withdrawal"
```

## Hosting

The Node service includes guarded URL import: HTTPS only, public DNS addresses
pinned to each connection, bounded redirects, response size and request duration.
HTML is returned as plain text for inspection. Protected pages, streaming video
and sites without readable transcripts require a text/file upload.

Cloudflare builds emit `dist/_worker.js/`. Bind `ACCOUNTS` to D1 and apply
`apps/site/worker/accounts.sql` followed by `packages/core/schema.sql`. Migrate the
semantic corpus before opening contribution. Cloudflare link import additionally
requires a `MEDIA_FETCHER` service implementing the Node importer contract; it is
not silently replaced with an unrestricted server fetch. The Node deployment is
the complete single-host path for the current corpus.

Account/email, external models, private storage and payments need actual provider
configuration. See the repository's `operations.txt` for the single operator checklist.
No secret belongs in a Vite variable or public asset.