Table of Contents
- Token Relations
- Observations, not catalogue entries
- A transfer has a direction; a relation has roles
- The table
- How relations are ingested
- Why this is NOT part of the token ingestion queue
- Why the burn/mint flags are NOT columns
- Why there are NO foreign keys to DeployedToken
- Deleting a deployed token leaves its relations in place
- Display implications
- Relations graph
- Human edits
- Known limitations
Token Relations
A token relation records that we witnessed a non-swapping interop
transfer between two token addresses: the same abstract asset moved between
them via a specific plugin, classified as a specific non-swapping
bridgeType. Relations are the raw material for reasoning about which
deployed tokens represent the same asset — including, crucially, for
spotting places where our current abstract-token assignments are wrong.
Observations, not catalogue entries
This is the single mental model that explains every design decision below:
A token relation is an observation; the token catalogue is an interpretation.
A relation says “this transfer happened on-chain”. That is a fact,
true regardless of whether we have catalogued either address as a
DeployedToken, and true especially when our abstract-token assignments
disagree with it. A token-ingestion conflict is precisely the situation
where our interpretation (two different abstract tokens) disagrees with the
observations (non-swapping transfers between them). The whole point of
collecting relations is to surface those disagreements — for example as a
graph where an edge between tokens of different abstract tokens is drawn
red, telling a human “these abstract tokens should probably be merged”
(see abstract token merging).
It follows that observation recording must never be gated on the interpretation being consistent. Any design where a token-level conflict can suppress a relation destroys the primary use of relations.
A transfer has a direction; a relation has roles
This is the second load-bearing distinction, and getting it wrong has already cost us one bug:
Which way a transfer moved is a property of the transfer. Which token is escrowed and which is a minted representation is a property of the pair.
An interop transfer travels from a source to a destination — a fact about one transaction, at one moment, in one direction. A relation says two token addresses are the same asset, bridged by a given mechanism. That is not directional in the transfer sense: users bridge both ways over the same route all day, and each of those transfers is evidence of the same relation.
What a lockAndMint relation does have is an asymmetry between the two
tokens: one is locked (escrowed) and released, the other is minted and
burned. Take a lockAndMint bridge between token X on ethereum, which is
escrowed, and token Y on arbitrum, which is its minted representation:
| observed transfer | srcWasBurned | dstWasMinted |
|---|---|---|
| deposit, X → Y | false (X locked) | true (Y minted) |
| withdrawal, Y → X | true (Y burned) | false (X released) |
Both transfers say exactly the same thing about the pair: X is escrowed,
Y is its representation. So the roles are a stable property of the pair,
while the direction is not — and the roles are complementary, so naming
one endpoint names both. That is the one bit of information a relation
needs, and lockedToken is where it lives.
The earlier design instead stored the endpoints in the order the sample transfer happened to travel and let readers infer roles from that order. That silently made every arrow on the graph and every role in the deployed token’s Relations tab a coin flip, and it stored one real relation twice — once per observed direction — in opposite orientations that nothing could tell apart. Do not reintroduce a directional reading of the endpoint columns.
The table
TokenRelation is keyed by the identity of the pair:
(tokenAChain, tokenAAddress, tokenBChain, tokenBAddress,
plugin, bridgeType)
tokenA*/tokenB* are not a direction. They are two slots holding an
unordered pair, always in lexicographic order of (chain, address), so that
a pair has exactly one possible identity and the primary key enforces one row
per pair. They are named A and B rather than from/to precisely so that no
reader can mistake a slot for an origin: A is simply whichever endpoint sorts
first. A CHECK constraint on the table rejects any other order,
which is also how anyone reading the schema learns the columns are not a
direction. normalizeTokenRelation in
TokenRelationRepository.ts
puts a relation into that order, and every write path calls it. Addresses
are stored normalized (lowercase, Address32 cropped to Ethereum
addresses, same normalization as token ingestion).
These columns were once called tokenFrom*/tokenTo*. That spelling survives
in one place and will forever: TokenDbHistory stores the executed command as
an immutable JSON snapshot, so entries recorded before the rename still use the
old field names. The history page reads both spellings for that reason — see
LEGACY_ENDPOINT_FIELDS in
TokenHistoryPage.tsx.
Nothing else should ever accept the old names.
The remaining columns:
-
bridgeTypeisNOT NULL— a relation only exists for non-swapping types, so every row has one. -
lockedTokennames the slot holding the locked token:'A','B', orNULL— aCHAR(1), since one letter says everything there is to say. It is deliberately outside the primary key, which is what makes both of the following possible:NULLcan later be resolved by a plain update, and the two observed directions of one route cannot fragment into two rows.NULLmeans “no endpoint is identified as the locked one”, which covers two cases that are the same thing to every reader, disambiguated bybridgeType:bridgeTypelockedTokenNULLmeansburnAndMintnothing is locked — the pair is symmetric, both sides burn and mint. A terminal value, not a gap. lockAndMintno observation has identified the locked endpoint yet. Resolved as soon as one does. -
transferholds one full sample interop transfer as evidence. It is embedded (not referenced by id) because the interop transfer table is a sliding ~7-day window — the same reasoning as thenon-swapping-transferassignment proof onDeployedToken. The sample keeps its own observed direction; readsrcChain/dstChainfrom inside the JSON when displaying it, never the relation’s endpoint columns.
A pair of identical endpoints (same chain, same address) is not recorded: a token is trivially the same asset as itself, so the row would carry no information, and it has no canonical order.
The table size is bounded by the number of distinct bridged pairs, not by transfer volume.
How relations are ingested
TokenRelationIngestion
(TokenRelationIngestion.ts)
runs as the first step of the same one-minute background loop that drives
automatic token ingestion. The steps run
sequentially — never in parallel — so failures and logs are attributable
to a single step. The order (relations before the queue drain) is not a
correctness requirement, since relations do not depend on the token
catalogue at all; relations simply go first because the step is fast and
bounded while the drain can run long.
The algorithm is deliberately trivial:
- Read the cursor (
token-relations:lastSerialIdinTokenDbSettings) — a separate cursor from the queue pre-step’sinterop-transfers:lastSerialId, so either step can fail without stalling the other. - Page through interop transfers with
serialIdgreater than the cursor, in fixed-size batches, ordered byserialId. Do not replace the paging with one big read: loading full rows for all retained transfers has caused out-of-memory crashes before. - For each transfer: normalize both token addresses (skip if either side
has none, or if both sides are the same token), classify the bridge type
(stored value, or inferred from the burn/mint flags), and keep only
non-swapping types (
lockAndMint,burnAndMint). - Derive
lockedTokenfrom the transfer’s burn/mint flags viaInteropTransferClassifier.inferLockedTransferSide, then normalize the pair into stored order. This is the only place transfer semantics are translated into relation semantics. Downstream code never opens the evidence JSON to work out roles or direction — that is precisely the mistake this step exists to prevent. - For each candidate pair not already present in
TokenRelation, commit anAddTokenRelationCommandthroughcommitTokenChanges— the shared write boundary — so every relation insert lands inTokenDbHistorylike every other TokenDB write. - For each candidate that already exists with
lockedToken = NULL, where this transfer does identify the locked endpoint, commit anUpdateTokenRelationCommandfilling it in. An already-identifiedlockedTokenis never overwritten, so a relation’s role cannot flap. - Advance the cursor after each batch.
There is no staging, approval state, or conflict concept. Relations are observations; there is nothing to approve, and the history table provides the audit trail.
The insert is check-then-insert rather than an upsert so that it can go through the write boundary. The race window is irrelevant: this loop is the only automatic writer and runs serially; if a human inserts the same relation in the same instant, the tick fails loudly and the next tick sees the relation exists.
Why this is NOT part of the token ingestion queue
An earlier version of this feature materialized relations inside the token ingestion processor, as a side effect of processing a queue entry. Do not go back to that design. It was removed for two reasons, and both still apply:
- The queue’s unit of work is the wrong shape. A queue entry is a token address — “there is potentially new knowledge about this address, reprocess it”. A relation is a property of a transfer (a route between two addresses). Deriving relations from an address-keyed queue meant every entry had to re-scan its transfer evidence, deduplicate candidate relations against both endpoints, and thread relation lists through every plan/fetch/apply outcome. It roughly doubled the processor’s complexity for what is, standalone, a ~40-line loop over new transfers.
- Token-level conflicts must never suppress relation evidence. In
the embedded design, relations were only written when a queue entry
reached a successful outcome. Entries that ended in
conflict— which is common in production, and is exactly the situation relations are meant to diagnose — wrote nothing. The result observed in production: the relations most needed for merging wrongly-split abstract tokens were systematically the ones missing. That defeats the purpose of the table.
If some future requirement seems to demand coupling relation creation to token ingestion again, re-read the observation/interpretation model above first: the requirement is almost certainly about interpreting relations, and belongs in a read path, not in ingestion.
Why the burn/mint flags are NOT columns
An earlier version stored sourceWasBurned and destinationWasMinted as
NOT NULL boolean columns, both part of the primary key. This was a bug;
do not add them back as columns.
The interop transfer table’s srcWasBurned / dstWasMinted are
nullable. Null means “we did not observe this side” — routinely the case
for one-sided transfers, where only the source or only the destination
event was captured. In production, ~85% of stored-lockAndMint transfers
have at least one of these flags null. The old code coerced null to
false (transfer.srcWasBurned ?? false), which was wrong in three ways:
- It fabricated observations.
falseasserts “we saw that it was not burned”. We saw no such thing. In a table whose entire justification is “relations are facts”, inventing a fact is the cardinal sin. - It self-contradicted. A stored-
lockAndMinttransfer with both flags null became a row withbridgeType = lockAndMintand flags(false, false)— flags the classifier itself reads asnonMinting. - It fragmented routes. Because the flags were in the primary key, one real-world route observed once via a two-sided transfer and once via a one-sided one produced two rows, split by an artifact of which events happened to be indexed rather than by anything on-chain.
The fix: the flags are not relation columns at all. A relation’s identity
is (pair, plugin, bridgeType), and bridgeType is the authoritative,
plugin-declared (or, absent that, inferred) classification of the bridge’s
mechanism. The honestly-observed flags are not lost — they remain in the
transfer evidence JSON exactly as seen: present when observed, absent
when not. No tri-state column, nothing to fabricate, no pair
fragmentation.
Note this means a stored bridgeType is trusted even when the flags are
unobserved — deliberately, and consistently with how the token ingestion
processor already trusts a stored bridgeType as non-swapping-transfer
assignment evidence. Demanding observed flags instead would drop the
~80k+ one-sided non-swapping transfers and recreate the very
“missing relations” problem this subsystem exists to fix.
The flags are also not copied verbatim onto the relation under new names, which is the other tempting shortcut. Two reasons:
- They are transfer-shaped, not pair-shaped.
srcWasBurnedis a fact about one leg of one transaction; a relation summarizes an unbounded set of transfers over a pair. Copying one sample’s flags onto the row would assert that they are a property of the pair. - They carry no information the relation needs beyond
lockedToken. Two nullable booleans encode nine states, of which only two identify a locked endpoint for alockAndMintpair — and every read site would have to re-run that truth table.lockedTokenis that truth table applied once, at ingestion.
Nothing is lost by not copying them: the flags stay in the evidence JSON verbatim, so any future question about a specific sample can still be answered from there. A future question about the pair should get its own pair-shaped column derived at ingestion, not a raw transfer field.
Why there are NO foreign keys to DeployedToken
TokenRelation.tokenA* / tokenB* deliberately do not reference
DeployedToken. This is not an oversight — the constraints existed and
were removed. Do not “fix” the schema by adding them back without
re-reading this section.
- Relations must be recordable before their endpoints are catalogued. The typical case: a transfer reveals a brand-new token, but cataloguing it hits an ingestion conflict that takes a human days or weeks to resolve. The relation observation is valid the whole time. With enforced foreign keys it cannot be stored, and by the time the conflict is resolved the source transfers may have aged out of the 7-day retention window — the evidence is gone forever. Without the constraints, the edges are already sitting in the table when the token is finally added; the moment it appears, its graph neighborhood is complete. The alternative (skip un-insertable relations and re-scan history later) is confusing — a freshly resolved token would appear with zero edges despite transfers having driven its creation — and fixing that requires a deferred-relations side table or periodic re-scans: real machinery to reproduce what “no constraint” gives for free.
- Postgres has no partial foreign key. A foreign key on non-nullable
columns strictly requires the referenced row to exist at insert time
(
NOT VALIDonly skips validating pre-existing rows). The choice is binary: either the constraint blocks relations for unknown tokens, or there is no constraint. - The constraints provided no query capability. All queries are
handwritten Kysely joining on
(chain, address); the two endpoint indexes serve them. Prisma relation fields were only used for migrations, not queries.
What is given up, honestly:
- No database-level guarantee that a relation’s endpoints exist as deployed tokens. Endpoint existence is resolved at read time (see below).
- No
RESTRICTprotection when deleting a deployed token — which is actually the semantics we want (next section). - A garbage address in a transfer would be persisted. Mitigated by the same address normalization token ingestion uses, and bounded by the primary key (one row per pair).
Deleting a deployed token leaves its relations in place
Deleting a DeployedToken does not delete relations that mention its
address (the user planner used to cascade-delete them when the foreign
keys demanded it; it no longer does). The transfers still happened —
deleting the catalogue entry does not un-happen them. The relation simply
degrades to mentioning an uncatalogued address, and if the token is ever
re-added its edges are intact. Bogus relations can still be deleted
individually via the relation delete intent.
Display implications
Because endpoints may be uncatalogued, read paths that show relations
resolve endpoints against DeployedToken at query time and must tolerate
a miss. The deployed-token getRelations endpoint returns
otherToken: null for unknown endpoints and the UI renders the raw
address instead of a token link.
The deployed-token set is small enough to resolve in memory; this small read-time cost is the entire price paid for the foreign-key decision above.
getRelations returns one flat list, not an inbound/outbound split: the
endpoint columns are not a direction, so there is nothing to split on. Each
entry instead carries the queried token’s role in that relation, derived
from bridgeType and lockedToken:
role | meaning |
|---|---|
locked | this token is escrowed; the other is its minted representation |
minted | this token is minted by the plugin: the representation side of a lockAndMint pair, or either side of a burnAndMint pair |
unknown | a lockAndMint pair whose locked endpoint is not identified |
There is deliberately no symmetric role (there used to be one, shown for
burnAndMint pairs). A burnAndMint pair is symmetric, but from each
endpoint’s point of view that fact reads “minted” — the question the role
answers — and the bridge type, shown alongside, is what carries the
symmetry. A relation that is neither burnAndMint nor lockAndMint (a
human-added nonMinting row; ingestion never writes one) mints nothing and
shows unknown.
This is the answer to “which plugin minted this token, and which token is it
a representation of” — the question the Relations tab exists for. Read it
from role, never by comparing the endpoint columns.
The narrower question “which plugins mint this token” — asked by the public
frontend, which reads the token database directly rather than through
token-backend — is answered by
TokenRelationRepository.getMintingPluginsFor: the distinct plugins of the
relations where the token’s role is minted. Deliberately excluded: relations
where the token is locked, relations with an unknown role (one of their
endpoints is minted, but nothing says it is this one), and human-added
nonMinting relations, which mint nothing. Token-UI exposes the same query
as deployedTokens.getMintingPlugins and shows the list above the Relations
table, so the summary can be eyeballed against the roles in the table.
Relations graph
The graph page in token-ui is a view of the relation observations resolved
against the current token catalogue. Every observed (chain, address)
endpoint is a node, including endpoints that do not yet have a
DeployedToken row. Catalogued nodes are green and labelled on two lines with
their deployed token symbol and chain; uncatalogued nodes are orange and use a
shortened address followed by their chain. An edge is an observed token
relation: burn-and-mint edges are blue and non-directional, while lock-and-mint
edges are pink and carry an arrowhead pointing from the locked token to the
minted one. The arrow follows lockedToken, never the endpoint column order
— that order is lexicographic and says nothing about roles. A lock-and-mint
edge whose lockedToken is unidentified is drawn without an arrowhead rather
than guessing which token is the original. Nodes can be dragged and the canvas
can be panned or zoomed. Edge stroke widths remain constant while zooming, and
node visuals stop growing beyond 2x zoom so additional zoom creates useful
space between them. Above 2.5x zoom, each edge shows its relation plugin name at
its midpoint.
Before drawing, the UI treats every connected component as a cluster and sorts the clusters by endpoint count (largest first, with a stable id tie-break). Each cluster gets its own force simulation, which is run to completion in memory so clusters do not repel each other and users never see the graph settle. The finished clusters are placed left-to-right in a square-ish grid, starting at the top-left, then the whole grid is fitted into the viewport. At low zoom levels each cluster is overlaid with its most common catalogued deployed-token symbol. The overlay stays readable through the mid-zoom range, then shrinks and fades at extreme zoom-out to avoid overlapping nearby cluster labels.
Clicking a node keeps the node, its incident edges, and its neighbors prominent while dimming the rest of the graph. A non-modal details panel loads that one deployed token and its abstract token on demand; the initial graph payload does not contain full token records. The panel also lists the relations already present in the graph — one list, each entry labelled with the selected token’s role — rather than issuing a second database query for the neighborhood. Uncatalogued nodes show their raw endpoint information instead of token details.
Edges are independently hoverable and clickable. Clicking one highlights its two endpoints and loads only that relation’s full transfer evidence, including source and destination transaction hashes used for explorer links. This keeps the evidence JSON out of the initial graph response. The evidence panel labels those hashes with the chains recorded inside the evidence, because the sample transfer keeps its own observed direction, which the relation’s endpoint order does not describe.
The relation panel also carries a delete button — the tool for removing a
bogus observation that a buggy plugin’s interop transfer ingested. Any
relation can be deleted this way, not only anomalies. The button goes through
the standard delete intent (see Human edits), so the user
confirms a plan first; the confirmation notes that the executed command lands
in TokenDbHistory with the full removed record, from which the relation can
be recovered in the worst case. On success the edge simply disappears from the
drawing and from the panel’s relation lists. The layout is deliberately not
re-run. Removing an edge can split a cluster in two, and re-clustering —
new cluster grid, new cluster labels, reset viewport — would yank the graph
out from under a user mid-investigation on a view that takes seconds to
build. Refreshing the page is how one sees the re-clustered graph. For the
same reason, the graph query is never refetched automatically while the page
is open — executing a plan only marks it stale without refetching active
instances, and the page opts out of the window-focus and reconnect refetches
that would otherwise pick that staleness up — so fresh data loads only on
the next visit to the page.
The graph header can search catalogued deployed tokens by symbol, chain, or address using the already-loaded graph payload. Choosing a result selects the node, opens its existing details panel, and animates the viewport to a readable zoom around it. Full token and abstract-token details remain selection-time queries rather than being added to the initial payload.
An edge is an assignment anomaly when both endpoints are assigned to abstract tokens and those abstract token IDs differ. An unassigned or uncatalogued endpoint is not considered an anomaly. The default view keeps the bridge-type colors and does not draw anomalies red. An anomaly switch changes conflicting edges to red and mutes other edges to gray, so anomaly inspection does not compete with the default bridge-mechanism view.
The initial graph query reads only relation identity fields, lockedToken,
and the minimal endpoint display data. It deliberately excludes full
deployed/abstract token records and the transfer evidence JSON; dedicated
selection-time queries fetch one node or one relation detail record when
requested. lockedToken is in that payload precisely so the graph never has
to open the evidence to know which way an arrow points.
Human edits
Humans can add, update, and delete relations through the standard
intent → plan → execute pipeline. The add intent still validates that
both endpoints exist as deployed tokens — a human hand-typing a relation
to an uncatalogued address is almost certainly a mistake, while the
ingestion loop observing one is the whole point. Validation belongs to
the pipeline, not the storage.
The add planner normalizes the record before storing it: a human names the
two endpoints in whichever order they happen to think of them, and the pair
is unordered, so the stored order is derived rather than taken. lockedToken
moves with the endpoints when they are swapped, so the role a human stated is
preserved. The update intent can also set lockedToken directly, which is
how a human corrects a role the flags got wrong.
Known limitations
- The serial-id cursor can permanently skip a transfer whose row committed out of order (same accepted risk as the queue pre-step’s cursor).
- Relations only capture routes observed while the loop runs; transfers that aged out of the ~7-day retention before the loop first ran are not represented.
- A
lockAndMintrelation can sit atlockedToken = NULLindefinitely if no transfer on that route ever identifies a side. Two things cause this: a plugin that declaresbridgeType: 'lockAndMint'while its burn/mint flags stay unobserved, and a plugin whose declaredlockAndMintcontradicts the flags it did observe (both sides locked, or both supply-changing). Such relations are still recorded — identity is the table’s primary purpose and matters more than the role — they just display asunknown. The ingestion log’sresolvedLockedTokenscounter shows how often this self-corrects.
All are acceptable on a living system: active routes recur, and a missed observation is re-created by the next transfer on the same route.