Suggestions

Why a separate tab

Mapping 1,000 local codes by hand against the 4 million concepts of the OMOP vocabulary is slow. Each algorithm family has its own bias:

• String comparison misses synonyms (“HR” vs “heart rate”).
• Semantic embeddings occasionally confuse close-but-different concepts (“serum creatinine” vs “urine creatinine”).
• An AI agent adds clinical context but is expensive to run over 4 million pairs.

The Suggestions tab combines several methods into a single score and surfaces the per-method details, so you keep the final decision.

Where suggestions come from

The two complement each other: the agent reads the precomputed scores when there are any, and uses them to find its candidates faster — especially for drugs.

The Suggestions tab in Linkr

Similarity methods

Four method families can feed the candidate ranking. Each attacks the problem from a different angle, and their failure modes don’t overlap — which is why combining them beats any single family.

In Linkr, every method is represented next to the combined score by a coloured dot (colours above) followed by its percentage. On hover, a tooltip names the exact algorithm. This helps understand why a score is high: a candidate scoring 92% from the syntactic method alone is likely a textual false friend, while one scoring 80% from semantic + AI is much more solid.

1. Syntactic — compare character strings

We compare label strings as-is, without any understanding of their meaning. Useful because source and OMOP labels often follow different writing conventions: case, spaces or underscores, accents, abbreviations, word order, punctuation. Three algorithms in the catalogue:

• Jaro-Winkler — letter-level proximity with a bonus for shared prefixes. Matches “hemoglobin_a1c” with “Hemoglobin A1c” despite the case and separator differences.
• Token-Sort — splits labels into words, sorts them alphabetically, then compares. Word-order agnostic (“blood pressure systolic” ≈ “systolic blood pressure”).
• N-gram IDF — splits into character n-grams (subsequences of N letters), weights each n-gram by its rarity in the vocabulary. Usagi’s historical method.

Strengths — catches near-literal matches despite writing variants (case, spaces, abbreviations, word order), very fast (milliseconds per pair).

Limits — does not account for synonyms: “creatinine” and “creat” are close, but “heart rate” and “pulse” score zero.

2. Semantic — compare meaning

Each label is encoded as a numerical vector through a sentence-embedding model, then we measure the angle between vectors via cosine similarity. Two labels with close meanings will produce close vectors, even if the words differ.

Default model: BioLORD-2023-M, trained on biomedical text and multilingual (French, English, Spanish, German…).

Configurable alternatives — any Sentence-Transformers compatible BERT model:

• PubMedBERT, SapBERT, ClinicalBERT — biomedical-specialised but English-only.
• all-MiniLM-L6-v2 — general-purpose, English-only, much faster.
• paraphrase-multilingual-MiniLM-L12-v2, LaBSE — general-purpose, multilingual.

Strengths — catches synonyms (“heart rate” ≈ “pulse”), rephrasings, terminology variants.

Limits — can conflate two close-but-distinct concepts (“serum creatinine” vs “urine creatinine”). Expensive to pre-compute over the ~4M OMOP concepts (GPU recommended).

3. Statistical — compare value distributions

For numerical variables (Measurement only), we compare the statistical distribution of values between source concept and candidates: min, P25, median, P75, max, mean, standard deviation. Two variables taking values in the same range with the same shape are probably the same measurement.

Concretely, we use a statistical test to quantify the gap between the two distributions:

• Kolmogorov-Smirnov test (KS) — maximum gap between cumulative distribution functions. KS = 0 → identical distributions; KS close to 1 → very different distributions.
• Wasserstein distance — “cost” of transporting the mass of one distribution onto the other. More robust than KS when supports are shifted (unit change).

Strengths — works even when the label is ambiguous or missing. Very discriminating: hard to confuse creatinine (60–120 µmol/L) with sodium (135–145 mmol/L).

Limits — only applicable to numerical variables with enough values for the distribution to be representative. Sensitive to data-entry artifacts (outliers, mixed units).

4. Agentic AI — let an LLM reason about it

An LLM driven agentically — it decides which tools to call: reading a source concept and its metadata, searching the OMOP vocabulary, browsing a candidate’s hierarchy — examines the best candidates with all the context available: source label, unit, value distribution, wards where the code is used. It proposes one or more mappings carrying a SKOS equivalence (exactMatch, closeMatch, broadMatch, narrowMatch, relatedMatch) and a comment justifying the choice.

Its proposals show up in the tab as one more method, named ai/<model> after the model that produced them, in the Agentic category. The Mapping with an AI agent page explains how to start it.

Strengths — the only method that “understands” clinical semantics (qualifiers, measurement context, nuanced equivalences). Can say “this is not exact, the target is more general” via broadMatch, and says in its comment what differs.

Limits — slow and compute-hungry, risk of hallucinations on rare cases, still requires human validation.

The combined score

The combined score (left column, green bar) is a weighted average of each method’s score. Default weights are:

The Manage suggestions button (top left) opens a dialog with sliders to tune each weight in real time — the table re-sorts automatically. It also shows the number of loaded scores, and lets you Import a file or Delete all.

In server mode, the Methods in the file section lists every method present — syntactic/jaro-winkler, semantic/biolord, ai/<model>… — each with a bin: Remove the suggestions of a method deletes all its rows from the scores file, keeping the other methods and the mappings. This is how you remove in one go what an agent left.

Selection and mapping

• Click a row to select it. The action bar at the top then shows the usual mapping buttons (Exact match + SKOS dropdown + comment button).
• Candidates already mapped for this source concept are greyed out and not re-selectable.
• The ⓘ icon at the row end opens the target OMOP concept’s detail (description, synonyms, hierarchy).

The main alignment button “Exact” is pre-filled with the selected suggestion’s SKOS equivalence — the one the agent proposed, for an ai/… row (exactMatch, closeMatch, broadMatch, narrowMatch, relatedMatch). You can accept it in one click, or open the dropdown to pick a different equivalence if you disagree. Linkr never maps on its own from a suggestion: that click is what creates the mapping.

AI agent suggestions

An AI agent can leave its own suggestions in the tab, each with a SKOS equivalence and a comment justifying the choice. The Mapping with an AI agent page explains how to put it to work, what it writes into the project and how to follow its work.

Precomputing the scores

Syntactic (Jaro-Winkler) and semantic (BioLORD embeddings) scores give every source concept a ranked list of candidates before any agent looks at it. They are computed outside Linkr, in a terminal, with the two Python scripts that ship with the skill, in its scripts/ folder. The resulting file is then loaded into the project; this part also works in an install without a backend.

Requirements

The two scripts

• embed_concepts.py — computes BioLORD embeddings of the OMOP concepts and writes concept_embeddings.parquet next to CONCEPT.parquet. Run it once per vocabulary release: the file serves every project.
• compute_scores.py — compares a project’s source concepts with the OMOP concepts and writes similarity-scores.parquet. Run it once per project. By default it computes Jaro-Winkler and BioLORD; without concept_embeddings.parquet, only the syntactic score.

The scoring script expects a source concepts CSV with three columns: terminology, concept_code and concept_name. The source-concepts.csv from a Linkr ZIP export of the project does the job, with its columns renamed if needed.

python scripts/compute_scores.py \
  --source source-concepts.csv \
  --concept /path/to/vocabulary/CONCEPT.parquet \
  --embeddings /path/to/vocabulary/concept_embeddings.parquet \
  --only-standard --only-valid --domain Measurement Observation

Run it in your own terminal, where you can watch memory and stop it with Ctrl-C. Each script lists all its options with --help.

Loading the scores into Linkr

In the mapping editor, Suggestions mode: Manage suggestions → Import a file, then pick similarity-scores.parquet.

Output files

The scoring script writes two files:

• source_embeddings.parquet — the embedding vectors of the source concepts. Kept so the embeddings can be reused on the next run without re-encoding everything.
• similarity-scores.parquet — one score per (source, candidate, method) triple. This is the file you load into Linkr, and also where Linkr stores the agents’ suggestions.

Preview of source_embeddings.parquet:

Preview of similarity-scores.parquet for one source concept (invasive systolic BP), after an agent’s pass:

Two more columns complete the file: concept_set_uid and concept_set_source_repo, filled when a suggestion was taken from a data-dictionary concept set.

Resume on interrupt

The scripts write their output incrementally, so a subsequent run resumes where the previous one stopped:

• embed_concepts.py checkpoints every 50 batches (~25,000 concepts) — you can interrupt without losing work.
• compute_scores.py checkpoints every 100 source concepts — (source_vocabulary_id, source_concept_code) pairs already scored are skipped on restart.

This matters for complete vocabularies (4M concepts) that can take hours.

Going further

• Mapping with an AI agent — putting an agent to work, following and reviewing its suggestions.
• Agents — what an agent can do in Linkr, and how you stay in control.
• MCP server — create your API key and connect LibreChat or OpenCode.
• Evaluation — review and validate mappings, including those created by an agent.