Match thresholds
Datum finds duplicates by scoring every plausible pair of records. Model › Match thresholds controls what happens to each score. It answers one question: where does the line fall between an automatic merge, a steward review and a silent rejection?
The match model in force for Party organisation: merge at 99%, review down to 35%.
The number is a probability, not a similarity
Read the percentages as probabilities. A pair scoring 92.5% is not "92.5% similar"; it is a pair the model believes is a true duplicate with about that probability, built from the evidence on each attribute. The model is measured against a labelled set of pairs whose answers are known whenever it changes, which is why the screen can report precision and recall (see Calibration).
The three bands
| Score | What happens | Where it goes |
|---|---|---|
| At or above the merge threshold (here 99.00%) | The records are merged without a steward seeing the pair. | Straight to the golden record. |
| Between the review floor and the merge threshold (35.00%–99.00%) | A person decides. | The Match review queue. |
| Below the review floor (35.00%) | The pair is rejected without a steward seeing it. | Nowhere — recorded as not a match. |
Raise the merge threshold and fewer pairs merge by themselves, so more reach stewards: safer, slower. Lower the review floor and more weak pairs reach the queue, which teaches reviewers to reject by reflex. Raise it and a true duplicate that scores low is rejected where no steward ever sees it, and stays in your data. That last failure is silent, so be most careful with this number.
Choose the entity in the Entity menu at the top. Each has its own model.
The cluster guard
Merging is transitive: if A matches B and B matches C, they end up in one cluster even if A and C look nothing alike. The cluster guard stops runaway chains:
| Setting | What it limits |
|---|---|
| Largest cluster | The most records one cluster may hold. |
| Minimum cohesion | How tightly connected the members must be. |
| Maximum diameter | The longest chain of links between any two members. |
A cluster that breaks the guard is suspended, not merged, and appears in the Suspended clusters queue for a steward.
The scoring parameters appear as a short digest (7fa24f…). A change to thresholds and guards carries them across unchanged, which is what keeps every stored score valid and every decided pair decided.
What the model compares
Choose the Comparisons icon (the second of the three at the top right).
What this model compares. Each attribute has a role and several levels; each level has a weight.
| Role | Meaning |
|---|---|
| Name | Compared as a name: exact, near, token overlap. |
| Identifier | A strong identifier: exact agreement bypasses scoring. |
| Discriminator | Disagreement counts against. |
| Supporting | Adds weight when it agrees; plays a smaller part. |
| Blocking only | Used to choose which pairs to compare, never to score them. |
The weight is the evidence, in base-2 log-odds. Positive pushes towards a match, negative against, and the bar beside it shows the size. In the screenshot, an exact registration number is worth +16.54, and a disagreeing one −4.32. "term-frequency adjusted" means an agreement on a rare word counts for more than on a common one.
Corporate groups produce genuinely different entities with near-identical names. Acme Holdings Ltd in Great Britain and Acme Holdings GmbH in Germany are rejected automatically despite an identical name, because disagreement on the legal-form suffix is strong negative evidence. Name similarity alone must never be enough to merge two organisations.
For the platform's own entities this view is read-only: the comparisons are defined in code and measured against a recorded baseline. A firm's own entity compares on whichever match roles its own attributes declare, and for these you can use Change what is compared to choose, per attribute, a method: As a name, As an identifier, As a discriminator, As supporting evidence or Not compared. You choose the method, never the numbers: the weights belong to the platform, because setting them by hand is tuning a model nobody has measured.
Normalisation
Choose the third icon, Normalisation. Before names are compared, Datum standardises them: LIMITED, LTD and LTD. all become LTD.
What the platform ships (214 rules) and This firm's rules, currently empty.
There are four kinds of rule:
| Kind | Does |
|---|---|
| Synonym | Treats two words as one (for example a firm's own abbreviation). |
| Stopword | Ignores a word in comparison. |
| Suffix | Standardises a legal-form ending to a canonical code. |
| Qualifier | Recognises a qualifying pattern in a name, so that it can be compared separately from the name itself. |
The platform's rules are reference data about company law across jurisdictions, not the firm's opinion, so they cannot be changed. A firm's rules add to them. Choose Propose change under This firm's rules, add rules (Find, Replace with, Priority), and choose Measure it.
The number that decides whether the change is safe is how many blocking keys it moves, and nobody can read that off the rules. If the draft moves any, the screen says so: "A pair that is never compared is the one failure no queue reports", and activating needs re-authentication. Discarding the draft afterwards does not put the keys back. If nothing has been loaded yet there is nothing to measure against and the screen says so.
The measurement also lists names that would normalise differently: As it arrived, Now, Would become.
Proposing a threshold change
- Choose Propose change at the top right of the Thresholds view.
The proposal form is pre-filled with the values in force.
- Give a new version identifier, change the values you need, and say why.
- Choose Propose change at the bottom. The draft appears under Drafts.
- Open the draft and choose Measure this change.
- Read the report (below). If it is what you intended, choose Activate. If it narrows oversight you will be asked to re-authenticate.
Reading the measurement
The measurement replays your proposal over the scored pairs held (for example 5,305 scored pairs available to measure against) and reports:
- Direction. Widens oversight (more decisions reach a steward), Narrows oversight (fewer do), Mixed or Unchanged.
- Decisions that would move. Pairs moving between Automatic merge, Steward review and Automatic rejection.
- Match review, once put into service. Pairs joining and leaving the queue. Pairs already there keep their place, age and deferral.
- Records moving cluster, and Master identifiers: how many would be retired and how many newly minted. A retired identifier is never handed back, not even by an unmerge, so every system holding a copy must repair its reference. No identifier would be retired or minted is the answer you want when changing thresholds lightly.
- Clusters the guard would suspend.
Calibration
Against labelled pairs the report gives Merge precision (of the pairs merged automatically, how many were right), Merge recall and Wrong merges, over n labelled pairs. If there are too few, the screen says "Below the … labelled pairs a precision figure needs to be worth quoting" rather than print a number it cannot defend.
Activating
| Direction | Re-authentication? |
|---|---|
| Narrows oversight (fewer pairs to a steward) | Yes. |
| Widens oversight (more pairs to a steward) | No. |
After activation the platform re-applies the new version to pairs already scored, in the background, so the review queue changes when that has run rather than at once.
Who can do what
Proposing, measuring and activating need the stewardship class. Without it the screen shows what is in force and says so.
A change of a point or two is usually safe and easily measured. Resist large changes on the strength of a single awkward pair; fix that with a comparison or a survivorship rule instead.