Independent synthetic data validation
Synthetic Data Validation Report
Full assessment — synthetic data against a real sample (Mode 1)
- Job
- e2e-credit-g-synthetic-holdout
- Category
- other
- Issued
- 2026-08-16 09:09 UTC
- Synthetic dataset
- 700 rows × 21 columns
- Real sample
- 700 rows × 21 columns
- Holdout sample
- 300 rows × 21 columns
Weighted across privacy (40%), fidelity (30%) and utility (30%). Structural integrity is reported separately as a precondition check.
Fidelity
Statistical similarity to the source data
Overall distributional similarity scores 93/100. Weakest match: duration x purpose at 70/100.
| Measurement | Value | Score | What it means |
|---|---|---|---|
| Column shapes | 0.968 score | 97 | How closely each column's marginal distribution matches the source (KS complement for numeric, total-variation complement for categorical). |
| Column pair trends | 0.9 score | 90 | Whether relationships between pairs of columns survived generation — correlations for numeric pairs, contingency for categorical pairs. |
Privacy
Resistance to re-identification attacks
Privacy scores 96/100 (full attack suite). Highest residual risk: singling out at 83/100.
| Measurement | Value | Score | What it means |
|---|---|---|---|
| Exact record copies | 0.00% % of synthetic rows | 100 | Share of synthetic rows that are byte-for-byte copies of a real record. Any non-zero value means real records were reproduced verbatim. Reported next to the real sample's own duplicate rate, because repeated rows arise by coincidence in low-cardinality data. Near copies — within 0.05 embedded distance of a real record — are reported alongside, against the same measurement between two halves of the real sample. |
| Nearest-neighbour distance | 1.219 ratio vs real-to-real baseline | 100 | Distance from synthetic records to their closest real record, divided by the same distance measured between two independent halves of the real sample. 1.0 or above means synthetic records are no closer to the source than unrelated real records are. |
| Nearest-neighbour distance ratio | -0.29 percentage points above the real-to-real baseline | — | A row is 'anchored' when its closest real record is less than 0.5x the distance of its second closest — it sits on one individual rather than between neighbours. Reported as the excess over how often real records anchor to each other, because that rate is far from zero on ordinary data. Not scored: it flags individual rows that a percentile averages away. |
| Membership inference | 0.053 attacker advantage | 95 | How much better than guessing an attacker does at deciding whether a given record was in the training data. 0 is a coin flip; 1 is perfect identification. |
| Singling out | 0.166 residual risk | 83 | Whether the synthetic data lets an attacker build a query that isolates exactly one real individual (Article 29 WP criterion 1). Measured against the holdout you supplied, so this is residual risk — the exposure attributable to the synthetic data itself. Measured risk falls between 0.106 and 0.226 at 95% confidence; the attack samples, so this interval is the measurement. |
| Linkability | 0.01 residual risk | 99 | Whether two separate datasets about the same people can be joined through the synthetic data (Article 29 WP criterion 2). Reported for the worst of 3 column splits. Measured against the holdout you supplied, so this is residual risk — the exposure attributable to the synthetic data itself. Measured risk falls between 0.000 and 0.043 at 95% confidence; the attack samples, so this interval is the measurement. |
| Attribute inference | 0.013 residual risk | 99 | Whether an attacker who knows some attributes of a real person can infer an unknown one from the synthetic data (Article 29 WP criterion 3). Averaged across columns. Measured against the holdout you supplied, so this is residual risk — the exposure attributable to the synthetic data itself. The worst column (employment) falls between 0.000 and 0.160 at 95% confidence; the attack samples, so this interval is the measurement. |
Utility
Train on synthetic, test on real
Models trained on the synthetic data retain 52% of the skill of models trained on real data, averaged over 3 target column(s).
| Measurement | Value | Score | What it means |
|---|---|---|---|
| TSTR — class | 0.596 roc_auc | 39 | Model trained on synthetic data scores 0.596 (roc_auc) on held-out real data, against 0.747 for the same model trained on real data (chance: 0.500). |
| TSTR — foreign_worker | 0.586 roc_auc | 60 | Model trained on synthetic data scores 0.586 (roc_auc) on held-out real data, against 0.644 for the same model trained on real data (chance: 0.500). |
| TSTR — own_telephone | 0.639 roc_auc | 57 | Model trained on synthetic data scores 0.639 (roc_auc) on held-out real data, against 0.745 for the same model trained on real data (chance: 0.500). |
Structural integrity
Internal consistency of the synthetic data
Structural integrity scores 99/100.
| Measurement | Value | Score | What it means |
|---|---|---|---|
| Duplicate rows | 0.00% % of rows | 100 | Repeated identical rows inside the synthetic data. A high rate means the generator collapsed onto a few records. |
| Missing values | 0.00% % of cells | 100 | Share of empty cells across the synthetic data. |
| Constant columns | 0.00% % of columns | 100 | Columns where every row holds the same value. These carry no information and usually indicate a generation failure. |
| Extreme values | 0.19% % of numeric cells | 96 | Values more than 4 standard deviations from their column mean — often impossible values invented by the generator. |
| Category collapse | 0.00% % of categorical columns | 100 | Categorical columns where a single value covers more than 95% of rows. Legitimate for genuinely skewed fields, a red flag when it appears across many columns. |
Findings
What the scores above mean in practice: where each weakness sits and the general direction of a fix. Findings are derived from the measurements and shape statistics in this report — not from the data, which is why they survive the deletion recorded in the footer.
Material
1Weaknesses worth acting on before this dataset is relied upon.
Individual records can be isolated by a query
purpose, credit_amount, age, duration — the columns the successful queries used most
Measured risk 0.17 on a 0–1 scale (Singling out), 95% confidence interval 0.11–0.23.
What to look at: Article 29 WP criterion 1. An attacker who can write a query matching exactly one person has singled them out, whether or not they learn a name. Reducing the precision of the columns that make records unique — coarsening dates to years, grouping small categories, banding rare numeric values — is the standard direction of travel.
Method and assumptions
- Fidelity uses Kolmogorov–Smirnov and total-variation complements per column, plus correlation and contingency similarity for every column pair.
- Privacy combines exact-copy detection, nearest-neighbour distance against a real-to-real baseline, a per-row distance ratio, membership inference, and the singling-out, linkability and attribute-inference attacks.
- Utility trains gradient-boosted models on the synthetic data and evaluates them on held-out real data, measured against the same model trained on real data and against chance.
- Attack metrics used the 300-row holdout you supplied as the control group, on the stated basis that those records were excluded from your generator's training data. The reported risks are therefore residual risk: exposure attributable to the synthetic data rather than to what any attacker could guess.
Verification
Scores are bound to the exact files measured by their SHA-256 fingerprints. Hash your copy of a file to confirm it is the one this report describes — the fingerprints outlive the source data, which is deleted.
- Synthetic dataset SHA-256
- e2536b1f45cefda02e86c4bc646ac5955358c6884181f6dd9cc653ab1b7baa55
- Real sample SHA-256
- 2d009642b66a86a2c336be538c353715bf8def674ecbfe2f83fc8a71cae2cb70
- Holdout sample SHA-256
- 634de9c8978d9e3ed252bb64080e681cf0f7640f8aad85fb7dc28c4cb7c7be94
- Metrics engine
- v0.6.0
- Benchmark snapshot
- not applicable — benchmarking not enabled
- Source data
- deletion pending
This report is independent third-party evidence intended to support your Data Protection Impact Assessment or anonymisation assessment. It is not a legal determination, a certification, or a statement that any dataset is compliant with the GDPR or any other regulation. Scores are reproducible from the metrics engine version stamped above.