Correlation preservation

Correlation preservation is where fidelity actually gets tested. Column shape asks whether each variable looks right on its own; correlation preservation asks whether the relationships between variables survived. It is the difference between a synthetic dataset that has the right ingredients and one that has the right recipe — and it is where most fidelity failures that matter in practice actually occur.

Definition

For every pair of columns, measure the strength of association in the real data and in the synthetic data, and compare. The comparison across all pairs — as a matrix, and as a summary score — is correlation preservation.

"Association" needs a different measure per pair type:

• Numeric–numeric: Pearson correlation for linear relationships, or Spearman rank correlation for monotone ones. Spearman is more robust to the outliers and non-linearities common in real data.
• Categorical–categorical: a contingency-based measure such as Cramér's V, or the difference between the real and synthetic contingency tables (the total variation distance between joint category frequencies).
• Numeric–categorical: the correlation ratio, or a comparison of the numeric column's distribution within each category — does synthetic income conditional on region look like real income conditional on region?

Each pair yields a real value and a synthetic value; the pair's score is a function of their difference (typically 1 minus the normalised absolute difference). The fidelity report shows the two matrices side by side, or their difference, and an average.

The two failure modes — and why both are findings

Collapsed correlations. The real data has a strong relationship (age and income at 0.62); the synthetic data has almost none (0.11). The generator learned each column but not their dependency. This is the classic failure of simpler generators and of any generator that treats columns as nearly independent. Its cost is paid downstream: every predictive task that relied on that relationship loses signal, and TSTR falls accordingly.

Invented correlations. The real data has no relationship between two columns; the synthetic data has one. This is subtler and easy to wave away as noise — but it isn't. It means the generator hallucinated structure, and a model trained on the synthetic data will learn a dependency that does not exist in reality. Anyone analysing the synthetic data will draw a conclusion that is simply false. Some quality frameworks apply a threshold below which correlation differences are ignored; that hides exactly this failure. A validation engine should report invented correlations as findings in their own right, not round them to zero.

Both directions matter equally, and the per-pair table is where you see which. The average score is a headline; the pairs with the largest real–synthetic gap are the information.

What a bad score usually means

The generator's model family can't represent the dependency. Marginal-first generators, and some copula and tree-based approaches, capture linear or monotone relationships and miss conditional, non-linear or interaction effects. The signature is: pairwise numeric correlations look decent, categorical–categorical and numeric–categorical pairs collapse.

Too little training data for the joint structure. With few rows, the joint distribution is undersampled and the generator learns marginals reliably and dependencies noisily — both collapsing real ones and inventing spurious ones.

Over-regularisation or a tight privacy budget. Some generators trade dependency fidelity for privacy on purpose. This is a legitimate trade, and the report should let you see its size: correlation preservation is where the utility cost of a privacy setting first shows.

Rare-category conditionals lost. The relationship holds in the bulk and vanishes for rare categories — income by region is right for the five common regions and random for the twenty rare ones. Aggregate correlation looks fine; conditional comparison per category exposes it. This is the same root cause as under-represented classes hurting utility.

The remedy is generator-agnostic: choose a model with explicit multivariate structure, condition generation on the variables whose relationships matter most, and, for privacy-driven collapse, decide consciously how much dependency fidelity you are willing to spend.

Reading correlation preservation alongside the other pillars

The most useful pattern is the gap between this metric and column shape. Column shape passes, correlation preservation fails: the generator got the ingredients and not the recipe, and utility on any relationship-dependent task will suffer. That gap should be reported explicitly, because it points straight at the cause.

The other pattern is correlation preservation too good. A generator that memorised its training data reproduces every correlation exactly, including the noise. Very high correlation preservation, like very high column shape, is consistent with both a faithful generator and a copying one — read it with the privacy pillar (DCR, membership inference) before it means anything.

Worked example

UCI Adult census, 4,200 rows × 15 columns, SDV Gaussian Copula, metrics engine 0.6.0. Column pair trends score 80.9/100 against 88.7/100 for column shapes — the fidelity pillar lands at 84.8 between them.

That eight-point gap is the finding. Marginals survived generation better than the relationships between them, which is the ordinary failure mode described above.

The weakest pairs on this dataset are all categorical associations rather than numeric correlations:

Pair	Measure	Score
education × education-num	contingency similarity	17.9
education × capital-gain	contingency similarity	48.7
relationship × capital-gain	contingency similarity	48.8
marital-status × capital-gain	contingency similarity	48.8
capital-gain × native-country	contingency similarity	48.8

education × education-num at 17.9 is the one to look at, because those two columns are the same fact twice — a label and its numeric code. In the real data the association is near-deterministic. The generator sampled them independently enough to break that correspondence, producing rows whose education label and education number disagree. The report raises this as a note-level finding covering ten pairs below 55.

The 15 numeric pairs tell a quieter story: the largest real–synthetic gap is age × hours-per-week at real +0.028 versus synthetic +0.093, a gap of 0.065. Nothing collapsed and nothing was invented, because there was little linear structure in the source to lose — Adult's numeric columns are weakly correlated to begin with. A dataset with genuine numeric dependencies would put its worst offenders here instead.

This engine does not currently link a collapsed pair to the utility target it damaged; the pillars are scored independently. Reading them side by side is manual for now — utility scored 35.1 on the same run.

How SynthProof reports it

SynthProof reports correlation preservation as a per-pair table with type-appropriate association measures, real and synthetic values side by side, and the largest gaps surfaced first — in both directions. No threshold hides small differences: an invented correlation is reported as a finding, not treated as noise. Conditional comparisons per category catch relationships that hold in the bulk and vanish for rare categories. When column shape passes and correlation preservation fails, the report states the gap explicitly and, where the engine can trace it, links the collapsed pair to the utility target it damaged.

Related metrics

Column shape is the marginal check this one extends. TSTR is where collapsed dependencies show their cost. Distance to closest record keeps a suspiciously perfect correlation score honest.

Back to the guide: Synthetic data validation: the complete guide.