Column shape: KS statistic and total variation distance

Column shape is the first thing anyone checks about synthetic data and the last thing that should reassure them. It asks, one column at a time: does the synthetic distribution of this variable look like the real one? It is measured with the Kolmogorov–Smirnov statistic for numeric columns and total variation distance for categorical ones, and it is the foundation of the fidelity pillar.

Most modern generators pass it comfortably. That is precisely why it matters to understand what it does and doesn't tell you.

Definitions

Kolmogorov–Smirnov (KS) statistic — for numeric columns. Build the empirical cumulative distribution function of the column in the real data and in the synthetic data. The KS statistic is the largest vertical gap between the two curves: the maximum difference, at any value, between the fraction of real rows at or below that value and the fraction of synthetic rows at or below it. It ranges from 0 (identical distributions) to 1 (no overlap). A column-shape score is usually reported as 1 − KS, so higher is better.

Total variation distance (TVD) — for categorical columns. Compute the frequency of each category in the real data and in the synthetic data. TVD is half the sum of the absolute differences across categories — equivalently, the largest amount of probability mass you'd have to move to turn one distribution into the other. Ranges from 0 to 1; the score is 1 − TVD.

Both are distribution-free (no assumption about the shape of the data), both are bounded and comparable across columns, and both are cheap. A fidelity report typically shows a per-column table and an average.

How to read a column-shape table

The average across columns is a headline; the per-column table is the information. Three patterns are worth recognising:

Uniformly high scores. The generator reproduced every marginal. Good — and expected of any competent generator. It tells you almost nothing about relationships between columns, which is where fidelity actually fails. Move on to correlation preservation.

One or two columns much worse than the rest. A specific failure. Common causes below. The finding is column-level and usually actionable.

Categorical columns systematically worse than numeric, or vice versa. A generator-architecture signature. Some models handle continuous data well and mangle high-cardinality categoricals; others the reverse. Useful for choosing between generators, less useful for fixing one.

What a bad score usually means

Missing tails. The KS gap sits at the extremes: the synthetic data has no values above the real 99th percentile, or the distribution is truncated. Generators that model numeric columns with a bounded transform, or that clip, do this. It matters more than it looks: tails are where fraud, high-value customers and rare events live, and it will show up again as poor TSTR on regression targets.

Mode collapse in categoricals. The TVD gap comes from rare categories that vanished — the synthetic column has the five common values and none of the thirty rare ones. Sometimes acceptable, sometimes fatal (if the rare categories are the ones you care about). Read the per-category breakdown, not just TVD.

Wrong shape entirely. A bimodal real column rendered as unimodal; a heavy-tailed column rendered as roughly normal. A generator that assumed a distributional family it shouldn't have. Visible in the KS gap sitting in the body of the distribution rather than the tail.

Serialisation artefacts, not generation failures. Integers written as floats, dates as strings, a categorical column encoded numerically. A naive comparison scores these harshly for what is a formatting issue. A careful validation engine reconciles dtypes before scoring so a generator isn't penalised for how it wrote the file — but if you're computing this yourself, check the dtypes before you blame the model.

The remedy for genuine failures is generator-agnostic: choose a model or transform that doesn't truncate, avoid clipping, and, for mode collapse, condition or oversample on the rare categories that matter.

Why passing column shape isn't enough

The most important thing about column-shape metrics is their blind spot. A generator can sample every column independently from its correct marginal distribution and score perfectly on KS and TVD — while producing data with no relationships at all. Age and income independent; diagnosis and medication independent; every pairwise structure gone. Such data passes column shape and fails everything downstream: correlation preservation collapses, TSTR falls to chance for any task that depends on relationships, and the dataset is useless despite looking, column by column, exactly right.

This is why column shape is a necessary fidelity check and never a sufficient one, and why a fidelity score built only from marginals overstates quality. It is also why the gap between "column shape passes" and "TSTR fails" is itself a finding — one that points directly at lost dependencies.

Reading column shape alongside the other pillars

Column shape has an odd relationship with privacy: a generator that memorised its training data has perfect column shape, because copies have exactly the right marginals. So high column-shape scores are consistent with both the best and the worst outcomes and should be read with the privacy pillar before they mean anything.

Low column-shape scores, on the other hand, usually cascade: missing tails hurt utility, mode collapse hurts utility on rare classes. When utility is poor and column shape is also poor on the relevant columns, the diagnosis is straightforward. When utility is poor and column shape is fine, look at dependencies.

Worked example

UCI Adult census, 4,200 rows × 15 columns, SDV Gaussian Copula, metrics engine 0.6.0. Column shapes score 88.7/100, inside a fidelity pillar of 84.8.

Column	Type	Metric	Score
capital-gain	numeric	KS complement	8.9
hours-per-week	numeric	KS complement	71.9
capital-loss	numeric	KS complement	82.1
education-num	numeric	KS complement	85.7
fnlwgt	numeric	KS complement	95.6
…
education	categorical	TV complement	98.8
relationship	categorical	TV complement	99.2
race	categorical	TV complement	99.2
class	categorical	TV complement	99.9
sex	categorical	TV complement	99.9

The shape of that table is the point. Every categorical column scores above 98 and every one of the five weakest is numeric — the generator reproduced category frequencies almost exactly and struggled with continuous distributions. capital-gain at 8.9/100 is the outlier by a wide margin, and it is the one finding the pillar raised: "Weakest is capital-gain at 9/100 distribution match; 1 column scores below 70." It is a heavily zero-inflated column — most people have no capital gain and a small minority have very large ones — and that is exactly the shape a copula fits worst.

The memorising-generator control scored 100.0 on column shapes. It is a verbatim copy of the real data, so every marginal matches perfectly by construction. That is the blind spot in one number: a column-shape table alone would have ranked the leaking generator above this one.

How SynthProof reports it

SynthProof reports column shape as a per-column table — KS for numeric, TVD for categorical, each as a 1 − distance score — with an average that feeds the fidelity pillar alongside pairwise and correlation measures, so marginals alone cannot produce a high fidelity score. Dtypes are reconciled before scoring so serialisation differences aren't penalised. Findings name the worst columns and their likely cause — truncated tail, dropped rare categories, wrong distributional shape — with a general remedy, and when column shape passes while TSTR fails, the report says so explicitly as a pointer to lost dependencies.

Related metrics

Correlation preservation is where fidelity actually gets tested once marginals pass. TSTR is where missing tails and dropped categories show their cost. Exact-copy detection is the check that keeps a perfect column-shape score honest.

Back to the guide: Synthetic data validation: the complete guide.