Fairness and Dependency Preservation in Synthetic Credit Data

German Credit, as it comes

After synthetic resampling

≈ 50%

Synthetic data fixes the class imbalance. The question is what it breaks along the way: the relationships between features, and the fairness of the model trained on it.

About the study

A closer look at synthetic data in lending

🧩

Features are tangled together

Loan approval models rely on applicant features that depend on each other. Even without sensitive attributes, correlated features can still lead to unfair outcomes.

🧪

Five datasets, one pipeline

The original data against SMOTE, CTGAN, TVAE and Gaussian Copula, measured on ROC AUC, three fairness metrics and a Dependency Preservation Score.

🛡️

The real data held up best

The baseline kept the relationships and the most stable fairness-utility trade-off. No synthetic method beat it.

Abstract. In this study, a baseline ensemble model- trained on the original German Credit dataset without synthetic augmentation- showed the most stable fairness-utility trade-off whereas SMOTE introduced the largest distortions. This study focuses on various techniques for generating synthetic data while preserving feature interdependence during fairness-aware preprocessing of loan approval models. Loan approval models rely on interdependent applicant features. Even without sensitive attributes, correlated features can induce unfair outcomes, making dependency preservation critical. Models were trained on the German Credit dataset without synthetic augmentation and compared against SMOTE, CTGAN, TVAE, and Gaussian Copula on the basis of ROC AUC and standard fairness metrics (comparative analysis only included methods that produced valid synthetic datasets). While the baseline consistently maintained interdependencies and provided stable fairness-utility trade-offs, synthetic methods distorted interdependencies and introduced instabilities, with no single method outperforming the baseline.

Explore

Explore the study

📊

Results

Every published number, method by method, with the spread across 10 runs.

→ ⚖️

The fairness metrics

Drag two groups’ rates and watch Disparate Impact and Predictive Parity react.

→ 🕸️

Dependencies

The real relationships in the data, computed live, and the Dependency Preservation Score.

→ 🧬

Method and limitations

The pipeline from raw data to metrics, and everything the paper says it couldn’t do.

→ ✨

Deluxe editionNEW

New follow-up experiments on the questions the paper left open.

→

Published results

Pick a method and see how it did

Every number here is from the paper: the mean ± standard deviation over 10 runs with different random seeds. The faint marks on each scale are the other methods.

Method	ROC AUC	Equalized Odds	Disparate Impact	Predictive Parity	Dependency Preservation Score

The paper reports no formal significance testing, so the differences are trends rather than statistically confirmed effects. Under a robustness check (normalization, Gaussian noise, small hyperparameter changes and an out-of-sample 80/20 split), SMOTE had a Dependency Preservation Score of 0.6573, and CTGAN was sensitive to small hyperparameter and data-split changes.

The metrics

What the fairness numbers mean

Move the sliders to build two groups and see how each metric reacts. The marks show where the paper’s three valid methods landed.

Disparate Impact

The ratio of desired outcomes (here, approvals) between the protected and unprotected groups. Closer to 1 means fairer.

Unprotected group approved

Protected group approved

Predictive Parity

The difference in positive predictive value between groups: of the applicants the model flags, how often it’s right in each group. Lower absolute differences mean better parity.

Group 1 predictive value

Group 2 predictive value

Equalized Odds

Measures differences in error rates (true positive and false positive rates) across groups. Lower values mean more equitable performance. The baseline scored 0.081 ± 0.007, SMOTE 0.118 ± 0.013 and CTGAN 0.102 ± 0.011.

Disparate Impact and Equalized Odds were computed mainly for personal_status_sex and foreign_worker; Predictive Parity and Equalized Odds for credit_history and employment_since. The first category of each attribute is the reference group for ratio-based measures.

Dependencies

The relationships synthetic data has to keep

The real relationships between the numeric columns in German Credit, computed live in your browser from all 1,000 applicants. Each cell is a Spearman rank correlation. Hover or tap a cell to read it.

Dependency Preservation Score

How the paper defines it:

For numerical features, Spearman correlation coefficients were computed for all feature pairs between the original and synthetic datasets, and their mean is the numerical dependency score.
For categorical features, Cramér’s V was computed for all feature pairs, and its mean is the categorical dependency score.
The final score averages the two. A companion measure, the Frobenius norm of the difference between the original and synthetic Spearman matrices, gives the overall distance; lower is better.

The heatmap covers the seven numeric columns only. The paper’s score also includes the categorical columns through Cramér’s V. Data: Hofmann, H. (1994), Statlog (German Credit Data), UCI Machine Learning Repository, doi.org/10.24432/C5NC77 (CC BY 4.0).

Method

The pipeline

German Credit (UCI Statlog)Target mapped to good credit = 0, bad credit = 1.

PreprocessingOne-hot encoding, with sensitive features such as gender and foreign_worker kept for fairness analysis.

Synthetic dataBaseline (none), SMOTE, CTGAN, TVAE, Gaussian Copula. CTGAN and TVAE: default settings, 300 epochs, batch size 500.

AutoGluon ensembleXGBoost, Random Forest, LightGBM, CatBoost and a PyTorch neural network; the selected model was XGBoost, stacking disabled.

EvaluationROC AUC, Equalized Odds, Disparate Impact, Predictive Parity and dependency preservation. 80/20 split, 10 runs.

Limitations the paper names

TVAE and Gaussian Copula failed. Both produced NaN for every metric because the synthetic data itself was invalid (for example constant columns, empty subgroups or undefined values).
Small dataset. Models trained on 800–1120 samples and were validated on 200–280, so a few errors can move the fairness estimates.
No significance testing. Results are means and standard deviations over 10 runs, so they should be read as trends.
Imbalanced baseline. The baseline used the original 70/30 data while the synthetic methods were balanced, which is not a like-for-like comparison.
Default generator settings. CTGAN and TVAE used default architectures and hyperparameters, with no tuning.
One selected model. Results come from the single model AutoGluon selected (XGBoost).
Messy attributes. foreign_worker is mislabeled in the UCI version, and personal status and sex are combined into one attribute.
No privacy check. Privacy leakage, such as membership inference, was not measured.

Bottom line from the paper: for fairness-critical applications like loan approval, it may be safer to rely on the original data with fairness-aware preprocessing than on synthetic data generation, which introduces dependency distortions and fairness instability, albeit improving class balance.

Deluxe edition

Picking up where the paper stopped

New experiments on the open questions from the paper’s limitations and future work.

⚠️

These are new runs, not the published results. They are a companion analysis made for this page, coded and run with AI assistance (Claude), and have not been peer reviewed. The published numbers above are unchanged. The paper leaves some details open, so each experiment states its exact definitions, and the absolute numbers here are not directly comparable with the paper’s.

EXPERIMENT 01 · REPRODUCTION

Running the study again, from scratch

The same five datasets, trained and tested again with XGBoost over the same 80/20 split design. TVAE and Gaussian Copula get another try with a current version of the SDV library. Published values are on the left of each pair; this re-run is in bold.

Method	ROC AUC paper	ROC AUC re-run	Disparate Impact paper	Disparate Impact re-run	Equalized Odds paper	Equalized Odds re-run

Accuracy reproduces closely; fairness is noisier. ROC AUC lands within about 0.01 of the paper: 0.770 vs 0.781 for the baseline, 0.765 vs 0.764 for SMOTE, 0.756 vs 0.751 for CTGAN, and the baseline again scores highest. This time TVAE and Gaussian Copula ran, but TVAE’s bad-credit rows were degenerate in all 20 runs (savings, other_debtors and foreign_worker never varied), consistent with the “constant columns” failure the paper describes for its NaNs (the original runs weren’t re-examined, so this can’t confirm their exact cause). Gaussian Copula produced valid data every time. The fairness numbers differ more, because the paper’s exact fairness definitions and groups aren’t fully specified, and they swing a lot from run to run on a 200-applicant test set.

EXPERIMENT 02 · A FAIR FIGHT

Does a balanced baseline change the story?

The paper notes its baseline trained on imbalanced 70/30 data while the synthetic methods trained on balanced data. Here the original data is balanced too, by class weighting and by plain random oversampling, with no new rows invented.

Balancing the real data is enough. With no rows invented, the class-weighted baseline scores ROC AUC 0.772 and the oversampled one 0.769, level with the imbalanced baseline (0.770) and at or above every synthetic method. So, for accuracy at least, the baseline’s edge in the paper doesn’t look like an artefact of the imbalance.

EXPERIMENT 03 · FAIRNESS

Disparate Impact for women vs men

Approval rate for women divided by approval rate for men, in each method’s model. The dashed line at 1 is equal treatment.

No synthetic method moves the needle. Every method lands between 0.844 (TVAE) and 0.909 (reweighing), and the error bars overlap. None of the synthetic methods is significantly different from the baseline on Disparate Impact; the only significant improvement is reweighing (experiment 05).

EXPERIMENT 04 · THE SYNTHETIC SLIDER

How much synthetic data can a model take?

Start with the real training set and swap a growing share of its rows for synthetic ones from each generator, keeping the size the same. At 100%, the model never sees a real applicant. Every model is tested on the same real held-out applicants. Points were measured at 0, 25, 50, 75 and 100%; the slider reads between them.

Synthetic share of the training data

ROC AUC

Disparate Impact

Equalized Odds

Swap in too much and the model stops working. CTGAN and Gaussian Copula lose accuracy steadily as synthetic rows replace real ones. At 100% synthetic, CTGAN’s ROC AUC is 0.497, no better than a coin flip, and Gaussian Copula’s is 0.571. TVAE holds up far better (0.742 at 100%). Disparate Impact drifts toward 1 as the synthetic share grows, but read that carefully: for CTGAN it happens as the model stops working, and a model that approves people at random treats every group the same.

EXPERIMENT 05 · THE PAPER’S RECOMMENDATION

Original data with fairness-aware preprocessing

The paper concludes that real data with fairness-aware preprocessing may be the safer path. Here that is tested with reweighing (Kamiran and Calders), which weights each sex and outcome combination so the two are independent in training.

The paper’s recommendation holds up. Reweighing kept ROC AUC level with the baseline (0.771 vs 0.770) while improving both fairness measures: Disparate Impact rose from 0.866 to 0.909 (p = 0.004) and Equalized Odds fell from 0.191 to 0.139 (p = 0.017). It was the only approach in these runs to significantly improve both, without inventing any data.

EXPERIMENT 06 · SIGNIFICANCE

Which differences are more than noise?

Paired Wilcoxon signed-rank tests across seeds, each method against the original-data baseline. A ✓ marks p < 0.05. With this few runs and no correction for multiple tests, treat a single ✓ as a hint, not proof.

Method	ROC AUC	Disparate Impact	Equalized Odds	Predictive Parity

What survives the test. On accuracy, CTGAN (p = 0.044) and Gaussian Copula (p = 0.003) are significantly below the baseline; SMOTE (p = 0.062) and TVAE (p = 0.985) are not. On fairness, no synthetic method differs significantly from the baseline on Disparate Impact; SMOTE-NC has a smaller Equalized Odds gap (p = 0.027). Reweighing is the standout on both fairness measures.

EXPERIMENT 07 · DEPENDENCIES, RE-MEASURED

How much structure each generator keeps

Our implementation of the Dependency Preservation Score as the paper describes it (1 means the synthetic rows keep exactly the real relationships), measured on rows each generator produced.

Frobenius distance between the real and synthetic Spearman matrices (lower is closer):

Compare the ranking, not the values. By our reading of the score, every generator keeps most pairwise structure (0.912 to 0.936), far above the paper’s 0.5365 and 0.3986, so the exact formula clearly differs. The Frobenius distance separates them more: CTGAN distorts the numeric relationships most (1.162), in line with its low score in the paper, while Gaussian Copula keeps them best (0.613).

EXPERIMENT 08 · A FIRST PRIVACY CHECK

Do synthetic rows sit too close to real people?

The paper didn’t measure privacy. A simple first check is the distance from each synthetic row to its nearest real training row, compared with how close genuinely new real applicants (the held-out set) sit. Much closer than real newcomers suggests copying.

No copies, but some rows sit close to real people. No method copied a real applicant exactly. SMOTE rows sit at a median distance of 3.00 from their nearest real applicant and TVAE rows at 3.69, both much closer than genuinely new applicants (5.70). SMOTE builds every row between two real people, so that is expected. CTGAN (6.65) and Gaussian Copula (6.32) sit about as far away as new applicants do, and SMOTE-NC sits furthest (11.14) because it creates unusual combinations of categories. This is a first check, not a privacy guarantee.

Exact definitions used in the deluxe experiments

Read the full paper

The whole study, in the journal

Published in NHSJS Reports, The National High School Journal of Science, and awarded a Gold CREST Award.

Read on NHSJS ↗PDF ↗

Upadhyay, R. (2026). Fairness and Dependency Preservation in Synthetic Credit Data. NHSJS Reports, The National High School Journal of Science. https://nhsjs.com/2026/fairness-and-dependency-preservation-in-synthetic-credit-data/