About
A reproduction-plus-analysis study by Kacper Lisowicz, Yunzhe Zheng and Daniel Lin. Done as a research project for Hunter College’s Big Data Analytics and Applied Privacy course.
Reproducibility
- Code and setup: all code lives in the
dpdisparityPython package. The notebooks only call it. See the repository README for setup. - Where results come from:
- Initial results from the original testing repository
- Adult and MNIST have also been retrained with this repository’s pipeline; their pages compare the two.
- Folktables results are this repository’s own (
results/folktables/); there are no earlier runs.
- To retrain a dataset:
uv run python scripts/sweep.py --dataset adult(ormnist; use a GPU for MNIST and Folktables).- Folktables:
--dataset folktables_sex(natural sex ratio),--dataset folktables_sex --balance,--dataset folktables_sex --n-train 5000 10000 25000 50000 100000(size sweep),--dataset folktables_race, the \(\varepsilon\) sweep--dataset folktables_sex --n-train 25000 --methods dpsgd global_adapt --target-epsilon 0.5 1 2 8, and the matched-learning-rate non-private baseline--methods nonprivate --lr 0.2(add the same--n-train/--balance/ dataset options). Runs at a non-preset learning rate are stored separately (..._lr0.2/).
- Folktables:
- Folktables data: the 2018 1-Year American Community Survey (ACS) Public Use Microdata Sample, California person file, from the US Census Bureau.
- The
folktablespackage (Ding et al., 2021) downloads it from census.gov intodata/folktables/(about 255 MB). Its SHA-256 is pinned in the loader. - Task: ACSIncome, income > $50K from 10 features (age, class of worker, education, marital status, occupation, place of birth, relationship, hours worked, sex, race), on employed adults (age > 16, income > $100, hours > 0). California has 195,665 such people.
- Preprocessing follows our Adult pipeline: age and hours standardized, the other categoricals one-hot encoded (815 inputs; 805 for the white/Black subset), sex kept as an input, 80/20 seeded split.
- The
- Folktables privacy: \(\varepsilon\) is fixed at 3.41 (Adult’s) and the noise multiplier \(\sigma\) is solved for each run’s sampling rate with Opacus’s
get_noise_multiplier, so runs of different sizes get the same guarantee. The solved \(\sigma\) is stored with each result. - Hardware: the MNIST rerun and all Folktables runs used an NVIDIA RTX 5060 Ti (CUDA 12.8, PyTorch 2.11, Opacus 1.6); exact versions are stored in every result file.
- Hyperparameters: Esipova et al. (2023), App. B.
- Privacy: Poisson sampling, RDP accountant, \(\delta = 10^{-6}\). Global-Adapt’s private count is included in \(\varepsilon\).
- Statistics: values are means over 5 seeds ± standard error (\(\text{SE} = \text{SD}/\sqrt{5}\)), matching the paper’s convention.
- The disparity measure is the privacy-cost gap \(\lvert \operatorname{mean}(\pi_a - \pi_b) \rvert\). It is not the raw accuracy gap.
Literature
| Paper | Role in this project |
|---|---|
| Esipova et al., Disparate Impact in Differential Privacy from Gradient Misalignment, ICLR 2023 | Main paper: mechanism, Global-Adapt, MNIST and Adult targets. Code: layer6ai-labs/fair-dp |
| Bagdasaryan et al., Differential Privacy Has Disparate Impact on Model Accuracy, NeurIPS 2019 | Original observation: DP-SGD hurts underrepresented groups more |
| Farrand et al., Neither Private Nor Fair, PPMLP 2020 | Even modest imbalance produces measurable disparity |
| Sanyal et al., How Unfair is Private Learning?, UAI 2022 | Part of the gap is fundamental: DP limits learning rare examples |
| Mangold et al., Differential Privacy has Bounded Impact on Fairness in Classification, ICML 2023 | The fairness gap from privacy is bounded and shrinks with more data; comparison point for the Folktables size sweep |
| Ding et al., Retiring Adult: New Datasets for Fair Machine Learning, NeurIPS 2021 | The Folktables package and the ACSIncome task, a modern, larger replacement for Adult |
| Yao & Juarez, SaTML 2025 | Survey of proposed causes of disparate impact |
| Demelius et al., TMLR 2025 | Conclusions depend on metric and hyperparameters |