Skip to main content

About

A reproduction-plus-analysis study by Kacper Lisowicz, Yunzhe Zheng and Daniel Lin. Done as a research project for Hunter College’s Big Data Analytics and Applied Privacy course.

Reproducibility

  • Code and setup: all code lives in the dpdisparity Python package. The notebooks only call it. See the repository README for setup.
  • Where results come from:
    • Initial results from the original testing repository
    • Adult and MNIST have also been retrained with this repository’s pipeline; their pages compare the two.
    • Folktables results are this repository’s own (results/folktables/); there are no earlier runs.
  • To retrain a dataset: uv run python scripts/sweep.py --dataset adult (or mnist; use a GPU for MNIST and Folktables).
    • Folktables: --dataset folktables_sex (natural sex ratio), --dataset folktables_sex --balance, --dataset folktables_sex --n-train 5000 10000 25000 50000 100000 (size sweep), --dataset folktables_race, the \(\varepsilon\) sweep --dataset folktables_sex --n-train 25000 --methods dpsgd global_adapt --target-epsilon 0.5 1 2 8, and the matched-learning-rate non-private baseline --methods nonprivate --lr 0.2 (add the same --n-train / --balance / dataset options). Runs at a non-preset learning rate are stored separately (..._lr0.2/).
  • Folktables data: the 2018 1-Year American Community Survey (ACS) Public Use Microdata Sample, California person file, from the US Census Bureau.
    • The folktables package (Ding et al., 2021) downloads it from census.gov into data/folktables/ (about 255 MB). Its SHA-256 is pinned in the loader.
    • Task: ACSIncome, income > $50K from 10 features (age, class of worker, education, marital status, occupation, place of birth, relationship, hours worked, sex, race), on employed adults (age > 16, income > $100, hours > 0). California has 195,665 such people.
    • Preprocessing follows our Adult pipeline: age and hours standardized, the other categoricals one-hot encoded (815 inputs; 805 for the white/Black subset), sex kept as an input, 80/20 seeded split.
  • Folktables privacy: \(\varepsilon\) is fixed at 3.41 (Adult’s) and the noise multiplier \(\sigma\) is solved for each run’s sampling rate with Opacus’s get_noise_multiplier, so runs of different sizes get the same guarantee. The solved \(\sigma\) is stored with each result.
  • Hardware: the MNIST rerun and all Folktables runs used an NVIDIA RTX 5060 Ti (CUDA 12.8, PyTorch 2.11, Opacus 1.6); exact versions are stored in every result file.
  • Hyperparameters: Esipova et al. (2023), App. B.
  • Privacy: Poisson sampling, RDP accountant, \(\delta = 10^{-6}\). Global-Adapt’s private count is included in \(\varepsilon\).
  • Statistics: values are means over 5 seeds ± standard error (\(\text{SE} = \text{SD}/\sqrt{5}\)), matching the paper’s convention.
  • The disparity measure is the privacy-cost gap \(\lvert \operatorname{mean}(\pi_a - \pi_b) \rvert\). It is not the raw accuracy gap.

Literature

Paper Role in this project
Esipova et al., Disparate Impact in Differential Privacy from Gradient Misalignment, ICLR 2023 Main paper: mechanism, Global-Adapt, MNIST and Adult targets. Code: layer6ai-labs/fair-dp
Bagdasaryan et al., Differential Privacy Has Disparate Impact on Model Accuracy, NeurIPS 2019 Original observation: DP-SGD hurts underrepresented groups more
Farrand et al., Neither Private Nor Fair, PPMLP 2020 Even modest imbalance produces measurable disparity
Sanyal et al., How Unfair is Private Learning?, UAI 2022 Part of the gap is fundamental: DP limits learning rare examples
Mangold et al., Differential Privacy has Bounded Impact on Fairness in Classification, ICML 2023 The fairness gap from privacy is bounded and shrinks with more data; comparison point for the Folktables size sweep
Ding et al., Retiring Adult: New Datasets for Fair Machine Learning, NeurIPS 2021 The Folktables package and the ACSIncome task, a modern, larger replacement for Adult
Yao & Juarez, SaTML 2025 Survey of proposed causes of disparate impact
Demelius et al., TMLR 2025 Conclusions depend on metric and hyperparameters