Empirical Evaluation of Membership Inference Attacks Against Differentially Private Deep Learning Models

Authors

  • Yuzhou Zhang Engineering University of PAP, Xi'an, China Author

DOI:

https://doi.org/10.70088/d0pka139

Keywords:

membership inference evaluation, differential privacy, deep learning, DP-SGD, privacy budget

Abstract

Membership inference evaluations (MIEs) determine whether a sample was used to train a machine-learning model and therefore provide an empirical measure of training-data leakage. This study evaluates black-box MIEs against non-private and differentially private convolutional neural networks trained on MNIST and CIFAR-10. Twenty-five shadow models were used to generate member and non-member prediction vectors, and class-specific LightGBM classifiers served as the inference models. Differentially private stochastic gradient descent (DP-SGD) was evaluated at privacy-budget settings of ε = 1, 2, 4, and 8. Across the evaluated configurations, DP-SGD reduced inference accuracy, precision, and recall relative to conventional SGD. On MNIST, inference accuracy decreased from 0.944 for the non-private model to 0.536 at ε = 1. On CIFAR-10, inference accuracy was 0.761 for the non-private model and 0.659–0.679 at ε = 2–8; the value of 0.370 at ε = 1 indicated substantial degradation of the inference classifier under the strongest privacy setting. These findings demonstrate the practical value of membership inference as an empirical privacy-testing tool and show that the effect of DP-SGD varies across datasets and privacy configurations.

References

R. Shokri, M. Stronati, C. Song, and V. Shmatikov, “Membership inference attacks against machine learning models,” in 2017 IEEE Symposium on Security and Privacy, 2017, pp. 3–18.

N. Carlini, S. Chien, M. Nasr, S. Song, A. Terzis, and F. Tramèr, “Membership inference attacks from first principles,” in 2022 IEEE Symposium on Security and Privacy, 2022, pp. 1897–1914.

A. Salem, Y. Zhang, M. Humbert, P. Berrang, M. Fritz, and M. Backes, “ML-Leaks: Model and data independent membership inference attacks and defenses on machine learning models,” in Proceedings of the 26th Annual Network and Distributed System Security Symposium, 2019.

M. Nasr, R. Shokri, and A. Houmansadr, “Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning,” in 2019 IEEE Symposium on Security and Privacy, 2019, pp. 739–753.

M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang, “Deep learning with differential privacy,” in *Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security*, 2016, pp. 308–318.

M. A. Rahman, T. Rahman, R. Laganière, N. Mohammed, and Y. Wang, “Membership inference attack against differentially private deep learning model,” Trans. Data Privacy, vol. 11, no. 1, pp. 61–79, 2018.

M. Jagielski, J. Ullman, and A. Oprea, “Auditing differentially private machine learning: How private is private SGD?” in Advances in Neural Information Processing Systems, vol. 33, 2020, pp. 22205–22216.

C. Dwork, “Differential privacy: A survey of results,” in Theory and Applications of Models of Computation, Lecture Notes in Computer Science, vol. 4978, 2008, pp. 1–19.

S. Yeom, I. Giacomelli, M. Fredrikson, and S. Jha, “Privacy risk in machine learning: Analyzing the connection to overfitting,” in 2018 IEEE 31st Computer Security Foundations Symposium, 2018, pp. 268–282.

L. Song and P. Mittal, “Systematic evaluation of privacy risks of machine learning models,” in 30th USENIX Security Symposium, 2021, pp. 2615–2632.

C. A. Choquette-Choo, F. Tramèr, N. Carlini, and N. Papernot, “Label-only membership inference attacks,” in Proceedings of the 38th International Conference on Machine Learning, vol. 139, 2021, pp. 1964–1974.

B. Jayaraman and D. Evans, “Evaluating differentially private machine learning in practice,” in 28th USENIX Security Symposium, 2019, pp. 1895–1912.

M. Nasr, J. Hayes, T. Steinke, B. Balle, F. Tramèr, M. Jagielski, and N. Carlini, “Tight auditing of differentially private machine learning,” in 32nd USENIX Security Symposium, 2023, pp. 1637–1654.

A. Yousefpour et al., “Opacus: User-friendly differential privacy library in PyTorch,” arXiv:2109.12298, 2021.

Downloads

Published

30 October 2026

Issue

Section

Article

How to Cite

Zhang, Y. (2026). Empirical Evaluation of Membership Inference Attacks Against Differentially Private Deep Learning Models. Artificial Intelligence and Digital Technology, 3(5), 118-124. https://doi.org/10.70088/d0pka139