Richard Plant R.Plant@napier.ac.uk
Research Student
Richard Plant R.Plant@napier.ac.uk
Research Student
Dr Dimitra Gkatzia D.Gkatzia@napier.ac.uk
Associate Professor
Valerio Giuffrida
Neural language models have contributed to state-of-the-art results in a number of downstream applications including sentiment analysis, intent classification and others. However, obtaining text representations or embeddings using these models risks encoding personally identifiable information learned from language and context cues that may lead to privacy leaks. To ameliorate this issue, we propose Context-Aware Private Embeddings (CAPE), a novel approach which combines differential privacy and adversarial learning to preserve privacy during training of embeddings. Specifically, CAPE firstly applies calibrated noise through differential privacy to maintain the privacy of text representations by preserving the encoded semantic links while obscuring sensitive information. Next, CAPE employs an adversarial training regime that obscures identified private variables. Experimental results demonstrate that our proposed approach is more effective in reducing private information leakage than either single intervention, with approximately a 3% reduction in attacker performance compared to the best-performing current method.
Plant, R., Gkatzia, D., & Giuffrida, V. (2021, November). CAPE: Context-Aware Private Embeddings for Private Language Learning. Presented at EMNLP 2021 Conference, Punta Cana, Dominican Republic [Online]
Presentation Conference Type | Conference Paper (published) |
---|---|
Conference Name | EMNLP 2021 Conference |
Start Date | Nov 7, 2021 |
End Date | Nov 11, 2021 |
Acceptance Date | Aug 26, 2021 |
Publication Date | 2021-11 |
Deposit Date | Aug 30, 2021 |
Publicly Available Date | Dec 3, 2021 |
Publisher | Association for Computational Linguistics (ACL) |
Pages | 7970-7978 |
Book Title | Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing |
Public URL | http://researchrepository.napier.ac.uk/Output/2797340 |
Publisher URL | https://aclanthology.org/2021.emnlp-main.628/ |
CAPE: Context-Aware Private Embeddings For Private Language Learning
(170 Kb)
PDF
Publisher Licence URL
http://creativecommons.org/licenses/by/4.0/
Evaluating Language Model Vulnerability to Poisoning Attacks in Low-Resource Settings
(2024)
Journal Article
Data-to-Text Generation Improves Decision-Making Under Uncertainty
(2017)
Journal Article
Multi-adaptive Natural Language Generation using Principal Component Regression
(2014)
Presentation / Conference Contribution
About Edinburgh Napier Research Repository
Administrator e-mail: repository@napier.ac.uk
This application uses the following open-source libraries:
Apache License Version 2.0 (http://www.apache.org/licenses/)
Apache License Version 2.0 (http://www.apache.org/licenses/)
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2025
Advanced Search