You Are What You Write: Author re-identification privacy attacks in the era of pre-trained language models

Plant, Richard; Giuffrida, Valerio; Gkatzia, Dimitra

Abstract

The widespread use of pre-trained language models has revolutionised knowledge transfer in natural language processing tasks. However, there is a concern regarding potential breaches of user trust due to the risk of re-identification attacks, where malicious users could extract Personally Identifiable Information (PII) from other datasets. To assess the extent of extractable personal information on popular pre-trained models, we conduct the first wide coverage evaluation and comparison of state-of-the-art privacy-preserving algorithms on a large multi-lingual dataset for sentiment analysis annotated with demographic information (including location, age, and gender). Our results suggest a link between model complexity, pre-training data volume, and the efficacy of privacy-preserving embeddings. We found that privacy-preserving methods demonstrate greater effectiveness when applied to larger and more complex models, with improvements exceeding over non-private baselines. Additionally, we observe that local differential privacy imposes serious performance penalties of in our test setting, which can be mitigated using hybrid or metric-DP techniques.

Citation

Plant, R., Giuffrida, V., & Gkatzia, D. (2025). You Are What You Write: Author re-identification privacy attacks in the era of pre-trained language models. Computer Speech and Language, 90, Article 101746. https://doi.org/10.1016/j.csl.2024.101746

• Journal Article Type: Article
• Acceptance Date: Oct 28, 2024
• Online Publication Date: Nov 16, 2024
• Publication Date: 2025-03
• Deposit Date: Oct 28, 2024
• Publicly Available Date: Nov 16, 2024
• Print ISSN: 0885-2308
• Publisher: Elsevier
• Peer Reviewed: Peer Reviewed
• Volume: 90
• Article Number: 101746
• DOI: https://doi.org/10.1016/j.csl.2024.101746
• Keywords: language models, privacy-preserving, differential privacy, adversarial learning, re-identification attacks