Exploring Dataset Diversity for GenAI Image Inpainting Localisation in Digital Forensics

Thomson, Matthew; McKeown, Sean; Macfarlane, Richard; Leimich, Petra

Abstract

Generative Artificial Intelligence (GenAI) has significantly increased the sophistication and ease of image tampering techniques, posing challenges for digital forensics in identifying manipulated images. A lack of dataset standardisation hinders the ability to effectively benchmark and compare GenAI inpainting localisation techniques, reducing their reliability in digital forensic applications. This paper aims to address this gap by exploring the need for standardised criteria for datasets in digital forensics for benchmarking detection techniques through preliminary experiments.
To address the limited diversity in existing datasets, a small-scale dataset was developed, consisting of 240 tampered images, 20 masks and 20 authentic images. This dataset includes four subject image classes (animals, objects, persons, scenery) and three inpainting tools (GLIDE, GalaxyAI, Photoshop). The dataset was evaluated against 13 localisation algorithms from the Image Forensics MATLAB Toolbox to determine key components that should be considered in the standardisation of testing environments.
The results show that the images in the animals and persons categories achieved the highest F1-Scores and accuracy over the other classes. Among tools, GLIDE inpainted images were consistently shown to be the most challenging to detect, underscoring the importance of further investigating these images. These findings provide foundational insights for identifying a set of criteria to establish robust testing environments, enabling the development of reliable and accurate GenAI inpainting localisation techniques.

Citation

Thomson, M., McKeown, S., Macfarlane, R., & Leimich, P. (2025, March). Exploring Dataset Diversity for GenAI Image Inpainting Localisation in Digital Forensics. Presented at DFDS 2025: Digital Forensics Doctoral Symposium, Brno, Czech Republic

• Presentation Conference Type: Conference Paper (published)
• Conference Name: DFDS 2025: Digital Forensics Doctoral Symposium
• Start Date: Mar 31, 2025
• Acceptance Date: Dec 14, 2024
• Publication Date: 2025-04
• Deposit Date: Mar 31, 2025
• Publicly Available Date: Mar 31, 2025
• Publisher: Association for Computing Machinery (ACM)
• Peer Reviewed: Peer Reviewed
• Pages: 1-7
• Book Title: DFDS '25: Proceedings of the Digital Forensics Doctoral Symposiu
• ISBN: 979-8-4007-1076-6
• DOI: https://doi.org/10.1145/3712716.3712724
• Public URL: http://researchrepository.napier.ac.uk/Output/4191251