2020 Nov 12th: Code is now available at
https://gitlab.com/omayer/forensic-graph
2020 June 3rd: This work has been selected to be published in the August 2020 IEEE
JSTSP Special Issue on Data Driven Media Authentication and Forensics!
Project code will be released to coincide with publication.
Overview
We introduce a novel
abstract, graph-based representation of an image, which we
call the Forensic Similarity Graph, that captures key forensic
relationships among regions in the image. In this representation,
small image patches are represented by graph vertices with edges
assigned according to the forensic similarity between patches.
Localized tampering introduces unique structure into this graph,
which aligns with a concept called “community structure” in
graph-theory literature. In the Forensic Similarity Graph, communities correspond to the tampered and unaltered regions in the
image. As a result, forgery detection is performed by identifying
whether multiple communities exist, and forgery localization is
performed by partitioning these communities.
An example of the forensic similarity graph representation of a forged image, exhibiting community structure. In the forged image, water stains were removed via local application of a brush tool.
The forensic similarity matrix shows the forensic similarity between each pair of sampled image patches.
The graph representation highlights the community structure, showing the patches associated with the tampered region appears as its own densely in-connected cluster.
Original image
Edited image, with vertex index overlay
Matrix of forensic similarities between pairs of patches in the edited image
The proposed graph representation, showing two distinct communities
Our method finally produces pixel-map highlights of the tampered region.
This example uses a higher spatial resolution than the above graph representation.
Editing credit to Reddit.com user "/u/rombouts."
In this work, we also present
community detection techniques to
detect and localize image forgeries utilizing the proposed graph representation. We experimentally show that
the community detection methods, when applied to the Forensic Similarity Graph, outperform existing
state-of-the-art forgery detection and localization methods, which
do not capture such community structure.
Forgery Localization Examples
Here are some examples of using our proposed algorithm for forgery localization, compared against other methods from literature.
Related Works
This article builds off of several works from the MISL group, including:
