\section{Open-Set Fraud Discovery in Production Traffic}

Adaptive fraud, in which attackers modify artifacts to evade deployed models, is the primary target of this work. This setting differs fundamentally from fraud detection on closed datasets such as FantasyID, where fraud categories are predefined and stable. Hence, we do not measure success by overall recall across all confirmed fraud cases, since that metric conflates easy and difficult regimes and is further confounded by incomplete production labels. Instead, we focus on incremental discovery: confirmed fraud surfaced by the embedding method beyond the incumbent stack, especially within the adaptive physical-fraud regime.

Fraud labels in our experiments are obtained through a multi-source ground-truth pipeline that combines programmatic signals with expert human review. The programmatic pipeline maintains a running set of fraudulent verifications identified via existing rule-based fraud signals, escalation workflows, and cases that trigger multiple downstream fraud checks simultaneously. Expert visual inspection is critical for validating cases that automated signals may not fully capture. Commonly inspected indicators include layout inconsistencies, saturation anomalies, color artifacts suggestive of digital replicas, evidence of tampering or image compositing, and portrait irregularities, image manipulation, etc. Fraud labels are most confidently confirmed when programmatic signals and human inspection concur, whereas benchmarking of new detection methods is conducted against the automated programmatic labels.

\subsection{Similarity-based fraud discovery}
The learned embedding space enables an effective similarity-driven fraud discovery mechanism. In practice, fraudulent IDs generated from a common template or fabrication pipeline often share similar features, such as suturation, fontsize, fontface, spacing tweaks, texture, etc. Their embeddings deviate from the centroid of their layout, but they exhibit strong mutual similarity.

Empirically, this embedding-based graph node substantially improves fraud discovery. Starting from a single confirmed fraudulent ID (a seed), the similarity edges enable the identification of related fraudulent documents that are not connected through conventional graph signals (e.g., shared metadata, device fingerprints, or user attributes). This demonstrates that visual embedding similarity captures complementary fraud patterns that are otherwise difficult to detect.

\subsection{Cluster-level fraud discovery}
In real-world fraud, successful fabrication pipelines are typically scaled, producing many visually consistent submissions. This induces redundant evidence: investigators can validate a suspected fraud campaign by inspecting a small number of representative samples from a cluster, rather than adjudicating each document independently. As a result, we evaluate our method not only by per-document scoring performance, but also by label efficiency—the number of fraudulent submissions identified per unit of human review—where a single cluster-level decision can cover tens to hundreds of related attempts. This human-in-the-loop framing allows the embedding model to prioritize high-recall discovery of coherent groups, while maintaining manageable review cost.

The global structure of the embedding space can reveal previously unknown fraud patterns. To study this structure, we cluster 20,448 real-world Canadian ID embeddings. We first project the embeddings into 2D space using t-SNE for visualization, and then perform k-means clustering with a suitable number of clusters. The resulting clusters are iteratively refined over multiple auditing rounds through splitting, merging, outlier removal, and percentile-based trimming to improve intra-cluster cohesion. Such outlier handling is especially important in real-world settings, where noisy samples are common.

Figure \ref{fig:fig1} shows a two-dimensional t-SNE projection of the embeddings colored by their cluster assignments. Fraudulent IDs (marked in red) are distributed across multiple layouts.

A particularly notable observation is the emergence of three dense clusters located far from primary legitimate layout centroids. Manual investigation reveals that these samples correspond to printed passport reproductions that were submitted as ID documents, and different clusters represent different passport layouts.

Remarkably, these three anomalous clusters surfaced a total of 276 confirmed cases of fraud. Incumbent supervised models and conventional metadata-graph signals only captured 54 of them. This finding highlights the effectiveness of embedding-space clustering for uncovering large-scale fraud operations and systematic fabrication sources, especially when no prior template labels are available.



\subsection{Borderline and High z-Score Cases}
We investigated samples in a cluster but are farthest from the centroids (having largest z-score where $z=(x-\mu)/\sigma$). We found many of them belong to the following situations. (1) Blurred, out of focus, snapshot during moving, and low resolution due to dim lighting condition, making it difficult or impossible to read ID information; (2) replay from screen, showing clear Moiré pattern; (3) physically printed IDs, paper printed IDs and deepfakes. The z-score alone can assist us detect unreal IDs.
Even without pairwise similarity analysis, the z-score itself provides a useful anomaly signal for identifying suspicious or low-quality submissions. Combined with similarity-based graph propagation and clustering analysis, the embedding space offers a unified and scalable framework for real-world fraud discovery.
