Common Inpainted Objects In-N-Out of Context

We present Common Inpainted Objects In-N-Out of Context (COinCO), a novel dataset addressing the scarcity of out-of-context examples in existing vision datasets. By systematically replacing objects in COCO images through diffusion-based inpainting, we create 97,722 unique images featuring both contextually coherent and inconsistent scenes, enabling effective context learning. Each inpainted object is meticulously verified and categorized as in- or out-of-context through Large Vision Language Model assessments. Our analysis reveals significant patterns in semantic priors that influence inpainting success across object categories. We demonstrate three key tasks enabled by COinCO: (1) developing a fine-grained context reasoning approach that classifies objects as in- or out-of-context based on three criteria; (2) a novel Objects-from-Context prediction task that determines which new objects naturally belong in given scenes at both instance and clique levels, and (3) context-enhanced fake detection on state-of-the-art methods without fine-tuning. COinCO provides a controlled testbed with contextual variations, establishing a foundation for advancing context-aware visual understanding in computer vision and image forensics.

This Supplementary Material provides additional examples, results, and details to supplement our main paper.

For more information on our inpainting procedure, please see:
Inpainting Procedure Details and Analysis

For more whole pipeline results, please see:
Additional Pipeline Results

For details on training our objects-from-context prediction model and fine-grained context classification models, please see:
Context Models & Training Details

For more results on fake detection and context enhancement, please see:
Fake Detection and Context Enhancement