Duplicates and split leakage
Find exact and near-duplicate images, including copies across dataset splits.
- Testing group
- Data quality
- Applies to
- Images
- App version
2.0.0- Channel
- stable
Required inputs
A dataset and its labels, task definition and train, validation or test split. Import checks run when tabular data is imported.
Test procedure
Run the check for the dataset's modality, review the flagged records and inspect the affected files or labels before changing the data.
Interpreting results
A flagged record is a prompt for review. Check whether it reflects a data error, an unusual valid example or a mismatch with the intended task.
Limitations
Evidence record
Keep the check settings, affected records, dataset version and any review decisions together.