Image Deduplicator

Find and remove duplicate or similar images in a folder using perceptual hashing. Use when user wants to clean up duplicate images, find near-duplicates, or deduplicate an image dataset.

Features

來源於7w4.net。

• Exact Duplicates: Find images with identical content
• Similar Images: Detect visually similar images (threshold configurable)
• Hash-based: Fast MD5 hashing for exact duplicates
• Perceptual Hash: pHash for finding similar images
• Batch Processing: Process large image folders
• Multiple Actions: List, delete, or move duplicates

Usage

# Find exact duplicates
python scripts/dedupe.py scan /path/to/images/

# Find similar images (90% similarity)
python scripts/dedupe.py scan /path/to/images/ --threshold 90

# Delete duplicates (keeps first occurrence)
python scripts/dedupe.py scan /path/to/images/ --action delete

# Move duplicates to a folder
python scripts/dedupe.py scan /path/to/images/ --action move --output /path/to/dupes/

Examples

$ python scripts/dedupe.py scan ./images/

Scanning images...
Found 150 images
Computing hashes...
Found 5 duplicate groups:

Group 1 (3 files):
  ./images/photo1.jpg
  ./images/photo1_copy.jpg
  ./images/photo1_final.jpg

Group 2 (2 files):
  ./images/screenshot.png
  ./images/screenshot (1).png

Total: 5 duplicate groups, 8 duplicate files

Installation

pip install pillow imagehash

Options

• --threshold: Similarity threshold (0-100), default: 100 (exact)
• --action: What to do with duplicates (list, delete, move)
• --output: Output folder for --action move
• --extensions: File extensions to scan (default: jpg,jpeg,png,bmp)

這是一款質量不錯的圖片去重工具，處理速度快，功能實用，文件說明詳細。它的優勢在於功能完整、使用靈活、安全性好，刪除前會提示確認。不足之處是依賴額外的庫，安裝時需要手動安裝依賴；另外只有命令列介面，對不熟悉命令列的使用者不太友好。總體而言，是一款專業可靠的圖片清理工具。