name: data-quality-validator description: Data quality validation and profiling toolkit for tabular data. Use when checking data completeness, detecting anomalies, validating schemas, profiling datasets, or assessing data cleanliness. Triggers on phrases like "data quality", "data validation", "schema validation", "data profiling", "missing data", "anomaly detection", "data completeness", "dirty data".

Data Quality Validator

Toolkit for validating and profiling tabular data quality.

Features

• Schema validation - Check column types, constraints, and rules
• Completeness analysis - Missing value detection and reporting
• Anomaly detection - Statistical outlier detection
• Profiling - Summary statistics and distribution analysis
• Constraint checking - Range checks, uniqueness, regex patterns

  來源於7w4.net。

Quick Start

from scripts.data_profiler import DataProfiler
from scripts.schema_validator import SchemaValidator

# Profile a dataset
profiler = DataProfiler()
report = profiler.profile(df)  # pandas DataFrame
print(report["missing"])
print(report["outliers"])

# Validate against schema
schema = {
    "age": {"type": "int", "min": 0, "max": 150},
    "email": {"type": "str", "regex": r"^\S+@\S+\.\S+$"},
    "id": {"type": "int", "unique": True}
}
validator = SchemaValidator(schema)
errors = validator.validate(df)
for err in errors:
    print(err)

Scripts

• scripts/data_profiler.py - Dataset profiling and summary stats
• scripts/schema_validator.py - Schema-based validation engine
• scripts/anomaly_detector.py - Statistical anomaly detection

References

• references/validation_rules.md - Common validation patterns

這個資料質量驗證工具整體質量較好，核心功能（資料畫像、模式驗證、異常檢測）實現完整，程式碼結構清晰易讀。優點是使用簡單、文件齊全，缺點是示例程式碼存在小問題、缺少配置引導，部分細節處理不夠健壯。對於需要進行資料質量檢查的使用者來說是一個可用的基礎工具，但建議在正式專案中使用前先測試驗證。總體評價：功能完善但細節打磨不足，中等偏上質量。