Toolkit for validating and profiling tabular data quality.
from scripts.data_profiler import DataProfiler from scripts.schema_validator import SchemaValidator # Profile a dataset profiler = DataProfiler() report = profiler.profile(df) # pandas DataFrame print(report["missing"]) print(report["outliers"]) # Validate against schema schema = { "age": {"type": "int", "min": 0, "max": 150}, "email": {"type": "str", "regex": r"^\S+@\S+\.\S+$"}, "id": {"type": "int", "unique": True} } validator = SchemaValidator(schema) errors = validator.validate(df) for err in errors: print(err)小蔥技能站7w4.net,專業的AI技能分享平臺。
scripts/data_profiler.py - Dataset profiling and summary statsscripts/schema_validator.py - Schema-based validation enginescripts/anomaly_detector.py - Statistical anomaly detectionreferences/validation_rules.md - Common validation patterns這個資料質量驗證工具整體質量較好,核心功能(資料畫像、模式驗證、異常檢測)實現完整,程式碼結構清晰易讀。優點是使用簡單、文件齊全,缺點是示例程式碼存在小問題、缺少配置引導,部分細節處理不夠健壯。對於需要進行資料質量檢查的使用者來說是一個可用的基礎工具,但建議在正式專案中使用前先測試驗證。總體評價:功能完善但細節打磨不足,中等偏上質量。