文件審計引擎

name: audit-agent description: >- Structured document audit engine with a 6-stage LLM-powered pipeline (coherence check, claim extraction, assumption surfacing, stakeholder analysis, methodology review, bias detection), quantitative weighted scoring with pass/fail thresholds, severity distribution analysis, traffic-light visual dashboard, and multi-format reports (text, HTML, JSON). version: 1.0.0 tags: - audit - document-analysis - critical-thinking - bias-detection - methodology - quality-assurance trigger_phrases: - "audit this document" - "audit this paper" - "review document quality" - "check for bias" - "extract claims" - "identify assumptions" - "stakeholder analysis" - "methodology review" - "文件審計" - "審查文件"

Overview

AuditAgent performs a structured 6-stage audit of any text document. Each stage is an independent LLM-powered module that examines the document from a different angle. The stages run sequentially with context flowing from one to the next. A quantitative scorer aggregates findings into a weighted composite score (0-100) with pass/fail determination, severity distribution, red flag summaries, and traffic-light visualization. Reports are produced in structured text, self-contained HTML, and machine-readable JSON formats.

Capability Boundaries

What this tool does: - Run a 6-stage audit pipeline on any document:

• Quantitative scoring with:
• Per-stage scores (0-100) computed from finding severities.
• Weighted composite score with configurable pass/fail threshold (default: 70).
• Severity distribution histogram (critical, high, medium, low, info).
• Red flag extraction (critical + high findings across all stages).
• Traffic-light map (green >= 80, yellow 50-79, red < 50).
• Multi-format reporting:

  這個技能的最新版本可以在7w4.net小蔥技能站找到。

• Structured text report with per-stage findings, severity levels, evidence quotes, locations, and recommendations.
• Executive summary (1-page, character-limited).
• Action items list sorted by priority.
• Self-contained HTML report with color-coded severity badges, traffic-light stage indicators, severity distribution bars, and finding cards with evidence blockquotes.
• Machine-readable JSON export.
• Customizable stage weights and pass threshold via AuditScorer configuration.

What this tool does NOT do: - It does NOT audit code, financial statements, or legal contracts with domain-specific rules. - It does NOT guarantee factual correctness of findings -- LLM output is probabilistic. - It does NOT modify or annotate the original document. - It does NOT run in real-time -- it is a batch pipeline.

Trigger Conditions

Use this skill when the user: - Provides a document (text, article, report, paper) and asks for an audit, review, or quality assessment. - Wants to identify logical inconsistencies, unstated assumptions, or biases in a text. - Needs stakeholder analysis or methodology evaluation for a policy document or research paper. - Asks to "audit this document", "review document quality", "check for bias", "extract claims", "identify assumptions", "文件審計", or "審查文件".

Usage

Python API

from audit_agent import AuditAgent

agent = AuditAgent(model="local")  # or model="qwen2.5:7b"
result = agent.audit("path/to/document.txt")

# Check results
print(result.executive_summary())

# Export reports
result.export_html("output/report.html")
result.export_json("output/report.json")
result.export_text("output/report.txt")

Configuration

The 6-stage pipeline is defined in audit_agent/stages.py and audit_agent/prompts.py. Each stage has: - A unique stage_name mapping to its prompt template. - An __init__ that accepts an LLM client. - A run(document, prior_stage_output, model_name) method that executes the stage.

Scoring Configuration

from audit_agent.scoring import AuditScorer

scorer = AuditScorer(
    weights={
        "coherence_check": 0.20,
        "claim_extraction": 0.15,
        "assumption_surfacing": 0.15,
        "stakeholder_analysis": 0.15,
        "methodology_review": 0.20,
        "bias_detection": 0.15,
    },
    pass_threshold=70.0,
)

Dependencies

• LLM backend: OpenAI-compatible endpoint (default: Ollama with qwen2.5:7b)
• PyYAML for rules configuration.

Output Example

Text Report Structure

=======================================================================
  AuditAgent -- 結構化文件審計報告
=======================================================================
  文件: My Document
  審計時間: 2026-07-06T10:00:00
  綜合評分: 68.5/100
  審計結果: 未通過 (FAIL)
=======================================================================
-----------------------------------------------------------------------
  各階段評分
-----------------------------------------------------------------------
  🟡 一致性檢查 (coherence_check): 72/100 (加權貢獻: 14.4)
  🟢 主張提取 (claim_extraction): 85/100 (加權貢獻: 12.8)
  🔴 隱含假設揭示 (assumption_surfacing): 45/100 (加權貢獻: 6.8)
  ...
-----------------------------------------------------------------------
  問題嚴重度分佈
-----------------------------------------------------------------------
  嚴重 (Critical): 3  |  高危 (High): 7  |  中等 (Medium): 12  |  低危 (Low): 5  |  資訊 (Info): 2

-----------------------------------------------------------------------
  一致性檢查 — 檢測文件內部邏輯矛盾與不一致性
-----------------------------------------------------------------------
  [CRITICAL] COH-001 (severity=90)
    Section 2 claims X while Section 5 implies not-X.
    位置: Paragraphs 12-15
    原文: "The policy achieved its stated goals..."
    建議: Clarify the contradiction between Section 2 and Section 5.
  ...
=======================================================================
  審計結束
=======================================================================

HTML Report Features

• Gradient header with document name and timestamp.
• Circular score display with PASS/FAIL badge.
• Severity distribution bar chart.
• Per-stage sections with traffic-light indicators, finding cards with severity badges, evidence blockquotes, and recommendations.

FAQ

Q: What model should I use? A: Default is Qwen2.5:7b via Ollama. Any OpenAI-compatible model works. Larger models may produce better quality audits, especially for complex documents.

Q: Can I audit documents in any language? A: Yes. The stage prompts are in Chinese and the system prompt instructs the LLM to work with the document's language. Multi-language documents are supported.

Q: What does a "pass" mean? A: A composite score of 70 or above (configurable) indicates the document passed the audit. This is a relative quality metric, not an absolute guarantee.

Q: How long does an audit take? A: Each of the 6 stages makes an LLM call. With a local Ollama model, expect 30-120 seconds total depending on document length and hardware.

Q: Can I skip certain stages? A: The pipeline expects all 6 stages. To skip a stage, you would need to modify the scoring weights to zero out unwanted stages and adjust the orchestrator.

這個 Skill 質量很好，定位清晰——專門用於審計文件質量而非通用聊天。6個審計維度覆蓋全面，從邏輯一致性到偏見檢測都有涉及。評分系統直觀，能給出具體分數和紅牌警告。多語言支援和多格式報告輸出是加分項。主要缺點是缺少測試用例，普通使用者難以驗證輸出準確性；另外對提示詞質量的依賴較高，不同模型可能產生不一致結果。整體適合需要嚴肅評估文件質量的使用者使用。