機器學習-英文文本二分類

👤 lzp 📦 v1.0.1 ⭐ 4.6 ⬇️ 32 下載

🤖 AI-Agent 免費

📖 技能介紹

name: text-binary-classification description: > English text binary classification end-to-end training and deployment. Triggers when the user mentions: text classification, sentiment analysis (binary/positive-negative), spam detection, 0/1-label text classification, "train a text classifier", "build a binary classification model", "NLP classification model", "train a sentiment model with CSV", or any task involving classifying English text into two categories.

Covers traditional ML (BoW/TF-IDF + SVM/LR/RF/NB), deep learning (CNN/LSTM/ GRU + GloVe/fastText embeddings, frozen/fine-tuned), and Transformers (BERT/RoBERTa/DeBERTa/DistilBERT/ALBERT/ELECTRA/XLNet, feature extraction, partial freeze, LoRA fine-tuning, full fine-tuning).

中文觸發詞：文本分類、情感分析（二分類/正負面）、垃圾郵件檢測、0/1標籤文本分類、"訓練文本分類器"、"構建二分類模型"、"NLP分類模型"、 "用CSV訓練情感模型"。

All training includes baseline defaults + Optuna hyperparameter tuning, MLflow experiment tracking, GPU auto-detection, progress bars, and deployment artifact generation. Use this skill when the user wants to train, tune, or deploy an English text binary classifier, even if they don't specify a model or framework.

English Text Binary Classification Skill

英文文本二分類技能

End-to-end binary text classifier training and deployment pipeline. Covers 150+ model variants across three categories: traditional ML, deep learning, and Transformers.

⛔ 常見 AI 違規行為 — 執行此技能前必讀

⛔ Common AI Violations — READ BEFORE EXECUTING THIS SKILL

The following 12 violations occur in >80% of AI executions of this skill. You MUST actively guard against each one:

#	Violation	Correct Behavior
1	Skipping `Read` when script already outputs to stdout — AI sees Bash stdout and decides the content is "already visible," so it skips the mandatory Read step.	stdout is NOT a substitute for Read. The all-153 path REQUIRES you to call the Read tool on `model_list.md`, regardless of what stdout shows.
2	"As shown above" shortcut — AI uses phrases like "如上所示" / "詳見輸出" / "已在對話中列出" instead of actually copying all 153 rows.	You MUST copy every single row verbatim. No summarization. No range abbreviations. No omission of P3 rows.
3	AskUserQuestion misuse — AI calls AskUserQuestion when the skill explicitly says "wait for text input in conversation." This happens most often after the all-153 list output.	After outputting the all-153 model list, you MUST NOT call AskUserQuestion. Just wait silently for the user to type their selection in the conversation.
4	Skipping the self-check — AI outputs the model list but never verifies the last line matches `> Total: 153 models (P1 ★ = N, P2 = N — recommended ✓, P3 = N)`.	After every all-153 output, you MUST output the self-check template (see below) and confirm every checkbox is ticked. If any box is unchecked, re-do the step from Read onward.
5	Ambiguous Step 2 transition — With the all-153 path, AI incorrectly assumes "training mode is encoded in the model number" means baseline/tune/both is also skipped.	The model number encodes full_ft/LoRA/feature_extraction/frozen/fine-tuned. The execution MODE (baseline/tune/both) is a SEPARATE question and MUST be asked via AskUserQuestion in Step 2 for every selected model.
6	No fallback when AskUserQuestion dialog fails — AskUserQuestion is a UI modal that can fail to render (network glitch, UI bug, platform compatibility). AI calls AskUserQuestion, user can't see/interact with it, rejects it — AI retries the same call or stalls. This creates a deadlock: the skill mandates AskUserQuestion, but the tool can't deliver.	After ANY AskUserQuestion is rejected (user denies) or fails twice, you MUST fall back to inline text prompts: output options as numbered plain text in the conversation body, wait for the user to type their choice. See "AskUserQuestion Fallback Protocol" below.
7	Skipping or truncating the smart recommendation table — AI rushes to AskUserQuestion without first displaying the full P1+P2 recommendation table. Instead, it embeds a partial list inside AskUserQuestion options (e.g. "僅訓練 #19 SVM + #21 LR") or shows only P1 models. The user sees nothing to choose from, so they reject the dialog.	Before the select/all AskUserQuestion, you MUST display the COMPLETE smart recommendation table with ALL P1+P2 models in every category (Traditional ML, Deep Learning, Transformer), each with its global #. The table must be shown as inline markdown text in the conversation body — NOT inside AskUserQuestion option labels.
8	Wrong category counts in smart recommendation headers — AI writes headers like "傳統機器學習（9 個）" / "深度學習（6 個）" / "Transformer（4 個）" by eyeballing the model list, without cross-referencing actual counts.	After assembling the recommendation table, you MUST verify `[N1]` / `[N2]` / `[N3]` header counts against `step3_scheme.py` stdout's "═══ P1+P2 by category ═══" section which prints ground-truth counts.
9	Using `python3` on Windows — Windows installs Python as `python`, not `python3`. Running `python3` causes exit code 49 and/or opens the Microsoft Store.	On Windows, use `python` (NOT `python3`) for ALL Bash tool calls. This rule has NO exceptions.
10	Inline Python to explore CSV files — AI writes `python -c "import csv; open('file.csv', encoding='utf-8')"` which hardcodes the wrong encoding and crashes on non-UTF-8 files.	NEVER use inline Python to read/explore CSV files. Use `step1_analyze.py` (auto-detects encoding from 8 codecs) or `head`/`wc` bash commands instead. If the script itself fails with encoding error, only then add `--encoding <detected>`.
11	Unix-style paths (`/c/Users/...`) passed to Python — Git Bash auto-translates `/c/Users/...` for shell commands, but Python's `open()`/`pd.read_csv()` bypass the shell and see the literal Unix path, which does not exist on Windows.	Bash commands (ls, head, wc) can use `/c/Users/...` paths. Python scripts MUST use Windows-native paths: `C:\Users\...` or `C:/Users/...`. Before any Python script call, verify `--csv` and `--output-dir` use Windows format.
12	Language mismatch — AI replies in wrong language — User's first message is in English but AI replies in Chinese, or vice versa. AI defaults to the skill document's language instead of detecting the user's language. This is the #1 most frequent violation across all executions.	⛔ Before ANY output (including the Stage 1 analysis summary), you MUST detect the user's language from their FIRST message in the conversation. See "⛔ LANGUAGE DETECTION" section below. At every user-facing output point, run the ⛔ 2-second language self-check: "What language did the user write their first message in?" → reply in THAT language.

⛔ When you encounter text marked with ⛔ in this skill, treat it as non-negotiable. ⛔ directives supersede any heuristic you may have learned from other contexts.

⛔ AskUserQuestion Fallback Protocol

AskUserQuestion renders as a UI modal dialog. This dialog can fail to display due to Claude Code UI bugs, network issues, or platform quirks. When this happens, the user sees nothing clickable — they can only type in the chat input.

Detection rule: if an AskUserQuestion is rejected (the tool result says "The user doesn't want to proceed with this tool use") twice for the same question, the dialog is likely invisible to the user. Do NOT call AskUserQuestion a third time for the same question.

Fallback procedure: 1. Output the options as numbered inline text in the conversation body, using the exact same labels and descriptions from the AskUserQuestion definition. 2. End with a clear prompt like: "請直接回複選項編號（如 1）" / "Reply with the option number (e.g. 1)." 3. Wait for the user's text reply in the conversation. 4. Parse the user's reply and proceed exactly as if AskUserQuestion had returned that value.

Example fallback (ZH):

對話方塊未能顯示，以下為文本選項：

[1] 從推薦模型中選擇（直接輸入編號）→ 如 21,19,22,32,81,82,92,96
[2] 顯示全部 153 個模型，我按編號選擇

請直接回複選項編號（如 1 或 2）。

This fallback applies to EVERY 🔴 阻斷點 in this skill. Each AskUserQuestion in the pipeline is a potential failure point. The fallback must preserve all option labels, descriptions, and semantics.

⛔ LANGUAGE DETECTION (READ THIS FIRST — VIOLATION #12)

⛔ This section is the SINGLE MOST VIOLATED rule in this skill. Read it before you produce ANY output — including any analysis summary, any table, any AskUserQuestion, any inline text. A single line of output in the wrong language is a violation.

Detection Rule

Detect the user's language from their first message in the conversation (the message that triggered this skill):

User's first message	ALL subsequent output language
Chinese (contains CJK characters as primary script)	Chinese (中文) — every line of text, every table header, every AskUserQuestion label/description/preview, every result summary, every model card, every training plan
English (Latin script only, or Latin-dominant)	English — every line of text, every table header, every AskUserQuestion label/description/preview, every result summary, every model card, every training plan

This rule applies to EVERY user-facing interaction point: Stage 1 analysis summary, Stage 2 split options, Stage 3 interactive model selection, Stage 4 training results, Stage 5 final artifact confirmation.

⛔ Mandatory Pre-Output Language Self-Check

Before EVERY user-facing output block (after each Bash tool call, before each AskUserQuestion, before each inline text prompt), run this 2-second mental check:

⛔ Language Self-Check:
1. What language did the user write their first message in? → [EN / ZH]
2. What language am I about to output? → [EN / ZH]
3. Do they match? → [YES / NO — if NO, STOP and rewrite]

If #3 is NO, do NOT output. Rewrite in the correct language.

Detection Heuristics

Chinese-first indicators (if ANY are true, treat as Chinese): - Primary content script is CJK (Hanzi): 中文, 文本分類, 情感分析, 訓練模型, etc. - Message contains CJK punctuation: 。、「」 - Message mixes Chinese + English but the INTENT is Chinese (e.g. "用這個 CSV train 一個模型" → Chinese)

English-first indicators (all must be true, otherwise default to Chinese if CJK present): - Primary content script is Latin: "train a model", "use this csv", etc. - NO CJK characters in the primary message content - Message may contain English + code/file paths (e.g. "use desktop nlp_test_5000.csv to model" → English)

Edge cases: - Pure file path / command only (e.g. /classify-text-binary-us): check the message BODY for language cues. If truly ambiguous, default to English. - Mixed script with CJK: ALWAYS treat as Chinese (CJK presence overrides Latin) - User switches language mid-conversation: STICK with the language detected from the FIRST message — do not switch

⛔ First-Output Gate

Before your FIRST text output after this skill is triggered, you MUST:

Re-read the user's first message
Identify the primary script (CJK or Latin)
Decide: Chinese or English
State the decision silently (do NOT output "Detected language: X" — just output in the correct language)
Produce ALL subsequent output in that language

⛔ If you catch yourself about to output in the wrong language, STOP immediately. Delete the draft. Rewrite in the user's language.

Common Failure Patterns (DO NOT REPEAT)

Pattern	Example	Why it happens	Fix
Skill doc influence	User writes English, AI replies in Chinese because skill doc is bilingual	AI defaults to skill doc's mixed language instead of detecting user's language	⛔ The skill doc's language is IRRELEVANT. Only the user's first message matters.
Template copy-paste	AI copies ZH template verbatim for an EN user	AI uses template without checking which language variant to use	Always check: "Which language templates should I use for THIS user?"
Mid-conversation drift	AI starts in EN, gradually adds ZH phrases	AI forgets the language rule after many tool calls	Re-run the self-check before EVERY output block
Mixed AskUserQuestion	AI creates AskUserQuestion with EN question but ZH labels	AI mixes templates from both languages	Use ONLY the language variant matching the user

DO NOT default to the language of this skill document. DO NOT mix languages. The user's first-message language overrides everything.

The templates below are provided in BOTH languages — use ONLY the set matching the user's language. Before using any template, verify it matches the detected language.

流水線概覽 / Pipeline Overview

共 5 個步驟 / 5 stages:

Stage 1 (分析+環境) → Stage 2 (劃分) → Stage 3 (方案) → Stage 4 (訓練) → Stage 5 (儲存)
Stage 1 (Analysis+Env) → Stage 2 (Split) → Stage 3 (Scheme) → Stage 4 (Train) → Stage 5 (Save)
       🔴 問答 Q&A            🔴 問答 Q&A          🔴 問答×4 Q&A×4       🔴 問答 Q&A        🔴 問答 Q&A

🔴 = 必須呼叫 AskUserQuestion 的互動步驟，任何情況下不得跳過。 🔴 = Mandatory AskUserQuestion interaction point. Must NOT be skipped under any circumstance.

所有指令碼位於 scripts/ 目錄下，可獨立執行。 All scripts are in the scripts/ directory and can run independently. 共享模組 / Shared modules: utils.py, preprocessing.py, model_factory.py, mlflow_utils.py, report.py, deploy.py.

⛔ 全域性輸出格式規則（GLOBAL — ALL STAGES）

⛔ Global Output Format Rule

訓練結果彙總表必須使用平鋪列名格式，違反此規則的輸出將被視為格式錯誤。 Training result summary tables MUST use flat column headers. Any output violating this format is an error.

列名格式 / Column format: {Dataset}_{Metric} (N={sample_count}) — 全部寫在一行表頭內 / all in a single header row.

3-way split example (applies to Stage 4 & Stage 5):

| Model | Mode | Train_Acc (N=3999) | Train_F1 (N=3999) | Train_AUC (N=3999) | Valid_Acc (N=501) | Valid_F1 (N=501) | Valid_AUC (N=501) | Test_Acc (N=500) | Test_F1 (N=500) | Test_AUC (N=500) | Time |

2-way split example:

| Model | Mode | Train_Acc (N=4500) | Train_F1 (N=4500) | Train_AUC (N=4500) | Val_Acc (N=CV) | Val_F1 (N=CV) | Val_AUC (N=CV) | Test_Acc (N=500) | Test_F1 (N=500) | Test_AUC (N=500) | Time |

禁止事項 / Prohibited (never use under any circumstance): - ❌ 兩行表頭 / two-row headers (group_header) - ❌ 合併單元格 / merged cells (colspan/rowspan) - ❌ 僅標註 Acc/F1/AUC 而不註明屬於哪個資料集 / metrics without dataset prefix - ❌ Train/Val/Test 作為父行、子行只有指標名的層級結構 / hierarchical header structure

Python enforcement: utils.print_table(group_header=...) with any non-None value raises ValueError.

中文亂碼預防 / UTF-8 Encoding (Windows)

關鍵 / Critical: On Windows, set UTF-8 encoding before running any Python script:

# Run before every Python script call:
export PYTHONIOENCODING=utf-8
# Or in PowerShell:
$env:PYTHONIOENCODING="utf-8"

所有指令碼在呼叫時都應加上 PYTHONIOENCODING=utf-8 環境變數
Always prefix python commands with PYTHONIOENCODING=utf-8
使用 Bash 工具執行命令時，始終在 python 命令前設定此變數

Windows 路徑處理（Git Bash / MSYS 環境）

Windows Path Handling (Git Bash / MSYS)

問題 / Problem: Git Bash auto-translates /c/Users/... → C:\Users\... for shell commands, but Python's open() / pd.read_csv() bypass the shell and see the literal Unix path, which does not exist on Windows.

head -3 /c/Users/YourUsername/Desktop/file.csv → Bash OK (shell translates)
pd.read_csv('/c/Users/YourUsername/Desktop/file.csv') → Python FileNotFoundError

規則 / Rule: In Windows, pass Windows-native paths to Python scripts:

# Correct ✓ — Windows native paths
python script.py --csv "C:\Users\YourUsername\Desktop\file.csv"
python script.py --csv "C:/Users/YourUsername/Desktop/file.csv"

# Wrong ✗ — Unix-style paths (Python cannot resolve)
python script.py --csv "/c/Users/YourUsername/Desktop/file.csv"

AI self-check before every Python call: 1. Is the current system Windows? 2. Are paths passed to --csv, --output-dir etc. in C:\... or C:/... format? 3. If a path came from a Bash tool output, has it been converted to Windows format?

⛔ Windows Python 命令（禁止 `python3`）/ ⛔ Windows Python Command (no `python3`)

Windows 上 Python 預設安裝為 python（不是 python3）。使用 python3 會導致 exit code 49 或彈出 Microsoft Store。

平臺	命令
Windows	`python`（永遠使用）
Linux / macOS	`python3` 或 `python`（優先 `python3`）

⛔ 在 Windows 下，所有 Bash 呼叫必須在 Python 命令前使用 python，不得使用 python3。 此規則無例外。

⛔ 禁止內聯 Python 探索 CSV（編碼安全）/ ⛔ No Inline Python for CSV Exploration

Bash 工具中編寫內聯 Python（python -c "..."）探索 CSV 檔案時，AI 經常硬編碼 encoding='utf-8'，導致 latin-1 等非 UTF-8 檔案解碼失敗（exit code 1）。

⛔ 規則：不得使用內聯 Python 讀取或探索 CSV 檔案。 以下為禁止示例：

# ✗ 禁止 — 內聯 Python 硬編碼編碼
python -c "import csv; ... open('file.csv', encoding='utf-8')"
python -c "import pandas as pd; pd.read_csv('file.csv')"

替代方案（按優先順序）：

首選：直接執行 step1_analyze.py（不加 --encoding，讓其自動檢測） bash PYTHONIOENCODING=utf-8 python scripts/step1_analyze.py --csv <path> --text-col <name> --label-col <name>
次選：使用 head / cut / wc 等 Bash 命令快速檢視檔案結構 bash head -5 /c/Users/.../file.csv wc -l /c/Users/.../file.csv
僅當指令碼本身報編碼錯誤時：才手動指定 --encoding bash PYTHONIOENCODING=utf-8 python scripts/step1_analyze.py --csv <path> ... --encoding latin-1

原因：所有 step 指令碼內部使用 utils.read_csv_safe()，該函式自動嘗試 ['utf-8', 'utf-8-sig', 'gbk', 'gb2312', 'gb18030', 'latin-1', 'cp1252', 'ISO-8859-1'] 編碼鏈。內聯 Python 跳過了此保護機制。

如何使用此技能 / How to Use This Skill

Trigger後，按照以下 4 步驟執行。 After triggering, follow the 4-stage pipeline below.

> 🚫 Core Rule / 核心規則: Every interaction step marked 🔴 or AskUserQuestion in any Stage is MANDATORY — must NOT be skipped under any circumstance. After outputting a table/result, you MUST immediately call AskUserQuestion to confirm with the user; do NOT stop at a text description and wait for the user to reply. If AskUserQuestion is rejected twice, trigger the AskUserQuestion Fallback Protocol (see above).

每個指令碼階段獨立執行；如需瞭解內部細節可以讀取指令碼檔案。 Each script stage runs independently; read the script file for internal details if needed. 指令碼會自動將自身加入 sys.path 以便直接匯入同級模組。 Scripts auto-add themselves to sys.path to import sibling modules directly.

⛔ Stage 0：操作前置檢查 / Pre-Flight Check (READ BEFORE EVERY BASH CALL)

在任何 Python 指令碼呼叫之前，必須完成以下 4 項檢查。違反任一項 = 必出錯。

┌─────────────────────────────────────────────────────────────┐
│                    ⛔ STAGE 0 CHECKLIST                      │
│                                                             │
│  □ 1. Python cmd: Windows → python / Linux → python3        │
│  □ 2. Path format: Python → Windows native (C:\... or C:/…) │
│  □ 3. Encoding: no inline Python for CSV → use step1_analyze│
│  □ 4. UTF-8 env: export PYTHONIOENCODING=utf-8              │
│                                                             │
│  ⛔ ALL 4 boxes MUST be [x] before running ANY python cmd.  │
└─────────────────────────────────────────────────────────────┘

#	Check	Why	Correct
1	Use `python` not `python3`	Windows only has `python`; `python3` → exit 49	`python script.py ...`
2	Windows-format paths for Python	`/c/Users/...` works for Bash, not Python	`--csv "C:\Users\..."` or `C:/Users/...`
3	No inline Python for CSV	`python -c "open('f.csv', encoding='utf-8')"` dies on latin-1/gbk	Run `step1_analyze.py` directly (auto-detects 8 encodings)
4	Set PYTHONIOENCODING=utf-8	Windows console GBK encoding causes garbled output	Prefix every python command with `PYTHONIOENCODING=utf-8`

最安全的啟動命令模板（Windows）：

# 第一步永遠是：直接執行 step1_analyze.py，不先探索 CSV
cd "技能的 scripts 目錄" && PYTHONIOENCODING=utf-8 python scripts/step1_analyze.py \
  --csv "C:\Users\...\file.csv" --text-col <列名> --label-col <列名> \
  --output-dir "C:\...\output"

如果 step1_analyze.py 報編碼錯誤（罕見），再加 --encoding：

# step1_analyze.py 內部已自動嘗試 8 種編碼。僅當它報錯時才手動加：
PYTHONIOENCODING=utf-8 python scripts/step1_analyze.py --csv "C:\..." ... --encoding latin-1

⛔ 此 Stage 0 檢查在每次新的 Bash 呼叫前必須在腦中過一遍。違反 #1-#3 中任一項 = 浪費 3-5 輪對話修復可避免的錯誤。

Stage 1：資料分析 / Data Analysis

Run scripts/step1_analyze.py to analyze the dataset and environment:

PYTHONIOENCODING=utf-8 python scripts/step1_analyze.py \
  --csv <path> --text-col <name> --label-col <name> \
  [--output-dir output] [--encoding <enc>]

Generates output/analysis.json and outputs to console. Analysis dimensions:

Part 1: Environment Analysis / 第 1 部分：環境分析

OS & Hardware / 作業系統與硬體：
OS type (Linux / Windows / macOS)
CPU: physical cores, logical cores, frequency (GHz)
RAM: total (GB)
GPU: name, count, VRAM (GB), CUDA cores, CUDA support
Storage / 儲存：
Free space (GB) / total (GB)
Disk type (SSD / HDD / NVMe)
Read/write speed (MB/s, via 50MB temp file benchmark)
Warns if free space < 10GB (large model downloads may fail)
Python Environment / Python 環境 (against references/requirements.yaml):
Total required, installed (version-matched)
Missing packages (not installed)
Outdated packages (installed but version mismatch)
"All dependencies ready" when all pass

Part 2: Modeling Sample Analysis / 第 2 部分：建模樣本分析

Data Scale & Distribution / 資料規模與分佈：
Total samples, class distribution (0/1 counts & ratio), class ratio
Text length distribution (mean, median, min/max, stddev, P25/P75/P90/P95/P99)
Missing label count
Data Quality & Features / 資料質量與特徵：
Missing rate, duplicate rate
Vocabulary Richness / 詞彙豐富度: vocab size, total tokens, Type-Token Ratio (TTR), Hapax Legomena ratio (words appearing only once), repeated word ratio
Syntactic Complexity / 句法複雜度: avg sentence length (words), avg sentences per text, avg clauses per sentence (heuristic subordinating conjunction detection)
Language Characteristics / 語言特性：
Is English dominant (langdetect preferred, Latin char ratio fallback)
Language distribution: sample count & ratio per language code (sampled 500 texts)

Warning Mechanisms / 警告機制: - Class imbalance (minority class <5%) - High missing rate - Non-English content >30% - Low TTR (highly repetitive vocabulary, likely narrow-domain text) - High TTR (extremely diverse vocabulary, potentially broad-domain or noisy text) - High Hapax ratio (many rare words, data may be sparse) - Very short/long sentences (social media fragments vs academic/legal text) - Very low clause count (text may be overly simple/informal) - Disk free space < 10GB - Python dependencies missing or version mismatch

If validation fails (wrong column names, labels not 0/1), the script exits with a clear error message. Non-English >30% triggers a warning but continues. Report the analysis summary to the user (matching the user's language per the Language Detection rule), highlighting all warnings, then proceed to Environment Confirmation below.

環境確認 / Environment Confirmation

分析結果展示後，根據分析結果與環境要求進行對比，與使用者互動確認後才能進入 Stage 2。 After showing analysis results, compare against environment requirements and get user confirmation before Stage 2.

Step 1: 環境評估 / Environment Assessment

Call utils.evaluate_environment(analysis) to evaluate.

Pass conditions / 通過條件： - Python >= 3.8 - RAM >= 4 GB - Disk free space >= 5 GB - Key packages (numpy, pandas, scikit-learn, torch, transformers) installed

Results display (ZH example): (use matching language per Language Detection rule)

## 環境評估結果

| 檢查項 | 要求 | 當前值 | 狀態 |
|--------|------|--------|------|
| Python 版本 | >= 3.8 | 3.11.9 | 通過 |
| 記憶體 | >= 4 GB | 16.0 GB | 通過 |
| 磁碟剩餘空間 | >= 5 GB | 167.4 GB | 通過 |
| GPU | 推薦 | RTX 4060 (8GB, 3840 CUDA) | 充足 |
| Python 包關鍵依賴 | 已安裝 | 18/19 | 通過 |

評估結論: 環境滿足要求，可以繼續。

Step 2: 不滿足要求時的處理 / Handling Unmet Requirements

If can_proceed == False, inform the user which conditions are not met:

EN: "The environment does not meet the minimum requirements for training. The following issues must be resolved first:" ZH: "當前環境不滿足建模訓練的最低要求，以下問題需要先解決："

列出所有 blockers，每條以 - [BLOCKED] 開頭。 / List blockers, each prefixed with - [BLOCKED]. 列出 warnings，每條以 - [WARN] 開頭。 / List warnings, each prefixed with - [WARN]. 列出 recommendations，每條以 - [TIP] 開頭。 / List recommendations, each prefixed with - [TIP].

Do NOT continue to Stage 2. Ask the user to resolve issues and re-run Stage 1.

Step 3: 滿足要求 — 確認安裝缺失的 Python 包 / Install Missing Packages

If can_proceed == True:

🚫 Blocking point — You MUST call AskUserQuestion here to ask whether to install missing packages. Do not skip.

Use AskUserQuestion to ask whether to install missing packages.

Only ask this question when python_env.missing is non-empty.

EN Question: "Some Python packages are missing. Do you want to install them now?" ZH Question: "部分 Python 包未安裝，是否現在安裝？"

Value	EN Label	ZH Label
yes	"Yes, install missing packages"	"是，安裝缺失的包"
no	"No, skip (I'll install manually)"	"否，跳過（我將手動安裝）"

如果使用者選擇 yes： - 執行 utils.install_missing_packages(missing_packages) - 顯示安裝結果（installed / failed）

PYTHONIOENCODING=utf-8 python -c "
import sys; sys.path.insert(0, 'scripts')
from utils import install_missing_packages
result = install_missing_packages(<missing_list>)
"

如果有包安裝失敗，提醒使用者手動安裝：pip install <failed_packages>

Step 4: 指定專案目錄 / Specify Project Directory

Prompt the user for the project output directory:

EN: "Where should the project output be saved? (Enter a path, or type 'auto' to use output/)" ZH: "專案輸出儲存到哪個目錄？（直接輸入路徑，或回覆「auto」使用預設 output/）"

User enters auto or blank: use output/ under the current working directory.
User enters a path: verify parent exists and is writable; ask whether to create if not.
All subsequent stages use --output-dir <user-specified-path>.

Step 5: 確認並進入 Stage 2 / Confirm and Proceed to Stage 2

After package installation and directory setup, output confirmation:

EN:

## Environment Setup Complete

- Project directory: <path>
- Python packages: all ready
- Ready to proceed to Stage 2: Model Scheme Generation.

ZH:

## 環境配置完成

- 專案目錄：<path>
- Python 依賴：已就緒
- 即將進入 Stage 2：模型方案生成。

然後直接進入 Stage 2。

Stage 2：樣本劃分 / Data Split

Run scripts/step2_split.py to perform train/validation/test split:

PYTHONIOENCODING=utf-8 python scripts/step2_split.py \
  --csv <path> --text-col <name> --label-col <name> \
  --split-type <random_2way|column_2way|random_3way|column_3way> \
  [--train-ratio 0.8] [--valid-ratio 0.1] [--test-ratio 0.2] \
  [--split-column <name>] [--train-value <val>] [--valid-value <val>] [--test-value <val>] \
  [--output-dir <dir>] [--encoding <enc>] [--seed 42]

Generates <project_dir>/split_info.json.

4 split types supported / 指令碼支援 4 種劃分方式：

Type	Description / 說明
`random_2way`	Random train-test (default 8:2), stratified / 隨機 train-test（預設 8:2），分層保持類別比例
`column_2way`	CSV column value-based train-test / 基於 CSV 列值劃分 train-test
`random_3way`	Random train-valid-test (default 8:1:1), stratified / 隨機 train-valid-test（預設 8:1:1），分層
`column_3way`	CSV column value-based train-valid-test / 基於 CSV 列值劃分 train-valid-test

Stage 2 Interactive Split Flow / 互動式樣本劃分流程：

🚫 Blocking point — You MUST call AskUserQuestion here to ask how the data should be split. Do not skip.

Present these options via AskUserQuestion:

EN Question: "How should the data be split?" ZH Question: "請選擇資料劃分方式："

Value	EN Label	ZH Label
random_2way	"Random train-test (8:2) — CV on train, test held out"	"隨機劃分 train-test（8:2）— train 上做 CV，test 保留不參與訓練"
column_2way	"Use CSV column to split train-test"	"使用 CSV 列值劃分 train-test"
random_3way	"Random train-valid-test (8:1:1) — tune on train+valid, test held out"	"隨機劃分 train-valid-test（8:1:1）— train+valid 上調優，test 保留"
column_3way	"Use CSV column to split train-valid-test"	"使用 CSV 列值劃分 train-valid-test"

If user chooses column_2way or column_3way: ask for the split column name and values for each set.
If user chooses random split: ask for custom ratios (optional, defaults 8:2 or 8:1:1).

After confirmation, run the script, display split summary (sample counts per set, class distribution), save split_info.json.

All subsequent stages use this split. The test set stays held out throughout training, used only for final comparison. Confirm with user before Stage 5 full-retrain.

Stage 3：模型方案 / Model Scheme

Run scripts/step3_scheme.py to generate model recommendations:

PYTHONIOENCODING=utf-8 python scripts/step3_scheme.py \
  --analysis <project_dir>/analysis.json \
  [--output-dir <project_dir>]

<project_dir> is the project directory from Stage 1 (default output/).

Important: NO web search needed before running this script. Default params come from references/model_params.md. Only search the web if that file lacks needed info or the user explicitly asks about specific parameter details.

Generates <project_dir>/model_scheme.json with three model categories:

A. Traditional ML: Sparse (Count/TF-IDF/OneHot, 1-gram+2-gram) with SVM(LinearSVC)/SVM(RBF)/LogisticRegression/RandomForest/MultinomialNB; plus Dense (GloVe/Word2Vec/fastText averaged) with SVM/LR/RF
B. Deep Learning: TextCNN, BiLSTM, StackedLSTM, LSTMAttention, BiGRU, StackedGRU, GRUAttention — each with GloVe/Word2Vec/fastText embeddings (frozen + fine-tuned)
C. Transformers: BERT, RoBERTa, DeBERTa, DistilBERT, ALBERT, ELECTRA, XLNet — each supports full fine-tuning, feature extraction, partial freeze, LoRA (PEFT)

Priority assignment via data-driven heuristic rules: - P1 (Baseline): Best traditional ML models — fast, reliable performance reference - P2 (Recommended): Best fit for the current data — covers multiple architectures - P3 (Exploratory): Worth trying if resources permit

After Stage 2 completes, follow the interactive model selection flow below. All choices are user-driven; P1 is no longer auto-trained by default.

Stage 3 Interactive Model Selection (Smart Defaults) / Stage 3 互動式模型選擇

以下所有 AskUserQuestion 選項、模型卡片和訓練計劃均有中英文兩套模板。 僅使用與使用者語言匹配的版本。

4 步互動流程：

第一步：智慧推薦+確認 → 第二步：訓練方式 → 第三步：調參配置 → 第四步：確認方案

核心設計：基於 Stage 1 資料分析結果自動篩選高優先順序模型組合，使用者只需確認或微調，無需從 153 個模型中逐一挑選。

關鍵規則：使用者第一步確認/調整後的模型，後續每一步對話方塊中都必須列出這些已選模型，讓使用者清楚當前正在為哪些模型做配置。

第一步：智慧推薦 + 使用者確認 / Step 1: Smart Recommendation + User Confirmation

執行 step3_scheme.py 後，讀取 model_scheme.json，根據以下啟發式規則自動篩選模型：

篩選規則 / Selection Rules（executed in priority order / 按優先順序從高到低執行）：

Rules based on 2024-2025 text classification benchmarks: - Reusens et al. (2024) "Evaluating text classification: a benchmark study" — comprehensive Trad ML vs Transformer comparison - 2024 scikit-learn + TF-IDF benchmarks: Linear SVM > Logistic Regression > Random Forest > Multinomial NB - SVM beat CNN/RNN/LSTM on 20 Newsgroups benchmark - BERT training ~360× slower than SVM, inference 50-200× slower, accuracy gain only 2-5% (0-3% on clear-topic text)

Rule 1 / 規則 1: Data Size Threshold (most critical decision factor / 最關鍵的決策依據)

Samples / 樣本量	Threshold / 閾值判斷
< 1,000	Traditional ML only / 傳統 ML 獨佔. Transformer/DL overfit too easily, excluded. Use only sparse vectorizers (Count/TF-IDF), no dense embeddings.
1,000 – 10,000	Transition zone / 過渡區. Traditional ML still primary. If GPU available, add 1-2 lightweight Transformers (DistilBERT / ALBERT) as reference.
10,000 – 100,000	Transformer advantage zone / Transformer 優勢區. Pre-trained models start consistently beating traditional ML. Traditional ML kept as baseline reference; Transformers become primary recommendations.
> 100,000	Large-scale zone / 大規模區. Transformers fully dominant. Traditional ML reduced to strongest baseline only (SVM + TF-IDF bigram). Consider large models and more DL variants.

Rule 2 / 規則 2: Mandatory Baselines (always included, regardless of sample size / 必選基線，始終包含，不論資料量)

SVM (LinearSVC) + TF-IDF (1,2-gram) — Traditional ML text classification ceiling (#1 in multiple 2024 benchmarks), seconds-level training, < 10ms inference
Logistic Regression + TF-IDF (1,2-gram) — Calibrated probability output, < 2% F1 gap vs SVM

Rule 3 / 規則 3: Class Imbalance (minority < 20%) → Add / 類別不平衡（少數類佔比 < 20%）→ 追加

Random Forest + TF-IDF (1,2-gram) — Tree models have different inductive bias for imbalanced data; class_weight='balanced' enabled by default
If GPU available and samples >= 5K: + SVM (RBF) + GloVe 300d — RBF kernel significantly outperforms sparse TF-IDF on dense vectors

Rule 4 / 規則 4: Graduated Additions by Sample Size / 按樣本量梯度追加

4a. Tiny (< 1,000) / 微小樣本： - Multinomial NB + TF-IDF (1-gram) — Extremely fast (< 1s training), Naive Bayes assumption is an advantage on small data - No dense embeddings, DL, or Transformers

4b. Small (1,000 – 5,000) / 小樣本： - Multinomial NB + Count (1,2-gram) — Discrete count features friendlier to NB - SVM (RBF) + GloVe 300d — Dense embeddings more expressive than sparse on small data - No DL/Transformers (insufficient data for stable training)

4c. Medium (5,000 – 10,000) / 中等樣本： - SVM (RBF) + GloVe 300d, Logistic Regression + GloVe 300d — Dense embedding variants - If GPU available + VRAM >= 4GB: DistilBERT (full_ft) — lightweight, only needs 4-6GB VRAM - If GPU available + VRAM >= 8GB: BERT base (full_ft) — standard text classification reference - If GPU available: TextCNN + FastText 300d (frozen + fine-tuned), BiLSTM/BiGRU + FastText 300d (frozen) — lightweight DL, fast training, good DL baselines

4d. Medium-Large (10,000 – 50,000) / 中大規模： - Random Forest + GloVe 300d — More data unlocks tree model potential - Logistic Regression + FastText 300d — Subword coverage helpful for domain text - If GPU + VRAM >= 4GB: DistilBERT (full_ft), ALBERT base (full_ft) - If GPU + VRAM >= 8GB: BERT base (full_ft), ELECTRA base (full_ft), RoBERTa base (full_ft) - DL: TextCNN + GloVe 300d (frozen), BiLSTM + GloVe 300d (frozen)

4e. Large (50,000 – 100,000) / 大規模： - On top of 4d, add: - Trad ML: SVM (RBF) + Word2Vec 300d, Random Forest + FastText 300d - DL: BiGRU + GloVe 300d (frozen), LSTM + Attention + GloVe 300d (frozen) - If GPU >= 12GB: DeBERTa v3 base (full_ft), RoBERTa large (full_ft)

4f. Very Large (> 100,000) / 超大規模： - On top of 4e, add: - Trad ML trimmed to 3-4 strongest baselines (SVM TF-IDF bigram + SVM GloVe + LR TF-IDF bigram + RF GloVe) - Transformers expanded: add BERT large, DeBERTa large, XLNet base - DL: add Stacked LSTM + FastText (fine-tuned), LSTM + Attention + FastText (fine-tuned)

Rule 5 / 規則 5: GPU VRAM Constraints / GPU 視訊記憶體約束

VRAM	Traditional ML	Deep Learning	Transformer
No GPU / CPU-only	All available (recommend 5-8 dense+sparse combos)	Skip (CPU DL training extremely slow)	Skip (BERT CPU training 10-50× slower)
< 4 GB	All available	TextCNN + frozen embeddings (lightweight)	DistilBERT feature_extraction / LoRA only
4-6 GB	All available	All frozen embedding variants	DistilBERT / ALBERT / ELECTRA small (full_ft + LoRA)
6-8 GB	All available	All frozen + fine-tuned	BERT base / RoBERTa base (full_ft, FP16, batch ≤ 16)
8-12 GB	All available	All	BERT/RoBERTa/ELECTRA/DeBERTa base (full_ft, FP32), XLNet base
>= 12 GB	All available	All	All base + large models (full_ft, FP32)

Emergency VRAM fallback / 視訊記憶體緊急回退規則： - If VRAM < 8GB but samples > 10K: Transformers default to LoRA (~0.1-0.5% params trained, 60-70% VRAM reduction) - If VRAM < 4GB and Transformer is required: only DistilBERT feature_extraction (fully frozen encoder)

Rule 6 / 規則 6: Text Feature Adaptation / 文本特徵適配

Feature / 特徵	Condition / 條件	Recommended Adjustment / 推薦調整
Short texts / 短文本	Avg length < 30 words	TextCNN ahead of BiLSTM (CNN more effective on short text); MNB performance improved
Long texts / 長文本	Avg length > 100 words	LSTM/GRU + Attention ahead of plain BiLSTM; XLNet ahead of BERT (better long-range dependencies)
High vocab diversity / 高詞彙多樣性	TTR > 0.7	FastText embeddings preferred (subword coverage for rare words)
Low vocab diversity / 低詞彙多樣性	TTR < 0.3	TF-IDF sparse features sufficient; dense embedding gains limited
Non-English > 10%	langdetect	FastText embeddings preferred (better multilingual subword coverage); warn user of data quality issue
High Hapax ratio / 高 Hapax 比率	> 30% words appear only once	Prefer FastText embeddings; skip-gram Word2Vec; Transformer feature extraction mode (freeze bottom layers, avoid overfitting noise)

Rule 7 / 規則 7: Domain Text Judgment / 領域文本判斷

General domain / 通用領域 (news, social media, reviews): Pre-trained model domain match high; Transformer recommendation strength +1
Vertical/Technical domain / 垂直/技術領域 (medical, legal, IT tickets, fintech compliance): Pre-trained models may underperform traditional ML (BERT subword tokenization breaks domain terminology)
Alert user: traditional ML often matches or beats BERT on domain text
If using Transformer: recommend LoRA/partial_ft (preserve general language knowledge + adapt to domain)
Prefer FastText embeddings (subword coverage for technical terms)

Rule 8 / 規則 8: Upper-Bound Control / 上限控制

Category / 類別	Min / 最小	Max / 最大	Note / 說明
Traditional ML / 傳統 ML	2 (SVM+LR mandatory baselines / 必選基線)	10	Sparse + dense embeddings total / 稀疏 + 稠密嵌入總計
Deep Learning / 深度學習	0 (no GPU / samples < 5K)	6	1 per encoder + embedding combo / 每個編碼器 + 詞向量組合計 1 個
Transformer	0 (no GPU / samples < 1K)	5	1 per model + training mode combo / 每個模型 + 訓練模式組合計 1 個
Total / 總計	4	15	When exceeding limit, trim by priority, keep P1 and P2 first / 超出上限時按優先順序裁剪，優先保留 P1 和 P2

Trim priority when exceeding limit / 裁剪優先順序（超出上限時從低優先順序開始移除）： 1. Remove P3 exploratory models first / 先移除 P3 探索模型 2. Deduplicate similar models (e.g., both SVM+TF-IDF unigram and bigram → keep bigram) / 同類模型去重 3. Remove redundant dense embedding variants (keep GloVe primary, Word2Vec/FastText secondary) / 移除冗餘稠密嵌入變體 4. Finally remove Transformer large variants, keep base / 最後移除 Transformer large 變體保留 base

⛔ Mandatory Display Rules (SMART RECOMMENDATION — non-negotiable / 強制展示規則 — 不可繞過):

Rule S1 — Must display complete recommendation table / 必須展示完整推薦表: Before calling the select/all AskUserQuestion, you MUST display the full smart recommendation table with ALL P1+P2 models in the conversation body (grouped by Traditional ML / Deep Learning / Transformer, listing every ✓-marked model in each group). Do NOT embed the table inside AskUserQuestion option labels. Do NOT show only the P1 subset. Do NOT abbreviate with "and N others."

Rule S2 — Category counts must be correct / 計數必須是正確的: The group header counts ### Traditional ML ([N1] models) / ### Deep Learning ([N2] models) / ### Transformer ([N3] models) MUST match the P1+P2 model counts per category from model_scheme.json. Before filling in counts, verify via either: - (A) Run python -c "import json; ..." to count models with priority<=2 per category from model_scheme.json - (B) Read step3_scheme.py stdout's "═══ P1+P2 by category ═══" section which gives ground-truth counts

Rule S3 — Post-display self-check / 展示後自檢: After displaying the smart recommendation table, you MUST output the following self-check template in conversation and verify every box:

✅ Smart Recommendation Self-Check:
- [ ] Displayed ALL P1+P2 models (total X: P1=N1, P2=N2)
- [ ] Traditional ML / Deep Learning / Transformer all three groups listed
- [ ] Each group header count [N] matches model_scheme.json category priority<=2 count
- [ ] Every row includes global model number
- [ ] Data diagnostics table displayed

Rule S4 — AskUserQuestion limited to select/all / AskUserQuestion 僅限 select/all: After displaying the table and completing self-check, call AskUserQuestion with options only select (choose from recommended) and all (show all 153 models). Do NOT provide baseline/tune/both options here. Do NOT pre-judge which models the user wants.

⛔ If any self-check checkbox is [ ] unchecked, do NOT proceed to AskUserQuestion. Fix and re-display.

篩選完成後，以文本表格展示推薦結果：

⛔ 強制規則：智慧推薦表的每一行必須包含全域性編號（# 列）。 編號從 step3_scheme.py 指令碼輸出的 # 列中獲取。指令碼已確保輸出包含編號列，你只需要照原樣使用。不得展示無編號列的推薦表。 此規則無例外。

ZH template:

## 智慧模型推薦

💡 直接輸入全域性編號選擇模型，如 `14,19,21,22,23,81,92`。支援逗號分隔和範圍選取。

### 資料診斷
| 維度 | 數值 | 判斷 |
|------|------|------|
| 樣本量 | [N] | [微小/小/中等/中大規模/大規模/超大] — [對應規則 4a-4f] |
| 平均文本長度 | [L] 詞 | [短文本/中等/長文本] — [對應編碼器偏好] |
| 詞彙量 / TTR | [V] 詞 / [TTR] | [低多樣性/正常/高多樣性] — [對應嵌入偏好] |
| GPU | [名稱 + 視訊記憶體]（或"僅 CPU"） | VRAM [檔位] — [可用模型級別] |
| 類別比 | [ratio] | [平衡/輕度不平衡/嚴重不平衡] — [追加 RF/不追加] |
| 語言 | [英語佔比]% 英語 | [純淨/混合] — [優先 FastText/標準] |
| Hapax 比率 | [H%] | [正常/偏高] — [凍結嵌入/特徵提取優先] |

### 傳統機器學習（[N1] 個）
| 編號 | 優先順序 | 模型 | 向量化器 | 推薦理由 |
|------|--------|------|---------|---------|
| 19 | P2 | SVM (LinearSVC) | TF-IDF (1,2-gram) | 多項 2024 基準第一；快速可靠 |
| 21 | P1 ★ | Logistic Regression | TF-IDF (1,2-gram) | 校準機率輸出，與 SVM 差距 < 2% F1 |
| 22 | P2 | Random Forest | TF-IDF (1,2-gram) | 類別不平衡，樹模型不同歸納偏置 |
| ... | ... | ... | ... | ... |

### 深度學習（[N2] 個）
| 編號 | 優先順序 | 編碼器 | 詞向量 | 推薦理由 |
|------|--------|--------|-------|---------|
| 81 | P2 | TextCNN | FastText 300d (微調) | CNN 擅長短文本，引數高效，VRAM 友好 |
| 82 | P2 | BiLSTM | FastText 300d (微調) | 序列建模標準基線 |
| ... | ... | ... | ... | ... |

### Transformer（[N3] 個）
| 編號 | 優先順序 | 模型 | 訓練方式 | 推薦理由 |
|------|--------|------|---------|---------|
| 92 | P2 | RoBERTa base (125M) | full_ft | 文本分類標準參照，VRAM 8GB 可跑 |
| 96 | P2 | DeBERTa v3 base (140M) | full_ft | 解耦注意力，文本分類常達最優 |
| ... | ... | ... | ... | ... |

> 💡 推薦理由基於 2024-2025 年多項基準研究。請在下方選項中選擇下一步操作。

EN template:

⛔ Mandatory rule: Every row of the smart recommendation table MUST include the global model number (# column). Numbers come from the # column output by step3_scheme.py. The script now guarantees this column is present — use it as-is. Never present a recommendation table without the ID column. No exceptions.

## Smart Model Recommendation

💡 Enter global model numbers to select, e.g. `14,19,21,22,23,81,92`. Supports comma-separated and ranges.

### Data Diagnostics
| Dimension | Value | Assessment |
|-----------|-------|------------|
| Samples | [N] | [tiny/small/medium/medium-large/large/very large] — [rule 4a-4f] |
| Avg text length | [L] words | [short/medium/long] — [encoder preference] |
| Vocab / TTR | [V] words / [TTR] | [low-diversity/normal/high-diversity] — [embedding preference] |
| GPU | [name + VRAM] (or "CPU-only") | VRAM [tier] — [available model level] |
| Class ratio | [ratio] | [balanced/mild imbalance/severe imbalance] — [+RF / skip] |
| Language | [EN%]% English | [pure/mixed] — [prefer FastText/standard] |
| Hapax ratio | [H%] | [normal/high] — [freeze embeddings/feature extraction preferred] |

### Traditional ML ([N1] models)
| # | Priority | Model | Vectorizer | Why |
|---|----------|-------|-----------|-----|
| 19 | P2 | SVM (LinearSVC) | TF-IDF (1,2-gram) | #1 in multiple 2024 benchmarks; fast, reliable |
| 21 | P1 ★ | Logistic Regression | TF-IDF (1,2-gram) | Calibrated probabilities; < 2% F1 gap vs SVM |
| 22 | P2 | Random Forest | TF-IDF (1,2-gram) | Class imbalance — different inductive bias |
| ... | ... | ... | ... | ... |

### Deep Learning ([N2] models)
| # | Priority | Encoder | Embedding | Why |
|---|----------|---------|----------|-----|
| 81 | P2 | TextCNN | FastText 300d (fine-tuned) | CNN excels at short texts; VRAM-efficient |
| 82 | P2 | BiLSTM | FastText 300d (fine-tuned) | Standard sequence modeling baseline |
| ... | ... | ... | ... | ... |

### Transformer ([N3] models)
| # | Priority | Model | Training Mode | Why |
|---|----------|-------|-------------|-----|
| 92 | P2 | RoBERTa base (125M) | full_ft | Standard reference for text classification; 8GB VRAM |
| 96 | P2 | DeBERTa v3 base (140M) | full_ft | Disentangled attention, often SOTA |
| ... | ... | ... | ... | ... |

> 💡 Recommendations based on 2024-2025 benchmark studies. Use the options below to proceed.

🚫 阻斷點 — 顯示完智慧推薦表格後，你必須在此處呼叫 AskUserQuestion 讓使用者選擇 select 或 all，不得跳過。若 AskUserQuestion 被拒兩次，立即執行 Fallback Protocol 切換為文本選項。

然後使用 AskUserQuestion 確認：

ZH Question: "以上推薦方案是否合適？" EN Question: "Does the recommendation above look good?"

Value	ZH Label	EN Label
select	"從推薦模型中選擇（直接輸入編號）"	"Select from recommended models (enter numbers directly)"
all	"顯示全部 153 個模型，我按編號選擇"	"Show all 153 models for manual selection by number"

select: User enters global model numbers directly from the smart recommendation table (e.g. 14,19,21,22,23,81,82,92,96). Do NOT use AskUserQuestion — just wait silently for the user to type their selection in the conversation. Parse the input, confirm, then proceed to Step 2 (training mode).
all: Display all 153 models for manual selection. After displaying all models, do NOT use AskUserQuestion — just wait silently for the user to type model numbers. Procedure:

Run step3_scheme.py --list-all <path to model_scheme.json> --output-dir <project_dir>.

⛔ Mandatory Display Rules (STRUCTURAL — non-negotiable / 強制展示規則 — 不可繞過):

The script outputs the full model list to stdout between  and , and also saves it to <project_dir>/model_list.md.

⛔ Critical Warning / 重要警告: Bash stdout output ≠ you have displayed it. Stdout is visible only to you, not to the user. You MUST use the Read tool to read the file separately and copy its contents into the conversation.

⛔ Step 1/3 — Read / 步驟 1/3: After running the script, use the Read tool on <project_dir>/model_list.md. Even if Bash stdout already showed the content, you MUST still call Read. ⛔ Step 2/3 — Copy / 步驟 2/3: Copy ALL content between  and  line-by-line into the conversation. No merging, abbreviation, or omission of P3 rows. ⛔ Step 3/3 — Self-Check / 步驟 3/3: After copying, output the ⛔ mandatory self-check template below and confirm every box. Any unchecked box = MUST redo from Step 1.

⛔ Prohibited (never do / 禁止，任何情況下不得使用): - Use phrases like "as shown above" / "如上所示" / "see output above" instead of actual display - Show only recommended model subset (P1+P2) and omit P3 - Merge rows, abbreviate, or use range notation instead of listing every row - Rely on Bash tool output as the display mechanism (user may not see it) - Skip the self-check template or output an incomplete one

⛔ No exceptions. Not outputting all 153 model rows in full = violation. Not outputting the self-check template = violation.

格式如下：

💡 推薦標註說明：P1 ★ + P2 = 智慧推薦（✓），P3 = 探索（—）。推薦總數顯示在表格末尾。

ZH template（全部模型展示）:

``` ## 全部 153 個模型（按模組分類）

💡 直接輸入編號選擇模型，如 `19,21,90,92,96,98`。支援逗號分隔和範圍選取（如 `1-5,8,12-15`）。

### A. 傳統機器學習 — 稀疏特徵（23個） | 編號 | 推薦 | 模型 | 向量化器 | 優先順序 | |------|------|------|---------|--------| | 1 | — | SVM (LinearSVC) | Count (1-gram) | P3 | | 2 | — | SVM (RBF kernel) | Count (1-gram) | P3 | | 3 | — | Logistic Regression | Count (1-gram) | P3 | | 4 | — | Random Forest | Count (1-gram) | P3 | | 5 | — | Multinomial NB | Count (1-gram) | P3 | | 6 | — | SVM (LinearSVC) | Count (1,2-gram) | P3 | | 7 | — | SVM (RBF kernel) | Count (1,2-gram) | P3 | | 8 | — | Logistic Regression | Count (1,2-gram) | P3 | | 9 | — | Random Forest | Count (1,2-gram) | P3 | | 10 | — | Multinomial NB | Count (1,2-gram) | P3 | | 11 | — | SVM (LinearSVC) | OneHot (1-gram) | P3 | | 12 | — | Logistic Regression | OneHot (1-gram) | P3 | | 13 | — | Multinomial NB | OneHot (1-gram) | P3 | | 14 | — | SVM (LinearSVC) | TF-IDF (1-gram) | P3 | | 15 | — | SVM (RBF kernel) | TF-IDF (1-gram) | P3 | | 16 | — | Logistic Regression | TF-IDF (1-gram) | P3 | | 17 | — | Random Forest | TF-IDF (1-gram) | P3 | | 18 | — | Multinomial NB | TF-IDF (1-gram) | P3 | | 19 | ✓ | SVM (LinearSVC) | TF-IDF (1,2-gram) | P2 | | 20 | — | SVM (RBF kernel) | TF-IDF (1,2-gram) | P3 | | 21 | ✓ | Logistic Regression | TF-IDF (1,2-gram) | P1 \u2605 | | 22 | ✓ | Random Forest | TF-IDF (1,2-gram) | P2 | | 23 | — | Multinomial NB | TF-IDF (1,2-gram) | P3 |

### B. 傳統機器學習 — 稠密嵌入（12個） | 編號 | 推薦 | 模型 | 詞嵌入 | 優先順序 | |------|------|------|--------|--------| | 24 | — | SVM (LinearSVC) | GloVe 300d | P3 | | 25 | ✓ | SVM (RBF kernel) | GloVe 300d | P2 | | 26 | ✓ | Logistic Regression | GloVe 300d | P2 | | 27 | — | Random Forest | GloVe 300d | P3 | | 28 | — | SVM (LinearSVC) | Word2Vec 300d | P3 | | 29 | — | SVM (RBF kernel) | Word2Vec 300d | P3 | | 30 | — | Logistic Regression | Word2Vec 300d | P3 | | 31 | — | Random Forest | Word2Vec 300d | P3 | | 32 | — | SVM (LinearSVC) | FastText 300d | P3 | | 33 | — | SVM (RBF kernel) | FastText 300d | P3 | | 34 | — | Logistic Regression | FastText 300d | P3 | | 35 | — | Random Forest | FastText 300d | P3 |

### C. 深度學習（54個） | 編號 | 推薦 | 編碼器 | 詞嵌入 | 嵌入方式 | 優先順序 | |------|------|--------|--------|---------|--------| | 36 | — | TextCNN | GloVe 300d | 凍結 | P3 | | 37 | BiLSTM | GloVe 300d | 凍結 | P3 | | 38 | LSTM | GloVe 300d | 凍結 | P3 | | 39 | Stacked LSTM | GloVe 300d | 凍結 | P3 | | 40 | LSTM + Attention | GloVe 300d | 凍結 | P3 | | 41 | BiGRU | GloVe 300d | 凍結 | P2 | | 42 | GRU | GloVe 300d | 凍結 | P3 | | 43 | Stacked GRU | GloVe 300d | 凍結 | P3 | | 44 | GRU + Attention | GloVe 300d | 凍結 | P3 | | 45 | TextCNN | GloVe 300d | 微調 | P2 | | 46 | BiLSTM | GloVe 300d | 微調 | P2 | | 47 | LSTM | GloVe 300d | 微調 | P3 | | 48 | Stacked LSTM | GloVe 300d | 微調 | P3 | | 49 | LSTM + Attention | GloVe 300d | 微調 | P3 | | 50 | BiGRU | GloVe 300d | 微調 | P3 | | 51 | GRU | GloVe 300d | 微調 | P3 | | 52 | Stacked GRU | GloVe 300d | 微調 | P3 | | 53 | GRU + Attention | GloVe 300d | 微調 | P2 | | 54 | TextCNN | Word2Vec 300d | 凍結 | P3 | | 55 | BiLSTM | Word2Vec 300d | 凍結 | P3 | | 56 | LSTM | Word2Vec 300d | 凍結 | P2 | | 57 | Stacked LSTM | Word2Vec 300d | 凍結 | P3 | | 58 | LSTM + Attention | Word2Vec 300d | 凍結 | P3 | | 59 | BiGRU | Word2Vec 300d | 凍結 | P3 | | 60 | GRU | Word2Vec 300d | 凍結 | P3 | | 61 | Stacked GRU | Word2Vec 300d | 凍結 | P3 | | 62 | GRU + Attention | Word2Vec 300d | 凍結 | P3 | | 63 | TextCNN | Word2Vec 300d | 微調 | P3 | | 64 | BiLSTM | Word2Vec 300d | 微調 | P3 | | 65 | LSTM | Word2Vec 300d | 微調 | P3 | | 66 | Stacked LSTM | Word2Vec 300d | 微調 | P3 | | 67 | LSTM + Attention | Word2Vec 300d | 微調 | P3 | | 68 | BiGRU | Word2Vec 300d | 微調 | P3 | | 69 | GRU | Word2Vec 300d | 微調 | P3 | | 70 | Stacked GRU | Word2Vec 300d | 微調 | P3 | | 71 | GRU + Attention | Word2Vec 300d | 微調 | P3 | | 72 | TextCNN | FastText 300d | 凍結 | P2 | | 73 | BiLSTM | FastText 300d | 凍結 | P3 | | 74 | LSTM | FastText 300d | 凍結 | P3 | | 75 | Stacked LSTM | FastText 300d | 凍結 | P3 | | 76 | LSTM + Attention | FastText 300d | 凍結 | P3 | | 77 | BiGRU | FastText 300d | 凍結 | P3 | | 78 | GRU | FastText 300d | 凍結 | P3 | | 79 | Stacked GRU | FastText 300d | 凍結 | P3 | | 80 | GRU + Attention | FastText 300d | 凍結 | P3 | | 81 | TextCNN | FastText 300d | 微調 | P3 | | 82 | BiLSTM | FastText 300d | 微調 | P3 | | 83 | LSTM | FastText 300d | 微調 | P3 | | 84 | Stacked LSTM | FastText 300d | 微調 | P3 | | 85 | LSTM + Attention | FastText 300d | 微調 | P3 | | 86 | BiGRU | FastText 300d | 微調 | P3 | | 87 | GRU | FastText 300d | 微調 | P3 | | 88 | Stacked GRU | FastText 300d | 微調 | P3 | | 89 | GRU + Attention | FastText 300d | 微調 | P3 |

### D. Transformer — Full Fine-tuning（16個） | 編號 | 推薦 | 模型 | 引數量 | 優先順序 | |------|------|------|--------|--------| | 90 | ✓ | BERT base (uncased) | 110M | P2 | | 91 | — | BERT large (uncased) | 340M | P3 | | 92 | RoBERTa base | 125M | P2 | | 93 | — | RoBERTa large | 355M | P3 | | 94 | — | DeBERTa base | 140M | P3 | | 95 | — | DeBERTa large | 400M | P3 | | 96 | DeBERTa v3 base | 140M | P2 | | 97 | — | DeBERTa v3 large | 400M | P3 | | 98 | DistilBERT base | 66M | P2 | | 99 | ALBERT base v2 | 12M | P2 | | 100 | — | ALBERT large v2 | 18M | P3 | | 101 | — | ELECTRA small | 13M | P3 | | 102 | ELECTRA base | 110M | P2 | | 103 | — | ELECTRA large | 335M | P3 | | 104 | XLNet base (cased) | 110M | P2 | | 105 | — | XLNet large (cased) | 340M | P3 |

### D. Transformer — Feature Extraction（16個） | 編號 | 推薦 | 模型 | 優先順序 | |------|------|------|--------| | 106 | — | — | BERT base — feature extraction | P3 | | 107 | BERT large — feature extraction | P3 | | 108 | RoBERTa base — feature extraction | P3 | | 109 | RoBERTa large — feature extraction | P3 | | 110 | DeBERTa base — feature extraction | P3 | | 111 | DeBERTa large — feature extraction | P3 | | 112 | DeBERTa v3 base — feature extraction | P3 | | 113 | DeBERTa v3 large — feature extraction | P3 | | 114 | DistilBERT base — feature extraction | P3 | | 115 | ALBERT base v2 — feature extraction | P3 | | 116 | ALBERT large v2 — feature extraction | P3 | | 117 | ELECTRA small — feature extraction | P3 | | 118 | ELECTRA base — feature extraction | P3 | | 119 | ELECTRA large — feature extraction | P3 | | 120 | XLNet base — feature extraction | P3 | | 121 | XLNet large — feature extraction | P3 |

### D. Transformer — Partial Fine-tuning（16個） | 編號 | 推薦 | 模型 | 優先順序 | |------|------|------|--------| | 122 | — | — | BERT base — partial fine-tuning | P3 | | 123 | BERT large — partial fine-tuning | P3 | | 124 | RoBERTa base — partial fine-tuning | P3 | | 125 | RoBERTa large — partial fine-tuning | P3 | | 126 | DeBERTa base — partial fine-tuning | P3 | | 127 | DeBERTa large — partial fine-tuning | P3 | | 128 | DeBERTa v3 base — partial fine-tuning | P3 | | 129 | DeBERTa v3 large — partial fine-tuning | P3 | | 130 | DistilBERT base — partial fine-tuning | P3 | | 131 | ALBERT base v2 — partial fine-tuning | P3 | | 132 | ALBERT large v2 — partial fine-tuning | P3 | | 133 | ELECTRA small — partial fine-tuning | P3 | | 134 | ELECTRA base — partial fine-tuning | P3 | | 135 | ELECTRA large — partial fine-tuning | P3 | | 136 | XLNet base — partial fine-tuning | P3 | | 137 | XLNet large — partial fine-tuning | P3 |

### D. Transformer — LoRA / PEFT（16個） | 編號 | 推薦 | 模型 | 優先順序 | |------|------|------|--------| | 138 | — | — | BERT base — LoRA | P3 | | 139 | BERT large — LoRA | P3 | | 140 | RoBERTa base — LoRA | P3 | | 141 | RoBERTa large — LoRA | P3 | | 142 | DeBERTa base — LoRA | P3 | | 143 | DeBERTa large — LoRA | P3 | | 144 | DeBERTa v3 base — LoRA | P3 | | 145 | DeBERTa v3 large — LoRA | P3 | | 146 | DistilBERT base — LoRA | P3 | | 147 | ALBERT base v2 — LoRA | P3 | | 148 | ALBERT large v2 — LoRA | P3 | | 149 | ELECTRA small — LoRA | P3 | | 150 | ELECTRA base — LoRA | P3 | | 151 | ELECTRA large — LoRA | P3 | | 152 | XLNet base — LoRA | P3 | | 153 | XLNet large — LoRA | P3 |

總計：153 個模型（P1 ★ = 1，P2 = 17 — 推薦 ✓，P3 = 135） ```

EN template:

``` ## All 153 Models (by Module)

💡 Enter numbers to select models, e.g. `19,21,90,92,96,98`. Supports comma-separated and ranges (e.g. `1-5,8,12-15`). 💡 Models marked ✓ are smart-recommended (P1+P2).

### A. Traditional ML — Sparse Features (23 models) | # | Rec | Model | Vectorizer | Priority | |---|-----|-------|-----------|----------| | 1 | — | SVM (LinearSVC) | Count (1-gram) | P3 | | 2 | — | SVM (RBF kernel) | Count (1-gram) | P3 | | 3 | — | Logistic Regression | Count (1-gram) | P3 | | 4 | — | Random Forest | Count (1-gram) | P3 | | 5 | — | Multinomial NB | Count (1-gram) | P3 | | 6 | — | SVM (LinearSVC) | Count (1,2-gram) | P3 | | 7 | — | SVM (RBF kernel) | Count (1,2-gram) | P3 | | 8 | — | Logistic Regression | Count (1,2-gram) | P3 | | 9 | — | Random Forest | Count (1,2-gram) | P3 | | 10 | — | Multinomial NB | Count (1,2-gram) | P3 | | 11 | — | SVM (LinearSVC) | OneHot (1-gram) | P3 | | 12 | — | Logistic Regression | OneHot (1-gram) | P3 | | 13 | — | Multinomial NB | OneHot (1-gram) | P3 | | 14 | — | SVM (LinearSVC) | TF-IDF (1-gram) | P3 | | 15 | — | SVM (RBF kernel) | TF-IDF (1-gram) | P3 | | 16 | — | Logistic Regression | TF-IDF (1-gram) | P3 | | 17 | — | Random Forest | TF-IDF (1-gram) | P3 | | 18 | — | Multinomial NB | TF-IDF (1-gram) | P3 | | 19 | ✓ | SVM (LinearSVC) | TF-IDF (1,2-gram) | P2 | | 20 | — | SVM (RBF kernel) | TF-IDF (1,2-gram) | P3 | | 21 | ✓ | Logistic Regression | TF-IDF (1,2-gram) | P1 \u2605 | | 22 | ✓ | Random Forest | TF-IDF (1,2-gram) | P2 | | 23 | — | Multinomial NB | TF-IDF (1,2-gram) | P3 |

### B. Traditional ML — Dense Embeddings (12 models) | # | Rec | Model | Embedding | Priority | |---|-----|-------|----------|----------| | 24 | — | SVM (LinearSVC) | GloVe 300d | P3 | | 25 | ✓ | SVM (RBF kernel) | GloVe 300d | P2 | | 26 | ✓ | Logistic Regression | GloVe 300d | P2 | | 27 | — | Random Forest | GloVe 300d | P3 | | 28 | — | SVM (LinearSVC) | Word2Vec 300d | P3 | | 29 | — | SVM (RBF kernel) | Word2Vec 300d | P3 | | 30 | — | Logistic Regression | Word2Vec 300d | P3 | | 31 | — | Random Forest | Word2Vec 300d | P3 | | 32 | — | SVM (LinearSVC) | FastText 300d | P3 | | 33 | — | SVM (RBF kernel) | FastText 300d | P3 | | 34 | — | Logistic Regression | FastText 300d | P3 | | 35 | — | Random Forest | FastText 300d | P3 |

### C. Deep Learning (54 models) | # | Rec | Encoder | Embedding | Mode | Priority | |---|-----|---------|----------|------|----------| | 36 | — | TextCNN | GloVe 300d | frozen | P3 | | 37 | BiLSTM | GloVe 300d | frozen | P3 | | 38 | LSTM | GloVe 300d | frozen | P3 | | 39 | Stacked LSTM | GloVe 300d | frozen | P3 | | 40 | LSTM + Attention | GloVe 300d | frozen | P3 | | 41 | BiGRU | GloVe 300d | frozen | P2 | | 42 | GRU | GloVe 300d | frozen | P3 | | 43 | Stacked GRU | GloVe 300d | frozen | P3 | | 44 | GRU + Attention | GloVe 300d | frozen | P3 | | 45 | TextCNN | GloVe 300d | fine-tuned | P2 | | 46 | BiLSTM | GloVe 300d | fine-tuned | P2 | | 47 | LSTM | GloVe 300d | fine-tuned | P3 | | 48 | Stacked LSTM | GloVe 300d | fine-tuned | P3 | | 49 | LSTM + Attention | GloVe 300d | fine-tuned | P3 | | 50 | BiGRU | GloVe 300d | fine-tuned | P3 | | 51 | GRU | GloVe 300d | fine-tuned | P3 | | 52 | Stacked GRU | GloVe 300d | fine-tuned | P3 | | 53 | GRU + Attention | GloVe 300d | fine-tuned | P2 | | 54 | TextCNN | Word2Vec 300d | frozen | P3 | | 55 | BiLSTM | Word2Vec 300d | frozen | P3 | | 56 | LSTM | Word2Vec 300d | frozen | P2 | | 57 | Stacked LSTM | Word2Vec 300d | frozen | P3 | | 58 | LSTM + Attention | Word2Vec 300d | frozen | P3 | | 59 | BiGRU | Word2Vec 300d | frozen | P3 | | 60 | GRU | Word2Vec 300d | frozen | P3 | | 61 | Stacked GRU | Word2Vec 300d | frozen | P3 | | 62 | GRU + Attention | Word2Vec 300d | frozen | P3 | | 63 | TextCNN | Word2Vec 300d | fine-tuned | P3 | | 64 | BiLSTM | Word2Vec 300d | fine-tuned | P3 | | 65 | LSTM | Word2Vec 300d | fine-tuned | P3 | | 66 | Stacked LSTM | Word2Vec 300d | fine-tuned | P3 | | 67 | LSTM + Attention | Word2Vec 300d | fine-tuned | P3 | | 68 | BiGRU | Word2Vec 300d | fine-tuned | P3 | | 69 | GRU | Word2Vec 300d | fine-tuned | P3 | | 70 | Stacked GRU | Word2Vec 300d | fine-tuned | P3 | | 71 | GRU + Attention | Word2Vec 300d | fine-tuned | P3 | | 72 | TextCNN | FastText 300d | frozen | P2 | | 73 | BiLSTM | FastText 300d | frozen | P3 | | 74 | LSTM | FastText 300d | frozen | P3 | | 75 | Stacked LSTM | FastText 300d | frozen | P3 | | 76 | LSTM + Attention | FastText 300d | frozen | P3 | | 77 | BiGRU | FastText 300d | frozen | P3 | | 78 | GRU | FastText 300d | frozen | P3 | | 79 | Stacked GRU | FastText 300d | frozen | P3 | | 80 | GRU + Attention | FastText 300d | frozen | P3 | | 81 | TextCNN | FastText 300d | fine-tuned | P3 | | 82 | BiLSTM | FastText 300d | fine-tuned | P3 | | 83 | LSTM | FastText 300d | fine-tuned | P3 | | 84 | Stacked LSTM | FastText 300d | fine-tuned | P3 | | 85 | LSTM + Attention | FastText 300d | fine-tuned | P3 | | 86 | BiGRU | FastText 300d | fine-tuned | P3 | | 87 | GRU | FastText 300d | fine-tuned | P3 | | 88 | Stacked GRU | FastText 300d | fine-tuned | P3 | | 89 | GRU + Attention | FastText 300d | fine-tuned | P3 |

### D. Transformers — Full Fine-tuning (16 models) | # | Rec | Model | Params | Priority | |---|-----|-------|--------|----------| | 90 | ✓ | BERT base (uncased) | 110M | P2 | | 91 | — | BERT large (uncased) | 340M | P3 | | 92 | RoBERTa base | 125M | P2 | | 93 | — | RoBERTa large | 355M | P3 | | 94 | — | DeBERTa base | 140M | P3 | | 95 | — | DeBERTa large | 400M | P3 | | 96 | DeBERTa v3 base | 140M | P2 | | 97 | — | DeBERTa v3 large | 400M | P3 | | 98 | DistilBERT base | 66M | P2 | | 99 | ALBERT base v2 | 12M | P2 | | 100 | — | ALBERT large v2 | 18M | P3 | | 101 | — | ELECTRA small | 13M | P3 | | 102 | ELECTRA base | 110M | P2 | | 103 | — | ELECTRA large | 335M | P3 | | 104 | XLNet base (cased) | 110M | P2 | | 105 | — | XLNet large (cased) | 340M | P3 |

### D. Transformers — Feature Extraction (16 models) | # | Rec | Model | Priority | |---|-----|-------|----------| | 106 | — | BERT base — feature extraction | P3 | | 107 | BERT large — feature extraction | P3 | | 108 | RoBERTa base — feature extraction | P3 | | 109 | RoBERTa large — feature extraction | P3 | | 110 | DeBERTa base — feature extraction | P3 | | 111 | DeBERTa large — feature extraction | P3 | | 112 | DeBERTa v3 base — feature extraction | P3 | | 113 | DeBERTa v3 large — feature extraction | P3 | | 114 | DistilBERT base — feature extraction | P3 | | 115 | ALBERT base v2 — feature extraction | P3 | | 116 | ALBERT large v2 — feature extraction | P3 | | 117 | ELECTRA small — feature extraction | P3 | | 118 | ELECTRA base — feature extraction | P3 | | 119 | ELECTRA large — feature extraction | P3 | | 120 | XLNet base — feature extraction | P3 | | 121 | XLNet large — feature extraction | P3 |

### D. Transformers — Partial Fine-tuning (16 models) | # | Rec | Model | Priority | |---|-----|-------|----------| | 122 | — | BERT base — partial fine-tuning | P3 | | 123 | BERT large — partial fine-tuning | P3 | | 124 | RoBERTa base — partial fine-tuning | P3 | | 125 | RoBERTa large — partial fine-tuning | P3 | | 126 | DeBERTa base — partial fine-tuning | P3 | | 127 | DeBERTa large — partial fine-tuning | P3 | | 128 | DeBERTa v3 base — partial fine-tuning | P3 | | 129 | DeBERTa v3 large — partial fine-tuning | P3 | | 130 | DistilBERT base — partial fine-tuning | P3 | | 131 | ALBERT base v2 — partial fine-tuning | P3 | | 132 | ALBERT large v2 — partial fine-tuning | P3 | | 133 | ELECTRA small — partial fine-tuning | P3 | | 134 | ELECTRA base — partial fine-tuning | P3 | | 135 | ELECTRA large — partial fine-tuning | P3 | | 136 | XLNet base — partial fine-tuning | P3 | | 137 | XLNet large — partial fine-tuning | P3 |

### D. Transformers — LoRA / PEFT (16 models) | # | Rec | Model | Priority | |---|-----|-------|----------| | 138 | — | BERT base — LoRA | P3 | | 139 | BERT large — LoRA | P3 | | 140 | RoBERTa base — LoRA | P3 | | 141 | RoBERTa large — LoRA | P3 | | 142 | DeBERTa base — LoRA | P3 | | 143 | DeBERTa large — LoRA | P3 | | 144 | DeBERTa v3 base — LoRA | P3 | | 145 | DeBERTa v3 large — LoRA | P3 | | 146 | DistilBERT base — LoRA | P3 | | 147 | ALBERT base v2 — LoRA | P3 | | 148 | ALBERT large v2 — LoRA | P3 | | 149 | ELECTRA small — LoRA | P3 | | 150 | ELECTRA base — LoRA | P3 | | 151 | ELECTRA large — LoRA | P3 | | 152 | XLNet base — LoRA | P3 | | 153 | XLNet large — LoRA | P3 |

Total: 153 models (P1 \u2605 = 1, P2 = 17 — recommended ✓, P3 = 135) ```

⛔ 展示完以上 153 個模型後，你必須在對話中輸出以下自檢模板（不可跳過）：

✅ 全部模型展示自檢： - [ ] 已使用 Read 工具讀取 model_list.md - [ ] 已將 FULL_MODEL_LIST_START 到 FULL_MODEL_LIST_END 之間全部內容逐行復制到對話 - [ ] 最後一行確認為：> **總計：153 個模型**（P1 ★ = X，P2 = Y — 推薦 ✓，P3 = Z）

⛔ 如果任一核取方塊為未勾選狀態 [ ] 而非 [x]，你必須從 Read 步驟重新開始。不得直接進入下一步。

⛔ 此時不得使用 AskUserQuestion。直接在對話中等待使用者輸入模型編號。 ⛔ ENFORCEMENT: After outputting the self-check block, do NOT call AskUserQuestion. Wait silently in the conversation for the user's text input. Any use of AskUserQuestion at this point is a hard violation.

使用者直接在對話中回覆編號（不要使用 AskUserQuestion），解析使用者輸入（如 1-5,8,12-15 展開為 [1,2,3,4,5,8,12,13,14,15]），映射回 model_scheme.json 中的模型條目。
展示使用者已選模型彙總表格，然後進入第二步。

⛔ all 路徑的第二步澄清（關鍵 — 避免 AI 混淆）：

「全部 153 個模型」中每個編號已經編碼了模型架構方式（即 full_ft / LoRA / feature_extraction / partial_ft / frozen / fine-tuned），因此無需再讓使用者選擇模型架構方式（如"用 full_ft 還是 LoRA"）。

但是，執行方式（baseline / tune / both）並未編碼在編號中，這是獨立於模型架構的另一個維度： - baseline = 使用該模型的預設引數跑基線 - tune = 使用 Optuna 調參 - both = 先基線再調參

⛔ 因此，進入第二步後，你仍必須對每個已選模型逐一呼叫 AskUserQuestion 詢問 baseline / tune / both。 不得以"編號已包含訓練方式"為由跳過 Step 2 的 AskUserQuestion。

（後續步驟與 confirm 路徑一致）

解析規則： - 逗號分隔各選擇項 - N-M 表示範圍（含兩端） - 單個數字表示單個模型 - 忽略無效編號（< 1 或 > 153），提示使用者修正 - 至少選擇 1 個模型，否則提示重新輸入

選擇完成後，彙總選定的模型列表，進入第二步（訓練方式選擇）。模型架構方式（full_ft/LoRA/feature_extraction/partial_ft/frozen/fine-tuned）已編碼在編號中無需再選，但執行方式（baseline/tune/both）仍必須通過 AskUserQuestion 逐一詢問。

第二步：訓練方式 / Step 2: Training Mode

對第一步確定的所有具體模型逐一選擇訓練方式。 Select the training mode for each model selected in Step 1.

Selection order / 選擇順序: Traditional ML → Deep Learning → Transformer.

🚫 Blocking point — You MUST call AskUserQuestion for each selected model to ask baseline/tune/both. Do not skip. This includes models selected via the all-153 path: even though model architecture mode (full_ft/LoRA etc.) is encoded in the number, baseline/tune/both MUST still be asked per-model here.

每個模型一個 question（multiSelect），超過 4 個模型時分多輪 AskUserQuestion。

每條 AskUserQuestion 開頭用文本列出所有已選模型：

ZH: "當前已選模型：\n- [傳統ML] SVM + TF-IDF bigram\n- [DL] TextCNN + GloVe（凍結/微調）\n- [TF] BERT base（全引數微調/LoRA）\n..." EN: "Currently selected models:\n- [Trad ML] SVM + TF-IDF bigram\n- [DL] TextCNN + GloVe (frozen/fine-tuned)\n- [TF] BERT base (full_ft/LoRA)\n..."

ZH Question: "{模型名稱} 的訓練方式？" EN Question: "Training mode for {model_name}?"

Value	ZH Label	EN Label	Description
baseline	"預設引數基線"	"Baseline (default params)"	"使用預設預設引數執行基線評估，不調參。快速獲得參照指標。" / "Run baseline with default params. No tuning. Quick reference."
tune	"Optuna 調參"	"Tune with Optuna"	"超引數自動調優（Optuna + MedianPruner 早停）。" / "Auto hyperparameter tuning with Optuna + MedianPruner early stopping."
both	"基線 + 調參"	"Baseline + Tune"	"先跑基線，再用 Optuna 調參，對比提升幅度。推薦。" / "Baseline first, then tune. Compare improvement. Recommended."

第三步：Optuna 調參配置 / Step 3: Optuna Tuning Configuration

僅對第二步中選擇了"Optuna 調參"或"基線 + 調參"的模型進行。 Only for models where "Tune with Optuna" or "Baseline + Tune" was selected in Step 2.

每個需要調參的模型一個 question，選擇 Optuna 試驗次數。 One question per model needing tuning: select number of Optuna trials.

🚫 Blocking point — You MUST call AskUserQuestion for each tune/both model to ask trial count. Do not skip.

超過 4 個模型時分多輪 AskUserQuestion。 / Split into multiple rounds if more than 4 models.

Each AskUserQuestion starts with a list of all models being tuned.

ZH Question: "{模型名稱} 的 Optuna 試驗次數？" EN Question: "Number of Optuna trials for {model_name}?"

每個 option 的 description 需根據模型型別說明預估時間。 Each option's description should include estimated time based on model type:

Value	ZH Label	EN Label
20	"20 次（快速）"	"20 trials (quick)"
30	"30 次（推薦）"	"30 trials (recommended)"
50	"50 次（徹底）"	"50 trials (thorough)"

各模型調參次數與預估時間參考 / Estimated Trial Time Reference:

Calibrated on 5K samples, ~200 word average text length, 5-fold CV, RTX 5060 8GB. Long texts (>150 words/sample) increase time 1.5-3×. Short texts (<50 words/sample) reduce time 30-50%.

Model Type / 模型型別	20 trials	30 trials (recommended)	50 trials
Traditional ML + TF-IDF / 傳統 ML + TF-IDF	~2 min	~3 min	~5 min
Traditional ML + embeddings / 傳統 ML + 嵌入	~5 min	~8 min	~15 min
DL + frozen embeddings / DL + 固定嵌入	~15 min	~25 min	~40 min
DL + fine-tuned embeddings / DL + 微調嵌入	~30 min	~45 min	~75 min
Transformer (DistilBERT/ALBERT)	~30 min	~45 min	~75 min
Transformer (BERT/RoBERTa/ELECTRA base)	~60 min	~90 min	~150 min
Transformer (DeBERTa/XLNet/large)	~90 min	~135 min	~220 min

Note: Times above are tuning phase only (not including baseline). both mode adds baseline time (~1× per-trial). Baseline reference: Traditional ML <1 min, DL ~3-8 min, BERT/XLNet base ~15-40 min (varies with text length and CV folds).

Early stopping / 早停機制: All Optuna tuning enables MedianPruner: - First 5 trials not pruned (establish baseline) - First 3 steps per trial (CV folds or epochs) not pruned - Split mode: DL/Transformer report intermediate values each epoch; bad trials terminated early - Traditional ML: reports after each CV fold; no early stopping within 3 folds (single-fold training is fast, low impact)

第四步：確認訓練計劃 / Step 4: Confirm Training Plan

Summarize all choices from the three steps above, output the training plan table / 彙總以上三步的所有選擇，輸出訓練計劃表：

ZH template:

## 訓練計劃彙總

| # | 模型 | 類別 | 嵌入/訓練方式 | 訓練模式 | 調參次數 | 預估時間 |
|---|------|------|-------------|---------|---------|---------|
| 1 | SVM + TF-IDF bigram | 傳統ML | — | 基線 + 調參 | 30 | ~5 分鐘 |
| 2 | 邏輯迴歸 + GloVe 300d | 傳統ML | — | 基線 + 調參 | 30 | ~10 分鐘 |
| 3 | TextCNN + GloVe | 深度學習 | 凍結 + 微調 | 基線 + 調參 | 30 | ~50 分鐘 |
| 4 | BERT base | Transformer | 全引數微調 + LoRA | 基線 + 調參 | 30 | ~120 分鐘 |
| ... | ... | ... | ... | ... | ... | ... |

總預估時間：約 X 小時 Y 分鐘（序列）/ Z 分鐘（並行 GPU 允許時）
預計磁碟佔用：~Z GB（含模型產物和嵌入檔案）

早停機制：已啟用 MedianPruner（5 trial 冷啟動 + 3 step 預熱）

EN template:

## Training Plan Summary

| # | Model | Category | Embedding/Mode | Training | Trials | Est. Time |
|---|-------|----------|---------------|----------|--------|-----------|
| 1 | SVM + TF-IDF bigram | Trad ML | — | baseline + tune | 30 | ~5 min |
| 2 | Logistic Regression + GloVe 300d | Trad ML | — | baseline + tune | 30 | ~10 min |
| 3 | TextCNN + GloVe | DL | frozen + fine-tuned | baseline + tune | 30 | ~50 min |
| 4 | BERT base | TF | full_ft + LoRA | baseline + tune | 30 | ~120 min |
| ... | ... | ... | ... | ... | ... | ... |

Total estimated time: ~X hr Y min (serial) / Z min (parallel where GPU allows)
Estimated disk usage: ~Z GB (including model artifacts and embeddings)

Early stopping: MedianPruner enabled (5 trial startup + 3 step warmup)

🚫 阻斷點 — 顯示完訓練計劃彙總表後，你必須在此處呼叫 AskUserQuestion 確認是否開始訓練，不得跳過。

使用 AskUserQuestion 確認：

EN Question: "Ready to start training with the plan above?" ZH Question: "確認以上訓練計劃，開始訓練？"

Value	EN Label	ZH Label
proceed	"Yes, start training"	"確認，開始訓練"
adjust	"No, let me adjust"	"否，我要調整"

如果選擇 adjust，回到第一步重新確認模型選擇。如果選擇 proceed，進入 Stage 4（訓練）。

模型描述參考

展示模型卡片時，根據互動語言使用以下描述：

中文描述（使用者使用中文時）：

模型	描述
SVM (LinearSVC)	強線性分類器；通常是傳統 ML 在文本分類上的效能上限。訓練快，無需 GPU。
邏輯迴歸 (Logistic Regression)	簡單高效的線性模型；小資料集上不易過擬合，適合作為快速基線。
隨機森林 (Random Forest)	決策樹整合；與線性模型有不同的歸納偏置，能捕獲非線性特徵互動。適合不平衡資料。
多項式樸素貝葉斯 (Multinomial NB)	機率模型；極快，適合短文本和小資料集。常作為基線比較。
SVM/ LR/ RF + GloVe/Word2Vec/fastText	傳統 ML + 稠密嵌入：用預訓練詞向量平均池化替代稀疏 BoW。RBF SVM 在稠密向量上顯著優於稀疏 TF-IDF。適合 5K+ 樣本。
BiLSTM	雙向 LSTM；捕獲長距離序列依賴。適合長文本和複雜句式。
BiGRU	雙向 GRU；與 BiLSTM 類似但收斂快約 15%，效能相當。
GRU + Attention	GRU 加註意力機制；學習哪些詞對分類決策最重要。適合長文本。
LSTM + Attention	LSTM 加註意力；與 GRU+Attention 類似但使用 LSTM 單元。大數據集上可能更有表達力。
Stacked LSTM / GRU	多層迴圈網路；對複雜模式有更強容量。小資料集上有過擬合風險。
BERT base	12 層 Transformer（110M 引數）；文本分類的標準參照點。綜合表現好。
RoBERTa base	BERT 最佳化版（125M 引數）；分類基準上通常優於 BERT。預訓練資料更多。
DeBERTa base/v3	解耦注意力（140M 引數）；文本分類上常達最優。v3 使用 ELECTRA 式預訓練。
DeBERTa large	24 層 DeBERTa（400M 引數）；比 base 好約 2-3%。推薦 12GB+ 視訊記憶體使用。
DistilBERT	BERT 蒸餾版（66M 引數）；體積小約 40%，快約 60%，保留約 95% BERT 效能。適合部署和低視訊記憶體場景。
ALBERT base	引數高效 BERT（12M 引數）；共享層權重 + 分解嵌入，視訊記憶體佔用低。適合視訊記憶體有限的 GPU。
ALBERT large	18M 引數 large 版（hidden 1024）；比 base 好約 2%，引數共享保持低視訊記憶體。
ELECTRA small	緊湊判別器（13M 引數）；hidden 256。極低視訊記憶體場景下快速實驗。適合 2-4GB 視訊記憶體。
ELECTRA base	判別器預訓練模型；中小資料集上常優於 BERT，訓練效率高。
ELECTRA large	24 層判別器（335M 引數）；比 base 好約 2-3%。推薦 12GB+ 視訊記憶體。
XLNet base	自迴歸 Transformer-XL（110M 引數）；無 [CLS] token，使用最後 token 表示。適合長文本中的長距離依賴。
XLNet large	24 層 XLNet（340M 引數）；比 base 好約 2-3%。推薦 12GB+ 視訊記憶體。
BERT/RoBERTa large	24 層版本（340M/355M 引數）；比 base 好約 2-3%，但慢 3-4 倍。僅推薦 12GB+ 視訊記憶體使用。

English descriptions (when user communicates in English):

Model	Description
SVM (LinearSVC)	Strong linear classifier; often the performance ceiling for traditional ML on text. Fast training, no GPU needed.
Logistic Regression	Simple, efficient linear model; less prone to overfitting on small datasets. Great quick baseline.
Random Forest	Decision tree ensemble; different inductive bias from linear models, captures non-linear feature interactions. Good for imbalanced data.
Multinomial NB	Probabilistic model; extremely fast, good for short texts and small datasets. Common baseline.
SVM / LR / RF + GloVe/Word2Vec/fastText	Traditional ML + dense embeddings: average pretrained word vectors instead of sparse BoW. RBF SVM significantly outperforms sparse TF-IDF on dense vectors. Best for 5K+ samples.
BiLSTM	Bidirectional LSTM; captures long-range sequential dependencies. Good for long texts and complex syntax.
BiGRU	Bidirectional GRU; similar to BiLSTM but converges ~15% faster with comparable performance.
GRU + Attention	GRU with attention mechanism; learns which words matter most for classification. Good for long texts.
LSTM + Attention	LSTM with attention; similar to GRU+Attention but uses LSTM cells. May be more expressive on large datasets.
Stacked LSTM / GRU	Multi-layer recurrent networks; higher capacity for complex patterns. Risk of overfitting on small datasets.
BERT base	12-layer Transformer (110M params); standard reference point for text classification. Strong all-around performance.
RoBERTa base	Optimized BERT (125M params); typically outperforms BERT on classification benchmarks. More pretraining data.
DeBERTa base/v3	Disentangled attention (140M params); often state-of-the-art on text classification. v3 uses ELECTRA-style pretraining.
DeBERTa large	24-layer DeBERTa (400M params); ~2-3% better than base. Recommended with 12GB+ VRAM.
DistilBERT	Distilled BERT (66M params); ~40% smaller, ~60% faster, retains ~95% of BERT performance. Great for deployment and low-VRAM.
ALBERT base	Parameter-efficient BERT (12M params); shared layer weights + factorized embeddings, low VRAM usage. Good for limited GPUs.
ALBERT large	18M param large variant (hidden 1024); ~2% better than base, shared params keep VRAM low.
ELECTRA small	Compact discriminator (13M params); hidden 256. Fast experiments on very low VRAM. Good for 2-4GB VRAM.
ELECTRA base	Discriminator-pretrained model; often outperforms BERT on small-medium datasets, efficient training.
ELECTRA large	24-layer discriminator (335M params); ~2-3% better than base. Recommended with 12GB+ VRAM.
XLNet base	Autoregressive Transformer-XL (110M params); no [CLS] token, uses last token representation. Good for long-range dependencies in long texts.
XLNet large	24-layer XLNet (340M params); ~2-3% better than base. Recommended with 12GB+ VRAM.
BERT/RoBERTa large	24-layer versions (340M/355M params); ~2-3% better than base but 3-4x slower. Only recommended with 12GB+ VRAM.

Stage 4：模型訓練 / Model Training

Run scripts/step4_train.py:

PYTHONIOENCODING=utf-8 python scripts/step4_train.py \
  --csv <path> --text-col <name> --label-col <name> \
  --scheme <project_dir>/model_scheme.json \
  [--split <project_dir>/split_info.json] \
  [--models <name1;name2>] \
  [--mode baseline|tune|both] \
  [--cv-folds 5] \
  [--tune-method cv|split] \
  [--tune-trials 50] \
  [--output-dir <project_dir>] \
  [--embedding-path <path>] \
  [--glove-path <path>] [--word2vec-path <path>] [--fasttext-path <path>] \
  [--epochs <N>] [--encoding <enc>] \
  [--no-mlflow] [--seed 42]

--mode defaults to both (baseline + tuning).
--models uses ; to separate model names (display_name may contain commas).
--encoding auto-detects (tries utf-8, utf-8-sig, gbk, latin-1, etc.); can manually specify.
--embedding-path is legacy; prefer --glove-path / --word2vec-path / --fasttext-path.

Training progress tracking / 訓練進度日誌: Dual progress tracking enabled: 1. Real-time log file / 即時日誌檔案: output/training.log — line-buffered, tail -f capable 2. Periodic progress reports / 定期進度報告: Every 5 minutes prints summary (completed models, current model, remaining, elapsed) to console and log

Baseline mode: Default params, stratified K-fold CV per model. Display per-fold metrics.

Tune mode: Optuna hyperparameter optimization with MedianPruner early stopping. tqdm progress bars show trial progress and current best. Best params re-evaluated with full CV.

Both mode (default): Baseline first, then tuning, then comparison. Shows absolute and relative improvement.

GPU auto-detection (CUDA > MPS > CPU). Conservative batch sizes for low-VRAM GPUs. Auto-fallback to CPU on OOM with warning.

Training Time Estimates / 訓練時間預估

Approximate times on a single GPU (e.g., RTX 3060/4060/5060 8GB). Actual times highly depend on text length — based on ~100 words/sample typical text. Long texts (>200 words) may double these times.

Data Size / 資料量	Traditional ML (all P1)	DL (1 model, 5-fold)	Transformer base (5-fold, 3 epoch)
5K samples	< 1 min	~8 min	~20-40 min
50K samples	~3 min	~40 min	~2-4 hr
200K+ samples	~8 min	~2 hr	~10+ hr (recommend --epochs 2)

Large dataset tips / 大數據集建議: - Use --epochs 2 to reduce Transformer/DL training time - Use --tune-method split for faster tuning (avoids CV per trial) - Use --cv-folds 3 instead of default 5-fold - For long-text data, prefer traditional ML or lightweight DistilBERT/ALBERT - Tuning total ≈ trials × baseline time (e.g. 20 trials × 8 min baseline ≈ 160 min)

China / Restricted Network Environments / 中國 / 網路受限環境

If Hugging Face is unreachable (common in mainland China), set the mirror before Stage 3 or 4:

# Linux / macOS
export HF_ENDPOINT=https://hf-mirror.com

# Windows PowerShell
$env:HF_ENDPOINT="https://hf-mirror.com"

# Windows CMD
set HF_ENDPOINT=https://hf-mirror.com

Script auto-detects network errors and prompts to use the mirror on download timeout.

MLflow tracking enabled by default (SQLite backend, stored at output/mlflow.db). All params, metrics, and per-fold results are logged.

Generates <project_dir>/training_results.json and saves intermediate model files in <project_dir>/models/.

After training completes, display baseline vs tuned results in the user's language:

Model metrics summary table — strictly follow the ⛔ Global Output Format Rule above. Flat single-row headers: {Dataset}_{Metric} (N={sample_count}). Copy directly from step4_train.py output; do not reformat.
Absolute and relative improvement for tuned models (vs baseline)
Best model annotation (sorted by F1/Accuracy)
Best model parameter table — below the summary table, list the best model's params:
Best model is tuned → Show Optuna best params with default value comparison column (Default → Best)
Best model is baseline → Show that model's default parameter table Extract params from training_results.json for the corresponding model (baseline.params or tuned.best_params).
Skipped models note — Below the summary, list any models that were skipped:

⚠️ The following models could not be trained: - {model name}: {reason skipped}. Solution: {specific steps}
Reasons must be specific (embedding download failed / network unreachable / insufficient VRAM / user cancelled)
Solutions must be actionable (e.g., manually download file then pass --xxx-path, set HF_ENDPOINT mirror, free VRAM)
Omit if all models trained successfully
Post-training diagnostic analysis — After displaying the summary table, run scripts/analyze_results.py:

bash PYTHONIOENCODING=utf-8 python scripts/analyze_results.py \ --results <project_dir>/training_results.json \ --analysis <project_dir>/analysis.json \ --output-dir <project_dir>

指令碼輸出以下維度的診斷分析：

模型綜合排名：按 Test F1 降序排列，顯示最佳模型及其領先幅度
過擬合分析：檢查每個模型的 Train/Val 指標差距。gap > 15pp 標記為嚴重（🔴），gap > 5pp 標記為中度（🟡）。對嚴重過擬合給出簡化結構/增加正則化的建議
交叉驗證穩定性：檢查各 CV 折之間的 Acc 極差。極差 > 5% 標記為不穩定
Test 集異常檢測：檢查 Test 指標是否顯著高於所有 CV 折的最高值。若 Test Acc > max(折 Acc) + 2pp，標記為異常，提示可能原因（全量重訓驗證劃分不一致 / 隨機種子效應）
模型-資料規模匹配：基於資料樣本量檢查模型複雜度是否合理。如 Stacked 模型在 <10K 樣本上的過擬合風險、Large 模型在 <20K 樣本上不推薦等
調參效率分析：對做過 Optuna 調參的模型，評估提升是否值得計算開銷。若超引數 ≤2 個且搜尋次數 ≥10 但 ΔF1 < 0.01，標記為價效比低，建議直接用預設引數
7w4.net小蔥技能站，你的AI助手技能庫。
📋 診斷建議（Diagnostic Recommendations）：綜合以上所有診斷維度，自動生成具體、可操作的建議。每項建議包含：
- 嚴重級別：critical（嚴重）/ high（高優先）/ moderate（中等）/ info（參考）
- 分類：convergence_failure（收斂失敗）/ cv_instability / severe_overfitting / test_anomaly / model_data_mismatch / tuning_inefficiency / seq_len_truncation
- 具體問題描述：包含相關指標數值
- 操作建議：按步驟給出可執行的修復動作（如"將 learning_rate 從 2e-5 提高到 5e-5"）
- 原因說明：解釋為什麼會出現該問題
覆蓋的診斷場景包括： - CV 部分收斂失敗（部分折 Acc≈0.50 隨機，其他折正常）→ 建議提高 LR、增加 warmup、使用 split 模式調參 - 嚴重/中度過擬合 → 按模型類別（Transformer/DL/傳統ML）給出針對性正則化建議 - Test 異常高於 CV → 識別是否為 CV 不穩定導致的假象，給出驗證建議 - 文本截斷（平均長度 > max_seq_len）→ 建議增加 seq_len 或使用長文本模型 - 總結建議：綜合以上各維度，給出最佳模型推薦和後續最佳化方向

指令碼生成的診斷結果儲存至 <project_dir>/post_analysis.json。將全部診斷內容逐段展示給使用者，不得省略任何維度。

⛔ 此分析步驟為強制步驟，每次 Stage 4 訓練完成後必須執行。 指令碼已整合所有分析邏輯，無需手工計算。

🚫 阻斷點 — 訓練結果展示 + 診斷分析完畢後，你必須在此處呼叫 AskUserQuestion 詢問使用者下一步操作，不得跳過。

然後詢問使用者下一步：

EN Question: "All tuning trials are complete. What would you like to do next?" ZH Question: "所有模型調優已完成。接下來做什麼？"

Value	EN Label	ZH Label
proceed	"Pick the best model → re-train on full dataset → save & deploy (Stage 5)"	"選擇最佳模型 → 全量資料重新訓練 → 儲存並部署（Stage 5）"
retune	"Re-tune some models with different parameters"	"調整引數重新調優某些模型"
more	"Train additional models from the scheme"	"從方案中增加訓練更多模型"

注意：使用者必須確認最佳模型後，才能在 Stage 5 中用全量資料重新訓練。這是因為調優階段使用交叉驗證/切分來公平評估，最終部署前應使用全部資料（在 Stage 5 中）重新訓練以獲得最佳泛化效能。

Stage 5：全量訓練 + 儲存最終模型 / Full Training + Save Final Model

Stage 4 中通過交叉驗證/切分確定了最佳超引數。Stage 5 使用全部資料 重新訓練最佳模型，以獲得生產環境的最佳泛化效能。

使用者確認最佳模型後，執行 scripts/step5_save.py：

PYTHONIOENCODING=utf-8 python scripts/step5_save.py \
  --csv <path> --text-col <name> --label-col <name> \
  --analysis <project_dir>/analysis.json \
  --training-results <project_dir>/training_results.json \
  --best-model "<模型展示名稱>" \
  [--split <project_dir>/split_info.json] \
  [--retrain-on-full] \
  [--output-dir <project_dir>] \
  [--embedding-path <path>] \
  [--glove-path <path>] [--word2vec-path <path>] [--fasttext-path <path>] \
  [--encoding <enc>] \
  [--no-mlflow] [--seed 42]

若提供了 --split：先在測試集上評估，--retrain-on-full 則用全部資料重訓
--encoding 預設自動檢測；--embedding-path 為相容引數，推薦用 per-type 引數
全量重訓前必須與使用者確認

<project_dir> 為 Stage 1 環境確認中使用者指定的專案目錄。

This script executes in order / 此指令碼按以下順序執行: 1. Full data training / 全量資料訓練: Retrain on all samples using Stage 4 best hyperparameters 2. Save model artifacts to <project_dir>/final_model/: - sklearn: .pkl files (model + vectorizer) via joblib - PyTorch: .pt files (state_dict) + vocab .pkl - Transformers: .pt file + tokenizer directory 3. Generate <project_dir>/training_report.html — standalone HTML report: - Dataset statistics summary - Training results comparison table for all models - Best model details (hyperparameters, per-fold/split metrics) - Confusion matrix (from Stage 4 validation set predictions) - Deployment info (API endpoints, model paths, dependencies) 4. Generate deployment artifacts at <project_dir>/deploy/: - api_server.py (FastAPI with /health, /predict, /predict_batch endpoints) - requirements.txt (pinned dependency versions) - Dockerfile (multi-stage build with health check) - monitoring.md (Prometheus metrics, alert rules, drift detection)

⚠️ step5_save.py internally calls webbrowser.open() to open the HTML report. Do NOT manually open it again or two browser windows will pop up. Print the output directory tree showing all generated files.

Output Directory Structure / 輸出目錄結構

output/
├── analysis.json              # Stage 1 output
├── split_info.json            # Stage 2 output
├── model_scheme.json          # Stage 3 output
├── training_results.json      # Stage 4 output
├── post_analysis.json         # Stage 4 post-training diagnostic analysis
├── mlflow.db                  # MLflow tracking database
├── models/                    # Stage 4 intermediate models
│   ├── <name>_baseline.pkl
│   └── <name>_tuned.pkl
├── final_model/               # Final model artifacts (Stage 5)
│   ├── <name>_model.pkl/.pt
│   └── <name>_vectorizer.pkl / tokenizer/
├── training_report.html       # HTML report (Stage 4)
└── deploy/                    # Deployment artifacts (Stage 5)
    ├── api_server.py
    ├── requirements.txt
    ├── Dockerfile
    └── monitoring.md

Encoding Strategy / 編碼策略

Python source code in English (ASCII-compatible); comments may use Chinese
All file I/O uses encoding='utf-8' with errors='replace'
JSON files use ensure_ascii=False for readable Unicode in reports
Reports use <meta charset="UTF-8">
Critical / 關鍵: All Bash calls must set PYTHONIOENCODING=utf-8 before the python command

Error Handling / 錯誤處理

Each script follows a unified pattern: 1. CLI arg validation — file existence, valid options 2. Data validation — column types, label values (0/1 only), language check 3. Training errors — GPU OOM fallback, convergence warnings, empty vocabulary 4. File I/O — atomic writes, directory creation (exist_ok)

Error / 錯誤	Message / 訊息	Handling / 處理
File not found — Windows path format	`FileNotFoundError` from Python with `/c/Users/...` path	Self-heal / 自愈: Convert path to `C:\Users\...` format and retry
File not found	`[ERROR] File not found: <path>`	Exit code 1
Column name error	`[ERROR] Column '<name>' not found. Available: [...]`	Exit code 1
Labels not 0/1	`[ERROR] Label column must contain only 0 and 1. Found: [...]`	Exit code 1
Non-English > 30%	`[WARN] Data may contain significant non-English text`	Warn, continue
GPU OOM	`[WARN] GPU out of memory, falling back to CPU`	Auto fallback
MLflow unavailable	`[WARN] MLflow is not installed. Experiment tracking disabled.`	Continue without MLflow
Empty vocabulary	`[ERROR] All texts became empty after cleaning`	Exit code 1

Dependencies / 依賴

Core / 核心: numpy pandas scikit-learn nltk joblib tqdm psutil Deep Learning / 深度學習: torch transformers tokenizers Optimization / 最佳化: optuna Tracking / 追蹤: mlflow Web API: fastapi uvicorn pydantic (for deployment) Optional / 可選: langdetect (better language detection) Environment check / 環境檢查: pyyaml packaging

One-liner install / 一鍵安裝:

pip install numpy pandas scikit-learn nltk joblib tqdm psutil \
  torch transformers tokenizers optuna mlflow \
  fastapi uvicorn pydantic pyyaml packaging

References / 參考資料

references/model_params.md — Hyperparameter recommendations for all model types, compiled from literature and web research. Reference when user asks about specific parameter choices or defaults.
references/requirements.yaml — Python dependency manifest; the reference baseline for Stage 1 environment checks. Update when adding new dependencies.
scripts/ — All Python modules. Read relevant files when debugging or extending functionality.

🤖 AI 評測

這個技能質量很高，幫你用CSV資料訓練文本分類模型時功能很全面，內建了150多種模型可選，訓練調參和自動部署都能搞定。最大亮點是預設了很多防錯機制，不容易出岔子。不足之處是功能選項太多太細，新手可能覺得有點複雜難選，整個流程跑下來步驟也比較多。總體來說，這是一個靠譜省心的選擇。

📊 多維度評分

適應性4.3

規範性4.5

有效性4.6

可靠性4.8

可信度4.8

📁 包含檔案 (17 個)

📄 SKILL.md 101 KB

📄 nlp_test_5000.csv 6.3 MB

📄 references/model_params.md 19.1 KB

📄 references/requirements.yaml 721 B

📄 scripts/__init__.py 18 B

📄 scripts/analyze_results.py 33.4 KB

📄 scripts/deploy.py 15.9 KB

📄 scripts/mlflow_utils.py 7.6 KB

📄 scripts/model_factory.py 51 KB

📄 scripts/preprocessing.py 35.6 KB

📄 scripts/report.py 14.6 KB

📄 scripts/step1_analyze.py 14.3 KB

📄 scripts/step2_split.py 14.9 KB

📄 scripts/step3_scheme.py 34.5 KB

📄 scripts/step4_train.py 106 KB

📄 scripts/step5_save.py 32.3 KB

📄 scripts/utils.py 55.5 KB

機器學習-英文文本二分類

📖 技能介紹

English Text Binary Classification Skill

英文文本二分類技能

⛔ 常見 AI 違規行為 — 執行此技能前必讀

⛔ Common AI Violations — READ BEFORE EXECUTING THIS SKILL

⛔ AskUserQuestion Fallback Protocol

⛔ LANGUAGE DETECTION (READ THIS FIRST — VIOLATION #12)

Detection Rule

⛔ Mandatory Pre-Output Language Self-Check

Detection Heuristics

⛔ First-Output Gate

Common Failure Patterns (DO NOT REPEAT)

流水線概覽 / Pipeline Overview

⛔ 全域性輸出格式規則（GLOBAL — ALL STAGES）

⛔ Global Output Format Rule

中文亂碼預防 / UTF-8 Encoding (Windows)

Windows 路徑處理（Git Bash / MSYS 環境）

Windows Path Handling (Git Bash / MSYS)

⛔ Windows Python 命令（禁止 python3）/ ⛔ Windows Python Command (no python3)

⛔ 禁止內聯 Python 探索 CSV（編碼安全）/ ⛔ No Inline Python for CSV Exploration

如何使用此技能 / How to Use This Skill

⛔ Stage 0：操作前置檢查 / Pre-Flight Check (READ BEFORE EVERY BASH CALL)

Stage 1：資料分析 / Data Analysis

環境確認 / Environment Confirmation

Step 1: 環境評估 / Environment Assessment

Step 2: 不滿足要求時的處理 / Handling Unmet Requirements

Step 3: 滿足要求 — 確認安裝缺失的 Python 包 / Install Missing Packages

Step 4: 指定專案目錄 / Specify Project Directory

Step 5: 確認並進入 Stage 2 / Confirm and Proceed to Stage 2

Stage 2：樣本劃分 / Data Split

Stage 3：模型方案 / Model Scheme

Stage 3 Interactive Model Selection (Smart Defaults) / Stage 3 互動式模型選擇

第一步：智慧推薦 + 使用者確認 / Step 1: Smart Recommendation + User Confirmation

第二步：訓練方式 / Step 2: Training Mode

第三步：Optuna 調參配置 / Step 3: Optuna Tuning Configuration

第四步：確認訓練計劃 / Step 4: Confirm Training Plan

模型描述參考

Stage 4：模型訓練 / Model Training

Training Time Estimates / 訓練時間預估

China / Restricted Network Environments / 中國 / 網路受限環境

Stage 5：全量訓練 + 儲存最終模型 / Full Training + Save Final Model

Output Directory Structure / 輸出目錄結構

Encoding Strategy / 編碼策略

Error Handling / 錯誤處理

Dependencies / 依賴

References / 參考資料

🤖 AI 評測

📊 多維度評分

📁 包含檔案 (17 個)

🔗 相關推薦

⛔ Windows Python 命令（禁止 `python3`）/ ⛔ Windows Python Command (no `python3`)