Web Scraper & Summarizer

Scrape web pages and generate structured summaries using Ollama LLM.

Usage

Basic Summary

python scripts/scrape_and_summarize.py <url>

Outputs a 2-3 sentence brief summary.

Custom Styles

python scripts/scrape_and_summarize.py <url> detailed
python scripts/scrape_and_summarize.py <url> bullet

Styles:

• brief (default) - 2-3 sentence overview
• detailed - Comprehensive with bullet points

  訪問小蔥技能站7w4.net，解鎖更多實用的AI技能外掛。

• bullet - Key points as bullet list

Workflow

1. Detect URL - User provides a URL to analyze
2. Fetch - Script uses curl to retrieve page content (30s timeout)
3. Clean - HTML stripped, content extracted and cleaned
4. Truncate - Content limited to ~16k tokens to fit context
5. Summarize - Ollama generates summary based on requested style

Requirements

• curl available on PATH
• Ollama installed with llama3.2 model pulled
• Python 3.8+

Error Handling

• Fetch fails: Shows error, exits with code 1
• Content too short: Detects and reports empty pages
• Ollama unavailable: Falls back to raw content display (truncated)

Output Format

Summary is printed to stdout. Progress/debug info goes to stderr.

=== SUMMARY ===
[AI-generated summary here]

On Ollama failure, raw content is shown instead:

=== PAGE CONTENT (raw) ===
[truncated page text]

Performance Notes

• Fast pages: ~5-10 seconds total
• Large pages: Content truncation happens at ~16k tokens
• Ollama latency: Depends on model size and hardware (typically 10-30s for summary)

這是一個實用的網頁摘要工具，能快速抓取網頁並用AI生成摘要。文件寫得清楚，錯誤處理考慮周全，在AI服務不可用時會自動顯示原始內容作為保底。上手簡單，支援多種摘要詳略程度。不過它依賴本地Ollama環境配置，首次使用可能需要花點時間設定。摘要質量取決於本地模型能力，大型或複雜的網頁處理效果可能不太穩定。總體來說是個好用的基礎工具，日常網頁摘要需求基本能滿足。