name: agent-lens description: "Track AI agent API calls, analyze token usage, and optimize costs. Use when user wants to monitor LLM spending, debug API calls, track token consumption, or generate cost reports for OpenAI/Anthropic/Google/DeepSeek APIs." version: 2.17.0 author: lrg913427-dot license: MIT metadata: hermes: tags: [llm, cost, tracking, observability, tokens, api, monitoring, agent] related_skills: [db-explorer]

Agent Lens

Track every AI API call, analyze token usage, and optimize costs.

When to Use

Activate this skill when the user: - Says "how much am I spending", "token usage", "API costs" - Wants to know which model is most expensive - Needs to optimize prompt costs - Wants to track API call latency or error rates - Mentions "budget", "cost optimization", or "token counting" - Asks "why is my API bill so high"

Quick Start

# Install
pip install git+https://github.com/lrg913427-dot/agent-lens.git

# Generate demo data and see it in action
agent-lens demo

# View stats
agent-lens stats
agent-lens cost
agent-lens recent

Three Ways to Track

1. Decorator (easiest)

from agent_lens import AgentLens

lens = AgentLens(agent_name="my-agent")

@lens.track(model="gpt-4o")
def call_api(prompt):
    return client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": prompt}],
    )

# Token usage is auto-extracted from OpenAI-style responses
result = call_api("Hello")

2. Context Manager (flexible)

from agent_lens import AgentLens

lens = AgentLens(agent_name="my-agent")

with lens.trace(model="claude-3.5-sonnet") as t:
    result = client.chat.completions.create(...)
    t.input_tokens = result.usage.prompt_tokens
    t.output_tokens = result.usage.completion_tokens

3. Direct Record (manual)

from agent_lens import AgentLens

lens = AgentLens(agent_name="my-agent")
lens.record(
    model="gpt-4o",
    input_tokens=1500,
    output_tokens=800,
    latency_ms=2300,
)

Global Shortcuts

from agent_lens import record, trace, track

record(model="gpt-4o", input_tokens=100, output_tokens=50)

with trace(model="gpt-4o") as t:
    ...

@track(model="gpt-4o")
def my_func():
    ...

CLI Commands

Cost Optimization Workflow

小蔥技能7w4.net有更新，你可以訪問看下。

When user asks "how can I save money":

1. Run cost report: agent-lens cost
2. Identify expensive models: Which models cost the most?
3. Check token efficiency: Are prompts too long?
4. Suggest cheaper alternatives:
5. gpt-4o → gpt-4o-mini (10x cheaper)
6. claude-3.5-sonnet → claude-3.5-haiku (4x cheaper)
7. gpt-4 → gpt-4o (2x cheaper)
8. Check caching: Are there repeated prompts?
9. Check error rate: agent-lens report --by status

Token Counting

import tiktoken

def count_tokens(text: str, model: str = "gpt-4o") -> int:
    """Count tokens for a given model."""
    try:
        enc = tiktoken.encoding_for_model(model)
    except KeyError:
        enc = tiktoken.get_encoding("cl100k_base")
    return len(enc.encode(text))

# Check before sending
prompt = "Your long prompt here..."
tokens = count_tokens(prompt)
print(f"Prompt: {tokens} tokens")
print(f"Estimated cost: ${tokens * 2.50 / 1_000_000:.4f}")

Supported Models

Pricing data for: OpenAI (GPT-4o, o1, o3), Anthropic (Claude 3.5/4), Google (Gemini 2.x), DeepSeek, Mistral, Qwen, GLM, MiMo.

Unknown models are tracked but cost shows "—".

Integration with Hermes

# Track Hermes agent API calls
from agent_lens import AgentLens

lens = AgentLens(agent_name="hermes-main")

# In your agent loop:
with lens.trace(model=config.model) as t:
    response = agent.run_conversation(message)
    t.input_tokens = response.get("input_tokens", 0)
    t.output_tokens = response.get("output_tokens", 0)

Data Storage

SQLite at ~/.agent-lens/traces.db. Fully local, no cloud service needed.

Pitfalls

• Token extraction auto-works only for OpenAI-compatible response format
• For non-OpenAI providers, manually set t.input_tokens and t.output_tokens
• Cost estimates use list prices; actual costs may differ with discounts
• Database grows over time; use agent-lens clean periodically

Verification

agent-lens demo        # Generate 20 sample records
agent-lens stats       # Should show 20 calls
agent-lens cost        # Should show cost breakdown by model

這個 Skill 文件質量不錯，提供了詳細的 API 追蹤和成本分析指南，支援多種主流 AI 服務，介面友好。但壓縮包裡只有說明文件，缺少實際的程式碼檔案，體驗不夠完整，需要進一步補充核心實現程式碼後才能真正投入使用。版本標註不一致也略顯粗糙。總體來說想法實用，但完成度還有提升空間。