JacobHuang91/prompt-refiner

Prompt Refiner

---

⚠️ PROJECT ARCHIVED (April 2026)

This project is no longer maintained. Token optimization is less critical now due to:

• Token costs dropped 12x (GPT-4: $2.50/M tokens)
• Context windows expanded to 200K-1M tokens
• Better alternatives: caching (90%+ savings), ML compression (20x), model routing

---

🚀 Lightweight Python library for AI Agents, RAG apps, and chatbots with smart context management and automatic token optimization.

Save 5-70% on API costs - 57% average reduction on function calling, 5-15% on RAG contexts.

---

🎯 Perfect for:

RAG Applications • AI Agents • Chatbots • Document Processing • Cost Optimization

---

Why use Prompt Refiner?

Build AI agents, RAG applications, and chatbots with automatic token optimization and smart context management. Here's a complete example (see examples/quickstart.py for full code):

from prompt_refiner import MessagesPacker, SchemaCompressor, ResponseCompressor, StripHTML, NormalizeWhitespace
1. Pack messages (automatic refining with default strategies)
packer = MessagesPacker(
    track_tokens=True,
    system="
You are a helpful AI assistant.
",
    context=(["
Installation Guide...
"], StripHTML() | NormalizeWhitespace()),
    query="Search for Python books."
)
messages = packer.pack()
2. Compress tool schema
tool_schema = pydantic_function_tool(SearchBooksInput, name="search_books")
compressed_schema = SchemaCompressor().process(tool_schema)
3. Call LLM with compressed schema
response = client.chat.completions.create(
    model="gpt-4o-mini", messages=messages, tools=[compressed_schema]
)
4. Compress tool response
tool_response = search_books(json.loads(tool_call.function.arguments))
compressed_response = ResponseCompressor().process(tool_response)

Key benefits:

• Default strategies - Automatic refining (MinimalStrategy for system/query, StandardStrategy for context/history)
• Tool schema compression - Save 10-70% tokens on AI agent function definitions (avg: 57%)
• Tool response compression - Save 30-70% tokens on agent tool outputs
• Compose operations with | - Chain multiple cleaners into a pipeline
• Save 5-15% tokens on RAG contexts - Remove HTML, whitespace, duplicates automatically
• All items included - No token budget limits, let LLM APIs handle final truncation
• Track savings - Measure token optimization impact with built-in savings tracking
• Production ready - Output goes directly to OpenAI without extra steps

| Module | Description | Components | |--------|-------------|------------| | Cleaner | Remove noise and save tokens | StripHTML(), NormalizeWhitespace(), FixUnicode(), JsonCleaner() | | Compressor | Reduce size aggressively | TruncateTokens(), Deduplicate() | | Scrubber | Protect sensitive data | RedactPII() | | Tools | Optimize AI agent function calling (tool schemas & responses) | SchemaCompressor(), ResponseCompressor() | | Packer | Smart message composition with priority-based ordering | MessagesPacker (chat APIs), TextPacker (completion APIs) | | Strategy | Benchmark-tested presets for quick setup | MinimalStrategy, StandardStrategy, AggressiveStrategy |

# Basic installation (lightweight, zero dependencies)
pip install llm-prompt-refiner
With precise token counting (optional, installs tiktoken)
pip install llm-prompt-refiner[token]

• strategy/ - Preset strategies (Minimal, Standard, Aggressive) with benchmark results
• cleaner/ - HTML cleaning, JSON compression, whitespace normalization, Unicode fixing
• compressor/ - Smart truncation, deduplication
• scrubber/ - PII redaction (emails, phones, credit cards, etc.)
• tools/ - Tool/API output cleaning for agent systems
• packer/ - Context budget management with OpenAI integration
• analyzer/ - Token counting and cost savings tracking

📖 Full documentation: examples/README.md

Prompt Refiner has been rigorously tested across 3 comprehensive benchmark suites covering function calling, RAG applications, and performance. Here's what the data shows:

SchemaCompressor was tested on 20 real-world API schemas from Stripe, Salesforce, HubSpot, Slack, OpenAI, Anthropic, and more:

Key Highlights:

• ✨ 56.9% average reduction across all schemas (15,342 tokens saved)
• 🔒 100% lossless compression - all protocol fields preserved (name, type, required, enum)
• ✅ 100% callable (20/20 validated) - all compressed schemas work correctly with OpenAI function calling
• 🏢 Enterprise APIs see 70%+ reduction - HubSpot, Salesforce, OpenAI File Search
• 📊 Real-world schemas from production APIs, not synthetic examples
• ⚡ Zero API cost** - local processing with tiktoken

SchemaCompressor achieves 60%+ reduction on complex APIs Estimated monthly savings for different agent sizes (GPT-4 pricing)

✅ Functional Validation:

We tested all 20 compressed schemas with real OpenAI function calling to prove they work correctly:

• 100% callable (20/20): Every compressed schema successfully triggers function calls
• 60% identical (12/20): Majority produce exactly the same arguments as original schemas
• 40% different but valid (8/20): Compressed descriptions may influence LLM's choice among valid options (e.g., default values, placeholders)
• Bottom line: Compression is safe for production - schemas remain functionally correct

💰 Cost Savings Example: A medium agent (10 tools, 500 calls/day) saves $541/month with SchemaCompressor.

> 📖 See full benchmark: benchmark/README.md#function-calling-benchmark

---

📚 RAG & Text Optimization Benchmark: 5-15% Token Reduction

Tested on 30 real-world test cases (SQuAD + RAG scenarios) to measure token reduction and quality preservation:

Key Insights:

• ✅ Standard strategy: 5% reduction with 98.4% cosine similarity and 90% judge approval
• 🚀 Aggressive strategy: 15% reduction while maintaining 96.4% semantic quality
• 📊 Individual tests: Up to 74% token savings on contexts with HTML and duplicates

💰 Cost Savings: At 1M tokens/month, 15% reduction saves $54/month on GPT-4 input tokens.

> 📖 See full benchmark: benchmark/README.md#rag-quality-benchmark

⚡ Performance & Latency

"What's the latency overhead?" - Negligible. Prompt Refiner adds < 0.5ms per 1k tokens of overhead.

Key Insights:

• ⚡ Minimal strategy: Only 0.05ms per 1k tokens (faster than a network packet)
• 🎯 Standard strategy: 0.25ms per 1k tokens - adds ~2.5ms to a 10k token prompt
• 📊 Context: Network + LLM TTFT is typically 600ms+, refining adds < 0.5% overhead
• 🚀 Individual operations (HTML, whitespace) are < 0.5ms per 1k tokens

10k token RAG context refining: ~2.5ms overhead
Network latency: ~100ms
LLM Processing (TTFT): ~500ms+
Total overhead: < 0.5% of request time

🔬 Run yourself: python benchmark/latency/benchmark.py (no API keys needed)

🎮 Interactive Demo

Try prompt-refiner in your browser - no installation required!

Play with different strategies, see real-time token savings, and find the perfect configuration for your use case. Features:

• 🎯 6 preset examples (e-commerce, support tickets, docs, RAG, etc.)
• ⚡ Quick strategy presets (Minimal, Standard, Aggressive)
• 💰 Real-time cost savings calculator
• 🔧 All 7 operations configurable
• 📊 Visual metrics dashboard

Star History

License

MIT