fkiene/llmtrim

# Linux / macOS
curl -fsSL https://raw.githubusercontent.com/fkiene/llmtrim/main/install.sh | sh
Windows (PowerShell)
irm https://raw.githubusercontent.com/fkiene/llmtrim/main/install.ps1 | iex
Package managers
brew install fkiene/tap/llmtrim
cargo binstall llmtrim
scoop install llmtrim
docker run -d -p 43117:43117 -v llmtrim-state:/data ghcr.io/fkiene/llmtrim

Full options: INSTALL.md.

Menu-bar / system-tray popover with the same savings numbers. Bundled in Homebrew, Scoop, and npm; setup can enable open-at-login. Open with llmtrim tray. On Linux desktops, interactive ensure can fetch the tray binary from the latest release (needs libwebkit2gtk-4.1 and libayatana-appindicator3).

Same technique as mitmproxy, scoped to LLM API hosts only. setup changes three things; llmtrim uninstall reverses all three:

1. Private CA in ~/.llmtrim/ (name-constrained; cannot intercept your bank or email) 2. Shell env: HTTPS_PROXY + CA trust 3. Login service: daemon at login

No API keys stored (your tool's auth is forwarded). Prompts never touch disk; only anonymous token counts. Recoverable tool results stay in bounded daemon RAM for five hours by default and disappear on restart. Full threat model: SECURITY.md.

llmtrim ca
openssl x509 -in ~/.llmtrim/ca.pem -noout -text | grep -A3 "Name Constraints"

Stages run in savings order. Nothing under a cache_control marker is rewritten.

| Stage | What it does | When it runs | |---|---|---| | tool-output | Lossless template fold first, then window logs · diffs · grep · dumps down to errors / changes / matches; opt-in first-arrival recall can restore omitted cache-boundary results with llmtrim recall | tool results | | cache discipline | Mark + stabilize the invariant prefix (sort tools/schema · OpenAI prompt_cache_key) so it stays cached | tools | | lexical retrieval | BM25+ ranking with RM3 feedback · TextTiling topic cuts · budgeted non-redundant selection; question protected | long context | | skeletonization | tree-sitter keeps relevant function bodies, drops the rest to signatures (14 languages) | code | | serialize + hygiene | Minify JSON, encode record arrays to TOON or CSV, Unicode-normalize | always · lossless | | json sample | Down-sample huge record arrays: first/last + outliers + a query-biased diverse sample | big JSON | | dedup | Collapse duplicate + near-duplicate lines (prose only) | always | | output control | Terse instruction · Chain-of-Draft · token budget · native JSON schema · anti-overthink directive (quantized reasoning) · agent-loop frugality directive | auto | | tool layer | Static tool selection + description trimming | tools | | multimodal | Downscale images to the provider's resolution cap | images |

Default auto enables each stage only where it pays. safe is lossless-only. Config →

Send Claude Code through CLIProxyAPI instead of Anthropic, or as a fallback hop when the current path fails. Login is CLIProxyAPI's TUI; decide for yourself whether that fits the provider ToS.

llmtrim sub on                  # install + start CLIProxyAPI, enable redirect
llmtrim sub on grok             # write opus/sonnet/haiku/fable → that CLI's models
llmtrim sub auth                # CLIProxyAPI TUI — sign in
llmtrim sub models              # official + live sidecar models
llmtrim sub map on opus grok-4.6
llmtrim sub chain anthropic,codex
llmtrim sub mode fallback       # try hops in order when the current one fails
llmtrim sub status
llmtrim sub off

Interactive: llmtrim status → tab 4 Sub — Off / Always / Fallback, then e to edit the input→output map (type to search CLIProxyAPI's catalog; s save, a add, d delete). [ ] rotate the fallback first hop. llmtrim update also updates CLIProxyAPI when you use it. Point LLMTRIM_CLIPROXY_URL at an existing instance to skip the managed sidecar.

Route only a delegated Claude Code subagent while leaving the parent window unchanged:

llmtrim agents install          # also installed/refreshed by setup, update, and ensure

Then ask naturally: Implement it using a Grok subagent, use Terra, or review this with GPT Terra. Provider-only agents preserve the child request's Claude tier through the configured mapping; an explicit model agent pins that provider model. Request-local agents override the window /sub and global policy only for their own requests. llmtrim agents uninstall removes only llmtrim-owned agent files and records an opt-out so ensure leaves them removed.

This window only (installed with ensure; includes subagents; survives /clear):

/sub on [optional:cli-or-model]  # gemini, codex, claude, grok, kimi, vertex, qwen, copilot, or a model id
/sub off
/sub status

Sidecar: ~/.llmtrim/cliproxy/ (binary + config). Auth: ~/.cli-proxy-api when present, else ~/.llmtrim/cliproxy/auth. Env: LLMTRIM_SUB, LLMTRIM_SUB_MODE, LLMTRIM_CLIPROXY_URL, LLMTRIM_CLIPROXY_KEY.

Anthropic /login vs claude.ai connectors: with global sub in always mode, by default llmtrim writes a dummy ANTHROPIC_AUTH_TOKEN into ~/.claude/settings.json (same idea as claude-code-proxy's ANTHROPIC_AUTH_TOKEN=unused) so Claude Code does not need a live Anthropic OAuth session. The MITM strips that dummy token and sends /v1/messages to CLIProxyAPI; non-messages Anthropic probes are answered locally so they never return 401 Invalid bearer token.

Claude Code treats any API-key auth as overriding claude.ai login, so claude.ai connectors are disabled while the dummy token is set. That is expected. To keep connectors (and accept Anthropic /login when the session expires):

llmtrim sub anthropic-login keep   # connectors OK; Anthropic login required
llmtrim sub anthropic-login skip   # default: no Anthropic /login; connectors off

Restart Claude Code after sub on / sub off / sub mode / sub anthropic-login for the settings change to take effect. Fallback whose first hop is "what's in use" still needs a real Anthropic login.

echo '{"model":"gpt-4o","messages":[...]}' | llmtrim compress --provider openai > out.json
echo '{"model":"gpt-4o","messages":[...]}' | llmtrim send --provider openai

use llmtrim_core::{compress, ir::ProviderKind};
let out = compress(request_json, Some(ProviderKind::OpenAi))?;

import llmtrim
out = llmtrim.compress(request_json, llmtrim.Provider.OPEN_AI, "aggressive")

import { compress } from "@llmtrim/js";
const out = compress(requestJson, "openai", "aggressive");

Bindings and WASM notes: crates/llmtrim-uniffi · crates/llmtrim-wasm.

llmtrim mcp install          # Claude Code
llmtrim mcp install --print  # paste into any client

{
  "mcpServers": {
    "llmtrim": { "command": "llmtrim", "args": ["mcp"] }
  }
}

Tools: llmtrim_compress, llmtrim_compress_text, llmtrim_stats (same ledger as status).

auto composes these per request shape, so most users never set them directly. Pick one when you know your traffic and want to skip shape detection:

| preset | for | | --- | --- | | agent | tool-calling loops: prunes the tool block first-turn-only so the prompt cache stays warm | | code | coding turns: skeletonize and minify code, compress pasted logs and diffs | | rag | long context with a question: sentence-level retrieval | | cache | a fixed prefix reused across many calls | | reasoning | math and step-by-step workloads | | frugal | isolates the agent-loop frugality directive alone, for clean benchmarking |

Every stage is individually tunable via config flags; preset wins over individual flags. The full table is long; see the field list in config.rs or run llmtrim compress --help. The most useful knobs:

| field | default | meaning | | --- | --- | --- | | toolout | on in agent/aggressive | tool-output compression (logs / diffs / grep / dumps) | | retrieve | false | lexical retrieval for long context (lossy) | | skeletonize | false | drop non-relevant function bodies to signatures | | serialize | true | TOON / CSV encoding of record arrays | | json_crush | on in agent/aggressive | sample huge record arrays | | output_control | false | terse-output instruction + cap | | output_anti_overthink | on in aggressive/rag/code/agent | commit-to-answer directive for quantized reasoning traffic | | output_frugal_tools | on in agent | steers agent loops toward fewer tool-call turns (batch, don't repeat) | | cache | false | cache_control breakpoints (lossless) | | dedup | true | collapse duplicate lines (lossless) | | quality_gate | true | revert any lossy cut whose query-relevant coverage drops too far |

Env: LLMTRIM_PRESET (preset), LLMTRIM_CONFIG (config-file path).

These knobs are orthogonal to compression. Each resolves env-first, then from the config file, so set whichever fits. The env var wins when both are present.

| env var | config key | meaning | | --- | --- | --- | | LLMTRIM_EXTRA_HOSTS | extra_hosts | extra exact LLM-API hosts to intercept (comma-separated env / array in file), e.g. a self-hosted OpenAI-compatible endpoint | | LLMTRIM_EXCLUDE_PROVIDERS | exclude_providers | wire shapes to skip compressing: openai / anthropic / google (e.g. anthropic to leave Claude Code untouched); coarse, covers every host of that shape | | LLMTRIM_EXCLUDE_HOSTS | exclude_hosts | exact hostnames to skip compressing (e.g. openrouter.ai); precise, leaves other hosts of the same shape compressed | | LLMTRIM_UPSTREAM_PROXY | upstream_proxy | route egress through another proxy (see below) | | LLMTRIM_DB_PATH | db_path | ledger location | | LLMTRIM_CAPTURE_DIR | capture_dir | before/after QA capture directory | | LLMTRIM_CAPTURE_MAX_MB | capture_max_mb | capture corpus size ceiling (0 disables) | | LLMTRIM_FIRST_ARRIVAL_RECALL | first_arrival_recall | recoverable first-arrival tool-output shaping (default false; set true to admit cache-boundary results for llmtrim recall) | | LLMTRIM_TOOL_OUTPUT | toolout_passthrough | skip tool-output compression for matching commands: passthrough (all) or command globs (gpt.sh); also honoured as a command assignment or a result line. LLMTRIM_KEEP: lines are always retained | | LLMTRIM_FIRST_ARRIVAL_RECALL_TTL_SECS | first_arrival_recall_ttl_secs | in-memory raw-result lifetime (default 18,000 seconds / five hours) | | LLMTRIM_BIND | bind | listen IP (default loopback; 0.0.0.0 for containers) | | LLMTRIM_BREAKDOWN_WINDOW | breakdown_window | context-window override for the cost breakdown | | LLMTRIM_RETENTION_DAYS | retention_days | ledger age-retention in days | | LLMTRIM_NO_UPDATE_CHECK | no_update_check | disable the passive update check |

extra_hosts entries must be exact hostnames (llm.acme.com, never a bare acme.com): each one widens the name-constrained MITM CA, which regenerates automatically on the next launch to cover them.

export LLMTRIM_UPSTREAM_PROXY=http://host:port
or with auth: http://user:pass@host:port  (redacted in logs)

Outbound calls use CONNECT + verifying TLS; the upstream only sees the encrypted stream. Looping to llmtrim's own listen address is rejected. Put the variable in the daemon's launch environment (launchd / systemd), not only your interactive shell. Profile secrets sit in plaintext.

Companion tools on another port (e.g. headroom) are fine.