tetherto/qvac

Run your first example using the JS/TS SDK.

1. Create the examples workspace:

mkdir qvac-examples
cd qvac-examples
npm init -y && npm pkg set type=module

2. Install the SDK:

npm i @qvac/sdk

3. Create qvac.config.json to enable client and server logs during the run:

{
  "loggerConsoleOutput": true,
  "loggerLevel": "info"
}

4. Create the quickstart.js script:

import { loadModel, LLAMA_3_2_1B_INST_Q4_0, completion, unloadModel } from '@qvac/sdk';
try {
  const modelId = await loadModel({
    modelSrc: LLAMA_3_2_1B_INST_Q4_0,
    onProgress: (p) => {
      const mb = (n) => (n / 1e6).toFixed(1);
      const line = ▸ Downloading ${p.percentage.toFixed(0)}% (${mb(p.downloaded)}/${mb(p.total)} MB);
      process.stderr.write(process.stderr.isTTY ? \r${line} : ${line}\n);
      if (p.percentage >= 100) process.stderr.write('\n');
    },
  });
  const history = [{ role: 'user', content: 'Explain quantum computing in one sentence' }];
  const result = completion({ modelId, history, stream: true });
  for await (const token of result.tokenStream) {
    process.stdout.write(token);
  }
  await unloadModel({ modelId });
} catch (error) {
  console.error('✖', error);
  process.exit(1);
}

5. Run the quickstart script:

QVAC_CONFIG_PATH=./qvac.config.json node quickstart.js

You'll see the model download first. Then QVAC will stream the response tokens and print them to the terminal.

Run your first example using the Python SDK.

1. Create the examples workspace:

mkdir qvac-examples-py
cd qvac-examples-py
python -m venv .venv
source .venv/bin/activate

2. Install the package (self-contained — bundles the QVAC worker and Bare runtime, no Node.js required):

# Replace  with the release you want, e.g. sdk-v0.17.0:
pip install tetherto-qvac-sdk \
  -f https://github.com/tetherto/qvac/releases/expanded_assets/sdk-v

3. Create the quickstart.py script:

import asyncio
import sys
from tetherto.qvac_sdk import Client, completion, load_model, unload_model
from tetherto.qvac_sdk.models import LLAMA_3_2_1B_INST_Q4_0
def print_progress(p):
    line = f"▸ Downloading {p.percentage:.0f}% ({p.downloaded / 1e6:.1f}/{p.total / 1e6:.1f} MB)"
    print(line, end="\r" if sys.stderr.isatty() else "\n", file=sys.stderr)
    if p.percentage >= 100:
        print(file=sys.stderr)
async def main():
    async with Client() as client:
        t = client.transport
        try:
            model_id = await load_model(
                t, model_src=LLAMA_3_2_1B_INST_Q4_0, on_progress=print_progress
            )
            run = completion(
                t,
                model_id=model_id,
                history=[
                    {"role": "user", "content": "Explain quantum computing in one sentence"},
                ],
            )
            async for event in run.events:
                if event.type == "contentDelta":
                    sys.stdout.write(event.text)
                    sys.stdout.flush()
            print()
            await unload_model(t, model_id)
        except Exception as error:
            print(f"✖ {error}", file=sys.stderr)
            return 1
    return 0
if __name__ == "__main__":
    sys.exit(asyncio.run(main()))

4. Run the quickstart script:

python quickstart.py

You'll see the model download first. Then QVAC will stream the response tokens and print them to the terminal.

Launch the server with the CLI, then use QVAC as model provider for OpenAI-compatible tools like OpenCode and OpenClaw.

1. Install the CLI globally (this also installs @qvac/sdk as a transitive dependency):

npm install -g @qvac/cli

2. Create the examples workspace:

mkdir qvac-server
cd qvac-server

3. Create the qvac.config.json declaring one model to serve:

{
  "serve": {
    "models": {
      "my-llm": {
        "model": "QWEN3_600M_INST_Q4",
        "default": true,
        "config": { "ctx_size": 8192 }
      }
    }
  }
}

4. Start the server (bound to 127.0.0.1:11434 by default):

qvac serve openai

The model downloads on first start and is preloaded into memory. You'll see progress in the server output.

5. From another terminal, hit it with any OpenAI-compatible client. A minimal curl:

curl http://localhost:11434/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "my-llm",
    "messages": [{"role": "user", "content": "Explain quantum computing in one sentence"}]
  }'

The response comes back as a single JSON payload with the model's answer. Add "stream": true to the body to get an SSE stream instead.

6. Point your AI tool at the server: open its model provider settings and add a new OpenAI-compatible provider with base URL http://localhost:11434/v1, any string as the API key, and my-llm as the model name.

[!IMPORTANT]

Setup varies by tool, and we ship dedicated plugins for some of them (like OpenCode and OpenClaw) that run the server for you. See Connect AI tools to QVAC for details.