UltraCompress documentation

Customer-side reproducible 5-bit transformer compression at near-lossless quality (~1% perplexity) with SHA-256 verifiable, reproducible reconstruction. 22 PPL-verified architectures (17 dense + 4 MoE + 1 SSM) plus 1 ViT cosine-verified — across 4 architecture classes. Drop into any PyTorch / vLLM / TensorRT-LLM / sglang inference stack.

Install

pip install ultracompress
# or with uv:
uv pip install ultracompress

Requires Python 3.10+, PyTorch 2.5+, CUDA 12.x for GPU paths. The verifier (uc verify) is CPU-only and works on any machine.

Download a compressed model

Every Sipsa-compressed model is hosted on the public HuggingFace org SipsaLabs. Use either the hf CLI or any HuggingFace download path:

# Option A — HF CLI (recommended)
hf download SipsaLabs/qwen3-8b-uc-v3-bpw5 --local-dir ./qwen3-8b

# Option B — Python
from huggingface_hub import snapshot_download
snapshot_download("SipsaLabs/qwen3-8b-uc-v3-bpw5", local_dir="./qwen3-8b")

Available models

22 PPL-verified architectures (17 dense + 4 MoE + 1 SSM) plus 1 ViT cosine-verified — across 4 architecture classes (0.6B–405B params). A small public verification set is on huggingface.co/SipsaLabs; the full catalog is available to customers under engagement. Highlights:

Verify SHA-256 download integrity

Every Sipsa-compressed artifact ships with a per-file SHA-256 manifest. Verify locally with zero trust in the vendor — this checks the pack is well-formed and the bytes you downloaded match the published manifest. It does not run the model or reproduce perplexity (that's the on-request reproduction harness):

uc verify ./qwen3-8b
# → structure + SHA-256 download integrity verified across N layer files
# → PASS (or per-file hash diffs on FAIL)

Runs on CPU only. No GPU required. This download-integrity + structural check is the cryptographic primitive that AWQ / GPTQ / EXL3 don't publish (they leave reproducibility ambiguous).

Benchmark TTFT / tokens-per-sec / VRAM

# → TTFT: 142 ms
# → tokens/sec: 87.3
# → peak VRAM: 6.2 GB

Measures time-to-first-token, tokens-per-second, and peak GPU memory. Useful for capacity planning and side-by-side comparison with bf16 / AWQ / GPTQ baselines.

Integrate with your inference stack

UltraCompress reconstructs the PyTorch model reproducibly — deterministic reconstruction of the validated artifact, verified via SHA-256 — before any inference framework runs. Once reconstructed, it's a standard transformers model that drops into any inference stack you already use.

vLLM

from vllm import LLM
from ultracompress import load_compressed

# Reconstruct the model from the compressed pack to a standard checkpoint dir:
model = load_compressed("./qwen3-8b")
model.save_pretrained("./qwen3-8b-reconstructed")

# Then load with vLLM as normal:
llm = LLM(model="./qwen3-8b-reconstructed", dtype="bfloat16")
outputs = llm.generate(["Hello, compressed world"])

TensorRT-LLM

# Reconstruct first via the Python loader, then build TRT-LLM engine as normal:
python -c "from ultracompress import load_compressed; load_compressed('./qwen3-8b').save_pretrained('./qwen3-8b-reconstructed')"
trtllm-build --checkpoint_dir ./qwen3-8b-reconstructed --output_dir ./qwen3-8b-engine

sglang

python -c "from ultracompress import load_compressed; load_compressed('./qwen3-8b').save_pretrained('./qwen3-8b-reconstructed')"
python -m sglang.launch_server --model-path ./qwen3-8b-reconstructed

Plain transformers

from ultracompress import load_compressed
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("./qwen3-8b")
model = load_compressed("./qwen3-8b")  # reconstructs in <10 sec for 7B-class

inputs = tokenizer("Hello, compressed world", return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(outputs[0]))

Use the managed API (self-serve)

The OpenAI-compatible inference API at api.sipsalabs.com/v1 is publicly self-serve. Subscribe Pro $20/mo, Max from $100/mo, or Team $25/seat/mo at sipsalabs.com/pricing, or grab free $5 credits with no card at sipsalabs.com/get-access. Once you have a key:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.sipsalabs.com/v1",
    api_key=os.environ["SIPSA_API_KEY"],
)

response = client.chat.completions.create(
    model="sipsa-qwen3-0.6b",  # instant. 70B and 405B available, see /pricing
    messages=[{"role": "user", "content": "hello, compressed world"}],
)
print(response.choices[0].message.content)

Streaming, function-calling, and the other endpoints the openai SDK speaks all work the same way. Model IDs are prefixed sipsa-* on the API — full list at /v1/models.

CLI reference

Reconstructing a compressed pack back into a standard transformers checkpoint is done by loading the pack with load_compressed() (shown above) or pointing your runtime directly at the pack directory. The legacy uc unpack / uc verify-org / uc status commands are not part of the v0.6.x public CLI.

FAQ

What's the difference between Sipsa and AWQ / GPTQ / EXL3?

Tightest PPL drift in the 4–5 bpw band, plus SHA-256 reproducible reconstruction (the cryptographic property that the customer-loaded model is SHA-256-identical to the validated artifact, every load). AWQ / GPTQ / EXL3 leave reproducibility ambiguous — you cannot bit-compare two runs. Sipsa proves reproducible reconstruction via per-tensor hash manifest. For SOC 2 / SR-11-7 / FDA / DoD audit, the difference is qualitative.

Does this require special hardware?

No. The reconstructed model runs on the standard PyTorch path. CUDA GPU for inference (consumer or datacenter), CPU only for uc verify. Tested on RTX 5090 / A100 / H100 / consumer Apple Silicon (Metal backend via PyTorch).

What's the inference speed overhead?

Reconstruction happens at load time (~5–10 sec for 70B-class). Two modes ship today: throughput mode reconstructs to bf16 in VRAM for bf16-parity inference speed (VRAM usage matches bf16); compact mode uses streaming dispatch at roughly half the VRAM, at reduced tokens/sec. On disk, packs are ~3× smaller than bf16 (16-bit → ~5 bits/weight). The long-context KV-cache savings are additive on top of either mode.

What's the license?

BUSL-1.1 with Additional Use Grant: free for personal use, research, and any company under $1M ARR. Auto-converts to Apache 2.0 four years after each release. Commercial license at scale via founder@sipsalabs.com.

What about 100B+ models?

Hermes-3-Llama-3.1-405B (405B dense) is the largest fully-verified artifact today. Mixtral-8x22B (141B MoE, 1.00611×), Qwen3-235B-A22B (235B MoE, 1.00377×), and Phi-3.5-MoE (42B MoE, 1.00129×) are PPL-verified and available as gated packs on request — they sit on top of the 23 in the public registry, not inside that count. The trillion-class roadmap targets DeepSeek-V3 685B once disk + MLA arch support land in the compressor (Q3 2026).

How do I cite Sipsa in a paper?

@software{sipsalabs_ultracompress_2026,
  title  = {UltraCompress: SHA-256 verifiable, reproducible near-lossless 5-bit transformer compression},
  author = {{Sipsa Labs}},
  year   = {2026},
  url    = {https://sipsalabs.com},
  note   = {Sipsa Labs, Inc.}
}

Read more

• Public benchmark dashboard (23 verified records)
• Pricing tiers
• Security & compliance posture
• Get free $5 API credits (no card)
• Subscribe Pro $20/mo, Max from $100/mo, or Team $25/seat/mo
• GitHub source