Verified inference benchmarks.

The verified matrix

22 PPL-verified architectures (17 dense + 4 MoE + 1 SSM) plus 1 ViT cosine-verified — across 4 architecture classes — at 5 bits per weight. Every PPL ratio measured under streaming-per-layer reconstruction comparator at seq_len=1024, seed=42, FineWeb-edu held-out tail (n=30–50 unless noted).

Mean across 22 verified records: 1.00537×. Median: 1.00527×.

Reproduce any row in 3 commands

Pick a row above. Copy the HF artifact name. Run:

pip install ultracompress
hf download SipsaLabs/qwen3-1.7b-base-uc-v3-bpw5 --local-dir ./pack
uc verify ./pack

For gated artifacts (10B+), click "Request access" on the HF page. Manual approval, usually within 24h. Free for sub-$1M ARR companies, individuals, research.

Eval methodology

• Comparator: per-layer streaming reconstruction. Both bf16 baseline and 5-bit compressed model use the same procedure on the same hardware (single 32 GB GPU), so the ratio isolates the codec contribution rather than confounding it with serving-stack differences.
• Dataset: FineWeb-edu held-out tail (no overlap with calibration). Seq length 1024 unless noted.
• n: 30–50 documents per row. Seed 42, deterministic.
• Hardware: dual RTX 5090 (32 GB each). 405B-class fits inside 32 GB peak via streaming compression — single consumer GPU is part of the value proposition.
• JSON receipts: full per-architecture PPL provenance is available to evaluators on request. We don't ship round numbers without source data behind them.

Reproducible, not cherry-picked

Reproducible weight reconstruction is independently SHA-256-verifiable on every openly-downloadable pack — uc verify confirms it on your own machine, and the PPL reproduction harness reproduces the perplexity comparison against the same held-out FineWeb-edu tail. (The 405B flagship pack is gated — access on request.)

• We publish only results that reproduce. If a number can’t be regenerated on demand from its pack, it doesn’t go on this page.
• No hand-tuned hero runs. Seed 42, deterministic, fixed n per row — the same harness for every architecture.
• State-space models past scalar-only: Mamba-2.8B canonical PPL ratio is 1.00593× with an SSM-compatible comparator (the canonical transformer streaming reconstruction is architecture-incompatible with SSMs). Comparator-note caveat is published with the record.
• Deferred is labelled deferred. Where a pack verifies clean but an eval is blocked, we say so explicitly instead of substituting an estimate.

The public streaming-compression eval JSONs ship in the repo; full per-architecture eval provenance is available to evaluators on request, so the table can be re-derived from source rather than taken on faith.

Want to use these via API?

Same model menu, OpenAI-compatible.

from openai import OpenAI
client = OpenAI(base_url="https://api.sipsalabs.com/v1", api_key="sk-...")
resp = client.chat.completions.create(
    model="sipsa-qwen3-8b",
    messages=[{"role": "user", "content": "test"}],
)
print(resp.choices[0].message.content)

$5 free credits on signup, no card. See /pricing for the full per-model token rates.

Questions about a specific row?

Direct line to the founder. Single solo founder; you'll hear back within 4–8h US business hours.

• Reproduce verification: founder@sipsalabs.com
• Architecture not in the list: Compression-as-a-Service, see /pricing
• Press / investor: press@sipsalabs.com