Sipsa Labs / Products / UltraCompress / Verify
Independent quality verification

Verify our claims yourself.

We publish PPL ratios. We ship the tool to verify them. We document failures alongside wins.

No "trust me" — every number is reproducible end-to-end on your hardware from its pack (the 405B flagship pack is gated — access on request). The shape contract ships in the public CLI today (uc verify: structure + SHA-256 download integrity); the quality contract (PPL reproduction) runs via a harness we share with evaluators on request; a productized third-party audit service is on the Q3 2026 roadmap.

/ self-serve · honest about gaps · everything traces to source data

SHAPE CONTRACT · uc verify, shipping today QUALITY CONTRACT · PPL harness on request AUDIT SERVICE · productized Q3 2026 REPRODUCIBLE · SHA-256-verifiable, no cherry-picking
/ section 01

The verification stack.

Three independent layers, each answering a different question. The first two ship in the public package; the third is on the roadmap as a paid service. If a claim doesn't survive all three, we don't ship it as a customer-facing number.

/ Layer 01 Live today

uc verify <pack>

Question it answers

"Did I download the exact artifact that was published?" Checks the pack is well-formed and confirms the bytes you downloaded match the published SHA-256 manifest. If a single byte drifts, it fails loudly. This is a structure + download-integrity check — it does not run the model or reproduce perplexity.

Ships in: pip install ultracompress
/ Layer 02 On request

a PPL reproduction harness (available to evaluators on request)

Question it answers

"Does the published 1.0066× ratio actually reproduce on my hardware?" Pulls the bf16 baseline from HuggingFace, runs the eval on the FineWeb-edu held-out tail (seq_len=1024, seed=42) on your GPU, computes the PPL ratio locally, and prints whether it matches the published value.

Closes the verifier-quality gap surfaced during a recent external evaluation.
/ Layer 03 Roadmap Q3 2026

Audit & Verification Service

Question it answers

"Can a third party reproduce these numbers under a contract?" Productized service tier where Sipsa runs an end-to-end re-eval against a customer-supplied baseline and signs the resulting receipt. Targets regulated buyers who need an external attestation, not just self-serve reproducibility.

Part of the Compression-as-a-Service catalog.
Why two contracts, not one? "Reproducible" is a load-bearing word. Public uc verify proves the bytes you downloaded equal the published manifest (structure + SHA-256 download integrity). The PPL reproduction harness proves those bytes behave the same way on a held-out evaluation. Either one alone is incomplete. The shape contract is in the public CLI; the quality-contract harness is shared with evaluators on request. We're saying that out loud here because the gap was real and we'd rather be precise about it than gloss over it.

Reproduce a published row in 3 commands

Today (shape contract):

# 1. install
pip install ultracompress

# 2. pull a published artifact
hf download SipsaLabs/qwen3-1.7b-base-uc-v3-bpw5 --local-dir ./pack

# 3. verify pack structure + SHA-256 download integrity
uc verify ./pack
# → VERIFY: PASS — structure + SHA-256 download integrity confirmed.
#   (public `uc verify` checks the pack is well-formed and the bytes you
#    downloaded match the published SHA-256 manifest. It does NOT run the
#    model or reproduce perplexity — that is the quality contract below.)

Quality contract (PPL reproduction — available to evaluators on request, not in the public CLI):

# 4. reproduce the published PPL ratio on your own hardware (GPU)
#    Runs via the PPL reproduction harness we share with evaluators —
#    it pulls the bf16 baseline, runs the held-out eval, and compares.
# → published: 1.00401×  measured: 1.0041×  match: PASS

Public uc verify covers the shape contract only (structure + SHA-256 download integrity). Headline PPL ratios are re-derivable with the PPL reproduction harness, which we share with evaluators on request along with full eval provenance. We don't ship round numbers without source data behind them.

/ section 02

Reproducible, not cherry-picked.

Most projects publish the wins and ask you to trust them. We publish only numbers you can regenerate yourself: every figure is independently reproducible and SHA-256-verifiable, so the published result and the result on your own machine are the same result.

/ Verify it, don't trust it

Every number is reproducible on demand.

uc verify confirms pack structure + download integrity against the public SHA-256 manifest — no GPU required — and a PPL reproduction harness is available to evaluators on request. We publish only results that reproduce; we don't cherry-pick the runs that looked good. If your reproduction disagrees with a published number beyond the eval tolerance, that's a bug report we want.

See the full verified matrix →

The README's "What doesn't work yet" section applies the same honesty to capabilities a reader might assume work because the rest of the package does. Both surfaces stay fresh.

/ section 03

What customers should ask us.

If you're evaluating a 5-bit near-lossless codec for production, these are the right questions to bring to the call. Our answers are the same in writing as they are on a sales call.

/ section 04 · for investors

For VCs specifically.

We know the verifier-quality gap is the #1 trust objection from technical diligence. Here's the timeline to close it — written in the present and near-future tense, not the aspirational quarter-plus-two.

/ Verification roadmap

Three contracts, one credibility surface.

The objection is real and we're closing it on a public schedule. Each step below is either shipped or has a hard near-term ship date attached, not a vague "later."

  • Public CLIShape contract. uc verify ships today in pip install ultracompress. Confirms pack structure + SHA-256 download integrity: the bytes you downloaded match the published manifest.
  • On requestQuality contract. — a PPL reproduction harness shared with evaluators on request (not in the public CLI). Reproduces the published PPL ratio on the customer's hardware against a HuggingFace-pulled bf16 baseline.
  • Q3 2026Productized audit service. — Third-party-runnable verification under a paid contract, with a signed receipt. Roadmap item on the Compression-as-a-Service catalog.

The pattern is the same on every other surface: ship the verifier alongside the claim, document what didn't make it, refuse to round the numbers. The verification stack is the product around the codec, not a footnote underneath it.

Want to verify a specific row?

Single solo founder, US business hours, 4–8h response. Reproduce a published benchmark, request a Phase 0 POC for an architecture not yet in the matrix, or ask anything diligence-grade about the verification stack.

/ press: press@sipsalabs.com · security: security@sipsalabs.com · HuggingFace org: huggingface.co/SipsaLabs