Verify our claims yourself.
We publish PPL ratios. We ship the tool to verify them. We document failures alongside wins.
No "trust me" — every number is reproducible end-to-end on your hardware from its pack (the 405B flagship pack is gated — access on request). The shape contract ships in the public CLI today (uc verify: structure + SHA-256 download integrity); the quality contract (PPL reproduction) runs via a harness we share with evaluators on request; a productized third-party audit service is on the Q3 2026 roadmap.
The verification stack.
Three independent layers, each answering a different question. The first two ship in the public package; the third is on the roadmap as a paid service. If a claim doesn't survive all three, we don't ship it as a customer-facing number.
uc verify <pack>
Question it answers
"Did I download the exact artifact that was published?" Checks the pack is well-formed and confirms the bytes you downloaded match the published SHA-256 manifest. If a single byte drifts, it fails loudly. This is a structure + download-integrity check — it does not run the model or reproduce perplexity.
a PPL reproduction harness (available to evaluators on request)
Question it answers"Does the published 1.0066× ratio actually reproduce on my hardware?" Pulls the bf16 baseline from HuggingFace, runs the eval on the FineWeb-edu held-out tail (seq_len=1024, seed=42) on your GPU, computes the PPL ratio locally, and prints whether it matches the published value.
Audit & Verification Service
Question it answers"Can a third party reproduce these numbers under a contract?" Productized service tier where Sipsa runs an end-to-end re-eval against a customer-supplied baseline and signs the resulting receipt. Targets regulated buyers who need an external attestation, not just self-serve reproducibility.
uc verify proves the bytes you downloaded equal the published manifest (structure + SHA-256 download integrity). The PPL reproduction harness proves those bytes behave the same way on a held-out evaluation. Either one alone is incomplete. The shape contract is in the public CLI; the quality-contract harness is shared with evaluators on request. We're saying that out loud here because the gap was real and we'd rather be precise about it than gloss over it.
Reproduce a published row in 3 commands
Today (shape contract):
# 1. install
pip install ultracompress
# 2. pull a published artifact
hf download SipsaLabs/qwen3-1.7b-base-uc-v3-bpw5 --local-dir ./pack
# 3. verify pack structure + SHA-256 download integrity
uc verify ./pack
# → VERIFY: PASS — structure + SHA-256 download integrity confirmed.
# (public `uc verify` checks the pack is well-formed and the bytes you
# downloaded match the published SHA-256 manifest. It does NOT run the
# model or reproduce perplexity — that is the quality contract below.)
Quality contract (PPL reproduction — available to evaluators on request, not in the public CLI):
# 4. reproduce the published PPL ratio on your own hardware (GPU)
# Runs via the PPL reproduction harness we share with evaluators —
# it pulls the bf16 baseline, runs the held-out eval, and compares.
# → published: 1.00401× measured: 1.0041× match: PASS
Public uc verify covers the shape contract only (structure + SHA-256 download integrity). Headline PPL ratios are re-derivable with the PPL reproduction harness, which we share with evaluators on request along with full eval provenance. We don't ship round numbers without source data behind them.
Reproducible, not cherry-picked.
Most projects publish the wins and ask you to trust them. We publish only numbers you can regenerate yourself: every figure is independently reproducible and SHA-256-verifiable, so the published result and the result on your own machine are the same result.
Every number is reproducible on demand.
uc verify confirms pack structure + download integrity against the public SHA-256 manifest — no GPU required — and a PPL reproduction harness is available to evaluators on request. We publish only results that reproduce; we don't cherry-pick the runs that looked good. If your reproduction disagrees with a published number beyond the eval tolerance, that's a bug report we want.
The README's "What doesn't work yet" section applies the same honesty to capabilities a reader might assume work because the rest of the package does. Both surfaces stay fresh.
What customers should ask us.
If you're evaluating a 5-bit near-lossless codec for production, these are the right questions to bring to the call. Our answers are the same in writing as they are on a sales call.
-
"Can I reproduce your headline number on my own hardware?"
Yes. The shape contract reproduces today via the public
uc verify(structure + SHA-256 download integrity). The quality contract (PPL ratio against the bf16 baseline pulled from HuggingFace) reproduces via a PPL reproduction harness we share with evaluators on request. Same JSON schema as our published numbers. -
"Is there an independent third-party audit?"
Coming as a productized service in Q3 2026 as part of the Compression-as-a-Service catalog. Until then: every published number is self-reproducible by the verification stack above, and full per-architecture eval provenance is available to any external auditor on request.
-
"What about regulatory-grade verification (SR 11-7, FDA, defense)?"
The on-prem MSA tier includes audit-trail logs, reproducibility manifests, and the reproducible reconstruction contract that maps onto SR 11-7 model validation, FDA-regulated healthcare AI dev/deploy equivalence, and defense deploy-bit-exactness. See /enterprise for the compliance architecture and the three engagement tiers.
-
"What if the architecture I care about isn't in your verified matrix?"
That's the Phase 0 POC. Bring a model, we deliver a UC pack inside 5 business days, and you self-verify it with
uc verify+ benchmark it with the PPL reproduction harness against your existing AWQ/GPTQ build on your hardware. If we don't materially help, you keep the diagnostic and we don't push Phase 1.
For VCs specifically.
We know the verifier-quality gap is the #1 trust objection from technical diligence. Here's the timeline to close it — written in the present and near-future tense, not the aspirational quarter-plus-two.
Three contracts, one credibility surface.
The objection is real and we're closing it on a public schedule. Each step below is either shipped or has a hard near-term ship date attached, not a vague "later."
- Public CLIShape contract. —
uc verifyships today inpip install ultracompress. Confirms pack structure + SHA-256 download integrity: the bytes you downloaded match the published manifest. - On requestQuality contract. — a PPL reproduction harness shared with evaluators on request (not in the public CLI). Reproduces the published PPL ratio on the customer's hardware against a HuggingFace-pulled bf16 baseline.
- Q3 2026Productized audit service. — Third-party-runnable verification under a paid contract, with a signed receipt. Roadmap item on the Compression-as-a-Service catalog.
The pattern is the same on every other surface: ship the verifier alongside the claim, document what didn't make it, refuse to round the numbers. The verification stack is the product around the codec, not a footnote underneath it.
Want to verify a specific row?
Single solo founder, US business hours, 4–8h response. Reproduce a published benchmark, request a Phase 0 POC for an architecture not yet in the matrix, or ask anything diligence-grade about the verification stack.