Use case · artifact storage
Reduce storage for binary-similar versions.
Backups, software catalogs, media asset variants, and AI model variants often share most of their bytes. DeltaGlider stores the differences while clients still write and read full S3 objects. This is storage deduplication, not S3 object versioning.
live product
How it runs
Three steps. No client rewrite.
Use your existing S3 tooling.
1
Point clients at the proxy
Keep aws-cli, boto3, SDKs, rclone, and MinIO Client. The wire protocol remains S3 + SigV4.
2
Upload normal artifacts
Delta-eligible archives and binary dumps route through xdelta3 against a per-prefix reference baseline.
3
Measure the truth
Dashboard and Prometheus metrics report original bytes, stored bytes, cache behavior, request rate, and errors.
Important distinction
Not S3 object versioning.
DeltaGlider does not restore old S3 object versions today. It stores repeated artifact releases more efficiently while preserving normal full-object reads.
What it does
Reduces storage for repeated binaries by storing deltas behind the S3 API.
What it does not do
Expose S3 version IDs or provide object-version restore workflows.
Real-world scenarios
Where the savings usually come from.
The benefit grows when you keep many versions that are internally similar. The more repeated binary structure you store, the more delta compression can compound.
Backup archives
Daily backup bundles often contain the same files, tables, or blocks with a small change set. Store every point in time without paying full price for repeated bytes.
Software catalogs
Release catalogs keep many builds, installers, packages, and archives. Adjacent versions are often highly similar, especially when packaged by the same pipeline.
Media and texture variants
Texture packs, asset bundles, and generated media variants can share large binary regions. Delta storage helps when variants are stored together over time.
AI model variants
Fine-tuned checkpoints and model variants can be binary-similar while still needing separate full-object reads. This is where the compression benefit can compound quickly.
Storage efficiency
Delta compression where it pays off.
Smart routing
Repeated archives and binary dumps use xdelta3 against a per-prefix baseline. Media and other poor-fit files pass through unchanged.
src/deltaglider/file_router.rsDrop-in S3 API
Existing S3 tools keep working. The proxy reconstructs full objects on GET, so applications never see delta files.
Live savings analytics
Per-bucket compression ratios, bytes saved, cache behavior, and request health from the dashboard. Prometheus metrics
src/api/handlers/status.rsdeltaglider_delta_compression_ratio and delta_bytes_saved_total are exposed at /metrics.Specifics
Default delta candidates, not a fixed promise.
These extensions are a starting point. You can add or remove file types for your workload. Savings depend on internal structure and binary similarity across versions.
.zip.tar.tgz.tar.gz.tar.bz2.jar.war.ear.sql.dump.bak.backup.rar.7z.dmg.isoDefault routing lives in src/deltaglider/file_router.rs. The right answer is workload-specific: a custom archive may delta well; a compressed archive with shuffled blocks may not.
Measure
Use your own artifact stream.
Savings depend on churn rate and file format. The dashboard shows original bytes, stored bytes, latency, and cache behavior.