docs: AI/GPU image comparison — CUDA ships in wheels (~98% of image); one-dep bump re-uploads ~11MB vs ~2.9GB for a uv sync Dockerfile

Co-authored-by: Jason Hall <imjasonh@users.noreply.github.com>

pymage/README.md

@@ -20,6 +20,13 @@ See [`DESIGN.md`](./DESIGN.md) for the full rationale.
   small layer (and the manifest).
 - **Reproducible**: same lock + same source + same base ⇒ same image digest.
 
+This pays off most on **large AI/GPU images**, where modern `torch` wheels carry
+the CUDA runtime (`nvidia-*` wheels) and the dependencies are ~98% of the image.
+Bumping one dependency re-uploads a single ~11 MB layer instead of re-pushing the
+whole ~2.9 GB `uv sync` venv layer — see
+[`docs/ai-image-comparison.md`](./docs/ai-image-comparison.md). For pure-wheel
+web apps see [`docs/real-world-comparison.md`](./docs/real-world-comparison.md).
+
 ## Usage (uv projects)
 
 pymage is designed for [uv](https://docs.astral.sh/uv/) projects. Configure it

pymage/docs/ai-image-comparison.md

@@ -0,0 +1,118 @@
+# Large AI images: where the size lives, and what pymage saves
+
+A common worry with GPU/AI Python images is that they're huge and slow to
+rebuild and ship. This study answers two questions with real numbers:
+
+1. In a CUDA/PyTorch image, is the size in the **base image** or in the **Python
+   wheels**?
+2. How do **incremental rebuilds** (bump one dependency, or edit app code)
+   compare between a normal `uv`/`pip` Dockerfile and pymage?
+
+**TL;DR:** for modern PyTorch installs the CUDA runtime ships *inside the
+wheels*, so the dependencies — not the base — are almost the entire image
+(2945 MB of 2988 MB here, **98.5%**). That plays directly to pymage's strength:
+each wheel is its own content-addressed layer, so bumping one dependency
+re-uploads **one layer (~11 MB)** while the ~2.9 GB of `torch`/CUDA layers are
+deduplicated by the registry. A `uv sync` Dockerfile puts the whole environment
+in a single layer, so the same one-line bump re-pushes the **entire ~2.9 GB**.
+
+## Where does the size come from?
+
+Modern `torch` wheels on PyPI are the CUDA build: `torch` itself plus a stack of
+`nvidia-*` CUDA-runtime wheels (cuDNN, cuBLAS, cuFFT, NCCL, …) and `triton`.
+These are pulled as ordinary wheel dependencies — the CUDA libraries are *in the
+wheels*, not in the base image.
+
+A representative `uv` project (`torch`, `transformers`, `numpy`, `pillow`,
+`fastapi`; 62 packages resolved) built on a `python:3.12-slim` base, pushed to a
+local registry (compressed/registry sizes):
+
+| Bucket | Compressed size | Share |
+| --- | --- | --- |
+| **Dependency (wheel) layers** | **2945 MB** | **98.5%** |
+| Base image (`python:3.12-slim`, 5 layers) | 43 MB | 1.5% |
+| **Image total** | **2988 MB** | 100% |
+
+Largest single layers (one wheel each):
+
+| Wheel | Layer (compressed) |
+| --- | --- |
+| `torch==2.12.0` | 564 MB |
+| `nvidia-cublas` | 434 MB |
+| `nvidia-cudnn-cu13` | 374 MB |
+| `triton` | 234 MB |
+| `nvidia-cufft` | 229 MB |
+| `nvidia-nccl-cu13` | 208 MB |
+| `nvidia-cusolver` | 204 MB |
+| `nvidia-cusparselt-cu13` | 174 MB |
+
+So the base is a rounding error; the wheels (dominated by `torch` + the CUDA
+stack, ~2.7 GB) are the image. (Even if you *do* use a multi-GB
+`nvidia/cuda:*-runtime` base, that base is stable and shared/cached — the churn
+that costs you on every rebuild is the dependency layer.)
+
+## Incremental rebuilds (the real win)
+
+Built `v1`, then made two one-line changes and rebuilt, pushing to the same
+registry and diffing the manifests. "Bytes re-uploaded" = the total size of
+layers whose digest changed (every other layer is already present and is
+skipped via a registry `HEAD`).
+
+| Change | Layers changed | Bytes re-uploaded | Rebuild wall time |
+| --- | --- | --- | --- |
+| Bump one dep (`transformers` 5.11.0 → 5.10.2) | **1 / 65** | **11.2 MB** | ~6 s |
+| Edit one app source file | **1 / 65** | **0.3 KB** | ~6 s |
+
+On the dependency bump, the push moved exactly two blobs: the new
+`transformers` layer (11 MB) and the image config; the other 63 layers —
+including all ~2.9 GB of `torch`/CUDA — were already present and skipped.
+**99.6% of the image (2977 MB) was deduplicated.**
+
+### Contrast with a `uv`/`pip` Dockerfile
+
+A typical CUDA Dockerfile installs the environment in a single step:
+
+```dockerfile
+RUN --mount=... uv sync --frozen --no-install-project   # ONE layer: ~2.9 GB
+COPY . /app
+```
+
+- **Editing app code** invalidates the `COPY` layer only, so it's cheap in both
+  approaches (assuming the multi-stage / deps-first ordering above). With a naive
+  `RUN pip install … && COPY .` in one layer, even an app edit re-pushes
+  everything.
+- **Bumping one dependency** re-runs `uv sync`, which rewrites the whole
+  virtualenv into a **single new image layer**. That entire layer — all ~2.9 GB,
+  including `torch` and every CUDA wheel that *didn't* change — becomes a new blob
+  that must be re-pushed and later re-pulled.
+
+|  | pymage | `uv sync` Dockerfile |
+| --- | --- | --- |
+| Bytes re-pushed on a one-dep bump | **~11 MB** (the changed wheel) | **~2.9 GB** (whole venv layer) |
+| Bytes a consumer re-pulls for the update | ~11 MB | ~2.9 GB |
+| Layer granularity | one layer per wheel (auto-bucketed to ≤127) | one layer per `RUN` |
+
+That's roughly a **260× reduction** in data moved for a routine dependency bump,
+and the savings repeat on every `docker pull` of the new revision across every
+node/replica. Build time drops too: pymage reuses the cached `torch`/CUDA layers
+and only fetches + layers the changed wheel (~6 s here), whereas re-running
+`uv sync` re-materializes and re-exports the multi-GB environment.
+
+You can't easily get pymage's granularity from a Dockerfile: splitting `torch`
+into its own layer means fragile, hand-maintained multi-`RUN` install ordering,
+and even then a patch to one CUDA wheel re-pushes its whole `RUN` group. pymage
+does per-wheel layering automatically and deterministically.
+
+## Method & caveats
+
+- Real `uv` project locked with `uv lock` (universal lock); pymage resolved the
+  linux/amd64 runtime closure and pulled wheels from the lock URLs. Sizes are
+  compressed (registry) bytes from the pushed manifests.
+- Built on `python:3.12-slim` to isolate the dependency story. In production
+  you'd typically run GPU images on a CUDA base (or rely on the host NVIDIA
+  driver + container toolkit, since the `torch`/`nvidia-*` wheels already bundle
+  the CUDA *runtime* libraries). The base choice doesn't change the
+  layer-reuse result — the dependency layers are what churn.
+- No GPU was needed: this measures build/layout/push behavior, not training.
+- The first (cold) build downloads the full ~2.8 GB of wheels once (~90 s here);
+  subsequent builds reuse the wheel and layer caches.

pymage/docs/real-world-comparison.md

@@ -76,6 +76,10 @@ Dockerfile:
   contains.
 - **Fast, docker-less builds** from any OS.
 
+> For the large **AI/GPU image** case (PyTorch + CUDA wheels, where deps are
+> ~98% of a ~3 GB image and a one-dep bump re-uploads ~11 MB vs ~2.9 GB), see
+> [`ai-image-comparison.md`](./ai-image-comparison.md).
+
 ## Incremental rebuilds: where layering pays off
 
 The study above compares *first* builds. The bigger day-to-day win is what