diff --git a/pymage/README.md b/pymage/README.md index f195f9f..8ebae3f 100644 --- a/pymage/README.md +++ b/pymage/README.md @@ -20,6 +20,13 @@ See [`DESIGN.md`](./DESIGN.md) for the full rationale. small layer (and the manifest). - **Reproducible**: same lock + same source + same base ⇒ same image digest. +This pays off most on **large AI/GPU images**, where modern `torch` wheels carry +the CUDA runtime (`nvidia-*` wheels) and the dependencies are ~98% of the image. +Bumping one dependency re-uploads a single ~11 MB layer instead of re-pushing the +whole ~2.9 GB `uv sync` venv layer — see +[`docs/ai-image-comparison.md`](./docs/ai-image-comparison.md). For pure-wheel +web apps see [`docs/real-world-comparison.md`](./docs/real-world-comparison.md). + ## Usage (uv projects) pymage is designed for [uv](https://docs.astral.sh/uv/) projects. Configure it diff --git a/pymage/docs/ai-image-comparison.md b/pymage/docs/ai-image-comparison.md new file mode 100644 index 0000000..6391da3 --- /dev/null +++ b/pymage/docs/ai-image-comparison.md @@ -0,0 +1,118 @@ +# Large AI images: where the size lives, and what pymage saves + +A common worry with GPU/AI Python images is that they're huge and slow to +rebuild and ship. This study answers two questions with real numbers: + +1. In a CUDA/PyTorch image, is the size in the **base image** or in the **Python + wheels**? +2. How do **incremental rebuilds** (bump one dependency, or edit app code) + compare between a normal `uv`/`pip` Dockerfile and pymage? + +**TL;DR:** for modern PyTorch installs the CUDA runtime ships *inside the +wheels*, so the dependencies — not the base — are almost the entire image +(2945 MB of 2988 MB here, **98.5%**). That plays directly to pymage's strength: +each wheel is its own content-addressed layer, so bumping one dependency +re-uploads **one layer (~11 MB)** while the ~2.9 GB of `torch`/CUDA layers are +deduplicated by the registry. A `uv sync` Dockerfile puts the whole environment +in a single layer, so the same one-line bump re-pushes the **entire ~2.9 GB**. + +## Where does the size come from? + +Modern `torch` wheels on PyPI are the CUDA build: `torch` itself plus a stack of +`nvidia-*` CUDA-runtime wheels (cuDNN, cuBLAS, cuFFT, NCCL, …) and `triton`. +These are pulled as ordinary wheel dependencies — the CUDA libraries are *in the +wheels*, not in the base image. + +A representative `uv` project (`torch`, `transformers`, `numpy`, `pillow`, +`fastapi`; 62 packages resolved) built on a `python:3.12-slim` base, pushed to a +local registry (compressed/registry sizes): + +| Bucket | Compressed size | Share | +| --- | --- | --- | +| **Dependency (wheel) layers** | **2945 MB** | **98.5%** | +| Base image (`python:3.12-slim`, 5 layers) | 43 MB | 1.5% | +| **Image total** | **2988 MB** | 100% | + +Largest single layers (one wheel each): + +| Wheel | Layer (compressed) | +| --- | --- | +| `torch==2.12.0` | 564 MB | +| `nvidia-cublas` | 434 MB | +| `nvidia-cudnn-cu13` | 374 MB | +| `triton` | 234 MB | +| `nvidia-cufft` | 229 MB | +| `nvidia-nccl-cu13` | 208 MB | +| `nvidia-cusolver` | 204 MB | +| `nvidia-cusparselt-cu13` | 174 MB | + +So the base is a rounding error; the wheels (dominated by `torch` + the CUDA +stack, ~2.7 GB) are the image. (Even if you *do* use a multi-GB +`nvidia/cuda:*-runtime` base, that base is stable and shared/cached — the churn +that costs you on every rebuild is the dependency layer.) + +## Incremental rebuilds (the real win) + +Built `v1`, then made two one-line changes and rebuilt, pushing to the same +registry and diffing the manifests. "Bytes re-uploaded" = the total size of +layers whose digest changed (every other layer is already present and is +skipped via a registry `HEAD`). + +| Change | Layers changed | Bytes re-uploaded | Rebuild wall time | +| --- | --- | --- | --- | +| Bump one dep (`transformers` 5.11.0 → 5.10.2) | **1 / 65** | **11.2 MB** | ~6 s | +| Edit one app source file | **1 / 65** | **0.3 KB** | ~6 s | + +On the dependency bump, the push moved exactly two blobs: the new +`transformers` layer (11 MB) and the image config; the other 63 layers — +including all ~2.9 GB of `torch`/CUDA — were already present and skipped. +**99.6% of the image (2977 MB) was deduplicated.** + +### Contrast with a `uv`/`pip` Dockerfile + +A typical CUDA Dockerfile installs the environment in a single step: + +```dockerfile +RUN --mount=... uv sync --frozen --no-install-project # ONE layer: ~2.9 GB +COPY . /app +``` + +- **Editing app code** invalidates the `COPY` layer only, so it's cheap in both + approaches (assuming the multi-stage / deps-first ordering above). With a naive + `RUN pip install … && COPY .` in one layer, even an app edit re-pushes + everything. +- **Bumping one dependency** re-runs `uv sync`, which rewrites the whole + virtualenv into a **single new image layer**. That entire layer — all ~2.9 GB, + including `torch` and every CUDA wheel that *didn't* change — becomes a new blob + that must be re-pushed and later re-pulled. + +| | pymage | `uv sync` Dockerfile | +| --- | --- | --- | +| Bytes re-pushed on a one-dep bump | **~11 MB** (the changed wheel) | **~2.9 GB** (whole venv layer) | +| Bytes a consumer re-pulls for the update | ~11 MB | ~2.9 GB | +| Layer granularity | one layer per wheel (auto-bucketed to ≤127) | one layer per `RUN` | + +That's roughly a **260× reduction** in data moved for a routine dependency bump, +and the savings repeat on every `docker pull` of the new revision across every +node/replica. Build time drops too: pymage reuses the cached `torch`/CUDA layers +and only fetches + layers the changed wheel (~6 s here), whereas re-running +`uv sync` re-materializes and re-exports the multi-GB environment. + +You can't easily get pymage's granularity from a Dockerfile: splitting `torch` +into its own layer means fragile, hand-maintained multi-`RUN` install ordering, +and even then a patch to one CUDA wheel re-pushes its whole `RUN` group. pymage +does per-wheel layering automatically and deterministically. + +## Method & caveats + +- Real `uv` project locked with `uv lock` (universal lock); pymage resolved the + linux/amd64 runtime closure and pulled wheels from the lock URLs. Sizes are + compressed (registry) bytes from the pushed manifests. +- Built on `python:3.12-slim` to isolate the dependency story. In production + you'd typically run GPU images on a CUDA base (or rely on the host NVIDIA + driver + container toolkit, since the `torch`/`nvidia-*` wheels already bundle + the CUDA *runtime* libraries). The base choice doesn't change the + layer-reuse result — the dependency layers are what churn. +- No GPU was needed: this measures build/layout/push behavior, not training. +- The first (cold) build downloads the full ~2.8 GB of wheels once (~90 s here); + subsequent builds reuse the wheel and layer caches. diff --git a/pymage/docs/real-world-comparison.md b/pymage/docs/real-world-comparison.md index d20bba2..65d214b 100644 --- a/pymage/docs/real-world-comparison.md +++ b/pymage/docs/real-world-comparison.md @@ -76,6 +76,10 @@ Dockerfile: contains. - **Fast, docker-less builds** from any OS. +> For the large **AI/GPU image** case (PyTorch + CUDA wheels, where deps are +> ~98% of a ~3 GB image and a one-dep bump re-uploads ~11 MB vs ~2.9 GB), see +> [`ai-image-comparison.md`](./ai-image-comparison.md). + ## Incremental rebuilds: where layering pays off The study above compares *first* builds. The bigger day-to-day win is what