From 971e955c35e149a28bf57b9176d2964dc7c5720a Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Thu, 11 Jun 2026 01:50:41 +0000 Subject: [PATCH] wheelhouse: don't build sdists; fail fast and direct to --find-links pre-built wheels Building sdists runs dependency build code on the host (RCE surface, no sandbox), is non-hermetic/non-reproducible, needs a toolchain, and is host-arch-only for compiled packages -- all at odds with pymage's guarantees. Remove the builder and --build-sdists opt-in; keep sdist parsing only so the error can explain the package is sdist-only and point at the --find-links pre-build workflow. Co-authored-by: Jason Hall --- pymage/DESIGN.md | 7 +- pymage/README.md | 51 +++--- pymage/docs/real-world-comparison.md | 46 ++--- pymage/internal/cli/cli.go | 4 +- pymage/internal/cli/defaults.go | 3 - pymage/internal/project/project.go | 1 - pymage/internal/wheelhouse/sdist.go | 222 ----------------------- pymage/internal/wheelhouse/sdist_test.go | 76 +------- pymage/internal/wheelhouse/wheelhouse.go | 49 +++-- 9 files changed, 88 insertions(+), 371 deletions(-) delete mode 100644 pymage/internal/wheelhouse/sdist.go diff --git a/pymage/DESIGN.md b/pymage/DESIGN.md index efb0c2b..2258e05 100644 --- a/pymage/DESIGN.md +++ b/pymage/DESIGN.md @@ -25,8 +25,11 @@ because it exploits content-addressed layering: ### Non-goals (initially) - Compiling C extensions / building wheels from sdists. We consume **pre-built - wheels** only. (sdist→wheel is a future extension; it would run a build, then - the resulting wheel re-enters the normal layer path.) + wheels** only. Building an sdist would run the dependency's own build code on + the host — non-hermetic, non-reproducible, an RCE surface with no sandbox, and + host-arch-only for compiled packages — so it is intentionally out of scope. + When a lock has no compatible wheel, pymage fails fast and directs the user to + pre-build a wheel out-of-band and supply it via `--find-links`. - Replacing dependency *resolution*. We delegate resolution to existing, correct tools (`uv` / `pip`) and consume their lockfile output. - A general-purpose Dockerfile interpreter. This is a focused Python app builder diff --git a/pymage/README.md b/pymage/README.md index d726831..f195f9f 100644 --- a/pymage/README.md +++ b/pymage/README.md @@ -73,7 +73,6 @@ the config value, which overrides the built-in default. | `max-wheel-layers` | `--max-wheel-layers` | *(derived from `max-layers`)* | | `push-concurrency` | `--push-concurrency` | auto (≥ 4, scales with CPUs) | | `no-cache` | `--no-cache` | `false` (caching is on by default) | -| `build-sdists` | `--build-sdists` | `false` (sdists are opt-in; see warning below) | | `extras` | `--extra` (repeatable) | — (enables uv project optional-dependency groups) | | `package` | `--package` | — (build a single uv workspace member) | | `python` | `--python` | auto-detected from the base | @@ -167,34 +166,35 @@ get from `uv sync --no-dev`), not every package in the lock: Windows-only or stale-Python-only packages. Markers are evaluated per platform, so each arch of a multi-arch build gets the correct set. -### Source distributions (sdists) — opt-in +### Source distributions (sdists) -By default pymage installs **only pre-built wheels**. If the lock pins a package -with no compatible wheel, the build fails fast and tells you to either supply a -wheel via `--find-links` or opt into sdist building. +pymage installs **pre-built wheels only — it does not build sdists.** This is a +deliberate choice: building a source distribution runs the dependency's own build +code (`setup.py` / build backend) on the build host, which would break pymage's +core guarantees — it's not hermetic or byte-reproducible, it's a remote-code- +execution surface with no container to sandbox it, it needs a build toolchain, +and compiled packages can only be built for the host architecture (no multi-arch). -Building from an sdist is **off by default** and must be enabled with -`--build-sdists` (or `build-sdists = true` in `[tool.pymage]`). When enabled, -pymage downloads the hash-verified sdist and builds a wheel with the host's `pip` -(`pip wheel --no-deps`), which requires a Python toolchain (`python3`/`python` -with `pip`) on the build host. It is a power-user feature with two important -caveats: +In practice this is rarely an issue: the modern ecosystem is wheel-first, so +mainstream dependencies on common targets all publish wheels. You only hit an +sdist for (a) older, low-maintenance pure-python packages that never uploaded a +wheel, (b) compiled packages on a brand-new Python or uncommon platform before +wheels are published, or (c) the occasional source-only package. -- **Security: building an sdist runs arbitrary code from the dependency** on the - build host. An sdist's `setup.py` / build backend executes during the build, - and — unlike a `docker build` — pymage does **not** sandbox it. A malicious or - compromised dependency can therefore achieve code execution on your build - machine. Only enable this for locks you trust, ideally in an ephemeral/CI - environment, and prefer pre-built wheels (`--find-links`, or a registry that - publishes wheels) whenever possible. -- **Architecture: compiled sdists are single-arch.** A pure-python sdist builds - to a `py3-none-any` wheel that works on any target. A *compiled* sdist can only - be built for the **host** platform, so a multi-arch build that needs to build - such a package will fail the compatibility check for the non-host arch. Use a - base/lock that provides pre-built wheels for every target arch instead. +If the lock pins a package with no compatible wheel, the build fails fast and +tells you exactly how to proceed: **pre-build the wheel out-of-band with your own +(trusted, ideally isolated) tooling and point `--find-links` at it.** -Built wheels are cached (keyed by the sdist hash and target) so they aren't -rebuilt on subsequent builds. +``` +# Build wheels once, with isolation/tooling you control: +uv pip wheel -r requirements.txt -w ./wheelhouse # or: pip wheel ... + +# Then build the image from the local wheelhouse: +pymage build --find-links ./wheelhouse -t latest +``` + +This keeps the escape hatch for the rare cases while keeping pymage's builds +hermetic, reproducible, multi-arch, and free of arbitrary build-time code. ### Base image requirements (OS / system libraries) @@ -255,7 +255,6 @@ base that advertises its version. | `--package` | Build a single uv workspace member by name (default: union of all members). | | `--cache-dir` | Cache root (default: `$PYMAGE_CACHE_DIR` or the per-user cache dir). Caches compressed layers, downloaded wheels, and base interpreter detection. | | `--no-cache` | Disable all caching (layers, downloaded wheels, interpreter detection). | -| `--build-sdists` | Allow building wheels from sdists when no compatible wheel exists. **Runs the dependency's build code on the host (no sandbox); single-arch for compiled packages.** Off by default. | | `--prefix` | install prefix / venv root (default `/app/.venv`). | | `--workdir` | image working dir and source destination (default `/app`). | | `--user` | image user, e.g. `65532`. | diff --git a/pymage/docs/real-world-comparison.md b/pymage/docs/real-world-comparison.md index c0f8b96..d20bba2 100644 --- a/pymage/docs/real-world-comparison.md +++ b/pymage/docs/real-world-comparison.md @@ -3,9 +3,9 @@ This is a hands-on study of migrating real, uv-based Python projects that ship a `Dockerfile` to `pymage`. The goals were to (a) see how the resulting images differ and (b) surface gaps/bugs real projects hit. It directly drove several -fixes — runtime-only dependency resolution, sdist→wheel building, `--extra` / -`--package` selection, and environment-marker evaluation — plus a ranked list of -the remaining migration blockers. +fixes — runtime-only dependency resolution, `--extra` / `--package` selection, +and environment-marker evaluation — plus a ranked list of the remaining +migration blockers. ## Method & caveats @@ -123,21 +123,18 @@ the worst case is one bucket rather than the whole environment.) 1. **(FIXED) Installed the whole `uv.lock`, including dev groups.** Caused 50–63% dependency bloat above. Now resolves the runtime closure. -2. **(FIXED, opt-in) sdist-only dependencies.** `imgpush` previously failed on - `timeout-decorator==0.5.0`, which publishes no wheel. With `--build-sdists` - (or `build-sdists` in `[tool.pymage]`) pymage builds a wheel from the sdist - (`pip wheel --no-deps`, host `python`/`pip` required) and feeds it into the - existing layer path; `imgpush` then builds with 52 runtime wheels including - the sdist-built `timeout-decorator`. It is **off by default and intentionally - a power-user feature** for two reasons: - - **Security:** building an sdist runs the dependency's own build code on the - host with no container isolation (pymage has no daemon to sandbox it), so a - malicious dependency could get code execution on the build machine. Without - the flag, a wheelless package fails fast with guidance to supply a wheel via - `--find-links` or opt in. - - **Single-arch:** compiled sdists can only be built for the host platform, so - enabling them doesn't make a compiled package multi-arch — pure-python - sdists build to `py3-none-any` and remain portable. +2. **sdist-only dependencies (by design — not built).** `imgpush` pins + `timeout-decorator==0.5.0`, which publishes no wheel. pymage installs + pre-built wheels only and **does not build sdists**: doing so would run the + dependency's build code on the host (RCE surface, no sandbox), be + non-reproducible and non-hermetic, need a build toolchain, and produce + host-arch-only output for compiled packages — all at odds with pymage's + guarantees. Instead the build fails fast and directs you to pre-build the + wheel out-of-band with tooling you control and pass `--find-links`: + `uv pip wheel timeout-decorator==0.5.0 -w ./wheelhouse` (or `pip wheel`), then + `pymage build --find-links ./wheelhouse …`. This is rare in practice (the + ecosystem is wheel-first); `timeout-decorator` was the only sdist across all + four projects studied. 3. **No system/OS packages (by design — documented).** pymage installs Python wheels, not apt/apk packages. `imgpush` needs `libmagickwand` (for `Wand`) and `nginx`; those must come from the **base image**. Projects with system-library @@ -169,9 +166,12 @@ the worst case is one bucket rather than the whole environment.) ## Verdict After the runtime-closure fix and this round of work, **pymage is a smaller, -faster, reproducible alternative to a uv Dockerfile** for the projects studied: -all three now build (including `imgpush`, via opt-in sdist building), with no daemon and -no build tooling in the image, and incremental rebuilds re-upload only changed -layers. The main remaining caveat is **runtime system libraries**, which must be -provided by the base image (documented), plus the multi-platform default for +faster, reproducible alternative to a uv Dockerfile** for wheel-based apps, with +no daemon and no build tooling in the image, and incremental rebuilds that +re-upload only changed layers. The FastAPI projects build directly; `imgpush` +builds once its one sdist-only dependency (`timeout-decorator`) is pre-built into +a `--find-links` wheelhouse, and still needs a base that ships ImageMagick at +runtime. The remaining caveats are **runtime system libraries** (must come from +the base image — documented), sdist-only dependencies (pre-build wheels +out-of-band; pymage won't run build code), and the multi-platform default for bases that advertise many architectures (item 5). diff --git a/pymage/internal/cli/cli.go b/pymage/internal/cli/cli.go index b41ed19..a7fa8a2 100644 --- a/pymage/internal/cli/cli.go +++ b/pymage/internal/cli/cli.go @@ -72,7 +72,6 @@ type buildFlags struct { insecure bool cacheDir string noCache bool - buildSdists bool } func buildCmd() *cobra.Command { @@ -120,7 +119,6 @@ func buildCmd() *cobra.Command { fs.BoolVar(&f.insecure, "insecure", false, "use plain HTTP for the registry") fs.StringVar(&f.cacheDir, "cache-dir", "", "cache root directory (default: per-user cache dir)") fs.BoolVar(&f.noCache, "no-cache", false, "disable all build caches (layers, downloaded wheels, interpreter detection)") - fs.BoolVar(&f.buildSdists, "build-sdists", false, "allow building wheels from source distributions when no compatible wheel exists (runs the dependency's build code on this host; single-arch for compiled packages)") return cmd } @@ -258,7 +256,7 @@ func buildOne(ctx context.Context, f *buildFlags, lk *lock.Lock, platform *v1.Pl return nil, nil, err } - wheels, err := wheelhouse.ResolveContext(ctx, reqs, f.findLinks, target, wheelCache, f.buildSdists) + wheels, err := wheelhouse.ResolveContext(ctx, reqs, f.findLinks, target, wheelCache) if err != nil { return nil, nil, err } diff --git a/pymage/internal/cli/defaults.go b/pymage/internal/cli/defaults.go index e81f42c..11f6b9c 100644 --- a/pymage/internal/cli/defaults.go +++ b/pymage/internal/cli/defaults.go @@ -75,9 +75,6 @@ func applyDefaults(cmd *cobra.Command, f *buildFlags) error { if !changed("package") && cfg.Package != "" { f.pkg = cfg.Package } - if !changed("build-sdists") && cfg.BuildSdists { - f.buildSdists = true - } if !changed("python") && cfg.Python != "" { f.pythonTag = cfg.Python } diff --git a/pymage/internal/project/project.go b/pymage/internal/project/project.go index 815ab5c..107269d 100644 --- a/pymage/internal/project/project.go +++ b/pymage/internal/project/project.go @@ -48,7 +48,6 @@ type Config struct { NoCache bool `toml:"no-cache"` Extras []string `toml:"extras"` Package string `toml:"package"` - BuildSdists bool `toml:"build-sdists"` } const defaultBase = "cgr.dev/chainguard/python:latest" diff --git a/pymage/internal/wheelhouse/sdist.go b/pymage/internal/wheelhouse/sdist.go deleted file mode 100644 index dbcc67b..0000000 --- a/pymage/internal/wheelhouse/sdist.go +++ /dev/null @@ -1,222 +0,0 @@ -package wheelhouse - -import ( - "context" - "crypto/sha256" - "encoding/hex" - "fmt" - "io" - "net/http" - "os" - "os/exec" - "path/filepath" - "strings" - "time" - - "github.com/imjasonh/terraform-playground/pymage/internal/lock" - "github.com/imjasonh/terraform-playground/pymage/internal/wheel" -) - -// buildFromSdist builds a wheel from a requirement's source distribution and -// returns it as a ResolvedWheel. The built wheel is cached (keyed by the sdist -// hash and target) so it isn't rebuilt on subsequent builds. -// -// This is only reached when the caller opted in (allowSdist), because building -// runs the dependency's own build code on the host with no isolation. Building -// runs the host's `pip wheel`, so a Python toolchain must be present. -// A pure-python sdist yields a `py3-none-any` wheel usable on any target; a -// compiled sdist is built for the host platform only, so cross-arch builds of -// such packages will fail the compatibility check with a clear error. -func buildFromSdist(ctx context.Context, req lock.Requirement, target wheel.Target, cache *wheelCache) (ResolvedWheel, error) { - if cache == nil { - return ResolvedWheel{}, fmt.Errorf("wheelhouse: building %s==%s from sdist requires a cache dir (or use --no-cache)", req.Name, req.Version) - } - key := req.Sdist.SHA256 + "-" + targetKey(target) - if p, ok := cache.getBuilt(key); ok { - sum, err := fileSHA256(p) - if err != nil { - return ResolvedWheel{}, err - } - return ResolvedWheel{Name: req.Name, Version: req.Version, Path: p, SHA256: sum}, nil - } - - work, err := os.MkdirTemp("", "pymage-sdist-") - if err != nil { - return ResolvedWheel{}, err - } - defer func() { _ = os.RemoveAll(work) }() - - sdistPath := filepath.Join(work, req.Sdist.Filename) - if err := downloadVerify(ctx, req.Sdist.URL, req.Sdist.SHA256, sdistPath); err != nil { - return ResolvedWheel{}, fmt.Errorf("wheelhouse: %s==%s sdist: %w", req.Name, req.Version, err) - } - - out := filepath.Join(work, "out") - if err := os.MkdirAll(out, 0o755); err != nil { - return ResolvedWheel{}, err - } - if err := buildWheel(ctx, sdistPath, out); err != nil { - return ResolvedWheel{}, fmt.Errorf("wheelhouse: build %s==%s from sdist: %w", req.Name, req.Version, err) - } - - built, err := selectBuiltWheel(out, req, target) - if err != nil { - return ResolvedWheel{}, err - } - - dst := cache.builtPath(key) - if err := os.MkdirAll(filepath.Dir(dst), 0o755); err != nil { - return ResolvedWheel{}, err - } - if err := os.Rename(built, dst); err != nil { - if err := copyFile(built, dst); err != nil { - return ResolvedWheel{}, err - } - } - sum, err := fileSHA256(dst) - if err != nil { - return ResolvedWheel{}, err - } - return ResolvedWheel{Name: req.Name, Version: req.Version, Path: dst, SHA256: sum}, nil -} - -func targetKey(t wheel.Target) string { - return fmt.Sprintf("%s-%s-cp%d%d", t.OS, t.Arch, t.PyMajor, t.PyMinor) -} - -func (c *wheelCache) builtPath(key string) string { - return filepath.Join(c.dir, "built", key+".whl") -} - -func (c *wheelCache) getBuilt(key string) (string, bool) { - if c == nil { - return "", false - } - if _, err := os.Stat(c.builtPath(key)); err == nil { - return c.builtPath(key), true - } - return "", false -} - -// buildWheel builds a wheel from an sdist into outDir using `pip wheel`. -func buildWheel(ctx context.Context, sdist, outDir string) error { - py := findPython() - if py == "" { - return fmt.Errorf("no python3/python on PATH to build the sdist (install Python+pip, supply a pre-built wheel via --find-links, or use a lock with wheels)") - } - cmd := exec.CommandContext(ctx, py, "-m", "pip", "wheel", "--no-deps", "--no-cache-dir", "--wheel-dir", outDir, sdist) - if out, err := cmd.CombinedOutput(); err != nil { - return fmt.Errorf("pip wheel: %w\n%s", err, tail(string(out), 20)) - } - return nil -} - -func findPython() string { - for _, name := range []string{"python3", "python"} { - if p, err := exec.LookPath(name); err == nil { - return p - } - } - return "" -} - -// selectBuiltWheel returns the built wheel in dir matching the requirement and -// compatible with the target. -func selectBuiltWheel(dir string, req lock.Requirement, target wheel.Target) (string, error) { - entries, err := os.ReadDir(dir) - if err != nil { - return "", err - } - var compatible []candidate - for _, e := range entries { - if e.IsDir() || !strings.HasSuffix(e.Name(), ".whl") { - continue - } - tags, err := wheel.ParseTags(e.Name()) - if err != nil { - continue - } - if lock.NormalizeName(tags.Name) != lock.NormalizeName(req.Name) || tags.Version != req.Version { - continue - } - if !tags.CompatibleWith(target) { - continue - } - compatible = append(compatible, candidate{path: filepath.Join(dir, e.Name()), tags: tags}) - } - if len(compatible) == 0 { - return "", fmt.Errorf("wheelhouse: built wheel for %s==%s is not compatible with %s/%s python%d.%d (a compiled package can only be built for the host platform)", - req.Name, req.Version, target.OS, target.Arch, target.PyMajor, target.PyMinor) - } - return pickBest(compatible).path, nil -} - -func downloadVerify(ctx context.Context, url, wantSum, dest string) error { - req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil) - if err != nil { - return err - } - client := &http.Client{Timeout: 5 * time.Minute} - resp, err := client.Do(req) - if err != nil { - return fmt.Errorf("download %s: %w", url, err) - } - defer func() { _ = resp.Body.Close() }() - if resp.StatusCode != http.StatusOK { - return fmt.Errorf("download %s: HTTP %s", url, resp.Status) - } - f, err := os.Create(dest) - if err != nil { - return err - } - h := sha256.New() - if _, err := io.Copy(io.MultiWriter(f, h), resp.Body); err != nil { - _ = f.Close() - return err - } - if err := f.Close(); err != nil { - return err - } - if got := hex.EncodeToString(h.Sum(nil)); !strings.EqualFold(got, wantSum) { - return fmt.Errorf("sdist hash %s != expected %s", got, wantSum) - } - return nil -} - -func fileSHA256(path string) (string, error) { - f, err := os.Open(path) - if err != nil { - return "", err - } - defer func() { _ = f.Close() }() - h := sha256.New() - if _, err := io.Copy(h, f); err != nil { - return "", err - } - return hex.EncodeToString(h.Sum(nil)), nil -} - -func copyFile(src, dst string) error { - in, err := os.Open(src) - if err != nil { - return err - } - defer func() { _ = in.Close() }() - out, err := os.Create(dst) - if err != nil { - return err - } - if _, err := io.Copy(out, in); err != nil { - _ = out.Close() - return err - } - return out.Close() -} - -func tail(s string, n int) string { - lines := strings.Split(strings.TrimRight(s, "\n"), "\n") - if len(lines) > n { - lines = lines[len(lines)-n:] - } - return strings.Join(lines, "\n") -} diff --git a/pymage/internal/wheelhouse/sdist_test.go b/pymage/internal/wheelhouse/sdist_test.go index 82c0636..2e4678d 100644 --- a/pymage/internal/wheelhouse/sdist_test.go +++ b/pymage/internal/wheelhouse/sdist_test.go @@ -2,8 +2,6 @@ package wheelhouse import ( "context" - "os" - "path/filepath" "strings" "testing" @@ -11,56 +9,10 @@ import ( "github.com/imjasonh/terraform-playground/pymage/internal/wheel" ) -func writeFiles(t *testing.T, dir string, names ...string) { - t.Helper() - for _, n := range names { - if err := os.WriteFile(filepath.Join(dir, n), []byte("x"), 0o644); err != nil { - t.Fatal(err) - } - } -} - -func TestSelectBuiltWheelPurePython(t *testing.T) { - dir := t.TempDir() - writeFiles(t, dir, "timeout_decorator-0.5.0-py3-none-any.whl", "notes.txt") - req := lock.Requirement{Name: "timeout-decorator", Version: "0.5.0"} - target := wheel.Target{OS: "linux", Arch: "arm64", PyMajor: 3, PyMinor: 12} - - got, err := selectBuiltWheel(dir, req, target) - if err != nil { - t.Fatal(err) - } - if filepath.Base(got) != "timeout_decorator-0.5.0-py3-none-any.whl" { - t.Fatalf("got %q", got) - } -} - -func TestSelectBuiltWheelIncompatibleCompiled(t *testing.T) { - dir := t.TempDir() - // A compiled wheel built for the host (linux x86_64) when targeting arm64. - writeFiles(t, dir, "ujson-5.0.0-cp312-cp312-manylinux_2_17_x86_64.whl") - req := lock.Requirement{Name: "ujson", Version: "5.0.0"} - target := wheel.Target{OS: "linux", Arch: "arm64", PyMajor: 3, PyMinor: 12} - - if _, err := selectBuiltWheel(dir, req, target); err == nil { - t.Fatal("expected incompatibility error for cross-arch compiled wheel") - } -} - -func TestSelectBuiltWheelWrongVersion(t *testing.T) { - dir := t.TempDir() - writeFiles(t, dir, "timeout_decorator-0.4.0-py3-none-any.whl") - req := lock.Requirement{Name: "timeout-decorator", Version: "0.5.0"} - target := wheel.Target{OS: "linux", Arch: "amd64", PyMajor: 3, PyMinor: 12} - - if _, err := selectBuiltWheel(dir, req, target); err == nil { - t.Fatal("expected error: built wheel version mismatch") - } -} - -func TestSdistRequiresOptIn(t *testing.T) { - // An sdist-only requirement (no wheels) must error unless opted in, rather - // than silently building (which runs the dependency's build code). +// pymage does not build sdists: a requirement available only as a source +// distribution must error with guidance to pre-build a wheel and use +// --find-links, rather than executing the dependency's build code. +func TestSdistOnlyErrors(t *testing.T) { req := lock.Requirement{ Name: "timeout-decorator", Version: "0.5.0", @@ -68,21 +20,13 @@ func TestSdistRequiresOptIn(t *testing.T) { } target := wheel.Target{OS: "linux", Arch: "amd64", PyMajor: 3, PyMinor: 12} - _, err := ResolveContext(context.Background(), []lock.Requirement{req}, nil, target, t.TempDir(), false) + _, err := ResolveContext(context.Background(), []lock.Requirement{req}, nil, target, t.TempDir()) if err == nil { - t.Fatal("expected error when sdist build is not opted in") + t.Fatal("expected an error for an sdist-only requirement") } - if !strings.Contains(err.Error(), "--build-sdists") { - t.Fatalf("error should point at the opt-in flag, got: %v", err) - } -} - -func TestBuiltPathKey(t *testing.T) { - c := &wheelCache{dir: "/tmp/c"} - target := wheel.Target{OS: "linux", Arch: "amd64", PyMajor: 3, PyMinor: 12} - key := "abc123" + "-" + targetKey(target) - want := filepath.Join("/tmp/c", "built", key+".whl") - if got := c.builtPath(key); got != want { - t.Fatalf("builtPath = %q, want %q", got, want) + for _, want := range []string{"source distribution", "--find-links", "timeout-decorator"} { + if !strings.Contains(err.Error(), want) { + t.Errorf("error %q should mention %q", err.Error(), want) + } } } diff --git a/pymage/internal/wheelhouse/wheelhouse.go b/pymage/internal/wheelhouse/wheelhouse.go index 4a64ef5..62ecb9c 100644 --- a/pymage/internal/wheelhouse/wheelhouse.go +++ b/pymage/internal/wheelhouse/wheelhouse.go @@ -47,7 +47,7 @@ type candidate struct { // per-user cache directory. Results are sorted by (normalized name, version) // for deterministic ordering. func Resolve(reqs []lock.Requirement, dirs []string, target wheel.Target) ([]ResolvedWheel, error) { - return ResolveContext(context.Background(), reqs, dirs, target, defaultWheelCacheDir(), false) + return ResolveContext(context.Background(), reqs, dirs, target, defaultWheelCacheDir()) } // defaultWheelCacheDir returns the per-user wheel download cache, or "" if the @@ -62,10 +62,8 @@ func defaultWheelCacheDir() string { } // ResolveContext is Resolve with an explicit context and on-disk wheel cache -// directory for lock-file downloads. When allowSdist is true, packages with no -// compatible wheel may be built from their source distribution (which executes -// the dependency's build code on this host); otherwise such packages error. -func ResolveContext(ctx context.Context, reqs []lock.Requirement, dirs []string, target wheel.Target, wheelCacheDir string, allowSdist bool) ([]ResolvedWheel, error) { +// directory for lock-file downloads. +func ResolveContext(ctx context.Context, reqs []lock.Requirement, dirs []string, target wheel.Target, wheelCacheDir string) ([]ResolvedWheel, error) { var index map[string][]candidate if len(dirs) > 0 { var err error @@ -82,7 +80,7 @@ func ResolveContext(ctx context.Context, reqs []lock.Requirement, dirs []string, out := make([]ResolvedWheel, 0, len(reqs)) for _, req := range reqs { - rw, err := resolveOne(ctx, req, index, target, cache, allowSdist) + rw, err := resolveOne(ctx, req, index, target, cache) if err != nil { return nil, err } @@ -99,7 +97,7 @@ func ResolveContext(ctx context.Context, reqs []lock.Requirement, dirs []string, return out, nil } -func resolveOne(ctx context.Context, req lock.Requirement, index map[string][]candidate, target wheel.Target, cache *wheelCache, allowSdist bool) (ResolvedWheel, error) { +func resolveOne(ctx context.Context, req lock.Requirement, index map[string][]candidate, target wheel.Target, cache *wheelCache) (ResolvedWheel, error) { key := lock.NormalizeName(req.Name) + "\x00" + req.Version if index != nil { if cands := index[key]; len(cands) > 0 { @@ -127,32 +125,33 @@ func resolveOne(ctx context.Context, req lock.Requirement, index map[string][]ca } } - if len(req.Wheels) == 0 && req.Sdist == nil { + if len(req.Wheels) == 0 { + if req.Sdist != nil { + // Distributed only as an sdist (no wheel anywhere). pymage installs + // pre-built wheels only; see errSdistOnly. + return ResolvedWheel{}, errSdistOnly(req, target) + } return ResolvedWheel{}, fmt.Errorf("wheelhouse: no wheel found for %s==%s (pass --find-links or use a uv.lock with wheel URLs)", req.Name, req.Version) } if cache == nil { return ResolvedWheel{}, fmt.Errorf("wheelhouse: %s==%s not in local wheelhouse and no wheel cache dir configured", req.Name, req.Version) } - // Prefer a pre-built lock wheel; fall back to building from the sdist. - if len(req.Wheels) > 0 { - rw, err := fetchRequirement(ctx, req, target, cache) - if err == nil { - return rw, nil - } - if req.Sdist == nil { - return ResolvedWheel{}, err - } - // No compatible lock wheel; consider building from the sdist below. + rw, err := fetchRequirement(ctx, req, target, cache) + if err == nil { + return rw, nil } + if req.Sdist == nil { + return ResolvedWheel{}, err + } + // The lock has wheels but none compatible with this target, and the only + // other artifact is an sdist, which pymage does not build. + return ResolvedWheel{}, errSdistOnly(req, target) +} - // Building from an sdist runs the dependency's own build code (setup.py / - // build backend) on this host with no container isolation, so it is gated - // behind explicit opt-in. - if !allowSdist { - return ResolvedWheel{}, fmt.Errorf("wheelhouse: %s==%s has no compatible pre-built wheel and would have to be built from its source distribution, which runs arbitrary build code from the dependency on this machine; pass --build-sdists (or set build-sdists in [tool.pymage]) to allow it, or supply a pre-built wheel via --find-links", req.Name, req.Version) - } - return buildFromSdist(ctx, req, target, cache) +func errSdistOnly(req lock.Requirement, target wheel.Target) error { + return fmt.Errorf("wheelhouse: %s==%s has no compatible pre-built wheel for %s/%s python%d.%d; it is only available as a source distribution (sdist), which pymage does not build. Pre-build a wheel out-of-band (e.g. `uv pip wheel %s==%s -w ./wheelhouse` or `pip wheel %s==%s -w ./wheelhouse`) and pass --find-links ./wheelhouse", + req.Name, req.Version, target.OS, target.Arch, target.PyMajor, target.PyMinor, req.Name, req.Version, req.Name, req.Version) } // pickBest chooses the most specific compatible wheel: platform-specific and