1
0
Fork 0
mirror of https://github.com/imjasonh/ocifuse synced 2026-07-07 00:12:43 +00:00
No description
Find a file
Jason Hall c59b7c7cbc Per-ref platform, @@meta dir, per-layer view, sibling expansion
Four user-visible additions, all working end-to-end against Docker Hub:

- Per-ref platform: append "~os-arch[-variant]" to a tag or digest
  segment (e.g. "ubuntu:22.04~linux-arm64") to select a specific
  manifest from a multi-arch index without touching PLATFORM.
- Synthetic "@@meta/" directory at every image root, exposing
  "@@meta/digest" (the manifest digest) and "@@meta/layers/<sha256>/"
  (raw per-layer tar contents). Inside each layer directory, a nested
  "@@meta/layer-digest" file.
- Per-layer view (under @@meta/layers/<sha256>/) shows the raw layer
  including whiteout markers. Built from a new merge.LayerTree helper
  that synthesizes parent dirs without applying whiteout filtering.
- Proactive sibling expansion: when a tag access resolves to a
  multi-arch index, the parent directory is populated with
  "<repo>:<tag>~<os>-<arch>" symlinks pointing at each platform's
  digest. Subsequent readdir shows them, so bash tab completion of
  "<tag>~<TAB>" enumerates the platforms (after at least one access of
  the bare tag).

Also fixed a pre-existing fspath bug: a segment containing "." was
classified as a registry, which broke shorthand tags with dotted
versions like "ubuntu:22.04". The new rule is: a ":" with all-digit
suffix means host:port, otherwise it's a name:tag pair regardless of
dots.
2026-04-30 20:22:42 -04:00
cmd/ocifuse Configurable TAG_TTL + slog text handler with structured logging 2026-04-30 20:04:39 -04:00
internal Per-ref platform, @@meta dir, per-layer view, sibling expansion 2026-04-30 20:22:42 -04:00
scripts Per-ref platform, @@meta dir, per-layer view, sibling expansion 2026-04-30 20:22:42 -04:00
.gitignore Initial implementation of ocifuse, a read-only FUSE driver for browsing 2026-04-30 18:43:16 -04:00
CLAUDE.md Configurable TAG_TTL + slog text handler with structured logging 2026-04-30 20:04:39 -04:00
future-work.md Per-ref platform, @@meta dir, per-layer view, sibling expansion 2026-04-30 20:22:42 -04:00
go.mod Drop vendored targz fork, use upstream 2026-04-30 18:58:37 -04:00
go.sum Drop vendored targz fork, use upstream 2026-04-30 18:58:37 -04:00
plan.md Drop vendored targz fork, use upstream 2026-04-30 18:58:37 -04:00
README.md Per-ref platform, @@meta dir, per-layer view, sibling expansion 2026-04-30 20:22:42 -04:00

ocifuse

Read-only FUSE driver that mounts remote OCI images so individual files can be browsed with normal POSIX tools, fetching only the bytes needed via HTTP Range requests on the layer blobs.

$ ocifuse /mnt/oci &
$ cat /mnt/oci/index.docker.io/library/alpine:latest/etc/os-release
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.23.4

Path layout

<mount>/<registry>/<repo...>/<ref>/<in-image-path>

<ref> is the first path segment containing : (a tag) or @ (a digest). Tags surface as symlinks to their repo@sha256:... digest sibling. A leading segment without . is treated as Docker Hub shorthand.

Multi-arch defaults to linux/amd64; override globally via PLATFORM or per-ref by appending ~os-arch[-variant], e.g. ubuntu:22.04~linux-arm64. After accessing a multi-arch tag, the parent directory shows <tag>~<os>-<arch> siblings for each platform in the index, so tab completion of <tag>~<TAB> enumerates them.

Each image root also exposes a synthetic @@meta/ directory:

  • @@meta/digest — the image manifest digest as text.
  • @@meta/layers/<sha256:...>/ — the raw tar contents of one layer (whiteouts visible). Useful for debugging overlays.
  • @@meta/layers/<sha256:...>/@@meta/layer-digest — that layer's digest as text.

Usage

ocifuse <mountpoint>

Env vars:

  • PLATFORM — default linux/amd64.
  • CACHE_DIR — default $XDG_CACHE_HOME/ocifuse.
  • CACHE_MAX_SIZE — disk LRU cap, default 1GB (K/M/G/T suffixes; 0 disables).
  • MEMORY_MAX_SIZE — in-memory chunk cache LRU cap, default 1GB.
  • TAG_TTL — kernel dentry TTL on tag symlinks, default 1m. Bounds tag-update detection latency. Standard Go duration syntax (30s, 5m, 1h).
  • DEBUG — verbose go-fuse logging plus debug-level structured logs.

Auth via go-containerregistry's authn.DefaultKeychain (~/.docker/config.json, gcloud, ECR, etc.).

How it stays fast

Four layers of cache, all optional, all preserving the partial-fetch contract:

  1. Per-layer tar+gzip index built with jonjohnsonjr/targz (gsip checkpoints + tarfs TOC), persisted to disk by layer digest. Indexing streams the layer once; afterwards individual files decompress from the nearest checkpoint.
  2. Chunk cache between gsip and HTTP. Range-fetched compressed bytes are remembered globally in an LRU; repeat reads of overlapping regions don't go back to the network.
  3. In-memory image cache keyed by manifest digest, so repeat lookups skip the manifest fetch + merged-tree rebuild.
  4. Manifest/config disk cache via a caching http.RoundTripper that intercepts only digest-keyed manifests/sha256:... and blobs/sha256:... responses (never layer blobs, never tags).

singleflight.Group deduplicates concurrent FUSE Lookups for the same ref.

The disk cache only ever holds indexes and small metadata — never layer blobs.

Building and testing

go build ./cmd/ocifuse
go test ./...

For end-to-end FUSE without Linux hardware:

scripts/smoke-linux.sh    # asserts ocifuse-read sha256 matches `crane export`
scripts/docker-run.sh     # interactive shell inside the mounted container

macFUSE on darwin is theoretically supported but unreliable in practice; the container path is far smoother.

Gotchas

  • Read-only.
  • Tab completion needs a tweak. Bash's default COMP_WORDBREAKS contains :, so alpine:latest/etc/<TAB> misfires. Fix:
    COMP_WORDBREAKS=${COMP_WORDBREAKS//:/}
    
    scripts/docker-run.sh does this inside the container.
  • Tab completion can't enumerate registries. First time you type a registry/repo/tag segment it's all manual; after that the kernel caches what you've touched.

Prior art

This project takes its central idea from jonjohnsonjr/dagdotdev, which uses the same gsip + tarfs + Range-request approach to serve OCI image content over HTTP. ocifuse rearranges those pieces behind a FUSE filesystem instead of a web UI.

See CLAUDE.md for the project contract on what optimizations are in-bounds, and future-work.md for what's been deferred.