1
0
Fork 0
mirror of https://github.com/imjasonh/git-k8s synced 2026-07-09 07:06:52 +00:00
No description
Find a file
Jason Hall 6cf845639e
Add PVC-backed Git workspace caching with fallback to in-memory (#21)
## Summary

This PR introduces a new `workspace` package that provides PVC-backed
caching for Git repositories with automatic fallback to in-memory
cloning. It enables controllers to optionally cache bare Git clones on
disk for improved performance while maintaining backward compatibility
with the existing in-memory-only behavior.

## Key Changes

- **New `workspace` package** (`pkg/workspace/workspace.go`):
- `Manager` type that handles both disk-backed and in-memory Git
repository acquisition
- `Acquire()` method that clones repos to disk (with SHA256-based path
hashing) or memory based on configuration
  - `Release()` method for reference counting on disk-backed repos
  - `GC()` method for garbage collecting stale cached repositories
- `Deepen()` method to convert shallow clones to full history when
needed
- Automatic fallback to in-memory if disk clone fails or caching is
disabled
- Context injection helpers (`WithManager`, `GetManager`) for dependency
injection

- **API extensions** (`pkg/apis/git/v1alpha1/types.go`):
  - New `CacheConfig` struct with `Enabled` boolean field
- Added `Cache` field to `GitRepositorySpec` for opt-in caching per
repository
  - Added `CacheStatus` struct to track cache health (clone/fetch times)
  - Updated generated deepcopy code

- **Reconciler updates** (sync, resolver, push controllers):
  - Integrated `workspace.Manager` into all three reconcilers
- Updated to use workspace manager for Git operations instead of direct
in-memory clones
  - Added auth resolution from Kubernetes secrets
  - Pass `cacheEnabled` flag based on repository configuration

- **Metrics enhancements** (`pkg/metrics/metrics.go`):
- `WorkspaceAcquireDuration` histogram tracking acquire time by mode
(disk/memory)
- `WorkspaceCacheHit` and `WorkspaceCacheMiss` counters for cache
effectiveness monitoring

- **Deployment configuration**:
- Added commented-out examples for enabling PVC-backed caching via
`GIT_CACHE_DIR` environment variable and volume mounts in all three
controller deployments
  - Updated CRD schema to include cache configuration

- **Comprehensive test coverage** (`pkg/workspace/workspace_test.go`):
  - 426 lines of unit tests covering all Manager functionality
- Tests for cache hits/misses, shallow clones, GC, context injection,
and error handling

- **E2E tests** (`test/e2e/workspace_test.go`):
  - Tests verifying push transactions work with and without caching
  - Tests for cache serialization and multiple sequential operations

## Implementation Details

- **Backward compatible**: Empty `basePath` or `cacheEnabled=false`
triggers in-memory fallback
- **Per-repo opt-in**: Caching is disabled by default; must be
explicitly enabled via `Cache.Enabled` in GitRepository spec
- **Graceful degradation**: If disk clone fails, automatically falls
back to in-memory without failing the operation
- **Reference counting**: Disk-backed workspaces use mutex-protected
reference counting to track active uses
- **Deterministic paths**: Repository cache paths are derived from
SHA256 hash of the URL for consistency
- **Shallow clone support**: Initial clones can use `--depth=1` when
`shallow=true`, with `Deepen()` to fetch full history later

https://claude.ai/code/session_01NULGxaCLPMT1yDVEAsffc5
2026-03-17 11:30:56 -04:00
.github Limit compile-workflows trigger to pull_request only 2026-02-28 17:46:35 +00:00
cmd Implement PVC-backed Git workspace cache (design-pvc-workspace.md) 2026-03-17 15:20:57 +00:00
config Implement PVC-backed Git workspace cache (design-pvc-workspace.md) 2026-03-17 15:20:57 +00:00
docs Add design doc for PVC-backed Git workspace cache 2026-03-17 14:46:01 +00:00
hack Implement Git-as-K8s control plane with Knative controllers 2026-02-27 23:58:44 +00:00
pkg Implement PVC-backed Git workspace cache (design-pvc-workspace.md) 2026-03-17 15:20:57 +00:00
test/e2e Implement PVC-backed Git workspace cache (design-pvc-workspace.md) 2026-03-17 15:20:57 +00:00
.gitattributes gh aw compile 2026-02-28 10:17:50 -05:00
.gitignore Add .gitignore for Go binaries and IDE files 2026-02-28 00:14:29 +00:00
.ko.yaml Implement Git-as-K8s control plane with Knative controllers 2026-02-27 23:58:44 +00:00
CLAUDE.md Add repo-watcher-controller for polling remote Git refs 2026-02-28 14:09:58 +00:00
COVERAGE_REPORT.md Improve test coverage from 10.3% to 42.7% with diffcover analysis 2026-03-04 19:49:59 +00:00
go.mod Add Prometheus metrics, health probes, Git timeouts, and unit tests 2026-03-17 14:32:02 +00:00
go.sum go get -u ./... && go mod tidy: update dependencies 2026-02-28 04:14:23 +00:00
README.md Add README with installation, auth, and architecture docs 2026-02-28 14:45:13 +00:00

git-k8s

Kubernetes-native controllers for declarative Git repository management. Define Git repositories, track branches, execute atomic pushes, and keep repositories in sync — all through Custom Resources.

Architecture

git-k8s runs four controllers that communicate through Kubernetes resources rather than direct APIs. Each controller is a separate deployment that watches specific CRDs and takes action when their state changes.

graph TB
    subgraph "Custom Resources"
        GR[GitRepository]
        GB[GitBranch]
        GPT[GitPushTransaction]
        GRS[GitRepoSync]
    end

    subgraph "Controllers"
        RW[Repo Watcher]
        SC[Sync Controller]
        RC[Resolver Controller]
        PC[Push Controller]
    end

    subgraph "External"
        Remote["Git Remote(s)"]
        Secret[K8s Secret]
    end

    RW -- "watches" --> GR
    RW -- "creates / updates / deletes" --> GB
    RW -- "polls refs via ls-remote" --> Remote

    SC -- "watches" --> GRS
    SC -- "reads" --> GB
    SC -- "creates" --> GPT

    RC -- "watches (Conflicted)" --> GRS
    RC -- "creates" --> GPT

    PC -- "watches" --> GPT
    PC -- "pushes commits" --> Remote
    PC -- "updates" --> GB

    PC -. "reads credentials" .-> Secret
    RW -. "reads credentials" .-> Secret

Controller responsibilities

Controller Watches Creates / Mutates Purpose
Repo Watcher GitRepository GitBranch Polls remotes on a configurable interval, mirrors branch state into GitBranch resources
Sync GitRepoSync, GitBranch GitPushTransaction Compares HEAD commits between two repos, calculates merge bases, creates push transactions to keep branches aligned
Resolver GitRepoSync (Conflicted) GitPushTransaction Performs automated 3-way merge, falls back to RequiresManualIntervention on file-level conflicts
Push GitPushTransaction GitBranch Clones into memory, executes atomic pushes with optional compare-and-swap, updates branch status

Resource lifecycle

stateDiagram-v2
    state "GitPushTransaction" as push {
        [*] --> Pending
        Pending --> InProgress
        InProgress --> Succeeded
        InProgress --> Failed
    }

    state "GitRepoSync" as sync {
        [*] --> InSync
        InSync --> Syncing : commits diverge
        Syncing --> InSync : push succeeds
        Syncing --> Conflicted : both sides changed
        Conflicted --> InSync : merge succeeds
        Conflicted --> RequiresManualIntervention : merge fails
    }

End-to-end flow

sequenceDiagram
    participant Remote as Git Remote
    participant RW as Repo Watcher
    participant GB as GitBranch
    participant Sync as Sync Controller
    participant GPT as GitPushTransaction
    participant Push as Push Controller

    RW->>Remote: git ls-remote
    Remote-->>RW: refs + SHAs
    RW->>GB: create / update branch

    GB-->>Sync: branch change triggers reconcile
    Sync->>Sync: compare commits, find merge base
    Sync->>GPT: create push transaction

    GPT-->>Push: new transaction triggers reconcile
    Push->>Remote: git push (in-memory clone)
    Push->>GB: update headCommit

Installation

Prerequisites

  • A Kubernetes cluster (KinD works for local development)
  • ko v0.15+
  • kubectl
  • Go 1.24.7+ (for building from source)

Deploy with ko

Set KO_DOCKER_REPO to a registry your cluster can pull from, then apply everything in order:

export KO_DOCKER_REPO=<your-registry>  # e.g. ghcr.io/you, kind.local

# Create namespace
kubectl create namespace git-system

# Install CRDs
kubectl apply -f config/crds/

# Install RBAC (ServiceAccount, ClusterRole, ClusterRoleBinding)
kubectl apply -f config/rbac/

# Build images and deploy all four controllers
ko apply -f config/deployments/

For local development with KinD, use the built-in local registry:

export KO_DOCKER_REPO=kind.local
ko apply -f config/deployments/

Verify

kubectl -n git-system get pods

You should see four controller pods running:

push-controller-...       1/1   Running
sync-controller-...       1/1   Running
resolver-controller-...   1/1   Running
repo-watcher-...          1/1   Running

Authentication

Controllers authenticate to Git remotes using Kubernetes Secrets referenced from GitRepository resources. The Secret must exist in the same namespace as the GitRepository.

Create a Secret

For HTTPS repositories using a username and personal access token:

kubectl create secret generic my-git-creds \
  --namespace=default \
  --from-literal=username=<git-username> \
  --from-literal=password=<personal-access-token>

Reference it from a GitRepository

apiVersion: git-k8s.imjasonh.com/v1alpha1
kind: GitRepository
metadata:
  name: my-repo
  namespace: default
spec:
  url: https://github.com/example/repo.git
  defaultBranch: main
  pollInterval: 30s
  auth:
    secretRef:
      name: my-git-creds

The auth field is optional — omit it for public repositories. When present, the Push Controller and Repo Watcher Controller both resolve the Secret to authenticate clone, push, and ls-remote operations.

RBAC

The controllers' ClusterRole already includes read access to Secrets:

- apiGroups: [""]
  resources: [secrets]
  verbs: [get, list, watch]

No additional RBAC configuration is needed.

Usage

Track a repository

apiVersion: git-k8s.imjasonh.com/v1alpha1
kind: GitRepository
metadata:
  name: upstream
spec:
  url: https://github.com/example/repo.git
  defaultBranch: main
  pollInterval: 1m

The Repo Watcher will poll the remote and create a GitBranch resource for each branch it discovers.

Push to a repository

apiVersion: git-k8s.imjasonh.com/v1alpha1
kind: GitPushTransaction
metadata:
  name: push-feature
spec:
  repositoryRef: upstream
  atomic: true
  refSpecs:
    - source: abc123def
      destination: refs/heads/main
      expectedOldCommit: 789fed456   # optional CAS check

Sync two repositories

apiVersion: git-k8s.imjasonh.com/v1alpha1
kind: GitRepoSync
metadata:
  name: keep-in-sync
spec:
  repoA:
    name: upstream
  repoB:
    name: fork
  branchName: main

The Sync Controller will detect when the branch diverges between the two repos and create GitPushTransaction resources to bring them back in sync. If both sides have diverged, the Resolver Controller attempts an automated 3-way merge.

Design decisions

  • Stateless controllers — All Git operations use in-memory storage (go-git with memory.NewStorage()). No persistent volumes required.
  • Separate binaries — Each controller is its own deployment. They communicate exclusively through Kubernetes resources.
  • Atomic pushes — Push transactions support compare-and-swap via expectedOldCommit to prevent race conditions.
  • No finalizers — Resource relationships use labels and owner references only.

License

Apache-2.0