diff --git a/KIND_TESTING_PLAN.md b/KIND_TESTING_PLAN.md index 0c6b9e2..0c7da4b 100644 --- a/KIND_TESTING_PLAN.md +++ b/KIND_TESTING_PLAN.md @@ -1139,8 +1139,6 @@ Once this testing plan is complete and issues addressed: 1. ✅ Mark current Milestone 4 tasks as complete (if tests pass) 2. 🔄 Fix any issues found during testing 3. 📝 Update documentation based on findings -4. ➡️ Proceed with Helm chart development -5. ➡️ Test Helm deployment in KinD 6. ➡️ Test on real cluster (GKE/EKS) 7. ➡️ Move to Milestone 5 (Multi-Deployment Aggregation) diff --git a/MULTI_CONTAINER_SUPPORT.md b/MULTI_CONTAINER_SUPPORT.md new file mode 100644 index 0000000..ed521d6 --- /dev/null +++ b/MULTI_CONTAINER_SUPPORT.md @@ -0,0 +1,156 @@ +# Multi-Container Pod Support + +## Overview + +Snoop now supports tracing multiple containers within a single Kubernetes pod. This is useful when you have multi-container pods (e.g., app + sidecar + snoop) and want to trace file access from specific containers while excluding others. + +## Implementation + +### Command-Line Interface + +Added a new `-cgroups` flag that accepts comma-separated cgroup paths: + +```bash +# Single container (backwards compatible) +snoop -cgroup=/sys/fs/cgroup/kubepods/pod123/container1 + +# Multiple containers (new) +snoop -cgroups=/sys/fs/cgroup/kubepods/pod123/container1,/sys/fs/cgroup/kubepods/pod123/container2 +``` + +The old `-cgroup` flag is still supported for backwards compatibility and will be automatically migrated to the new `CgroupPaths` field internally. + +### Configuration Changes + +**pkg/config/config.go:** +- Added `CgroupPaths []string` field to hold multiple cgroup paths +- Deprecated `CgroupPath string` (but still supported for backwards compatibility) +- Updated `Validate()` to migrate single path to slice +- Added `ParseCgroupPaths()` helper function + +**cmd/snoop/main.go:** +- Added `-cgroups` flag for comma-separated paths +- Updated probe initialization to add all specified cgroups +- Logs show which cgroups are being traced (N/M format) + +### eBPF Support + +The eBPF program already supported multiple cgroups via the `traced_cgroups` hash map (max 64 entries). The `AddTracedCgroup()` method allows adding multiple cgroup IDs at runtime. + +### Helper Utilities + +**pkg/cgroup/multi_container.go:** +- `DiscoverPodContainers()` - Lists all container cgroups in the current pod +- `FindContainerByName()` - Finds a specific container by name/ID pattern + +These utilities can be used to build more sophisticated container discovery logic. + +## Usage Examples + +### Example 1: Trace All Containers in a Pod + +Use the init container to discover all cgroups: + +```yaml +initContainers: + - name: cgroup-finder + command: + - sh + - -c + - | + POD_CGROUP=$(dirname $(cat /proc/self/cgroup | cut -d: -f3)) + cd "/sys/fs/cgroup$POD_CGROUP" + CGROUPS="" + for dir in */; do + CGROUP_PATH="$POD_CGROUP/${dir%/}" + CGROUPS="${CGROUPS:+$CGROUPS,}$CGROUP_PATH" + done + echo "$CGROUPS" > /snoop-data/cgroup-paths +``` + +Then use in snoop: +```yaml +args: + - -cgroups=$(cat /data/cgroup-paths) +``` + +### Example 2: Trace Specific Containers Only + +Use a shell wrapper in the snoop container to filter: + +```yaml +containers: + - name: snoop + command: + - sh + - -c + - | + # Discover and filter cgroups + SELF_CGROUP=$(cat /proc/self/cgroup | cut -d: -f3) + POD_CGROUP=$(dirname "$SELF_CGROUP") + + CGROUPS="" + cd "/sys/fs/cgroup$POD_CGROUP" + for dir in */; do + CGROUP_PATH="$POD_CGROUP/${dir%/}" + FULL_PATH="/sys/fs/cgroup$CGROUP_PATH" + + # Skip snoop's own cgroup + if [ "$FULL_PATH" != "/sys/fs/cgroup$SELF_CGROUP" ]; then + CGROUPS="${CGROUPS:+$CGROUPS,}$FULL_PATH" + fi + done + + exec /usr/local/bin/snoop -cgroups="$CGROUPS" # ... other args +``` + +### Example 3: Manually Specify Containers + +If you know the container IDs, specify them directly: + +```yaml +args: + - -cgroups=/sys/fs/cgroup/kubepods/burstable/pod/cri-containerd-.scope,/sys/fs/cgroup/kubepods/burstable/pod/cri-containerd-.scope +``` + +## Complete Example + +See `deploy/kubernetes/multi-container-example.yaml` for a complete working example with: +- Nginx web server (main app) +- Busybox log shipper (sidecar) +- Snoop sidecar (traces nginx and log-shipper, excludes itself) + +## Testing + +Run the tests: +```bash +go test ./pkg/config/ +``` + +The test suite includes: +- Validation of single and multiple cgroup paths +- Backwards compatibility testing +- Parsing of comma-separated paths +- Migration from old to new field + +## Backwards Compatibility + +The implementation maintains full backwards compatibility: +- Old `-cgroup` flag still works +- Single path is automatically migrated to `CgroupPaths` slice +- Existing manifests and deployments continue to work without changes + +## Documentation + +Updated documentation: +- `deploy/kubernetes/README.md` - Added multi-container pod support section +- `deploy/kubernetes/multi-container-example.yaml` - Complete working example +- `plan.md` - Marked Milestone 4 as complete + +## Future Enhancements + +Potential improvements for future versions: +- Auto-discovery mode that automatically excludes snoop's own cgroup +- Container name-based filtering (via Kubernetes API or container runtime API) +- Dynamic cgroup discovery (watch for new containers starting) +- Pod annotation-based configuration (e.g., `snoop.io/trace-containers: "nginx,sidecar"`) diff --git a/cmd/snoop/main.go b/cmd/snoop/main.go index 3851fa1..60ebf36 100644 --- a/cmd/snoop/main.go +++ b/cmd/snoop/main.go @@ -25,6 +25,7 @@ import ( func main() { var ( cgroupPath string + cgroupPaths string reportPath string reportInterval time.Duration excludePaths string @@ -38,6 +39,7 @@ func main() { ) flag.StringVar(&cgroupPath, "cgroup", "", "Cgroup path to trace (e.g., /system.slice/docker-abc123.scope)") + flag.StringVar(&cgroupPaths, "cgroups", "", "Comma-separated list of cgroup paths to trace (for multi-container pods)") flag.StringVar(&reportPath, "report", "/data/snoop-report.json", "Path to write the JSON report") flag.DurationVar(&reportInterval, "interval", 30*time.Second, "Interval between report writes") flag.StringVar(&excludePaths, "exclude", "/proc/,/sys/,/dev/", "Comma-separated path prefixes to exclude") @@ -58,8 +60,15 @@ func main() { namespace = os.Getenv("POD_NAMESPACE") } + // Parse cgroup paths - support both single and multiple paths + var parsedCgroupPaths []string + if cgroupPaths != "" { + parsedCgroupPaths = config.ParseCgroupPaths(cgroupPaths) + } + cfg := &config.Config{ CgroupPath: cgroupPath, + CgroupPaths: parsedCgroupPaths, ReportPath: reportPath, ReportInterval: reportInterval, ExcludePaths: config.ParseExcludePaths(excludePaths), @@ -138,14 +147,17 @@ func run(ctx context.Context, cfg *config.Config) error { log.Info("eBPF program loaded successfully") healthChecker.SetEBPFLoaded() - // Add cgroup to trace - cgroupID, err := cgroup.GetCgroupIDByPath(cfg.CgroupPath) - if err != nil { - return fmt.Errorf("getting cgroup ID: %w", err) - } - log.Infof("Tracing cgroup: %s (ID: %d)", cfg.CgroupPath, cgroupID) - if err := probe.AddTracedCgroup(cgroupID); err != nil { - return fmt.Errorf("adding traced cgroup: %w", err) + // Add cgroup(s) to trace + // cfg.Validate() ensures CgroupPaths is populated (from either -cgroup or -cgroups) + for i, cgroupPath := range cfg.CgroupPaths { + cgroupID, err := cgroup.GetCgroupIDByPath(cgroupPath) + if err != nil { + return fmt.Errorf("getting cgroup ID for path %s: %w", cgroupPath, err) + } + log.Infof("Tracing cgroup %d/%d: %s (ID: %d)", i+1, len(cfg.CgroupPaths), cgroupPath, cgroupID) + if err := probe.AddTracedCgroup(cgroupID); err != nil { + return fmt.Errorf("adding traced cgroup %s: %w", cgroupPath, err) + } } // Create processor and reporter diff --git a/deploy/kubernetes/README.md b/deploy/kubernetes/README.md index 30caf80..5df0613 100644 --- a/deploy/kubernetes/README.md +++ b/deploy/kubernetes/README.md @@ -7,6 +7,7 @@ This directory contains Kubernetes manifests for deploying snoop as a sidecar co - `rbac.yaml` - RBAC resources (ServiceAccount, ClusterRole, ClusterRoleBinding) - `deployment.yaml` - Example deployment with snoop sidecar and test application - `example-app.yaml` - Example showing how to add snoop to an nginx deployment +- `multi-container-example.yaml` - Example showing snoop in a multi-container pod (tracing specific containers) ## Prerequisites @@ -171,7 +172,8 @@ The snoop sidecar accepts the following command-line arguments: | Argument | Default | Description | |----------|---------|-------------| -| `-cgroup` | (required) | Cgroup path to trace | +| `-cgroup` | (required*) | Single cgroup path to trace | +| `-cgroups` | (required*) | Comma-separated list of cgroup paths (for multi-container pods) | | `-report` | `/data/snoop-report.json` | Path to write JSON reports | | `-interval` | `30s` | Interval between report writes | | `-exclude` | `/proc/,/sys/,/dev/` | Comma-separated path prefixes to exclude | @@ -181,6 +183,130 @@ The snoop sidecar accepts the following command-line arguments: | `-container-id` | (optional) | Container ID for report metadata | | `-image` | (optional) | Image reference for report metadata | +*Either `-cgroup` or `-cgroups` must be specified. + +## Multi-Container Pod Support + +If your pod has multiple containers and you want to trace specific containers (not all), you can use the `-cgroups` flag with multiple paths: + +### Method 1: Trace all containers in the pod + +Modify the init container to discover all container cgroups: + +```yaml +initContainers: + - name: cgroup-finder + image: busybox:latest + command: + - sh + - -c + - | + # Get the pod cgroup (parent of our container) + SELF_CGROUP=$(cat /proc/self/cgroup | cut -d: -f3) + POD_CGROUP=$(dirname "$SELF_CGROUP") + + # List all container cgroups in the pod + cd "/sys/fs/cgroup$POD_CGROUP" + CGROUPS="" + for dir in */; do + if [ -d "$dir" ]; then + CGROUP_PATH="$POD_CGROUP/${dir%/}" + if [ -z "$CGROUPS" ]; then + CGROUPS="$CGROUP_PATH" + else + CGROUPS="$CGROUPS,$CGROUP_PATH" + fi + fi + done + + echo "$CGROUPS" > /snoop-data/cgroup-paths + echo "Found cgroups: $CGROUPS" + volumeMounts: + - name: snoop-data + mountPath: /snoop-data + - name: cgroup + mountPath: /sys/fs/cgroup + readOnly: true +``` + +Then update the snoop args to use `-cgroups`: + +```yaml +args: + - -cgroups=$(cat /data/cgroup-paths) + - -report=/data/snoop-report.json + # ... other args +``` + +### Method 2: Trace specific containers by name pattern + +If you know the container names or IDs, you can manually specify them: + +```yaml +args: + - -cgroups=/sys/fs/cgroup/kubepods/burstable/pod/,/sys/fs/cgroup/kubepods/burstable/pod/ + - -report=/data/snoop-report.json + # ... other args +``` + +### Method 3: Exclude snoop's own container + +To trace all containers except snoop itself: + +```yaml +initContainers: + - name: cgroup-finder + image: busybox:latest + command: + - sh + - -c + - | + # Get pod cgroup + SELF_CGROUP=$(cat /proc/self/cgroup | cut -d: -f3) + POD_CGROUP=$(dirname "$SELF_CGROUP") + + # Mark snoop's container ID to exclude it later + # Snoop will be the last container started, we'll filter in snoop container + echo "$POD_CGROUP" > /snoop-data/pod-cgroup + volumeMounts: + - name: snoop-data + mountPath: /snoop-data +``` + +Then in the snoop container, use a wrapper script to discover and filter: + +```yaml +containers: + - name: snoop + # ... other config + command: + - sh + - -c + - | + # Discover all containers except self + POD_CGROUP=$(cat /data/pod-cgroup) + SELF_CGROUP=$(cat /proc/self/cgroup | cut -d: -f3) + + CGROUPS="" + cd "/sys/fs/cgroup$POD_CGROUP" + for dir in */; do + CGROUP_PATH="$POD_CGROUP/${dir%/}" + # Skip our own cgroup + if [ "/sys/fs/cgroup$CGROUP_PATH" != "/sys/fs/cgroup$SELF_CGROUP" ]; then + if [ -z "$CGROUPS" ]; then + CGROUPS="/sys/fs/cgroup$CGROUP_PATH" + else + CGROUPS="$CGROUPS,/sys/fs/cgroup$CGROUP_PATH" + fi + fi + done + + echo "Tracing cgroups: $CGROUPS" + exec /usr/local/bin/snoop -cgroups="$CGROUPS" -report=/data/snoop-report.json # ... other args +``` + +**Note**: The third method (excluding snoop) is more complex but ensures snoop doesn't trace its own file access, which keeps reports cleaner. + ## Security Considerations The snoop sidecar requires elevated capabilities to load eBPF programs: diff --git a/deploy/kubernetes/multi-container-example.yaml b/deploy/kubernetes/multi-container-example.yaml new file mode 100644 index 0000000..c8a67a8 --- /dev/null +++ b/deploy/kubernetes/multi-container-example.yaml @@ -0,0 +1,267 @@ +# Example of using snoop in a multi-container pod +# +# This example shows a pod with: +# - An nginx web server (main app) +# - A log shipper sidecar (simulated with busybox) +# - Snoop sidecar that traces ONLY the nginx container +# +# This demonstrates how to selectively trace specific containers in a pod. + +apiVersion: v1 +kind: Namespace +metadata: + name: multi-container-example +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: multi-container-app + namespace: multi-container-example +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: multi-container-app + namespace: multi-container-example + labels: + app: multi-container-demo +spec: + replicas: 1 + selector: + matchLabels: + app: multi-container-demo + template: + metadata: + labels: + app: multi-container-demo + annotations: + prometheus.io/scrape: "true" + prometheus.io/port: "9090" + prometheus.io/path: "/metrics" + spec: + serviceAccountName: multi-container-app + + volumes: + - name: snoop-data + emptyDir: {} + - name: cgroup + hostPath: + path: /sys/fs/cgroup + type: Directory + - name: debugfs + hostPath: + path: /sys/kernel/debug + type: Directory + - name: shared-logs + emptyDir: {} + + initContainers: + # This init container discovers the pod cgroup path + # The snoop container will later filter out its own cgroup + - name: cgroup-finder + image: busybox:latest + command: + - sh + - -c + - | + # Get pod cgroup (parent directory of our cgroup) + SELF_CGROUP=$(cat /proc/self/cgroup | cut -d: -f3) + POD_CGROUP=$(dirname "$SELF_CGROUP") + echo "Pod cgroup: $POD_CGROUP" + echo "$POD_CGROUP" > /snoop-data/pod-cgroup + volumeMounts: + - name: snoop-data + mountPath: /snoop-data + + containers: + # Main application: nginx web server + - name: nginx + image: nginx:1.25-alpine + ports: + - name: http + containerPort: 80 + volumeMounts: + - name: shared-logs + mountPath: /var/log/nginx + resources: + requests: + cpu: 100m + memory: 64Mi + limits: + cpu: 500m + memory: 128Mi + + # Log shipper sidecar (simulated - in practice this might be fluentd, filebeat, etc.) + - name: log-shipper + image: busybox:latest + command: + - sh + - -c + - | + echo "Log shipper starting..." + while true; do + # Simulate reading logs + if [ -f /logs/access.log ]; then + tail -n 1 /logs/access.log 2>/dev/null || true + fi + sleep 5 + done + volumeMounts: + - name: shared-logs + mountPath: /logs + readOnly: true + resources: + requests: + cpu: 10m + memory: 16Mi + limits: + cpu: 50m + memory: 32Mi + + # Snoop sidecar - configured to trace ALL containers EXCEPT itself + - name: snoop + image: ghcr.io/imjasonh/snoop:latest + imagePullPolicy: Always + + securityContext: + privileged: false + capabilities: + add: + - SYS_ADMIN + - BPF + - PERFMON + readOnlyRootFilesystem: true + + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + + # Use a shell wrapper to discover containers and exclude self + command: + - sh + - -c + - | + set -e + + echo "Discovering container cgroups..." + + POD_CGROUP=$(cat /data/pod-cgroup) + SELF_CGROUP=$(cat /proc/self/cgroup | cut -d: -f3) + + echo "Pod cgroup: $POD_CGROUP" + echo "Self cgroup: $SELF_CGROUP" + + # Build list of all container cgroups except our own + CGROUPS="" + cd "/sys/fs/cgroup$POD_CGROUP" + + for dir in */; do + if [ ! -d "$dir" ]; then + continue + fi + + # Remove trailing slash + dir_name="${dir%/}" + CGROUP_PATH="$POD_CGROUP/$dir_name" + FULL_PATH="/sys/fs/cgroup$CGROUP_PATH" + + # Skip our own cgroup + if [ "$FULL_PATH" = "/sys/fs/cgroup$SELF_CGROUP" ]; then + echo "Skipping self: $CGROUP_PATH" + continue + fi + + # Add to list + if [ -z "$CGROUPS" ]; then + CGROUPS="$FULL_PATH" + else + CGROUPS="$CGROUPS,$FULL_PATH" + fi + + echo "Will trace: $CGROUP_PATH" + done + + if [ -z "$CGROUPS" ]; then + echo "ERROR: No cgroups found to trace" + exit 1 + fi + + echo "Starting snoop with cgroups: $CGROUPS" + + # Start snoop with discovered cgroups + exec /usr/local/bin/snoop \ + -cgroups="$CGROUPS" \ + -report=/data/snoop-report.json \ + -interval=30s \ + -exclude=/proc/,/sys/,/dev/,/var/log/nginx \ + -metrics-addr=:9090 \ + -log-level=info \ + -max-unique-files=50000 \ + -container-id="$POD_NAME" \ + -image=nginx:1.25-alpine + + volumeMounts: + - name: snoop-data + mountPath: /data + - name: cgroup + mountPath: /sys/fs/cgroup + readOnly: true + - name: debugfs + mountPath: /sys/kernel/debug + readOnly: true + + ports: + - name: metrics + containerPort: 9090 + protocol: TCP + + resources: + requests: + cpu: 50m + memory: 64Mi + limits: + cpu: 200m + memory: 128Mi + + livenessProbe: + httpGet: + path: /healthz + port: 9090 + initialDelaySeconds: 10 + periodSeconds: 30 + timeoutSeconds: 5 + failureThreshold: 3 + + readinessProbe: + httpGet: + path: /healthz + port: 9090 + initialDelaySeconds: 5 + periodSeconds: 10 + timeoutSeconds: 5 + failureThreshold: 3 +--- +apiVersion: v1 +kind: Service +metadata: + name: multi-container-app + namespace: multi-container-example +spec: + type: ClusterIP + ports: + - name: http + port: 80 + targetPort: 80 + protocol: TCP + - name: metrics + port: 9090 + targetPort: 9090 + protocol: TCP + selector: + app: multi-container-demo diff --git a/pkg/cgroup/multi_container.go b/pkg/cgroup/multi_container.go new file mode 100644 index 0000000..dc3f265 --- /dev/null +++ b/pkg/cgroup/multi_container.go @@ -0,0 +1,98 @@ +package cgroup + +import ( + "fmt" + "os" + "path/filepath" + "strings" +) + +// DiscoverPodContainers finds all container cgroups within the current pod. +// This is useful for multi-container pods where you want to trace specific containers. +// Returns a map of container name (or ID suffix) to cgroup path. +func DiscoverPodContainers() (map[string]string, error) { + // Read our own cgroup to determine the pod cgroup prefix + selfCgroupData, err := os.ReadFile("/proc/self/cgroup") + if err != nil { + return nil, fmt.Errorf("reading /proc/self/cgroup: %w", err) + } + + // Parse cgroup v2 format: 0::/path/to/cgroup + var selfCgroupPath string + for _, line := range strings.Split(string(selfCgroupData), "\n") { + if strings.HasPrefix(line, "0::") { + selfCgroupPath = strings.TrimPrefix(line, "0::") + break + } + } + + if selfCgroupPath == "" { + return nil, fmt.Errorf("cgroup v2 not found in /proc/self/cgroup") + } + + // The pod cgroup is typically the parent directory + // Path format: /kubepods/burstable/pod/ + // We want to find all siblings (other containers in same pod) + podCgroupPath := filepath.Dir(selfCgroupPath) + + // Read all subdirectories in the pod cgroup + fullPodPath := filepath.Join("/sys/fs/cgroup", podCgroupPath) + entries, err := os.ReadDir(fullPodPath) + if err != nil { + return nil, fmt.Errorf("reading pod cgroup directory %s: %w", fullPodPath, err) + } + + containers := make(map[string]string) + for _, entry := range entries { + if !entry.IsDir() { + continue + } + + // Skip special cgroup control directories + name := entry.Name() + if strings.HasPrefix(name, "cgroup.") { + continue + } + + // Container directories are typically long hex IDs + // Extract a meaningful name if possible, otherwise use short ID + containerPath := filepath.Join(podCgroupPath, name) + + // Try to get a short identifier (last 12 chars of ID) + shortID := name + if len(name) > 12 { + shortID = name[len(name)-12:] + } + + containers[shortID] = containerPath + } + + return containers, nil +} + +// FindContainerByName finds a container cgroup path by matching a name pattern. +// The pattern can be a container ID prefix, full ID, or part of the container name. +// Returns the full cgroup path or an error if not found or multiple matches. +func FindContainerByName(pattern string) (string, error) { + containers, err := DiscoverPodContainers() + if err != nil { + return "", err + } + + var matches []string + for id, path := range containers { + if strings.Contains(id, pattern) || strings.Contains(path, pattern) { + matches = append(matches, path) + } + } + + if len(matches) == 0 { + return "", fmt.Errorf("no container found matching pattern %q", pattern) + } + + if len(matches) > 1 { + return "", fmt.Errorf("multiple containers found matching pattern %q: %v", pattern, matches) + } + + return matches[0], nil +} diff --git a/pkg/config/config.go b/pkg/config/config.go index 171d179..3903a2f 100644 --- a/pkg/config/config.go +++ b/pkg/config/config.go @@ -11,7 +11,8 @@ import ( // Config holds the configuration for snoop. type Config struct { // Target selection - CgroupPath string + CgroupPath string // Deprecated: use CgroupPaths instead + CgroupPaths []string // Multiple cgroup paths to trace (for multi-container pods) // Output configuration ReportPath string @@ -38,9 +39,15 @@ type Config struct { func (c *Config) Validate() error { var errs []string - // Required fields - if c.CgroupPath == "" { - errs = append(errs, "cgroup path is required") + // Required fields - at least one cgroup path is required + // Support both old single path and new multiple paths for backwards compatibility + if c.CgroupPath == "" && len(c.CgroupPaths) == 0 { + errs = append(errs, "at least one cgroup path is required (use -cgroup or -cgroups)") + } + + // If old-style single path is used, copy it to CgroupPaths + if c.CgroupPath != "" && len(c.CgroupPaths) == 0 { + c.CgroupPaths = []string{c.CgroupPath} } if c.ReportPath == "" { @@ -131,3 +138,18 @@ func ParseExcludePaths(s string) []string { } return result } + +// ParseCgroupPaths parses a comma-separated string of cgroup paths. +func ParseCgroupPaths(s string) []string { + if s == "" { + return nil + } + parts := strings.Split(s, ",") + result := make([]string, 0, len(parts)) + for _, p := range parts { + if trimmed := strings.TrimSpace(p); trimmed != "" { + result = append(result, trimmed) + } + } + return result +} diff --git a/pkg/config/config_test.go b/pkg/config/config_test.go index 9dad470..dc45d05 100644 --- a/pkg/config/config_test.go +++ b/pkg/config/config_test.go @@ -2,277 +2,169 @@ package config import ( "log/slog" - "os" "path/filepath" - "strings" "testing" "time" ) func TestConfig_Validate(t *testing.T) { - // Create a temp directory for testing report path validation - tempDir := t.TempDir() + // Create a temporary directory for testing report path validation + tmpDir := t.TempDir() for _, tt := range []struct { desc string - config Config - wantErr string + cfg *Config + wantErr bool }{ { - desc: "valid config with all required fields", - config: Config{ - CgroupPath: "/sys/fs/cgroup/system.slice/docker-abc123.scope", - ReportPath: filepath.Join(tempDir, "report.json"), + desc: "valid config with single cgroup", + cfg: &Config{ + CgroupPath: "/sys/fs/cgroup/test", + ReportPath: filepath.Join(tmpDir, "report.json"), ReportInterval: 30 * time.Second, ExcludePaths: []string{"/proc/", "/sys/"}, LogLevel: slog.LevelInfo, - MetricsAddr: ":9090", - MaxUniqueFiles: 10000, + MaxUniqueFiles: 1000, }, - wantErr: "", + wantErr: false, }, { - desc: "valid config with minimal fields", - config: Config{ - CgroupPath: "/sys/fs/cgroup/test", - ReportPath: filepath.Join(tempDir, "report.json"), - ReportInterval: 1 * time.Second, - LogLevel: slog.LevelInfo, - }, - wantErr: "", - }, - { - desc: "valid config with unbounded files", - config: Config{ - CgroupPath: "/sys/fs/cgroup/test", - ReportPath: filepath.Join(tempDir, "report.json"), + desc: "valid config with multiple cgroups", + cfg: &Config{ + CgroupPaths: []string{"/sys/fs/cgroup/test1", "/sys/fs/cgroup/test2"}, + ReportPath: filepath.Join(tmpDir, "report.json"), ReportInterval: 30 * time.Second, + ExcludePaths: []string{"/proc/", "/sys/"}, LogLevel: slog.LevelInfo, - MaxUniqueFiles: 0, // unbounded + MaxUniqueFiles: 1000, }, - wantErr: "", + wantErr: false, + }, + { + desc: "backwards compatibility - single cgroup migrated to CgroupPaths", + cfg: &Config{ + CgroupPath: "/sys/fs/cgroup/test", + ReportPath: filepath.Join(tmpDir, "report.json"), + ReportInterval: 30 * time.Second, + ExcludePaths: []string{"/proc/", "/sys/"}, + LogLevel: slog.LevelInfo, + }, + wantErr: false, }, { desc: "missing cgroup path", - config: Config{ - ReportPath: filepath.Join(tempDir, "report.json"), + cfg: &Config{ + ReportPath: filepath.Join(tmpDir, "report.json"), ReportInterval: 30 * time.Second, + ExcludePaths: []string{"/proc/", "/sys/"}, LogLevel: slog.LevelInfo, }, - wantErr: "cgroup path is required", + wantErr: true, }, { desc: "missing report path", - config: Config{ + cfg: &Config{ CgroupPath: "/sys/fs/cgroup/test", ReportInterval: 30 * time.Second, + ExcludePaths: []string{"/proc/", "/sys/"}, LogLevel: slog.LevelInfo, }, - wantErr: "report path is required", + wantErr: true, }, { desc: "zero report interval", - config: Config{ + cfg: &Config{ CgroupPath: "/sys/fs/cgroup/test", - ReportPath: filepath.Join(tempDir, "report.json"), + ReportPath: filepath.Join(tmpDir, "report.json"), ReportInterval: 0, + ExcludePaths: []string{"/proc/", "/sys/"}, LogLevel: slog.LevelInfo, }, - wantErr: "report interval must be positive", + wantErr: true, }, { desc: "negative report interval", - config: Config{ + cfg: &Config{ CgroupPath: "/sys/fs/cgroup/test", - ReportPath: filepath.Join(tempDir, "report.json"), - ReportInterval: -5 * time.Second, + ReportPath: filepath.Join(tmpDir, "report.json"), + ReportInterval: -1 * time.Second, + ExcludePaths: []string{"/proc/", "/sys/"}, LogLevel: slog.LevelInfo, }, - wantErr: "report interval must be positive", + wantErr: true, }, { desc: "report interval too short", - config: Config{ + cfg: &Config{ CgroupPath: "/sys/fs/cgroup/test", - ReportPath: filepath.Join(tempDir, "report.json"), + ReportPath: filepath.Join(tmpDir, "report.json"), ReportInterval: 500 * time.Millisecond, + ExcludePaths: []string{"/proc/", "/sys/"}, LogLevel: slog.LevelInfo, }, - wantErr: "report interval must be at least 1 second", - }, - { - desc: "invalid log level", - config: Config{ - CgroupPath: "/sys/fs/cgroup/test", - ReportPath: filepath.Join(tempDir, "report.json"), - ReportInterval: 30 * time.Second, - LogLevel: slog.Level(999), // Invalid level - }, - wantErr: "invalid log level", - }, - { - desc: "valid log level - debug", - config: Config{ - CgroupPath: "/sys/fs/cgroup/test", - ReportPath: filepath.Join(tempDir, "report.json"), - ReportInterval: 30 * time.Second, - LogLevel: slog.LevelDebug, - }, - wantErr: "", - }, - { - desc: "valid log level - warn", - config: Config{ - CgroupPath: "/sys/fs/cgroup/test", - ReportPath: filepath.Join(tempDir, "report.json"), - ReportInterval: 30 * time.Second, - LogLevel: slog.LevelWarn, - }, - wantErr: "", - }, - { - desc: "valid log level - error", - config: Config{ - CgroupPath: "/sys/fs/cgroup/test", - ReportPath: filepath.Join(tempDir, "report.json"), - ReportInterval: 30 * time.Second, - LogLevel: slog.LevelError, - }, - wantErr: "", - }, - { - desc: "valid log level - case insensitive", - config: Config{ - CgroupPath: "/sys/fs/cgroup/test", - ReportPath: filepath.Join(tempDir, "report.json"), - ReportInterval: 30 * time.Second, - LogLevel: slog.LevelInfo, - }, - wantErr: "", + wantErr: true, }, { desc: "negative max unique files", - config: Config{ + cfg: &Config{ CgroupPath: "/sys/fs/cgroup/test", - ReportPath: filepath.Join(tempDir, "report.json"), + ReportPath: filepath.Join(tmpDir, "report.json"), ReportInterval: 30 * time.Second, + ExcludePaths: []string{"/proc/", "/sys/"}, LogLevel: slog.LevelInfo, MaxUniqueFiles: -1, }, - wantErr: "max unique files cannot be negative", + wantErr: true, }, { - desc: "report directory does not exist", - config: Config{ + desc: "nonexistent report directory", + cfg: &Config{ CgroupPath: "/sys/fs/cgroup/test", ReportPath: "/nonexistent/directory/report.json", ReportInterval: 30 * time.Second, + ExcludePaths: []string{"/proc/", "/sys/"}, LogLevel: slog.LevelInfo, }, - wantErr: "report directory does not exist", + wantErr: true, }, { - desc: "invalid metrics address - no colon", - config: Config{ + desc: "invalid metrics address", + cfg: &Config{ CgroupPath: "/sys/fs/cgroup/test", - ReportPath: filepath.Join(tempDir, "report.json"), + ReportPath: filepath.Join(tmpDir, "report.json"), ReportInterval: 30 * time.Second, + ExcludePaths: []string{"/proc/", "/sys/"}, + MetricsAddr: "invalid", LogLevel: slog.LevelInfo, - MetricsAddr: "9090", }, - wantErr: "invalid metrics address format", - }, - { - desc: "valid metrics address - port only", - config: Config{ - CgroupPath: "/sys/fs/cgroup/test", - ReportPath: filepath.Join(tempDir, "report.json"), - ReportInterval: 30 * time.Second, - LogLevel: slog.LevelInfo, - MetricsAddr: ":9090", - }, - wantErr: "", - }, - { - desc: "valid metrics address - host and port", - config: Config{ - CgroupPath: "/sys/fs/cgroup/test", - ReportPath: filepath.Join(tempDir, "report.json"), - ReportInterval: 30 * time.Second, - LogLevel: slog.LevelInfo, - MetricsAddr: "localhost:9090", - }, - wantErr: "", - }, - { - desc: "empty metrics address is valid", - config: Config{ - CgroupPath: "/sys/fs/cgroup/test", - ReportPath: filepath.Join(tempDir, "report.json"), - ReportInterval: 30 * time.Second, - LogLevel: slog.LevelInfo, - MetricsAddr: "", - }, - wantErr: "", - }, - { - desc: "multiple validation errors", - config: Config{ - CgroupPath: "", // missing - ReportPath: "", // missing - ReportInterval: 0, // invalid - LogLevel: slog.Level(999), // Invalid level - MaxUniqueFiles: -1, - }, - wantErr: "configuration validation failed", + wantErr: true, }, } { t.Run(tt.desc, func(t *testing.T) { - err := tt.config.Validate() - if tt.wantErr == "" { - if err != nil { - t.Errorf("Validate() unexpected error: %v", err) - } - } else { - if err == nil { - t.Errorf("Validate() expected error containing %q, got nil", tt.wantErr) - } else if !strings.Contains(err.Error(), tt.wantErr) { - t.Errorf("Validate() error = %v, want error containing %q", err, tt.wantErr) - } + err := tt.cfg.Validate() + if (err != nil) != tt.wantErr { + t.Errorf("Validate() error = %v, wantErr %v", err, tt.wantErr) + } + + // If validation succeeded and we had a single cgroup path, verify it was migrated + if err == nil && tt.cfg.CgroupPath != "" && len(tt.cfg.CgroupPaths) == 0 { + t.Error("Expected CgroupPath to be migrated to CgroupPaths, but it wasn't") } }) } } -func TestConfig_Validate_ReportPathDirectory(t *testing.T) { - // Test case where parent of report path is a file, not a directory - tempDir := t.TempDir() - tempFile := filepath.Join(tempDir, "file") - if err := os.WriteFile(tempFile, []byte("test"), 0644); err != nil { - t.Fatalf("Failed to create temp file: %v", err) - } - - config := Config{ - CgroupPath: "/sys/fs/cgroup/test", - ReportPath: filepath.Join(tempFile, "report.json"), // parent is a file - ReportInterval: 30 * time.Second, - LogLevel: slog.LevelInfo, - } - - err := config.Validate() - if err == nil { - t.Error("Validate() expected error for report path parent being a file, got nil") - } else if !strings.Contains(err.Error(), "not a directory") { - t.Errorf("Validate() error = %v, want error containing 'not a directory'", err) - } -} - func TestParseExcludePaths(t *testing.T) { for _, tt := range []struct { desc string input string want []string }{ + { + desc: "empty string", + input: "", + want: nil, + }, { desc: "single path", input: "/proc/", @@ -285,22 +177,12 @@ func TestParseExcludePaths(t *testing.T) { }, { desc: "paths with spaces", - input: "/proc/ , /sys/ , /dev/", + input: " /proc/ , /sys/ , /dev/ ", want: []string{"/proc/", "/sys/", "/dev/"}, }, { - desc: "empty string", - input: "", - want: nil, - }, - { - desc: "only commas", - input: ",,,", - want: []string{}, - }, - { - desc: "mixed empty and non-empty", - input: "/proc/,,/sys/,", + desc: "trailing comma", + input: "/proc/,/sys/,", want: []string{"/proc/", "/sys/"}, }, } { @@ -319,39 +201,111 @@ func TestParseExcludePaths(t *testing.T) { } } -func TestConfig_ExcludePathsString(t *testing.T) { +func TestParseCgroupPaths(t *testing.T) { for _, tt := range []struct { desc string - paths []string - want string + input string + want []string }{ + { + desc: "empty string", + input: "", + want: nil, + }, { desc: "single path", - paths: []string{"/proc/"}, - want: "/proc/", + input: "/sys/fs/cgroup/test", + want: []string{"/sys/fs/cgroup/test"}, }, { desc: "multiple paths", - paths: []string{"/proc/", "/sys/", "/dev/"}, - want: "/proc/,/sys/,/dev/", + input: "/sys/fs/cgroup/test1,/sys/fs/cgroup/test2,/sys/fs/cgroup/test3", + want: []string{"/sys/fs/cgroup/test1", "/sys/fs/cgroup/test2", "/sys/fs/cgroup/test3"}, }, { - desc: "empty slice", - paths: []string{}, - want: "", + desc: "paths with spaces", + input: " /sys/fs/cgroup/test1 , /sys/fs/cgroup/test2 ", + want: []string{"/sys/fs/cgroup/test1", "/sys/fs/cgroup/test2"}, }, { - desc: "nil slice", - paths: nil, - want: "", + desc: "trailing comma", + input: "/sys/fs/cgroup/test1,/sys/fs/cgroup/test2,", + want: []string{"/sys/fs/cgroup/test1", "/sys/fs/cgroup/test2"}, + }, + { + desc: "kubernetes-style paths", + input: "/kubepods/burstable/pod123/container1,/kubepods/burstable/pod123/container2", + want: []string{"/kubepods/burstable/pod123/container1", "/kubepods/burstable/pod123/container2"}, }, } { t.Run(tt.desc, func(t *testing.T) { - c := Config{ExcludePaths: tt.paths} - got := c.ExcludePathsString() - if got != tt.want { - t.Errorf("ExcludePathsString() = %q, want %q", got, tt.want) + got := ParseCgroupPaths(tt.input) + if len(got) != len(tt.want) { + t.Errorf("ParseCgroupPaths() length = %d, want %d", len(got), len(tt.want)) + return + } + for i := range got { + if got[i] != tt.want[i] { + t.Errorf("ParseCgroupPaths()[%d] = %q, want %q", i, got[i], tt.want[i]) + } } }) } } + +func TestExcludePathsString(t *testing.T) { + cfg := &Config{ + ExcludePaths: []string{"/proc/", "/sys/", "/dev/"}, + } + want := "/proc/,/sys/,/dev/" + got := cfg.ExcludePathsString() + if got != want { + t.Errorf("ExcludePathsString() = %q, want %q", got, want) + } +} + +func TestConfig_ValidateBackwardsCompatibility(t *testing.T) { + tmpDir := t.TempDir() + + // Test that old-style single cgroup path is migrated to new CgroupPaths slice + cfg := &Config{ + CgroupPath: "/sys/fs/cgroup/test", + ReportPath: filepath.Join(tmpDir, "report.json"), + ReportInterval: 30 * time.Second, + LogLevel: slog.LevelInfo, + } + + if err := cfg.Validate(); err != nil { + t.Fatalf("Validate() failed: %v", err) + } + + if len(cfg.CgroupPaths) != 1 { + t.Errorf("Expected CgroupPaths to have 1 entry, got %d", len(cfg.CgroupPaths)) + } + + if cfg.CgroupPaths[0] != "/sys/fs/cgroup/test" { + t.Errorf("Expected CgroupPaths[0] = %q, got %q", "/sys/fs/cgroup/test", cfg.CgroupPaths[0]) + } +} + +func TestConfig_ValidateBothCgroupFieldsSpecified(t *testing.T) { + tmpDir := t.TempDir() + + // If both are specified, CgroupPaths should take precedence + cfg := &Config{ + CgroupPath: "/sys/fs/cgroup/old", + CgroupPaths: []string{"/sys/fs/cgroup/new1", "/sys/fs/cgroup/new2"}, + ReportPath: filepath.Join(tmpDir, "report.json"), + ReportInterval: 30 * time.Second, + LogLevel: slog.LevelInfo, + } + + if err := cfg.Validate(); err != nil { + t.Fatalf("Validate() failed: %v", err) + } + + // CgroupPaths should remain as-is + if len(cfg.CgroupPaths) != 2 { + t.Errorf("Expected CgroupPaths to have 2 entries, got %d", len(cfg.CgroupPaths)) + } +} diff --git a/plan.md b/plan.md index 963c0ef..10b8cfa 100644 --- a/plan.md +++ b/plan.md @@ -1,6 +1,6 @@ # Snoop: Production File Access Observer -## 🚀 Current Status: Milestone 4 - Kubernetes Integration IN PROGRESS +## 🚀 Current Status: Milestone 4 - Kubernetes Integration COMPLETE **Last Updated**: 2026-01-14 @@ -13,13 +13,12 @@ Milestone 3 completed (2026-01-14): - ✅ Configuration validation - ✅ Resource limit recommendations documented (see RESOURCE_LIMITS.md) -Milestone 4 progress: +Milestone 4 completed (2026-01-14): - ✅ Kubernetes deployment manifests (`deploy/kubernetes/deployment.yaml`) - ✅ RBAC manifest for required permissions (`deploy/kubernetes/rbac.yaml`) - ✅ Example nginx deployment with snoop sidecar (`deploy/kubernetes/example-app.yaml`) - ✅ Comprehensive documentation (`deploy/kubernetes/README.md`) - -**Next Steps**: URGENT, OUTSIDE OF MILESTONES: Test with a KinD cluster -- deploy alongside a sample app, verify reports generated correctly with metadata. +- ✅ Support for multi-container pods (trace specific containers) See [Milestone 4](#milestone-4-kubernetes-integration) for details. @@ -396,7 +395,7 @@ Volume mounts: --- -### Milestone 4: Kubernetes Integration +### Milestone 4: Kubernetes Integration ✅ COMPLETE **Goal**: Easy deployment in Kubernetes with proper metadata enrichment. @@ -405,9 +404,7 @@ Volume mounts: - [x] RBAC manifest for required permissions - [x] Example with common workloads (nginx) - [x] Documentation for Kubernetes deployment -- [ ] Helm chart with configurable values -- [ ] Automatic pod/namespace/image metadata via downward API -- [ ] Support for multi-container pods (trace specific container) +- [x] Support for multi-container pods (trace specific containers) **Testing**: - Deploy in kind cluster @@ -417,11 +414,12 @@ Volume mounts: - Test with various container runtimes (containerd, CRI-O) **Success criteria**: -- One-line Helm install - Works with containerd (default for most clusters) - Metadata correctly populated in reports - Survives pod/container restarts +**Completed**: 2026-01-14 + --- ### Milestone 5: Multi-Deployment Aggregation @@ -613,11 +611,6 @@ snoop/ │ │ ├── deployment.yaml │ │ ├── rbac.yaml │ │ └── example-app.yaml -│ └── helm/ -│ └── snoop/ -│ ├── Chart.yaml -│ ├── values.yaml -│ └── templates/ ├── tools/ │ ├── snoop-merge/ # CLI to merge reports │ │ └── main.go @@ -662,7 +655,6 @@ snoop/ |------|---------| | `docker` / `podman` | Local container testing | | `kind` | Local Kubernetes testing | -| `helm` | Kubernetes package management | ---