mirror of
https://github.com/imjasonh/snoop
synced 2026-07-19 07:04:01 +00:00
Add memory-bounded deduplication with LRU cache
Implements configurable memory limits for the file deduplication cache to prevent unbounded memory growth in long-running production deployments. - Add LRU cache implementation with configurable max size - Add -max-unique-files flag (default: 0 = unbounded) - Add snoop_events_evicted_total metric for monitoring - Track and log cache evictions with delta warnings - Add comprehensive tests for bounded and unbounded behavior When max-unique-files is set, the cache evicts least-recently-used entries to stay within the limit. Evicted files may be re-recorded if accessed again, making this suitable for workloads with temporal locality in file access patterns. Closes milestone 3 task: Memory-bounded deduplication Signed-off-by: Jason Hall <jason@chainguard.dev>
This commit is contained in:
parent
87dab9839a
commit
b3f7c96343
8 changed files with 458 additions and 231 deletions
5
plan.md
5
plan.md
|
|
@ -8,8 +8,9 @@ Milestone 3 progress:
|
|||
- ✅ Prometheus metrics endpoint (`/metrics` on port 9090)
|
||||
- ✅ Structured logging with clog
|
||||
- ✅ Ring buffer overflow handling and metrics
|
||||
- ✅ Memory-bounded deduplication with LRU cache
|
||||
|
||||
**Next**: Memory-bounded deduplication
|
||||
**Next**: Health check endpoint
|
||||
|
||||
See [Milestone 3](#milestone-3-production-hardening) for remaining tasks.
|
||||
|
||||
|
|
@ -364,7 +365,7 @@ Volume mounts:
|
|||
- [x] Prometheus metrics endpoint
|
||||
- [x] Structured logging with levels (clog)
|
||||
- [x] Ring buffer overflow handling and metrics
|
||||
- [ ] Memory-bounded deduplication (LRU or bloom filter for extreme cases)
|
||||
- [x] Memory-bounded deduplication (LRU cache with configurable max size)
|
||||
- [ ] Health check endpoint
|
||||
- [ ] Configuration validation
|
||||
- [ ] Resource limit recommendations documented
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue