Add tracking for data sections, DWARF debug info, and symbol tables to provide comprehensive binary size attribution to packages.
- Parse and attribute data sections (.data, .rodata, .bss)
- Track DWARF debug info distributed by compile units
- Track symbol tables using Go function tables
- Fix Mach-O symbol attribution (use gosym instead of C symbols)
- Add "internal" category, color stdlib/internal/other gray in SVG
Improves attribution coverage to 89% (gobinsize) and 75% (kubectl).
Replaced exec.Command("go", "list", "std") with packages.Load(nil, "std")
from golang.org/x/tools/go/packages as suggested.
Source attribution:
- https://stackoverflow.com/a/53541580
- Posted by Martin Tournoij, modified by community
- Retrieved 2025-11-17, License - CC BY-SA 4.0
This approach is more idiomatic and doesn't require spawning a subprocess.
Added golang.org/x/tools dependency to go.mod.
Co-authored-by: imjasonh <210737+imjasonh@users.noreply.github.com>
- Removed hardcoded stdlibPackages map (anti-pattern)
- Added getStdlibPackages() that calls `go list std` dynamically
- Removed deprecated getPackageName(), isExternalDependency(), getModuleName() functions
- Updated findModuleForSymbol() to use dynamic stdlib list
- Fixed assumption that stdlib packages don't have slashes (encoding/json does)
- Updated tests to use findModuleForSymbol instead of deprecated functions
- Removed unused net/url import
All stdlib packages are now properly detected including those with slashes.
No hardcoded lists needed.
Co-authored-by: imjasonh <210737+imjasonh@users.noreply.github.com>
Completely rewrote the attribution logic per feedback. Instead of complex
string parsing to extract package names, now simply check if any module
from BuildInfo or stdlib package is contained in the symbol name.
Key changes:
- New findModuleForSymbol() function uses simple substring matching
- Check BuildInfo modules first (longest first for best match)
- Then check stdlib packages with prefix matching
- Removed all complex string manipulation logic
- Added stdlib package map to avoid guessing based on dots
Fixes:
- google.golang.org symbols now correctly attributed (was: "google")
- Simpler, more maintainable code
- No more edge cases with domains containing dots
Results on Hugo:
- google.golang.org/protobuf: 726.84 KB (was split as "google" 601KB + protobuf 125KB)
- All symbols properly attributed
Co-authored-by: imjasonh <210737+imjasonh@users.noreply.github.com>
Removed isLikelyPartialDomain function and complex domain detection logic.
New simpler algorithm:
- If first component has no dot → stdlib package
- If single component with dot → extract base (e.g., unicode.map → unicode)
- If multiple components with dot in first → other
This eliminates the hardcoded list of partial domains which was an anti-pattern.
The module prefix matching handles most cases, and edge cases like go.uber
now extract to their base (go) which is acceptable given the simplicity gain.
Co-authored-by: imjasonh <210737+imjasonh@users.noreply.github.com>
Completely rewrote the attribution algorithm as requested:
1. Get module dependencies from BuildInfo (including main module)
2. Sort module paths by length (longest first)
3. Iterate through modules to find matches via prefix matching
4. If no match and from stdlib, attribute to stdlib package (e.g., "unicode")
5. Otherwise, attribute to "other" with verbose logging
Key fixes:
- github.com/gohugoio/hugo/common/hstrings → github.com/gohugoio/hugo
- unicode.map → unicode (stdlib)
- go.uber → other (detected as partial domain)
- google.golang.org/protobuf/internal → google.golang.org/protobuf
Added tests covering all edge cases including partial domains, stdlib packages
with .map suffixes, and module subpackages.
Co-authored-by: imjasonh <210737+imjasonh@users.noreply.github.com>
- Split stdlib packages into individual modules (runtime, net, crypto, etc.) instead of grouping as 'stdlib'
- Added -verbose flag that logs packages attributed to 'other' for debugging
- This helps identify potential attribution errors and understand what's in 'other'
Example verbose output shows:
- Helper types (*.map packages)
- Partial package names (go.uber instead of go.uber.org/automaxprocs)
- Internal subpackages not listed as separate BuildInfo deps
Co-authored-by: imjasonh <210737+imjasonh@users.noreply.github.com>
Changed getModuleName to only report modules that exist in BuildInfo.
Packages not in BuildInfo are grouped as:
- "stdlib" for standard library packages (no domain in path)
- "other" for unrecognized packages
This fixes the issue where partial package names like "google", "google.golang",
and stdlib packages like "encoding", "net", "crypto" were being reported as
separate modules when they don't exist in BuildInfo.
Co-authored-by: imjasonh <210737+imjasonh@users.noreply.github.com>
- Handle .init functions before Type.method detection to avoid false positives
- Fix Type.method detection for external packages with 4+ parts
- Add special handling for (*Type).Method followed by type.method patterns
- Add comprehensive tests for all problematic patterns found in Hugo
- Fixes issues like github.com/spf13/pflag.timeValue.Add and nested closures
Co-authored-by: imjasonh <210737+imjasonh@users.noreply.github.com>
- Added handling for trailing semicolons in function names (e.g., io.Reader;)
- Removed analyze-binaries.yml workflow as CI tests now cover everything needed
- Added tests for patterns with semicolons and the new symbols mentioned
Co-authored-by: imjasonh <210737+imjasonh@users.noreply.github.com>
Changed go:struct and go:itab patterns to extract package information
instead of filtering them out entirely. This ensures symbols like
go:struct { github.com/gohugoio/hugo are attributed to Hugo.
Co-authored-by: imjasonh <210737+imjasonh@users.noreply.github.com>
- Make symtab extraction failures non-fatal for modern Go binaries
- Handle cases where symtab symbols are missing or equal (empty symtab)
- Extract symtab as named constant noNextSectionRVA for better readability
- Add comments explaining that modern Go can work without separate symtab
Co-authored-by: imjasonh <210737+imjasonh@users.noreply.github.com>
- Fixed PE symbol handling: symbols are section-relative, not RVAs
- Implemented multi-section data extraction for pclntab/symtab
- Added gap handling between PE sections
- Handle empty symtab (modern Go uses pclntab-only)
- Added ImageBase calculation for correct text address
- Added Windows binary analysis steps to CI workflow
- All three platforms (Linux ELF, Darwin Mach-O, Windows PE) now fully supported
Co-authored-by: imjasonh <210737+imjasonh@users.noreply.github.com>
This is work in progress to support Windows binary analysis. Modern Go (1.16+)
embeds runtime tables within sections rather than as separate PE sections. The
code now attempts to extract pclntab/symtab using runtime symbols, but needs
further refinement to handle tables that span multiple sections.
Co-authored-by: imjasonh <210737+imjasonh@users.noreply.github.com>
- Made analyzeSymbolsMachO consistent with analyzeSymbolsPE
- Returns error instead of silently failing with empty report
- Provides clear feedback when symbol table is not found
Co-authored-by: imjasonh <210737+imjasonh@users.noreply.github.com>
- Created processSymbolTable() function to handle common symbol analysis
- Updated analyzeSymbols (ELF), analyzeSymbolsMachO, and analyzeSymbolsPE to use shared function
- Reduces code duplication and improves maintainability
Co-authored-by: imjasonh <210737+imjasonh@users.noreply.github.com>
- Extract module dependencies from BuildInfo and use for attribution
- Remove .init and .init.N suffixes from package names
- Add error checking for file seek operations
- Improve error message for unsupported binary format
- Fix analyzeLineTable to return error for unsupported case
- Add named constants for truncation suffix
- Fix go.mod version to 1.21
- Add comprehensive tests for .init suffix removal and BuildInfo integration
Co-authored-by: imjasonh <210737+imjasonh@users.noreply.github.com>
