nescript/docs/architecture.md

Compiler Architecture

An overview of the NEScript compiler internals for contributors and maintainers.

---

Pipeline

Source (.ne) --> Lexer --> Parser --> Analyzer --> IR Lowering --> Optimizer --> Codegen --> Assembler --> Linker --> ROM (.nes)

Each phase is a pure function (input to output) with no global state, making every stage independently testable.

.ne source ---------> Lexer -----> Parser -----> Analyzer ------>
                     (tokens)     (AST)       (annotated AST)

IR Lowering -----> Optimizer -----> Codegen -----> Linker -----> ROM
   (IR)          (optimized IR)   (6502 insns)               (.nes file)

Phase Summary

Phase	Input	Output	Responsibility
Lexer	Source text	Token stream	Tokenization, number/string literal parsing
Parser	Token stream	AST	Syntax validation, tree construction
Analyzer	AST	Annotated AST	Type checking, scope resolution, call graph analysis
IR Lowering	Annotated AST	NEScript IR	Flatten expressions, expand u16 ops, desugar
Optimizer	IR	Optimized IR	Constant folding, dead code, ZP promotion, inlining
Codegen	Optimized IR	6502 instruction list	Register allocation, instruction selection
Assembler	6502 instructions	Byte sequences + fixups	Opcode encoding, address resolution
Linker	Bytes + assets	.nes file	Bank layout, vectors, iNES header

---

Modules

lexer/

Tokenizes NEScript source text into a stream of typed tokens with source spans. Handles decimal, hex, and binary integer literals, string literals, all keywords, and operators.

parser/

Recursive descent parser that converts the token stream into an AST. Defines all AST node types (expressions, statements, declarations) in ast.rs.

analyzer/

Performs semantic analysis on the AST: type checking (types.rs), scope and symbol table management (scope.rs), and call graph construction with depth analysis (call_graph.rs). Detects recursion, type mismatches, undefined references, and call depth violations.

ir/

Defines the intermediate representation and the lowering pass (lowering.rs) that translates the annotated AST into IR. Flattens nested expressions, expands 16-bit operations into 8-bit sequences, and resolves syntactic sugar.

optimizer/

Runs optimization passes over the IR: constant folding (const_fold.rs), dead code elimination (dead_code.rs), zero-page promotion analysis (zp_promote.rs), and function inlining (inliner.rs).

codegen/

Translates optimized IR into 6502 instructions. Includes register allocation for the A/X/Y registers (regalloc.rs) and instruction pattern matching for idiomatic 6502 code (patterns.rs).

asm/

The built-in assembler. Encodes 6502 instructions (encode.rs) with all addressing modes (addressing.rs), using a complete opcode table (opcodes.rs -- 56 instructions across all modes).

linker/

Assigns addresses to code and data segments, resolves fixups/relocations (fixups.rs), and handles bank allocation (banks.rs) for banked mappers.

rom/

Builds the final iNES ROM file. Generates the 16-byte iNES header (header.rs) and places the NMI/RESET/IRQ vector table (vectors.rs).

runtime/

Contains built-in runtime code that the compiler emits into every ROM: NES hardware initialization (init.rs), NMI handler generation (nmi.rs), controller read routines (input.rs), OAM DMA setup (oam.rs), PPU helper routines (ppu.rs), and software multiply/divide (math.rs).

assets/

The asset pipeline. Converts PNG images to CHR tile data (chr.rs), generates nametables from full-screen images (nametable.rs), extracts and maps palettes (palette.rs), and handles audio import stubs (audio.rs).

debug/

Debug instrumentation output. Generates source maps relating ROM addresses to source locations (source_map.rs), symbol tables compatible with Mesen (symbols.rs), and runtime check code for debug builds (checks.rs).

errors/

Error reporting infrastructure. Defines the Diagnostic struct with error codes, severity levels, source spans, labels, help text, and notes (diagnostic.rs). Renders diagnostics with color and source context for terminal output (render.rs).

---

Testing

Test Organization

Tests are co-located with each module in tests.rs files:

src/lexer/tests.rs        -- lexer unit tests
src/parser/tests.rs       -- parser unit tests
src/analyzer/tests.rs     -- semantic analysis tests
src/ir/tests.rs           -- IR lowering tests
src/optimizer/tests.rs    -- optimizer tests
src/codegen/tests.rs      -- code generation tests
src/asm/tests.rs          -- assembler tests
src/linker/tests.rs       -- linker tests
src/rom/tests.rs          -- ROM builder tests
src/assets/tests.rs       -- asset pipeline tests

Integration tests live in the tests/ directory:

tests/integration/        -- full pipeline tests with .ne source files
tests/error_tests/        -- tests that verify specific error codes
tests/asm_tests/          -- 6502 opcode and addressing mode tests

Running Tests

# Run all tests
cargo test

# Run tests for a specific module
cargo test --lib lexer
cargo test --lib parser
cargo test --lib analyzer

# Run integration tests only
cargo test --test '*'

# Run a specific test by name
cargo test test_name

Test Strategy

Each compiler phase is designed as a pure function, so unit tests provide isolated input and verify output without side effects. Integration tests compile complete .ne source files and verify the output ROM matches expected golden files in tests/integration/expected/.

Error tests in tests/error_tests/ contain intentionally broken programs and verify that the correct error code is produced (e.g., recursion.ne should produce E0402).