mirror of
https://github.com/imjasonh/nescript
synced 2026-07-08 17:06:04 +00:00
d7092f703d
docs/language-guide.md (895 lines): Complete reference for every language feature with code examples. Covers: program structure, types, expressions, statements, assets, mappers, error codes, and compiler flags. docs/architecture.md (130 lines): Compiler pipeline overview, module descriptions, testing guide. docs/nes-reference.md (190 lines): NES hardware quick reference: CPU, memory map, PPU, iNES format. https://claude.ai/code/session_01W6eQFStA66EuMKHUFo2rx3
130 lines
6.2 KiB
Markdown
130 lines
6.2 KiB
Markdown
# Compiler Architecture
|
|
|
|
An overview of the NEScript compiler internals for contributors and maintainers.
|
|
|
|
---
|
|
|
|
## Pipeline
|
|
|
|
```
|
|
Source (.ne) --> Lexer --> Parser --> Analyzer --> IR Lowering --> Optimizer --> Codegen --> Assembler --> Linker --> ROM (.nes)
|
|
```
|
|
|
|
Each phase is a pure function (input to output) with no global state, making every stage independently testable.
|
|
|
|
```
|
|
.ne source ---------> Lexer -----> Parser -----> Analyzer ------>
|
|
(tokens) (AST) (annotated AST)
|
|
|
|
IR Lowering -----> Optimizer -----> Codegen -----> Linker -----> ROM
|
|
(IR) (optimized IR) (6502 insns) (.nes file)
|
|
```
|
|
|
|
### Phase Summary
|
|
|
|
| Phase | Input | Output | Responsibility |
|
|
|-----------------|----------------------|------------------------|------------------------------------------------------|
|
|
| **Lexer** | Source text | Token stream | Tokenization, number/string literal parsing |
|
|
| **Parser** | Token stream | AST | Syntax validation, tree construction |
|
|
| **Analyzer** | AST | Annotated AST | Type checking, scope resolution, call graph analysis |
|
|
| **IR Lowering** | Annotated AST | NEScript IR | Flatten expressions, expand u16 ops, desugar |
|
|
| **Optimizer** | IR | Optimized IR | Constant folding, dead code, ZP promotion, inlining |
|
|
| **Codegen** | Optimized IR | 6502 instruction list | Register allocation, instruction selection |
|
|
| **Assembler** | 6502 instructions | Byte sequences + fixups| Opcode encoding, address resolution |
|
|
| **Linker** | Bytes + assets | .nes file | Bank layout, vectors, iNES header |
|
|
|
|
---
|
|
|
|
## Modules
|
|
|
|
### `lexer/`
|
|
Tokenizes NEScript source text into a stream of typed tokens with source spans. Handles decimal, hex, and binary integer literals, string literals, all keywords, and operators.
|
|
|
|
### `parser/`
|
|
Recursive descent parser that converts the token stream into an AST. Defines all AST node types (expressions, statements, declarations) in `ast.rs`.
|
|
|
|
### `analyzer/`
|
|
Performs semantic analysis on the AST: type checking (`types.rs`), scope and symbol table management (`scope.rs`), and call graph construction with depth analysis (`call_graph.rs`). Detects recursion, type mismatches, undefined references, and call depth violations.
|
|
|
|
### `ir/`
|
|
Defines the intermediate representation and the lowering pass (`lowering.rs`) that translates the annotated AST into IR. Flattens nested expressions, expands 16-bit operations into 8-bit sequences, and resolves syntactic sugar.
|
|
|
|
### `optimizer/`
|
|
Runs optimization passes over the IR: constant folding (`const_fold.rs`), dead code elimination (`dead_code.rs`), zero-page promotion analysis (`zp_promote.rs`), and function inlining (`inliner.rs`).
|
|
|
|
### `codegen/`
|
|
Translates optimized IR into 6502 instructions. Includes register allocation for the A/X/Y registers (`regalloc.rs`) and instruction pattern matching for idiomatic 6502 code (`patterns.rs`).
|
|
|
|
### `asm/`
|
|
The built-in assembler. Encodes 6502 instructions (`encode.rs`) with all addressing modes (`addressing.rs`), using a complete opcode table (`opcodes.rs` -- 56 instructions across all modes).
|
|
|
|
### `linker/`
|
|
Assigns addresses to code and data segments, resolves fixups/relocations (`fixups.rs`), and handles bank allocation (`banks.rs`) for banked mappers.
|
|
|
|
### `rom/`
|
|
Builds the final iNES ROM file. Generates the 16-byte iNES header (`header.rs`) and places the NMI/RESET/IRQ vector table (`vectors.rs`).
|
|
|
|
### `runtime/`
|
|
Contains built-in runtime code that the compiler emits into every ROM: NES hardware initialization (`init.rs`), NMI handler generation (`nmi.rs`), controller read routines (`input.rs`), OAM DMA setup (`oam.rs`), PPU helper routines (`ppu.rs`), and software multiply/divide (`math.rs`).
|
|
|
|
### `assets/`
|
|
The asset pipeline. Converts PNG images to CHR tile data (`chr.rs`), generates nametables from full-screen images (`nametable.rs`), extracts and maps palettes (`palette.rs`), and handles audio import stubs (`audio.rs`).
|
|
|
|
### `debug/`
|
|
Debug instrumentation output. Generates source maps relating ROM addresses to source locations (`source_map.rs`), symbol tables compatible with Mesen (`symbols.rs`), and runtime check code for debug builds (`checks.rs`).
|
|
|
|
### `errors/`
|
|
Error reporting infrastructure. Defines the `Diagnostic` struct with error codes, severity levels, source spans, labels, help text, and notes (`diagnostic.rs`). Renders diagnostics with color and source context for terminal output (`render.rs`).
|
|
|
|
---
|
|
|
|
## Testing
|
|
|
|
### Test Organization
|
|
|
|
Tests are co-located with each module in `tests.rs` files:
|
|
|
|
```
|
|
src/lexer/tests.rs -- lexer unit tests
|
|
src/parser/tests.rs -- parser unit tests
|
|
src/analyzer/tests.rs -- semantic analysis tests
|
|
src/ir/tests.rs -- IR lowering tests
|
|
src/optimizer/tests.rs -- optimizer tests
|
|
src/codegen/tests.rs -- code generation tests
|
|
src/asm/tests.rs -- assembler tests
|
|
src/linker/tests.rs -- linker tests
|
|
src/rom/tests.rs -- ROM builder tests
|
|
src/assets/tests.rs -- asset pipeline tests
|
|
```
|
|
|
|
Integration tests live in the `tests/` directory:
|
|
|
|
```
|
|
tests/integration/ -- full pipeline tests with .ne source files
|
|
tests/error_tests/ -- tests that verify specific error codes
|
|
tests/asm_tests/ -- 6502 opcode and addressing mode tests
|
|
```
|
|
|
|
### Running Tests
|
|
|
|
```bash
|
|
# Run all tests
|
|
cargo test
|
|
|
|
# Run tests for a specific module
|
|
cargo test --lib lexer
|
|
cargo test --lib parser
|
|
cargo test --lib analyzer
|
|
|
|
# Run integration tests only
|
|
cargo test --test '*'
|
|
|
|
# Run a specific test by name
|
|
cargo test test_name
|
|
```
|
|
|
|
### Test Strategy
|
|
|
|
Each compiler phase is designed as a pure function, so unit tests provide isolated input and verify output without side effects. Integration tests compile complete `.ne` source files and verify the output ROM matches expected golden files in `tests/integration/expected/`.
|
|
|
|
Error tests in `tests/error_tests/` contain intentionally broken programs and verify that the correct error code is produced (e.g., `recursion.ne` should produce `E0402`).
|