# NEScript Compiler — Engineering Design & Implementation Plan
**Version 0.1 — Draft**
---
## 1. Executive Summary
This document describes the architecture, implementation strategy, and milestone plan for the NEScript compiler: a Rust-based toolchain that compiles NEScript source files into iNES-format NES ROMs. The compiler owns the entire pipeline from source text to playable ROM, with no external assembler or linker dependencies.
The plan is structured around five milestones, each producing a working compiler that generates a playable ROM demonstrating progressively more language features. The first milestone targets a "sprite on screen, moving with the d-pad" demo in approximately 4–6 weeks of focused development.
1.**Single binary, zero dependencies.** The compiler is one Rust binary. No need to install ca65, Python, or Node. Asset conversion (PNG → CHR) is built in.
2.**Fast compilation.** NES games are small. Compilation should be under 1 second for any reasonable project. No incremental compilation needed for v1.
3.**Excellent errors.** Every error message includes the source location, a clear explanation, and a `help:` suggestion where possible. Errors are the primary teaching tool.
4.**Testable at every layer.** Each compiler phase is a pure function (input → output) with no global state, enabling unit testing in isolation.
pub source: Option<Span>, // link back to NEScript source
}
#[derive(Debug, Clone, Copy, PartialEq)]
pub enum Opcode {
LDA, LDX, LDY, STA, STX, STY,
ADC, SBC, AND, ORA, EOR,
ASL, LSR, ROL, ROR,
INC, DEC, INX, INY, DEX, DEY,
CMP, CPX, CPY, BIT,
JMP, JSR, RTS, RTI,
BEQ, BNE, BCC, BCS, BMI, BPL, BVC, BVS,
CLC, SEC, CLI, SEI, CLV, CLD, SED,
PHA, PLA, PHP, PLP,
TAX, TAY, TXA, TYA, TSX, TXS,
NOP, BRK,
}
#[derive(Debug, Clone)]
pub enum AddressingMode {
Implied, // CLC, RTS, etc.
Accumulator, // ASL A
Immediate(u8), // LDA #$FF
ZeroPage(u8), // LDA $10
ZeroPageX(u8), // LDA $10,X
ZeroPageY(u8), // LDX $10,Y
Absolute(u16), // LDA $8000
AbsoluteX(u16), // LDA $8000,X
AbsoluteY(u16), // LDA $8000,Y
Indirect(u16), // JMP ($FFFC)
IndirectX(u8), // LDA ($10,X)
IndirectY(u8), // LDA ($10),Y
Relative(i8), // BEQ +5 (branch offset)
// Pre-resolution symbolic forms (resolved by linker)
Label(String), // JMP some_label
LabelRelative(String), // BEQ some_label
SymbolLo(String), // LDA #<some_address
SymbolHi(String), // LDA #>some_address
}
```
---
## 5. Phase Details
### 5.1 Lexer
**Implementation:** Hand-written scanner (no parser generator). NEScript's lexical grammar is simple enough that a hand-written lexer is clearer and produces better error messages than a generated one.
**Key behavior:**
- Tracks byte offset, line, and column for every token.
-`asm { ... }` blocks: when the lexer encounters `asm` followed by `{`, it switches to a raw-capture mode that collects everything until the matching `}`, respecting nested braces. The content is emitted as a single `AsmBody` token.
- Unterminated strings and unknown characters produce error tokens with descriptive messages rather than panicking.
**Estimated size:** ~400 lines of Rust.
### 5.2 Parser
**Implementation:** Recursive descent, single-token lookahead. The grammar is LL(1)-friendly by design. No ambiguities requiring backtracking.
**Key decisions:**
-`draw` statements use a keyword-argument syntax that looks like `draw SpriteName frame: expr at: (expr, expr)`. The parser recognizes `draw` as a statement keyword, then consumes named property pairs until it hits a statement-ending token (newline at same indentation, `}`, or another statement keyword). Properties can appear in any order.
- Operator precedence is handled via Pratt parsing (precedence climbing) for expressions.
- Error recovery: on a parse error, the parser skips to the next statement boundary (`;`, `}`, or a keyword that starts a new declaration) and continues. This allows reporting multiple errors per compilation.
**Estimated size:** ~800 lines of Rust.
### 5.3 Semantic Analyzer
This is the most complex phase. It performs multiple passes over the AST:
**Pass 1 — Symbol Resolution:**
Build a symbol table. Resolve all identifier references to their declarations. Detect undefined variables, duplicate declarations, and scope violations.
**Pass 2 — Type Checking:**
Verify that all operations have compatible types. Check assignments, function arguments, return types, and array index types. Insert explicit cast nodes where the programmer wrote `as`. Flag implicit coercion attempts as errors.
**Pass 3 — Call Graph Analysis:**
Build a directed graph of function calls. Detect recursion (cycles in the graph). Compute the maximum call depth for every entry point (each `on frame`, `on enter`, etc.). Compare against `stack_depth` and emit errors if exceeded.
**Pass 4 — State Machine Validation:**
Verify that all `transition` targets name real states. Verify that `start` names a real state. Warn if any state is unreachable from the start state.
**Pass 5 — Memory Planning:**
Compute variable lifetimes. Determine which state-local variables can share RAM addresses. Count zero-page demand vs. supply. Assign preliminary memory locations (finalized during codegen/linking).
**Estimated size:** ~1,200 lines of Rust.
### 5.4 IR Lowering
Translates the annotated AST into the flat IR representation:
- Expressions become sequences of IrOps operating on virtual temps.
-`u16` operations expand into paired `u8` operations with carry propagation.
-`if`/`while`/`loop` become basic blocks with branch terminators.
-`on frame` desugars into a loop + wait_frame.
-`draw` statements become `DrawSprite` ops with evaluated sub-expressions.
-`debug.*` statements are either lowered to debug ops (debug mode) or discarded entirely (release mode).
-`button.X_pressed` expressions expand to `(current & mask) AND NOT (previous & mask)`.
**Estimated size:** ~600 lines of Rust.
### 5.5 Optimizer
Runs a sequence of optimization passes over the IR. Each pass is a function `fn optimize(ir: &mut IrProgram)`.
**Constant Folding:** Evaluate operations on known constants at compile time. `3 + 5` becomes `8`. `score > 255` on a `u8` becomes `false`.
**Dead Code Elimination:** Remove ops whose results are never used. Remove basic blocks that are never jumped to.
**Zero-Page Promotion:** Analyze access frequency across hot paths (frame handlers). Rank variables by access count. Assign the hottest variables to zero-page, respecting the `fast`/`slow` hints as hard constraints. Variables marked `fast` get ZP unconditionally (or error if no room). Variables marked `slow` are excluded. Unmarked variables are auto-promoted by rank.
**Function Inlining:** Inline functions marked `inline` or functions under a size threshold (~8 IR ops) that are called from only 1–2 sites.
**Strength Reduction:** Replace multiply/divide by powers of 2 with shifts. Replace `x * 3` with `x + x + x` (three adds are cheaper than a software multiply on the 6502).
**Estimated size:** ~800 lines of Rust.
### 5.6 Code Generation
Translates IR ops into 6502 instructions. This is the heart of the compiler.
**Register Allocation Strategy:**
The 6502 has three registers: A (accumulator, used for all arithmetic), X and Y (index registers, used for array indexing and loops). There is no general register file.
The codegen uses a simple strategy:
1. A is the primary working register. Most IR temps map to A.
2. X is used for array indexing and loop counters.
3. Y is used as a secondary index register.
4. When all three are in use, spill to a set of compiler-reserved zero-page temporaries ($00–$0F).
This is not a full graph-coloring allocator — the 6502's constraints make that overkill. Instead, the codegen walks each basic block linearly, tracking what's currently in A/X/Y, and emitting loads/stores as needed.
- Instructions are 1, 2, or 3 bytes depending on addressing mode.
- Labels are resolved in two passes: first pass collects label addresses, second pass fills in references.
- Branch instructions (BEQ, BNE, etc.) use relative addressing (signed 8-bit offset). If a branch target is out of range (>127 bytes away), the assembler automatically rewrites it as the inverse branch over a JMP (e.g., BEQ +3 / JMP far_target).
**Opcode table dimensions:** 56 unique instructions × ~12 addressing modes = ~150 valid combinations. The full table fits in a static array.
**Estimated size:** ~500 lines of Rust (including the opcode table).
### 5.8 Linker / ROM Builder
Arranges compiled code and data into the final ROM image.
**For NROM (no banking):**
1. Lay out PRG ROM starting at $8000 (32 KB) or $C000 (16 KB).
2. Place the runtime init code at the RESET vector entry.
3. Place the NMI handler.
4. Place all compiled functions and state handlers.
5. Place constant data (lookup tables, palette data, nametable data).
6. Write the interrupt vector table at $FFFA: NMI, RESET, IRQ addresses.
7. Place CHR data (tiles) into the CHR ROM section.
8. Prepend the 16-byte iNES header.
9. Write the .nes file.
**For banked mappers:**
The linker manages multiple PRG banks. The fixed bank contains the runtime, NMI handler, and trampoline stubs. Switchable banks contain state code and associated data. The linker verifies that no single bank exceeds its size limit and that cross-bank references go through trampolines.
**Estimated size:** ~400 lines of Rust.
### 5.9 Runtime Library
The compiler emits a small runtime library embedded in every ROM. This is not a separate file — the codegen emits these routines directly.
**Color mapping:** The compiler includes a reference table of all 64 NES colors in RGB. Source image colors are mapped to the nearest NES color by Euclidean distance in RGB space. The compiler warns if a source color is far from any NES color.
### 6.2 PNG → Nametable Conversion
1. Convert the full 256×240 image into 8×8 tiles as above.
2. Deduplicate tiles (NES nametable is 960 tile indices, but CHR ROM holds at most 256 unique tiles per bank).
3. Build the 960-byte nametable (tile indices) and 64-byte attribute table (palette assignments per 16×16 pixel region).
4. Emit tile data to CHR ROM and nametable data to PRG ROM.
5. Error if more than 256 unique tiles are needed (NES hardware limit).
### 6.3 Audio (Deferred)
For v1, audio asset handling is stubbed. The `@sfx()` and `@music()` directives will accept pre-formatted binary data via `@binary()` as a workaround. Full FamiTracker import is a v2 feature.
let bytes = assemble_instruction(Opcode::LDA, AddressingMode::Immediate(0xFF));
assert_eq!(bytes, vec![0xA9, 0xFF]);
}
```
### 8.2 Integration Tests (Golden File Tests)
Each `.ne` test file in `tests/integration/` has a corresponding expected output. The test harness compiles the source and compares the resulting ROM byte-for-byte against the golden file. If the golden file doesn't exist, the test creates it and marks itself as "needs review."
Additionally, integration tests can run the ROM in an embedded NES CPU emulator (using the `mos6502` crate or a minimal custom implementation) for a fixed number of frames and assert on memory state. For example: "after 10 frames with the right button held, the byte at the player_x address should have increased by 20."
### 8.3 Error Tests
Each `.ne` file in `tests/error_tests/` is expected to fail compilation. The test asserts that the correct error code is produced and the error message contains expected substrings.
### 8.4 Fuzzing
The lexer and parser are fuzz-tested using `cargo-fuzz` to ensure they don't panic on arbitrary input. The compiler should always produce a clean error or a valid ROM — never a crash.
### 8.5 Hardware Validation
Select milestone ROMs are tested on real NES hardware via flash cart (manual process, not automated). This catches emulator-specific behavior that doesn't match real hardware (e.g., PPU timing edge cases).
| `ariadne` | Beautiful error message rendering | small |
| `serde` | Config/debug symbol serialization | small |
| `serde_json` | Debug symbol output format | small |
Total external dependencies: 5 crates. No native/C dependencies. The compiler builds and runs on Windows, macOS, and Linux without any platform-specific setup.
---
## 10. Milestone Plan
### Milestone 1 — "Hello Sprite" (Weeks 1–6)
**Goal:** Compile a minimal NEScript program that displays a sprite on screen and moves it with the d-pad. The resulting .nes file runs in an emulator.
**Language subset:**
-`game` declaration (NROM only)
-`var` (u8 only, global only)
-`const` (u8 only)
-`on frame` (single state, no state machine)
-`if` / `else`
- Arithmetic: `+`, `-`, `+=`, `-=`
- Comparison: `==`, `!=`, `<`, `>`, `<=`, `>=`
-`button.*` input (held only, no pressed/released)
-`draw` (single sprite, hardcoded CHR data)
- Inline `asm` blocks
**Compiler phases built:**
- Lexer (full)
- Parser (subset)
- Analyzer (basic type checking, no call graph yet)
- Codegen (direct AST → 6502, skip IR for this milestone)
| M5 | 25–32 | 32 weeks | Bank switching, audio, public release |
**Total estimated timeline: ~8 months to v0.1 release** (one developer, focused). With two developers, milestones 3 and 4 can partially overlap (asset pipeline and optimizer are independent), reducing the timeline to approximately 6 months.
| Asset pipeline color mapping produces ugly results | Medium | Allow manual palette override; provide clear warning when colors are approximated |
| Optimizer introduces bugs | Medium | Every optimization pass has its own test suite; optimizer can be disabled with `--no-opt` flag |
| FamiTracker format parsing is complex | Medium | Defer to M5; use binary include as workaround; consider using existing FT export tools |
| Scope creep (too many language features too early) | Medium | Strict milestone scoping; "reserved for future" list in spec prevents premature implementation |
| Single-developer bus factor | Low | Thorough documentation; clean module boundaries; public repo from M1 |
---
## 13. Open Questions
1.**Inline asm label syntax:** Should labels in `asm {}` blocks use `.label:` (ca65 style) or `label:` (generic)? This affects the assembler's label parser.
2.**Debug port address:** $4800 is the conventional choice for debug output in homebrew NES emulators, but not all emulators support it. Should we support multiple debug output methods?
3.**OAM allocation strategy:** Currently the compiler allocates OAM slots per `draw` call in order. Should we support priority-based allocation or automatic sprite cycling to mitigate the 8-sprites-per-scanline hardware limit?
4.**Error recovery granularity:** How aggressively should the parser recover from errors? More recovery = more errors reported per compile = faster iteration. But poor recovery can produce cascading false errors.
5.**WASM build target:** Should the compiler itself compile to WASM for a future browser-based IDE? This would require avoiding file system access in the core pipeline (using an in-memory VFS instead). Worth considering in the architecture now even if not implemented until post-v1.
