1
0
Fork 0
mirror of https://github.com/imjasonh/nescript synced 2026-07-08 08:55:38 +00:00

cleanup: fix silent miscompiles and delete dead code exposed by code review

Two correctness bugs were silently producing wrong ROMs:

  - `x << n` / `x >> n` always shifted by 1, regardless of `n`, because
    the IR lowering for `BinOp::ShiftLeft`/`ShiftRight` hardcoded the
    count. Now eval_const the RHS into a compile-time count; fall back
    to a new `IrOp::ShiftLeftVar` / `ShiftRightVar` (runtime loop) when
    the amount isn't constant. Strength reduction folds the variable
    form back to a fixed count once the optimizer knows the value.

  - `x / n` / `x % n` always returned 0, because the lowering emitted
    `LoadImm(t, 0)` for `BinOp::Div`/`Mod` with a comment saying the
    runtime call was "TODO for now". Added real `IrOp::Div` and
    `IrOp::Mod`, wired them through use-counting and DCE, gave codegen
    `__divide`-based implementations, and taught strength reduction to
    rewrite power-of-two divisors into shifts and modulo-by-2ⁿ into
    AND masks. Constant folding now handles `Mul`/`Div`/`Mod`/shifts
    too, which were previously left for the codegen to emit inefficient
    software calls.

Dead code removed (no backward-compat shims kept):

  - `src/debug/` entirely. `DebugSymbols`, `SourceMap`, and the
    Mesen/.sym emitters had no callers outside their own tests;
    `main.rs` never wrote a symbol file. Documented the intent in
    `docs/future-work.md` so it comes back intentionally if needed.
  - `ErrorCode::E0202` (invalid cast) and `E0403` (unreachable state):
    defined, formatted, and marked `#[allow(dead_code)]` but never
    emitted. W0104 now carries the unreachable-state semantics too.
  - `Level::Info`: never constructed.
  - `load_background` / `set_palette` statements and their
    `BackgroundDecl` / `PaletteDecl` parser support: parsed and
    silently dropped by IR lowering (`// TODO: implement in asset
    pipeline`). Removed keywords, AST variants, parser paths, analyzer
    arms, and tests. `docs/future-work.md` documents the runtime
    palette/nametable design for when it comes back.

Doc cleanup:

  - `docs/architecture.md` was describing files that don't exist
    (`analyzer/types.rs`, `optimizer/const_fold.rs`, `codegen/regalloc.rs`,
    `rom/header.rs`, `debug/symbols.rs`, …). Rewrote it to match the
    real flat `mod.rs` + `tests.rs` layout and the real pipeline order.
  - `docs/future-work.md` was a hybrid of open work and "recently
    completed" entries that duplicated the active stubs at the top of
    the file. Collapsed to just the gaps that are actually still open.
  - `README.md` claimed Mesen symbol export and 210 tests; updated both.
  - `docs/language-guide.md` and `spec.md` described `palette` decls,
    `set_palette` / `load_background`, `debug.overlay`, and error codes
    that were never emitted. Trimmed.
  - Stale comments on `Statement::Play`/`StartMusic`/`StopMusic`
    claimed the audio subsystem was "a no-op at codegen time".

Tests:

  - Regression tests for every fix above (`lower_shift_left_with_literal
    _count_uses_that_count`, `lower_shift_right_with_variable_count
    _uses_runtime_variant`, `lower_divide_emits_div_op_not_load_imm
    _zero`, `lower_modulo_emits_mod_op_not_load_imm_zero`,
    `strength_reduce_div_by_power_of_two`, `strength_reduce_mod_by
    _power_of_two`, `strength_reduce_shift_var_with_constant_amount`).
  - Renamed the `program_with_sprites_and_palette` integration test
    (which was exercising the now-removed `load_background`/`set_palette`)
    to `program_with_inline_sprite_chr`.

`examples/sprites_and_palettes.ne` lost its `palette`/`set_palette`
usage. Nothing in the emulator test presses A, so the headless
jsnes render shouldn't move, but the golden may need regeneration
via `UPDATE_GOLDENS=1` if it does.

https://claude.ai/code/session_012fKB251HvEUQwG3tizFyqt
This commit is contained in:
Claude 2026-04-13 02:47:37 +00:00
parent 971a128d0b
commit fdb1ec7c91
No known key found for this signature in database
29 changed files with 699 additions and 1157 deletions

View file

@ -7,19 +7,11 @@ An overview of the NEScript compiler internals for contributors and maintainers.
## Pipeline
```
Source (.ne) --> Lexer --> Parser --> Analyzer --> IR Lowering --> Optimizer --> Codegen --> Assembler --> Linker --> ROM (.nes)
Source (.ne) --> Lexer --> Parser --> Analyzer --> IR Lowering --> Optimizer --> Codegen --> Peephole --> Linker --> ROM (.nes)
```
Each phase is a pure function (input to output) with no global state, making every stage independently testable.
```
.ne source ---------> Lexer -----> Parser -----> Analyzer ------>
(tokens) (AST) (annotated AST)
IR Lowering -----> Optimizer -----> Codegen -----> Linker -----> ROM
(IR) (optimized IR) (6502 insns) (.nes file)
```
### Phase Summary
| Phase | Input | Output | Responsibility |
@ -28,53 +20,52 @@ IR Lowering -----> Optimizer -----> Codegen -----> Linker -----> ROM
| **Parser** | Token stream | AST | Syntax validation, tree construction |
| **Analyzer** | AST | Annotated AST | Type checking, scope resolution, call graph analysis |
| **IR Lowering** | Annotated AST | NEScript IR | Flatten expressions, expand u16 ops, desugar |
| **Optimizer** | IR | Optimized IR | Constant folding, dead code, ZP promotion, inlining |
| **Codegen** | Optimized IR | 6502 instruction list | Register allocation, instruction selection |
| **Assembler** | 6502 instructions | Byte sequences + fixups| Opcode encoding, address resolution |
| **Linker** | Bytes + assets | .nes file | Bank layout, vectors, iNES header |
| **Optimizer** | IR | Optimized IR | Constant folding, dead code, strength reduction, inlining |
| **Codegen** | Optimized IR | 6502 instruction list | Slot allocation, instruction selection |
| **Peephole** | 6502 instructions | 6502 instructions | Dead-load elimination, branch folding, INC/DEC fold |
| **Linker** | Instructions + assets | .nes file | Bank layout, vectors, iNES header |
---
## Modules
Each module has a `mod.rs` (implementation) and a co-located `tests.rs` with unit tests.
### `lexer/`
Tokenizes NEScript source text into a stream of typed tokens with source spans. Handles decimal, hex, and binary integer literals, string literals, all keywords, and operators.
`mod.rs`, `token.rs`, `tests.rs`. Tokenizes NEScript source into a stream of typed tokens with source spans. Handles decimal/hex/binary integer literals, string literals, keywords, operators, and the raw-capture mode for `asm { ... }` bodies.
### `parser/`
Recursive descent parser that converts the token stream into an AST. Defines all AST node types (expressions, statements, declarations) in `ast.rs`.
`mod.rs`, `ast.rs`, `preprocess.rs`, `tests.rs`. Recursive descent parser that converts the token stream into an AST. `ast.rs` defines every AST node type (expressions, statements, declarations). `preprocess.rs` inlines `include "path.ne"` directives before parsing.
### `analyzer/`
Performs semantic analysis on the AST: type checking (`types.rs`), scope and symbol table management (`scope.rs`), and call graph construction with depth analysis (`call_graph.rs`). Detects recursion, type mismatches, undefined references, and call depth violations.
`mod.rs`, `tests.rs`. Performs semantic analysis on the AST: type checking, scope and symbol table management, call graph construction with depth analysis, state reachability, unused-variable detection, and dead-code-after-terminator warnings. Emits all user-facing diagnostics beyond lexer/parser syntax errors.
### `ir/`
Defines the intermediate representation and the lowering pass (`lowering.rs`) that translates the annotated AST into IR. Flattens nested expressions, expands 16-bit operations into 8-bit sequences, and resolves syntactic sugar.
`mod.rs` (types), `lowering.rs` (AST → IR), `tests.rs`. The IR is a flat, register-agnostic representation built from virtual temps and basic blocks. Lowering flattens nested expressions, expands 16-bit operations, desugars `for` into `while`, and resolves constant expressions early.
### `optimizer/`
Runs optimization passes over the IR: constant folding (`const_fold.rs`), dead code elimination (`dead_code.rs`), zero-page promotion analysis (`zp_promote.rs`), and function inlining (`inliner.rs`).
`mod.rs`, `tests.rs`. Runs passes over the IR in order: strength reduction (mul/div by powers of two, mod by powers of two, ShiftVar → ShiftLeft/Right), function inlining (trivial-function elimination), constant folding (arithmetic + comparisons + shifts), and dead code elimination.
### `codegen/`
Translates optimized IR into 6502 instructions. Includes register allocation for the A/X/Y registers (`regalloc.rs`) and instruction pattern matching for idiomatic 6502 code (`patterns.rs`).
`ir_codegen.rs`, `peephole.rs`, `mod.rs`. `ir_codegen.rs` walks the optimized IR and emits 6502 instructions; variables land in allocated addresses and IR temps land in a recycling zero-page slot pool. `peephole.rs` runs after codegen to clean up the temp-heavy output (dead-load elimination, branch folding, INC/DEC fold, copy propagation).
### `asm/`
The built-in assembler. Encodes 6502 instructions (`encode.rs`) with all addressing modes (`addressing.rs`), using a complete opcode table (`opcodes.rs` -- 56 instructions across all modes).
`mod.rs`, `opcodes.rs`, `inline_parser.rs`, `tests.rs`. The built-in assembler and the inline-asm parser. `opcodes.rs` defines the 6502 opcode table with addressing modes. `inline_parser.rs` parses the body of `asm { ... }` blocks so codegen can splice real instructions in-line.
### `linker/`
Assigns addresses to code and data segments, resolves fixups/relocations (`fixups.rs`), and handles bank allocation (`banks.rs`) for banked mappers.
`mod.rs`, `tests.rs`. Assigns addresses to code and data segments, resolves label/symbol fixups, lays out banks for banked mappers (MMC1/UxROM/MMC3), and emits the final iNES byte stream via `rom::RomBuilder`.
### `rom/`
Builds the final iNES ROM file. Generates the 16-byte iNES header (`header.rs`) and places the NMI/RESET/IRQ vector table (`vectors.rs`).
`mod.rs`, `tests.rs`. Builds the final iNES ROM file. Generates the 16-byte iNES header and places the NMI/RESET/IRQ vector table.
### `runtime/`
Contains built-in runtime code that the compiler emits into every ROM: NES hardware initialization, NMI handler generation, controller read routines, OAM DMA setup, software multiply/divide, and the frame-walking audio driver (`gen_audio_tick`, `gen_period_table`, `gen_data_block`).
`mod.rs`, `tests.rs`. Contains built-in runtime code emitted into every ROM: NES hardware init, NMI handler, controller reads, OAM DMA, software multiply/divide, the frame-walking audio driver (`gen_audio_tick`, `gen_period_table`, `gen_data_block`), and the MMC1/UxROM/MMC3 mapper init and bank-switch helpers.
### `assets/`
The asset pipeline. Converts PNG images to CHR tile data (`chr.rs`), maps RGB colors to the NES palette (`palette.rs`), resolves `sprite`/`background` declarations into tile-indexed CHR blocks (`resolve.rs`), and compiles `sfx`/`music` declarations into ROM-ready envelope and note-stream byte tables (`audio.rs`) — plus the builtin effect/track tables used as fallbacks when programs reference audio names without declaring them.
### `debug/`
Debug instrumentation output. Generates source maps relating ROM addresses to source locations (`source_map.rs`), symbol tables compatible with Mesen (`symbols.rs`), and runtime check code for debug builds (`checks.rs`).
`mod.rs`, `chr.rs`, `palette.rs`, `resolve.rs`, `audio.rs`, `tests.rs`. The asset pipeline. `chr.rs` converts PNGs to CHR tile data. `palette.rs` maps RGB to NES palette indices. `resolve.rs` resolves `sprite` declarations into tile-indexed CHR blocks. `audio.rs` compiles `sfx`/`music` declarations into ROM-ready envelope and note-stream byte tables, plus the builtin effect/track tables used when programs reference audio names they haven't declared.
### `errors/`
Error reporting infrastructure. Defines the `Diagnostic` struct with error codes, severity levels, source spans, labels, help text, and notes (`diagnostic.rs`). Renders diagnostics with color and source context for terminal output (`render.rs`).
`mod.rs`, `diagnostic.rs`, `render.rs`. Defines the `Diagnostic` struct (error codes, severity, spans, labels, help text). Renders diagnostics with color and source context for terminal output using ariadne.
---
@ -82,7 +73,7 @@ Error reporting infrastructure. Defines the `Diagnostic` struct with error codes
### Test Organization
Tests are co-located with each module in `tests.rs` files:
Tests are co-located with each module in `tests.rs` files under `src/`:
```
src/lexer/tests.rs -- lexer unit tests
@ -90,20 +81,14 @@ src/parser/tests.rs -- parser unit tests
src/analyzer/tests.rs -- semantic analysis tests
src/ir/tests.rs -- IR lowering tests
src/optimizer/tests.rs -- optimizer tests
src/codegen/tests.rs -- code generation tests
src/asm/tests.rs -- assembler tests
src/linker/tests.rs -- linker tests
src/rom/tests.rs -- ROM builder tests
src/runtime/tests.rs -- runtime code emission tests
src/assets/tests.rs -- asset pipeline tests
```
Integration tests live in the `tests/` directory:
```
tests/integration/ -- full pipeline tests with .ne source files
tests/error_tests/ -- tests that verify specific error codes
tests/asm_tests/ -- 6502 opcode and addressing mode tests
```
End-to-end and error-code tests live in `tests/integration_test.rs`, which compiles representative `.ne` snippets through the full pipeline and asserts on ROM/diagnostic shape. The emulator smoke test (`tests/emulator/run_examples.mjs`) runs every example through `jsnes` and byte-compares the resulting screenshot and audio hash against goldens in `tests/emulator/goldens/`.
### Running Tests
@ -117,7 +102,7 @@ cargo test --lib parser
cargo test --lib analyzer
# Run integration tests only
cargo test --test '*'
cargo test --test integration_test
# Run a specific test by name
cargo test test_name
@ -125,6 +110,4 @@ cargo test test_name
### Test Strategy
Each compiler phase is designed as a pure function, so unit tests provide isolated input and verify output without side effects. Integration tests compile complete `.ne` source files and verify the output ROM matches expected golden files in `tests/integration/expected/`.
Error tests in `tests/error_tests/` contain intentionally broken programs and verify that the correct error code is produced (e.g., `recursion.ne` should produce `E0402`).
Each compiler phase is a pure function, so unit tests provide isolated input and verify output without side effects. Integration tests compile complete `.ne` programs and either assert on shape (length, presence of specific labels) or byte-compare output against checked-in goldens. Emulator goldens catch regressions that pass type-check but corrupt the final executable image.

View file

@ -1,524 +1,180 @@
# Future Work
This document catalogs known gaps, incomplete features, and planned improvements
in the NEScript compiler. Items are organized by priority and area.
This document tracks the gaps between what NEScript currently compiles and
what the spec describes. Items are grouped by area. Anything implemented and
tested is omitted — `git log` is the authoritative record of what shipped.
---
## 1. IR-Based Code Generation
## Runtime palette / nametable updates
**Status**: Complete. The AST → IR lowering, optimizer, and
`src/codegen/ir_codegen.rs` all work end-to-end; the legacy AST
codegen has been removed. See "Recently completed" below.
**Status.** Parsed away. Previously the compiler accepted `palette Name
{ colors: [...] }`, `background Name { chr: @chr(...) }`, `load_background
Name`, and `set_palette Name` statements, but the lowering silently dropped
them: the declarations were never resolved into CHR or palette blobs and the
statements emitted zero instructions. They were removed during cleanup to
avoid quietly misleading users.
**What a proper implementation needs.**
- New AST declarations for palette and background/nametable data, with
analyzer validation for color indices and nametable dimensions.
- An asset pipeline pass that compiles PNG or inline byte data into a
ROM-resident blob with a known label.
- Runtime helpers that write to PPU `$2006/$2007` during vblank from a
source pointer.
- IR ops `LoadBackground(BlobId)` and `SetPalette(BlobId)` that the
codegen emits inside an NMI-safe window.
- A `--memory-map` breakdown that shows palette and nametable budgets,
since they eat into the PRG-ROM cost model that `memory_map` reports
for CHR bytes.
Until this exists, programs should use `poke(0x2006, ...)` / `poke(0x2007, ...)`
directly inside an `on frame` handler (runs immediately after vblank) to push
palette or nametable updates.
---
## 3. Sprite Name Resolution
## User code distribution across switchable banks
**Status**: `draw SpriteName at: (x, y)` parses the sprite name but codegen
ignores it. All draws use OAM slot 0 with CHR tile index 0 (the built-in smiley).
**Status.** `mapper: MMC1 / UxROM / MMC3` plus top-level `bank Name { prg }`
declarations are honored by the iNES header and by the linker, which reserves
each declared bank as a 16 KB switchable slot. However, the IR codegen puts
every user function and state handler into the fixed bank at `$C000-$FFFF`
the declared banks exist only as empty space. Programs outgrowing the fixed
16 KB have nowhere to put their code.
**What's needed**:
- Track a mapping from sprite name → CHR tile index in the linker
- When a `sprite` declaration provides inline CHR data or `@chr("file.png")`,
write that data to the CHR ROM at a known tile index
- In codegen, look up the sprite name to get the tile index and write it to
OAM byte 1 (tile number)
- Support multiple OAM slots: track a `next_oam_slot` counter per frame,
allocate slots 0..63 as draws are emitted, warn if >64
**What's needed.**
- A bank-assignment step (analyzer or a new pass) that maps each user function
/ state handler to a target bank, either via explicit `bank Foo { fun bar()
... }` nesting or by greedy size-packing.
- Codegen support for emitting into non-fixed banks and for generating
cross-bank trampolines (the runtime helper scaffold already exists in
`runtime/mod.rs::gen_bank_trampoline`; it just isn't invoked).
- Linker changes so that functions in a switchable bank are found by the
JSR fix-up logic when the call crosses bank boundaries.
---
## 4. Multi-OAM Sprite Support
**Status**: Every `draw` writes to the same OAM bytes ($0200-$0203). Only one
sprite is visible at a time.
**What's needed**:
- Frame-level OAM slot allocator: each `draw` gets the next available 4-byte slot
- Counter reset at the start of each frame handler
- Warn at compile time if a frame handler has >64 static draws
- Runtime: clear unused OAM slots to Y=$FE (off-screen) at frame start
---
## 5. State Machine Dispatch
**Status**: The codegen only generates code for the `start` state. Other states
are parsed and analyzed but their frame handlers are never called.
**What's needed**:
- A `current_state` zero-page variable holding the active state index
- A dispatch table at the start of the main loop: load `current_state`, branch to
the corresponding state's frame handler
- `transition StateName` writes the new state index to `current_state`
- Generate `on_enter` call on transition, `on_exit` call before leaving
---
## 6. Include Directive
**Status**: The `include` keyword is lexed (`KwInclude`) but not parsed or
implemented.
**What's needed**:
- Parser: `include "path.ne"` at top level
- File resolution: relative to the including file's directory
- Circular include detection (track include stack)
- Merge included declarations into the main `Program` AST
- Span tracking: included files need their own `file_id` for error messages
---
## 7. Debug Mode
**Status**: `--debug` flag is accepted by the CLI but has no effect.
**What's needed**:
- `debug.log(expr, ...)`: write values to emulator debug port ($4800)
- `debug.assert(expr)`: emit runtime check, halt on failure
- `debug.overlay(x, y, text)`: render text to a reserved nametable region
- Array bounds checking in debug mode (compare index against array size)
- Frame overrun detection: count cycles per frame, warn if approaching vblank
- All debug code stripped in release mode (already designed: `DebugLog`/`DebugAssert`
in the `Statement` enum are defined in the spec but not yet in the AST)
---
## 8. Scroll Hardware Writes
**Status**: `scroll(x, y)` is parsed but produces no output.
**What's needed**:
- Write X scroll value to PPU register $2005
- Write Y scroll value to PPU register $2005 (second write)
- Must happen during vblank (inside NMI handler or after `wait_frame`)
- Split-screen scroll requires MMC3 scanline IRQ (`on scanline`)
---
## 9. Asset Pipeline Completion
**Status**: PNG → CHR conversion exists (`src/assets/chr.rs`) but is never called
from the compilation pipeline.
### 9a. Wire `@chr("file.png")` to actual PNG loading
- When a sprite/background declares `chr: @chr("path.png")`, call `png_to_chr()`
during compilation
- Resolve the path relative to the source file
- Store resulting CHR data in the ROM's CHR section at a known tile index
### 9b. Wire `@binary("file.bin")` to raw file inclusion
- Read the file as raw bytes and include in CHR or PRG ROM
### 9c. Palette extraction from PNG
- `@palette("file.png")`: analyze image colors, map to nearest NES palette entries
- Already have `nearest_nes_color()` in `src/assets/palette.rs`
### 9d. Nametable conversion
- Full 256×240 PNG → 960-byte nametable + 64-byte attribute table
- Tile deduplication (max 256 unique tiles per pattern table)
- Error if >256 unique tiles
---
## 10. Error Message Polish
**Status**: Errors work and render with ariadne, but many error paths use generic
messages.
### Unused error codes
These are defined in `ErrorCode` but never emitted:
- `E0202` — invalid cast
- `E0203` — invalid operation for type
- `E0301` — zero-page overflow
- `E0403` — unreachable state
- `E0505` — multiple start declarations
- `W0101` — expensive multiply/divide operation
- `W0102` — loop without break or wait_frame
- `W0103` — unused variable
- `W0104` — unreachable code after return/break/transition
### Missing validations
- No error for assigning to a `const`
- No error for `break`/`continue` outside a loop
- No warning for variables declared but never read
- No error for `return` with wrong type vs function signature
- No error for calling a function with wrong argument count/types
---
## 11. Scanline IRQ (MMC3)
**Status**: `on scanline(N)` is in the spec and `on_scanline` field exists in
`StateDecl`, but parsing and codegen are not implemented.
**What's needed**:
- Parser: `on scanline(N) { ... }` event handler in state bodies
- MMC3 IRQ setup: write scanline counter to $C000/$C001/$E000/$E001
- IRQ handler generation: branch to the scanline handler code
- Only valid with `mapper: MMC3`
---
## 12. Audio
**Status**: A full data-driven audio subsystem is in place. User programs
can declare `sfx Name { duty, pitch, volume }` blocks (frame-accurate
pulse-1 envelopes) and `music Name { duty, volume, repeat, notes }`
blocks (pulse-2 note streams with rests and looping). The resolver
compiles these into ROM-ready byte tables; the IR codegen emits
trigger sequences that load pointers into ZP; the runtime NMI tick
walks the envelope and note stream each frame, indexing into a
builtin 60-note period table. Builtin effects (`coin`, `jump`, `hit`,
`click`, `cancel`, `shoot`, `step`) and tracks (`theme`, `battle`,
`victory`, `gameover`) are synthesized from the same data path so
programs that don't declare their own audio still make sound.
Programs that touch no audio pay zero ROM or cycle cost — the whole
subsystem elides when the `__audio_used` marker is absent.
**Still TODO for richer audio**:
- Triangle/noise/DMC channels (currently only pulse 1 and 2 are used)
- Multi-channel tracker playback (one `notes` list per channel)
- `@sfx("file.nsf")` / `@music("file.ftm")` asset directives
- FamiTracker export format parsing
- Per-note pitch changes within a sfx (currently pitch is latched once)
---
## 13. Language Features (Post-v0.1)
## Language feature gaps (post-v0.1)
From the spec's "Reserved for Future Versions" section:
| Feature | Description |
|---------|-------------|
| **Structs** | `struct Vec2 { x: u8, y: u8 }` — composite types with known layout |
| **Enums** | `enum Direction { Up, Down, Left, Right }` — mapped to u8 values |
| **Fixed-point** | `fixed8.8` type for sub-pixel movement |
| **Text/HUD** | Font sheet declarations, layout system for scores/health/menus |
| **Metasprites** | Multi-tile sprite groups with relative positioning |
| **Tilemaps** | Declarative level data with collision queries |
| **SRAM/saves** | Persistent storage declarations for battery-backed save data |
| **NES 2.0** | Extended iNES header format |
| Feature | Description |
|--------------------|-----------------------------------------------------------------------|
| **Fixed-point** | `fixed8.8` type for sub-pixel movement with operator support. |
| **Text / HUD** | Font sheet declarations + layout system for scores, health, menus. |
| **Metasprites** | Multi-tile sprite groups with relative positioning. |
| **Tilemaps** | Declarative level data with built-in collision queries. |
| **SRAM / saves** | Persistent storage declarations for battery-backed save data. |
| **NES 2.0 header** | Extended iNES header format for additional metadata. |
### Struct / array field widths
Struct and array element types are currently restricted to the single-byte
primitives (`u8`, `i8`, `bool`). `u16`, nested struct fields, and array fields
are rejected with `E0201`. The analyzer's field-layout machinery needs to
grow multi-byte offsets, and IR lowering needs to treat wide fields as the
existing wide-var path already does for `u16` globals.
---
## 14. Inline Assembly
## Audio pipeline
**Status**: `asm { }` blocks are lexed (`KwAsm`, `AsmBody`) but not parsed or
compiled. The lexer has raw-capture mode for asm content.
**What ships today.** Frame-walking pulse driver with `sfx Name { duty, pitch,
volume }` and `music Name { duty, volume, repeat, notes }` blocks; builtin
effects and tracks; a 60-entry period table; `__audio_used` marker that
elides the whole subsystem when no program statement references it.
**What's needed**:
- Parser: capture asm body text, parse `{variable_name}` substitutions
- Codegen: emit raw 6502 instructions with variable address substitution
- Labels: local to the asm block scope
- `raw asm { }` variant with no substitution or safety checks
**Still TODO for richer audio.**
- Triangle / noise / DMC channels (today the driver only uses pulse 1 and
pulse 2).
- Multi-channel tracker playback (one `notes` list per channel).
- `@sfx("file.nsf")` / `@music("file.ftm")` asset directives — neither the
NSF nor the FamiTracker format is parsed yet.
- Per-note pitch changes within a sfx (currently `pitch` latches once at
trigger time).
---
## 15. Compiler Performance
## Debug instrumentation
**Status**: Compilation is fast (<100ms for all examples) but has no benchmarks.
**What ships today.** `debug.log(...)` and `debug.assert(...)` lower to $4800
writes when `--debug` is passed, and are stripped entirely in release builds.
**What's needed**:
- `cargo bench` benchmarks for each pipeline phase
- Regression test: compilation must stay under 500ms for any reasonable project
- Profile-guided optimization of hot paths (lexer, parser)
**Not yet implemented.**
- Mesen-compatible symbol export (`.mlb` / `.sym` files) — the CLI does not
emit them, and the previous `DebugSymbols` helper was removed as dead code
during cleanup.
- Source maps relating ROM addresses to source lines — the `SourceLoc`
IR op exists but is not consumed by the linker or CLI.
- Array bounds checking in debug mode.
- Frame overrun detection (cycles-per-frame counting).
- `debug.overlay(x, y, text)` — needs the text/HUD subsystem above.
---
## 16. WASM Build Target
## Code quality / tooling
**What's needed**:
- Factor out all file I/O behind a trait (`FileSystem` / `VFS`)
- Core pipeline takes `&str` source → `Vec<u8>` ROM with no filesystem access
- Compile the compiler to WASM for a browser-based IDE
- In-browser NES emulator integration for instant preview
### Register allocator
All IR temps currently spill to a recycled zero-page slot (`$80-$FF`). The
peephole pass mops up the most obvious waste, but a real CFG-aware allocator
that holds short-lived temps in `A`/`X`/`Y` would cut a noticeable number of
LDA/STA pairs.
### Cross-block temp live-range analysis
The slot recycler is function-local per-block. Temps that flow across block
boundaries get a dedicated slot for the entire function, even if a later
block could reuse the slot.
### `--no-opt` flag
There is no way to disable the optimizer from the CLI. Adding one would make
optimizer-introduced bugs easier to bisect.
### Compilation benchmarks
Compilation is fast (<100 ms for every example today) but has no `cargo
bench` harness, so regressions would slip through.
### WASM build target
To build a browser IDE we would need to route file I/O through a trait so the
core pipeline works on `&str → Vec<u8>` without touching `std::fs`. Today the
parser's preprocess pass and the asset resolver read files directly.
---
## 17. Open Design Questions
## Error message polish
From the engineering plan:
### Unused error codes
1. **Inline asm label syntax**: `.label:` (ca65 style) vs `label:` (generic)?
2. **Debug port address**: $4800 is conventional but not universal. Support
multiple debug output methods?
3. **OAM allocation strategy**: Sequential allocation vs priority-based with
`ErrorCode` only defines codes that are actually emitted. Previously there
were placeholder variants (`E0202` invalid cast, `E0403` unreachable state)
marked `#[allow(dead_code)]`; those were removed during cleanup. If those
semantics come back, add the codes at that point.
### Missing diagnostics
- No warning for implicit-drop of a function return value (`my_fun()` at
statement position when `my_fun` returns non-void).
- `W0102` ("loop without yield") is only emitted for bare `loop`, not for
`while true` or `loop { if cond { continue } }`.
- No warning for `fast` variables that never justify the zero-page slot
(could cross-reference access counts).
---
## Open design questions
1. **Inline asm label syntax.** `.label:` (ca65 style) vs `label:` (generic)?
Today the inline-asm parser accepts `label:` but not `.label`; migrating
would be cheap but would invalidate any copy-pasted ca65 fragments.
2. **Debug port address.** $4800 is conventional but not universal. Should
we support multiple debug output methods?
3. **OAM allocation strategy.** Sequential allocation vs priority-based with
automatic sprite cycling for the 8-per-scanline limit?
4. **Error recovery granularity**: How aggressively should the parser recover?
More recovery = more errors per compile, but risk of cascading false errors.
---
## 18. Missing Assignment Operators
`<<=` and `>>=` (shift-assign) are lexed as `ShiftLeftAssign`/`ShiftRightAssign`
tokens but have no corresponding variants in the `AssignOp` AST enum. They can
never appear in parsed code. Adding them requires:
- New `AssignOp::ShiftLeftAssign` and `AssignOp::ShiftRightAssign` variants
- Parser handling in `parse_assign_or_call` (alongside the other compound ops)
- Codegen: load value, shift, store back
---
## 19. Player 2 Controller
`Player::P1` and `Player::P2` are defined in the AST but marked `#[allow(dead_code)]`.
The parser always produces `ButtonRead(None, ...)` — it never parses `p1.button.X`
or `p2.button.X` syntax. The runtime only reads controller 1 ($4016).
**What's needed**:
- Parser: `p1.button.X` and `p2.button.X` syntax producing `Player::P1`/`P2`
- Runtime: read controller 2 from $4017 into a second zero-page byte
- Codegen: select the correct input byte based on player
---
## 20. Register Allocator
The plan describes `src/codegen/regalloc.rs` for managing A/X/Y allocation, but
no register allocator exists. The current codegen uses A for everything and
spills to zero-page $02 for comparisons. A proper allocator would:
- Track A/X/Y liveness across basic blocks
- Use X for array indexing and loop counters
- Use Y as secondary index
- Spill to zero-page temps only when all three are live
---
## Priority Order
### Recently completed (removed from backlog)
These items were documented as future work but have since been implemented:
- **Full audio subsystem**`src/runtime/mod.rs::gen_audio_tick`,
`gen_period_table`, and `gen_data_block` implement a frame-walking
pulse driver. `src/assets/audio.rs` compiles user `sfx`/`music`
declarations (and builtins referenced via `play coin` etc.) into
ROM-ready envelope and note-stream byte tables. `IrCodeGen::with_audio`
threads the compile-time trigger constants into `play`/`start_music`,
which emit pointer loads against per-blob labels. The linker splices
driver body, period table, and every blob into PRG gated on the
`__audio_used` marker so silent programs pay no cost. Full
parser/analyzer/codegen/linker/runtime test coverage.
- **u16 arithmetic and comparisons** — new IR ops `LoadVarHi`,
`StoreVarHi`, `Add16`, `Sub16`, `CmpEq16` through `CmpGtEq16`. The
lowering context tracks variable types via the analyzer's symbol
table and routes each expression through the 8-bit or 16-bit path
based on operand width. Initializers, compound assignments, and
comparisons all preserve both bytes. The codegen emits
`CLC;ADC;ADC` for Add16 with carry propagating naturally, and
compare-high-then-compare-low dispatch for the six comparison
variants.
- **Multi-scanline on_scanline per state**`gen_scanline_irq` now
dispatches on `(current_state, ZP_SCANLINE_STEP)` and reloads the
MMC3 counter with the delta to the next scanline in the same
state. `gen_scanline_reload` resets the step counter at the top
of each NMI so a state with multiple handlers fires them in
ascending line order.
- **IR temp slot recycling**`build_use_counts` pre-scans each
function to count per-temp uses; `retire_op_sources` decrements
the counts after each op runs and pushes dead slots back onto
`free_slots` for later allocation. Previously, `bitwise_ops.ne`
crashed (debug) or silently miscompiled (release) once it
allocated more than 128 concurrent temps. With recycling the
same function now uses ~4 slots instead of 136.
- **INC/DEC peephole fold**`fold_inc_dec` collapses
`LDA addr; CLC; ADC #1; STA addr` into a single `INC addr`
(and the SEC/SBC/#1 variant into `DEC addr`). Saves 5 bytes and
5 cycles per increment. The fold is suppressed when the next
instruction is a carry-dependent branch (`BCC`/`BCS`) since
INC/DEC don't update the carry flag.
- **Peephole dead-load elimination across passive ops** — the
old `remove_dead_loads` only dropped an LDA if the very next
instruction was another A-writer. Now it walks past
INC/DEC/STX/STY (which don't touch A) to find the actual
next A-use, catching more dead loads produced by copy
propagation.
- **State machine dispatch** — CMP + BNE + JMP trampoline dispatch table,
`current_state` in ZP $03, on_enter/on_exit handlers as JSR targets
- **Function call codegen** — JSR to function labels, ZP $04-$07 param passing,
RTS for returns
- **Break/continue** — loop_stack with JMP to continue/break labels
- **Return** — evaluate expr to A + RTS
- **Transition** — write state index + JMP to main loop
- **Array indexing** — TAX + LDA/STA with ZeroPageX or AbsoluteX
- **Scroll** — PPU $2005 writes (X then Y)
- **Multiply/divide/modulo** — JSR __multiply/__divide with shift-and-add/restoring division
- **Shift left/right** — ASL A / LSR A
- **Multi-OAM sprites** — sequential slot allocation (0-63), reset per frame
- **Const assignment error** — E0203 for assigning to constants
- **Break outside loop error** — E0203 for break/continue without enclosing loop
- **Math routines wired into linker** — gen_multiply/gen_divide included in ROM
- **Sprite name resolution** — sprite declarations map to CHR tile indices,
draw statements use the correct tile number
- **Inline sprite CHR data** — sprite decls with `chr: [0x..., ...]` work
- **Include directive**`include "path"` inlines files at parse time,
with circular include detection
- **Shift-assign operators**`<<=` and `>>=` work in all contexts
- **Player 2 controller**`p1.button.X` / `p2.button.X` syntax, P2 input
read from $4017 into ZP $08
- **Unused variable warning** — W0103 emits for declared-but-never-read
globals (underscore-prefix silences)
- **Unreachable state warning** — W0104 emits for states not reachable from
the start state via transitions
- **E0502 "did you mean" suggestions** — undefined variable errors include
a suggestion for nearby-named symbols
- **debug.log / debug.assert** — parses into `Statement::DebugLog` /
`Statement::DebugAssert`, codegen emits runtime writes to $4800 when
`--debug` is set, stripped in release mode
- **--debug CLI flag wired** — threads through `IrCodeGen::with_debug`
- **IR-based codegen**`src/codegen/ir_codegen.rs` walks `IrProgram` and
emits 6502 for every IR op: load/store, arithmetic, comparisons, arrays,
calls, draws, input (P1 and P2), scroll, debug.log/assert, state
dispatch, runtime OAM cursor for looped draws, transitions + on_enter
handlers. It's the only codegen — the legacy AST-based path and the
`--use-ast` flag were removed once the IR pipeline was proven correct
by the jsnes emulator smoke test.
- **IR lowering bug fixes**`ReadInput` now has a destination temp,
`ButtonRead` uses the proper input temp, logical AND/OR use a new
`emit_move` helper instead of the buggy raw VarId temp storage
- **IR Player 2 controller**`ReadInput(temp, player)` selects $01
or $08 based on player index
- **IR scroll support**`scroll(x, y)` lowers to `IrOp::Scroll(x, y)`
which emits two PPU $2005 writes in IR codegen
- **IR debug.log / debug.assert** — new `IrOp::DebugLog(temps)` and
`IrOp::DebugAssert(cond)` variants, emitted as $4800 writes in
debug mode and stripped in release
- **Asset pipeline @binary / @chr loading**`resolve_sprites()` reads
raw binary files and converts PNGs via `png_to_chr()`. Missing files
are silently skipped (documentation-friendly)
- **Call arity validation** — E0203 when `Statement::Call` or
`Expr::Call` has the wrong number of arguments or a mismatched
argument type (uses a `function_signatures` map)
- **Return type validation**`return value` is type-checked against
the function's declared return type (E0201); returning a value from a
void function emits E0203
- **W0102 loop-without-yield warning** — emitted when a `loop { ... }`
body contains no `break`, `return`, `transition`, or `wait_frame`
- **W0101 expensive mul/div/mod warning** — flags multiply/divide/modulo
with two non-constant operands; literal operands are silent because
the optimizer strength-reduces them
- **W0104 dead-code-after-terminator warning** — statements after
`return`, `break`, `continue`, or `transition` in the same block
emit W0104 with a label pointing at the terminator
- **E0301 RAM overflow** — the zero-page user region is now bounded
above by `$80` (leaving `$80-$FF` for IR temps) and the main RAM
allocator stops at `$0800`; overflow emits E0301 at the declaration
- **E0505 multiple start declarations** — parser rejects a second
`start X` token
- **`on scanline(N)` handlers** — `state { on scanline(240) { ... } }`
parses and populates `StateDecl::on_scanline`; analyzer emits E0203
if the game isn't using MMC3. The IR codegen now emits the full
MMC3 IRQ vector glue: per-state dispatch in `__irq_user` and a
`__ir_mmc3_reload` helper that picks the right `$C000` latch value
based on `current_state`. See `examples/scanline_split.ne` and
`examples/mmc3_per_state_split.ne`.
- **Inline assembly**`asm { ... }` blocks. The lexer captures the
body as a raw `AsmBody` token; `src/asm/inline_parser.rs` provides a
minimal 6502 mnemonic parser that handles every addressing mode the
codegen emits. The IR codegen splices parsed instructions directly
into the output stream
- **Enum types**`enum Name { V1, V2, ... }` declares u8 constants
with values equal to declaration order. Variant names are flattened
into the global symbol table
- **Struct types**`struct Vec2 { x: u8, y: u8 }` with field
access (`pos.x = 5`) and contiguous u8-only field layout. The
analyzer synthesizes per-field VarAllocations so the rest of the
compiler treats field access as ordinary variable access.
- **`for i in start..end { ... }` loops** — half-open range with a
u8 index variable. Desugared in IR lowering to a while loop with
a proper continue-edge block so `break`/`continue` work.
- **Audio subsystem** — full data-driven pulse driver with
`sfx`/`music` block declarations, builtin effects, and an
NMI-time tick that walks envelope and note-stream tables.
See section 12 above for the full writeup.
- **Constant expression folding**`const B: u8 = A + 3` evaluates
at compile time and feeds through to variable initializers too.
- **`on scanline` codegen (minimal)** — MMC3 IRQ setup at startup
plus a `__irq_user` dispatcher that saves registers, ACKs via
`$E000`, dispatches on `current_state` to the right scanline
handler, and restores. `__ir_mmc3_reload` helper re-arms the
counter each frame from NMI.
- **Peephole optimizer**`src/codegen/peephole.rs` runs to fixed
point after codegen. Current passes:
- copy propagation for IR temps (rewrites loads to their source)
- dead-LDA elimination (drops overwritten LDAs)
- redundant STA/LDA pair removal
- LDA-then-STA-same-address removal
- dead-store elimination for IR temp slots (function-wide scan)
- A-value tracking eliminates redundant LDAs (ZP and absolute)
- branch folding: `Bxx L; JMP M; L:``Byy M`
- dead JMP to next label removal
- **`--dump-ir` CLI flag** — prints the lowered IR program after the
optimizer pass for debugging
- **Function-local variables** — IR codegen allocates backing
storage for `var`s declared inside function bodies, using a
per-function RAM range at `$0300+` so nested calls don't clobber
each other.
- **E0502 on assignment to undefined variable** — previously was
silently creating a new variable.
- **Function call ABI fix** — IR codegen was JSRing to `__fn_name`
but functions were defined as `__ir_fn_name`, and param VarIds
weren't in `var_addrs` so callees read temp slots instead of
parameters. Both bugs are now fixed with an integration test
guard.
- **Struct literal syntax**`Vec2 { x: 100, y: 50 }` in both
variable initializers and assignments. Desugars in lowering to
per-field stores. Restricted to non-condition expression contexts
(if/while/for conditions) to avoid ambiguity with block `{`.
- **Match statement**`match x { pat => body, _ => default }`
parses to an if/else-if chain at parse time, so no new AST
variant is needed. Supports any expression patterns and an
underscore catch-all.
- **For loops**`for i in start..end { body }` (half-open range).
Desugars in IR lowering to a while loop with a proper
continue-edge block.
- **Semicolon statement separators** — short statements can share
a line: `a += 1; b += 2`.
- **Inline asm `{var}` substitution** — inside an `asm { ... }`
block, `{name}` is replaced with the hex address of the variable
`name`. The lexer balances nested braces so `{counter}` inside
an asm body is captured correctly.
- **`raw asm { ... }` blocks** — variant of inline asm that skips
`{var}` substitution, passing the body through verbatim.
- **`poke(addr, value)` / `peek(addr)` intrinsics** — hardware
register access without needing an asm block. Compile to a
single LDA/STA against a compile-time-constant address.
- **`--memory-map` CLI flag** — prints a human-readable variable
allocation table showing what's in ZP vs main RAM.
- **`--call-graph` CLI flag** — prints a call-tree view with max
depth reached from each entry point handler.
### Remaining priority order
For someone picking up this codebase, the recommended order of work:
1. **u16 / array / nested struct fields** — u16 *globals* now work
end-to-end (load/store, +/-, comparisons all propagate through
16-bit IR ops). Struct fields and array elements are still u8-only
— the layout machinery needs to grow multi-byte field offsets.
2. **Triangle / noise / DMC channels** — the current audio engine
plays sfx on pulse 1 and music on pulse 2 with a full
data-driven tracker model (envelope walk, period table,
`(pitch, duration)` note streams, loop-back). Wiring triangle
and noise channels into the same model would unblock richer
multi-part compositions.
3. **Register allocator** — proper A/X/Y allocation to replace
zero-page spills used by the current IR codegen. Partially
mitigated by peephole passes + the new slot recycler, but still
wasteful in some cases (every temp spills to a ZP slot even if
its live range is one op wide).
4. **Text / HUD layer** — font sheet + layout system for scores.
5. **Cross-block temp live-range analysis** — the current slot
recycler is function-local; temps that flow across block
boundaries always get a dedicated slot for the full function.
A proper CFG-aware live range interference graph would let more
temps share slots.
6. **Peephole: drop LDA dead across unconditional JMPs** — after
the INC/DEC fold we sometimes leave an `LDA #1` whose value is
consumed by nothing before the next `JMP __ir_main_loop`. Local
analysis can't prove it's dead; a cross-block pass could.
4. **Error recovery granularity.** How aggressively should the parser
recover? More recovery means more errors per compile but also risks
cascading false errors.

View file

@ -641,22 +641,6 @@ Set the PPU scroll position:
scroll(scroll_x, scroll_y)
```
### Load Background
Load a background nametable:
```
load_background TitleBG
```
### Set Palette
Apply a palette:
```
set_palette GamePalette
```
### Function Calls as Statements
```
@ -680,29 +664,6 @@ sprite Coin {
}
```
### Palette Declarations
Define color palettes using NES PPU color indices (`$00`-`$3F`):
```
palette GamePalette {
colors: [0x0F, 0x00, 0x10, 0x30,
0x0F, 0x07, 0x17, 0x27,
0x0F, 0x09, 0x19, 0x29,
0x0F, 0x01, 0x11, 0x21]
}
```
Each group of 4 values is a sub-palette. The first color is typically `0x0F` (black) as the shared background color.
### Background Declarations
```
background TitleBG {
chr: @chr("assets/title_screen.png")
}
```
### Asset Sources
Three ways to provide asset data:
@ -964,7 +925,6 @@ reference NEScript variables.
| Code | Description |
|--------|----------------------------|
| E0201 | Type mismatch |
| E0202 | Invalid cast |
| E0203 | Invalid operation for type |
### Memory Errors (E03xx)
@ -979,7 +939,6 @@ reference NEScript variables.
|--------|----------------------------|
| E0401 | Call depth exceeded |
| E0402 | Recursion detected |
| E0403 | Unreachable state |
| E0404 | Transition to undefined state |
### Declaration Errors (E05xx)
@ -999,7 +958,7 @@ reference NEScript variables.
| W0101 | Expensive multiply/divide operation |
| W0102 | Loop without break or wait_frame |
| W0103 | Unused variable |
| W0104 | Unreachable code |
| W0104 | Unreachable code (after return/break/transition, or state unreachable from start) |
### Example Error Output