1
0
Fork 0
mirror of https://github.com/imjasonh/nescript synced 2026-07-08 08:55:38 +00:00

pipeline: share a single compile function across CLI, bench, and tests

The compile bench had a hand-maintained parallel copy of
`src/main.rs::compile`, and that copy went out of sync after
bank switching landed — the bench kept handing the linker
`PrgBank::empty(...)` slots even though the CLI started
populating per-bank instruction streams + trampoline requests.
The assembler then panicked with `unresolved label:
'__tramp_step_animation'` on `uxrom_user_banked.ne` under
`cargo test --all-targets`, which is what CI runs. A plain
`cargo test --release` (what CLAUDE.md used to document) never
builds the bench so the bug slipped through local validation.

Fix:

- New `nescript::pipeline` module with `compile_source(source,
  source_dir, &CompileOptions)` that owns the full
  `parse → analyze → lower → optimize → codegen → peephole →
  link` pipeline including the per-bank stream + trampoline
  reconstruction. Returns a `CompileOutput` carrying the ROM,
  the linker result, analysis, IR, assets, instructions, and
  source-loc markers so downstream tools have one place to
  pull metadata from.
- `src/main.rs::compile` reduces to file I/O + preprocessing +
  a single `compile_source` call + CLI-only side effects
  (`--dump-ir`, `--call-graph`, `--asm-dump`, `--memory-map`,
  `--symbols`, `--source-map`).
- `benches/compile.rs::compile_pipeline` becomes a one-line
  `compile_source` call. It is now structurally impossible for
  the bench to drift from the CLI path.
- `tests/integration_test.rs::compile_with_debug_artifacts`
  likewise delegates to `compile_source`. This also fixes a
  latent bug in the helper where it used `Linker::with_mapper`
  without `.with_header(...)` — programs opting into
  `header: nes2` would have quietly got an iNES 1.0 header
  through this path.
- `CLAUDE.md`: updated the "Running the basics" section to
  specify `cargo test --all-targets` (plain `cargo test` skips
  benches) and to point at `scripts/pre-commit` with the exact
  install command. Also installed the hook in this worktree.

All 24 existing `examples/*.nes` rebuild byte-identical through
the new pipeline. 624 tests + all 25 emulator goldens pass.

https://claude.ai/code/session_01MaNVcDmK9gsspRkdxowQAM
This commit is contained in:
Claude 2026-04-14 13:02:58 +00:00
parent 889074a415
commit 2966a6ab17
No known key found for this signature in database
6 changed files with 362 additions and 359 deletions

View file

@ -1,30 +1,31 @@
//! End-to-end compilation benchmarks.
//!
//! Each `examples/*.ne` file becomes its own Criterion group that
//! times the full `parse → analyze → lower → optimize → codegen →
//! peephole → link` pipeline the `nescript build` CLI runs. The goal
//! is to catch compile-time regressions — today every example
//! compiles in well under 100 ms, so a change that doubles that
//! shows up as a large red bar in `cargo bench`'s output.
//! times the full `preprocess → parse → analyze → lower →
//! optimize → codegen → peephole → link` pipeline the `nescript
//! build` CLI runs. The goal is to catch compile-time regressions
//! — today every example compiles in well under 100 ms, so a
//! change that doubles that shows up as a large red bar in
//! `cargo bench`'s output.
//!
//! The harness pre-reads every source file into memory before any
//! measurement starts. Criterion's sample iterations then run only
//! the in-memory compile path, so disk I/O never shows up on the
//! hot loop.
//!
//! The bench calls [`nescript::pipeline::compile_source`]
//! directly so it's impossible for it to drift away from the CLI
//! compile path — a 2026-04 regression where a hand-maintained
//! parallel copy of the pipeline missed a new bank-switching step
//! is exactly what this refactor prevents.
use std::fs;
use std::path::{Path, PathBuf};
use criterion::{criterion_group, criterion_main, BenchmarkId, Criterion};
use nescript::analyzer;
use nescript::assets;
use nescript::codegen::{peephole, IrCodeGen};
use nescript::ir;
use nescript::linker::{BankTrampoline, Linker, PrgBank};
use nescript::optimizer;
use nescript::parser;
use nescript::parser::ast::BankType;
use nescript::pipeline::{compile_source, CompileOptions};
/// Pre-loaded `.ne` source plus the directory it was read from. The
/// directory matters because sprite `@binary` / `@chr` paths resolve
@ -55,8 +56,12 @@ fn load_examples() -> Vec<Example> {
entries
.into_iter()
.map(|path| {
let source = fs::read_to_string(&path)
let raw = fs::read_to_string(&path)
.unwrap_or_else(|e| panic!("failed to read {}: {e}", path.display()));
// Preprocess once up front (include inlining, etc.)
// so the hot loop never touches the filesystem.
let source = parser::preprocess_source(&raw, Some(&path))
.unwrap_or_else(|e| panic!("preprocess failed for {}: {e}", path.display()));
let name = path
.file_stem()
.and_then(|s| s.to_str())
@ -74,102 +79,14 @@ fn load_examples() -> Vec<Example> {
.collect()
}
/// Run the full CLI compile pipeline on an in-memory source string.
/// Mirrors `compile` in `src/main.rs`: parse → analyze → IR lower →
/// optimize → IR codegen → peephole → link. Panics on any error so
/// a regression that breaks the pipeline surfaces immediately instead
/// of silently skewing the measurements.
/// Run the full compile pipeline on an in-memory source string
/// via the shared library entry point so the bench can't drift
/// away from the CLI build path.
fn compile_pipeline(source: &str, source_dir: &Path) -> Vec<u8> {
let preprocessed = parser::preprocess_source(source, None)
.unwrap_or_else(|e| panic!("preprocess failed: {e}"));
let (program, parse_diags) = parser::parse(&preprocessed);
assert!(
!parse_diags
.iter()
.any(nescript::errors::Diagnostic::is_error),
"parse errors: {parse_diags:?}"
);
let program = program.expect("parse produced no program");
let analysis = analyzer::analyze(&program);
assert!(
!analysis
.diagnostics
.iter()
.any(nescript::errors::Diagnostic::is_error),
"analysis errors: {:?}",
analysis.diagnostics
);
let mut ir_program = ir::lower(&program, &analysis);
optimizer::optimize(&mut ir_program);
let sprites = assets::resolve_sprites(&program, source_dir).expect("sprite resolution failed");
let sfx = assets::resolve_sfx(&program).expect("sfx resolution failed");
let music = assets::resolve_music(&program).expect("music resolution failed");
let palettes =
assets::resolve_palettes(&program, source_dir).expect("palette resolution failed");
let backgrounds =
assets::resolve_backgrounds(&program, source_dir).expect("background resolution failed");
let mut codegen = IrCodeGen::new(&analysis.var_allocations, &ir_program)
.with_sprites(&sprites)
.with_audio(&sfx, &music);
let mut instructions = codegen.generate(&ir_program);
peephole::optimize(&mut instructions);
// Pull the per-bank instruction streams out of the codegen and
// reconstruct the trampoline requests for each banked function,
// mirroring the real CLI compile path in `src/main.rs`. A
// bench that left the switchable banks empty would panic in
// the assembler's fixup pass for any program that nests a
// function inside a `bank` block (e.g. `uxrom_user_banked`),
// because the `__tramp_<name>` label emitted by IR codegen
// would have nowhere to resolve to.
let mut banked_streams = codegen.banked_streams().clone();
for stream in banked_streams.values_mut() {
peephole::optimize(stream);
match compile_source(source, source_dir, &CompileOptions::default()) {
Ok(out) => out.rom,
Err(e) => panic!("pipeline failed: {e:?}"),
}
let mut bank_trampolines: std::collections::HashMap<String, Vec<BankTrampoline>> =
std::collections::HashMap::new();
for func in &ir_program.functions {
if let Some(bank_name) = &func.bank {
bank_trampolines
.entry(bank_name.clone())
.or_default()
.push(BankTrampoline {
tramp_label: format!("__tramp_{}", func.name),
entry_label: format!("__ir_fn_{}", func.name),
});
}
}
let linker = Linker::with_mapper(program.game.mirroring, program.game.mapper)
.with_header(program.game.header);
let switchable_banks: Vec<PrgBank> = program
.banks
.iter()
.filter(|b| b.bank_type == BankType::Prg)
.map(|b| {
let stream = banked_streams.remove(&b.name).unwrap_or_default();
let tramps = bank_trampolines.remove(&b.name).unwrap_or_default();
if stream.is_empty() && tramps.is_empty() {
PrgBank::empty(&b.name)
} else {
PrgBank::with_instructions(&b.name, stream, tramps)
}
})
.collect();
linker.link_banked_with_ppu(
&instructions,
&sprites,
&sfx,
&music,
&palettes,
&backgrounds,
&switchable_banks,
)
}
/// Criterion entry point. One benchmark group per example so the