1
0
Fork 0
mirror of https://github.com/imjasonh/nescript synced 2026-07-08 00:45:38 +00:00

ci: emulator job must still rebuild ROMs from source

The previous commit wired the emulator job to read the committed
examples/*.nes directly, with the rationale that the `examples`
job's reproducibility diff would catch "stale committed ROMs"
at PR time. That reasoning is wrong and defeats the point of
the emulator harness.

Failure mode: a compiler change lands with correctly-rebuilt
ROMs (passing the examples job) but introduces a rendering
regression that flips a golden. The emulator job would catch
this today because it runs the harness against freshly-compiled
ROMs. With the previous commit's shortcut, the harness would
boot the committed ROMs — which the PR author just rebuilt to
match — so the mismatch wouldn't show up until a second commit
touched anything else. That's exactly the kind of silent-failure
hole the golden harness exists to plug.

Fix: emulator job rebuilds the compiler (toolchain + cache +
`cargo build --release`) and compiles every .ne into the
workspace before running the harness, same as the pre-ROMs-
committed era. The committed ROMs keep their review/demo role
(clone-and-play, diff visibility in PRs) but the test job
always validates the working compiler, not a frozen snapshot.

CLAUDE.md updated to match: the harness runs against whatever
sits in examples/*.nes, so iterating locally still means
rebuilding the ROM(s) you care about first.

https://claude.ai/code/session_01BcCcHi6FUmTh8jC7UgkA3A
This commit is contained in:
Claude 2026-04-13 15:21:38 +00:00
parent 57faf9e36a
commit 12b54a715e
No known key found for this signature in database
2 changed files with 35 additions and 18 deletions

View file

@ -97,6 +97,8 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- uses: actions/setup-node@v4
with:
node-version: '22'
@ -122,13 +124,26 @@ jobs:
libxkbcommon0 \
libxrandr2 \
libasound2t64
# No compiler build needed — the `.nes` files are committed
# alongside their `.ne` sources, and the `examples` CI job
# fails on any stale ROM, so the emulator job can trust them.
- name: Build compiler
run: cargo build --release
- name: Compile all .ne examples
# Overwrites the committed examples/*.nes in the CI workspace
# (the workspace is ephemeral; the committed ROMs aren't
# touched). This is load-bearing: the goldens must be diffed
# against what *this* compiler produces, not against the ROMs
# an earlier commit happened to ship, otherwise a broken
# compiler change can still pass the emulator job as long as
# the contributor forgot to rebuild the ROMs. The `examples`
# job catches that second case via its reproducibility diff.
run: |
for f in examples/*.ne; do
echo "==> Compiling $f"
./target/release/nescript build "$f"
done
- name: Install emulator harness dependencies
working-directory: tests/emulator
run: npm ci
- name: Diff each ROM's framebuffer against its committed golden
- name: Diff each ROM's framebuffer against its golden
working-directory: tests/emulator
run: node run_examples.mjs
- name: Upload actual + diff PNGs on failure

View file

@ -70,29 +70,31 @@ tests/emulator/
The harness is **separate** from `cargo test`. You have to run it by hand:
```bash
# 1. Install node deps (once per worktree; node_modules/ is gitignored).
# 1. Rebuild every example with the current compiler. The harness
# reads whatever sits under examples/*.nes — if you want to test
# your working copy you have to rebuild them first.
cargo build --release
for f in examples/*.ne; do ./target/release/nescript build "$f"; done
# 2. Install node deps (once per worktree; node_modules/ is gitignored).
cd tests/emulator
npm install # or `npm ci` in CI
# 2. Verify every committed ROM still matches its golden.
# 3. Verify every ROM still matches its golden.
node run_examples.mjs
# → "22/22 ROMs match their goldens" on success
# → FAIL / MISS lines + `actual/<name>.png`, `actual/<name>.diff.png`,
# `actual/<name>.wav` written for any ROM that mismatched
```
The harness reads the committed `examples/*.nes` files directly —
since those are rebuilt and diffed by the `examples` CI job, the
emulator job doesn't need a compiler toolchain at all. If you're
iterating on the compiler locally, rebuild the example ROM(s) you
care about before re-running the harness:
```bash
cargo build --release
./target/release/nescript build examples/hello_sprite.ne
# then:
(cd tests/emulator && node run_examples.mjs)
```
The harness always runs against whatever sits in `examples/*.nes`,
so iterating on the compiler means rebuilding the example first.
CI's `emulator` job does this too — it builds the compiler, compiles
every `.ne` into the workspace (overwriting the committed ROMs,
which are ephemeral in the CI checkout), and then runs the harness.
The committed ROMs are a PR-review convenience and a "did this
change affect codegen" tripwire via the `examples` job's
reproducibility diff; they are **not** what the emulator job tests.
### Updating goldens