mirror of
https://github.com/imjasonh/nescript
synced 2026-07-08 00:45:38 +00:00
ci: emulator job must still rebuild ROMs from source
The previous commit wired the emulator job to read the committed examples/*.nes directly, with the rationale that the `examples` job's reproducibility diff would catch "stale committed ROMs" at PR time. That reasoning is wrong and defeats the point of the emulator harness. Failure mode: a compiler change lands with correctly-rebuilt ROMs (passing the examples job) but introduces a rendering regression that flips a golden. The emulator job would catch this today because it runs the harness against freshly-compiled ROMs. With the previous commit's shortcut, the harness would boot the committed ROMs — which the PR author just rebuilt to match — so the mismatch wouldn't show up until a second commit touched anything else. That's exactly the kind of silent-failure hole the golden harness exists to plug. Fix: emulator job rebuilds the compiler (toolchain + cache + `cargo build --release`) and compiles every .ne into the workspace before running the harness, same as the pre-ROMs- committed era. The committed ROMs keep their review/demo role (clone-and-play, diff visibility in PRs) but the test job always validates the working compiler, not a frozen snapshot. CLAUDE.md updated to match: the harness runs against whatever sits in examples/*.nes, so iterating locally still means rebuilding the ROM(s) you care about first. https://claude.ai/code/session_01BcCcHi6FUmTh8jC7UgkA3A
This commit is contained in:
parent
57faf9e36a
commit
12b54a715e
2 changed files with 35 additions and 18 deletions
23
.github/workflows/ci.yml
vendored
23
.github/workflows/ci.yml
vendored
|
|
@ -97,6 +97,8 @@ jobs:
|
|||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: dtolnay/rust-toolchain@stable
|
||||
- uses: Swatinem/rust-cache@v2
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '22'
|
||||
|
|
@ -122,13 +124,26 @@ jobs:
|
|||
libxkbcommon0 \
|
||||
libxrandr2 \
|
||||
libasound2t64
|
||||
# No compiler build needed — the `.nes` files are committed
|
||||
# alongside their `.ne` sources, and the `examples` CI job
|
||||
# fails on any stale ROM, so the emulator job can trust them.
|
||||
- name: Build compiler
|
||||
run: cargo build --release
|
||||
- name: Compile all .ne examples
|
||||
# Overwrites the committed examples/*.nes in the CI workspace
|
||||
# (the workspace is ephemeral; the committed ROMs aren't
|
||||
# touched). This is load-bearing: the goldens must be diffed
|
||||
# against what *this* compiler produces, not against the ROMs
|
||||
# an earlier commit happened to ship, otherwise a broken
|
||||
# compiler change can still pass the emulator job as long as
|
||||
# the contributor forgot to rebuild the ROMs. The `examples`
|
||||
# job catches that second case via its reproducibility diff.
|
||||
run: |
|
||||
for f in examples/*.ne; do
|
||||
echo "==> Compiling $f"
|
||||
./target/release/nescript build "$f"
|
||||
done
|
||||
- name: Install emulator harness dependencies
|
||||
working-directory: tests/emulator
|
||||
run: npm ci
|
||||
- name: Diff each ROM's framebuffer against its committed golden
|
||||
- name: Diff each ROM's framebuffer against its golden
|
||||
working-directory: tests/emulator
|
||||
run: node run_examples.mjs
|
||||
- name: Upload actual + diff PNGs on failure
|
||||
|
|
|
|||
30
CLAUDE.md
30
CLAUDE.md
|
|
@ -70,29 +70,31 @@ tests/emulator/
|
|||
The harness is **separate** from `cargo test`. You have to run it by hand:
|
||||
|
||||
```bash
|
||||
# 1. Install node deps (once per worktree; node_modules/ is gitignored).
|
||||
# 1. Rebuild every example with the current compiler. The harness
|
||||
# reads whatever sits under examples/*.nes — if you want to test
|
||||
# your working copy you have to rebuild them first.
|
||||
cargo build --release
|
||||
for f in examples/*.ne; do ./target/release/nescript build "$f"; done
|
||||
|
||||
# 2. Install node deps (once per worktree; node_modules/ is gitignored).
|
||||
cd tests/emulator
|
||||
npm install # or `npm ci` in CI
|
||||
|
||||
# 2. Verify every committed ROM still matches its golden.
|
||||
# 3. Verify every ROM still matches its golden.
|
||||
node run_examples.mjs
|
||||
# → "22/22 ROMs match their goldens" on success
|
||||
# → FAIL / MISS lines + `actual/<name>.png`, `actual/<name>.diff.png`,
|
||||
# `actual/<name>.wav` written for any ROM that mismatched
|
||||
```
|
||||
|
||||
The harness reads the committed `examples/*.nes` files directly —
|
||||
since those are rebuilt and diffed by the `examples` CI job, the
|
||||
emulator job doesn't need a compiler toolchain at all. If you're
|
||||
iterating on the compiler locally, rebuild the example ROM(s) you
|
||||
care about before re-running the harness:
|
||||
|
||||
```bash
|
||||
cargo build --release
|
||||
./target/release/nescript build examples/hello_sprite.ne
|
||||
# then:
|
||||
(cd tests/emulator && node run_examples.mjs)
|
||||
```
|
||||
The harness always runs against whatever sits in `examples/*.nes`,
|
||||
so iterating on the compiler means rebuilding the example first.
|
||||
CI's `emulator` job does this too — it builds the compiler, compiles
|
||||
every `.ne` into the workspace (overwriting the committed ROMs,
|
||||
which are ephemeral in the CI checkout), and then runs the harness.
|
||||
The committed ROMs are a PR-review convenience and a "did this
|
||||
change affect codegen" tripwire via the `examples` job's
|
||||
reproducibility diff; they are **not** what the emulator job tests.
|
||||
|
||||
### Updating goldens
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue