mirror of
https://github.com/imjasonh/nescript
synced 2026-07-08 08:55:38 +00:00
The smoke test used to check a per-example `nonBlack` floor — "at
least one sprite rendered," plus a per-example minimum for the
multi-sprite examples. That catches gross regressions (a compiler
bug that makes everything go black) but silently lets through
anything that changes a handful of pixels without dropping below
the sprite floor. The whole point of this harness is to catch
compiler miscompiles before they land; a softer check means bugs
can still sneak in.
This swap makes every run diff the raw canvas framebuffer against
a committed PNG golden. One mismatched byte at pixel (120, 119)
is enough to fail CI — there's nowhere for a regression to hide.
Workflow:
# normal — fails on any pixel change
node tests/emulator/run_examples.mjs
# when the diff is intentional, rewrite the goldens
UPDATE_GOLDENS=1 node tests/emulator/run_examples.mjs
# or: node tests/emulator/run_examples.mjs --update-goldens
git diff tests/emulator/goldens/ # review the change
git add tests/emulator/goldens/
git commit # explain WHY in the message
When a run fails without `UPDATE_GOLDENS`, the runner writes:
tests/emulator/actual/<name>.png the run's raw output
tests/emulator/actual/<name>.diff.png red-highlighted pixel diff
so reviewers can eyeball what changed without rerunning locally.
`actual/` is gitignored and re-created on every run. The CI job
now uploads `actual/`, `goldens/`, and `report.json` together as
a single `emulator-diff` artifact on failure — side by side means
the "what changed" story is obvious without cloning.
Implementation:
- `tests/emulator/screenshots/` is renamed to `tests/emulator/goldens/`.
All 18 existing PNGs are preserved as the initial goldens (git
detected them as pure renames).
- `harness.html` gets a new `window.nesHarness.rawPixelsBase64()`
that returns the 245760-byte (256 × 240 × 4 bytes) RGBA canvas
buffer as base64. The runner compares raw pixels, not PNG
bytes, so encoder quirks (zlib level, filter heuristics) can't
cause false positives across Chrome versions or platforms.
- The runner uses `pngjs` (pure-JS, no native deps) to decode
goldens and to write diff PNGs. `PNG.sync.write` is
byte-deterministic for identical pixels, so `git diff` on a
committed golden only ever shows up when the actual rendered
pixels changed — not because two machines produced slightly
different compression.
- The committed goldens were re-encoded with pngjs in this commit
so the baseline is consistent from day one. File sizes are a
touch larger than Chrome's output (~1KB vs ~800B on average),
but that's negligible and it eliminates one entire class of
flaky-looking diffs in the future.
Determinism verification: I ran each of the 18 ROMs twice
through fresh `NES` instances in fresh puppeteer pages, hashed
the 245760-byte framebuffers at frame 180 with SHA-256, and
confirmed `run1 == run2` for every single one. Exact-pixel diffs
are safe for this ROM set.
Negative path verification: I corrupted one golden (flipped one
pixel to pure red via pngjs) and reran the runner. It printed
DIFF hello_sprite 1/61440 pixels differ; first at (120,120)
expected [255,0,0] got [0,0,0]
actual: tests/emulator/actual/hello_sprite.png
diff: tests/emulator/actual/hello_sprite.diff.png
and exited 1 as expected. The diff PNG shows a dim-grayscale
silhouette of the expected frame with a bright-red dot on the
one mismatched pixel — enough visual context to locate the
regression at a glance.
All 18 examples match their goldens in strict mode. `cargo fmt
--check`, `cargo clippy --release --all-targets -- -D warnings`,
and `cargo test --release` (313 unit + 37 integration) are all
still green.
https://claude.ai/code/session_014Z5y3Q9krLcAxYpZQJhZ5V
140 lines
4 KiB
YAML
140 lines
4 KiB
YAML
name: CI
|
|
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
pull_request:
|
|
branches: [main]
|
|
|
|
env:
|
|
CARGO_TERM_COLOR: always
|
|
RUSTFLAGS: "-D warnings"
|
|
|
|
jobs:
|
|
check:
|
|
name: Check
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- uses: dtolnay/rust-toolchain@stable
|
|
- uses: Swatinem/rust-cache@v2
|
|
- run: cargo check --all-targets
|
|
|
|
fmt:
|
|
name: Format
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- uses: dtolnay/rust-toolchain@stable
|
|
with:
|
|
components: rustfmt
|
|
- run: cargo fmt --all -- --check
|
|
|
|
clippy:
|
|
name: Clippy
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- uses: dtolnay/rust-toolchain@stable
|
|
with:
|
|
components: clippy
|
|
- uses: Swatinem/rust-cache@v2
|
|
- run: cargo clippy --all-targets -- -D warnings
|
|
|
|
test:
|
|
name: Test
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- uses: dtolnay/rust-toolchain@stable
|
|
- uses: Swatinem/rust-cache@v2
|
|
- run: cargo test --all-targets
|
|
|
|
examples:
|
|
name: Build Examples
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- uses: dtolnay/rust-toolchain@stable
|
|
- uses: Swatinem/rust-cache@v2
|
|
- run: cargo build --release
|
|
- name: Compile all .ne examples
|
|
run: |
|
|
for f in examples/*.ne; do
|
|
echo "==> Compiling $f"
|
|
cargo run --release -- build "$f"
|
|
done
|
|
- name: Verify ROMs are valid iNES
|
|
run: |
|
|
for f in examples/*.nes; do
|
|
echo "==> Checking $f"
|
|
# iNES magic: first 4 bytes must be "NES\x1a"
|
|
header=$(od -A n -t x1 -N 4 "$f" | tr -d ' ')
|
|
if [ "$header" != "4e45531a" ]; then
|
|
echo "FAIL: $f is not a valid iNES ROM (header: $header)"
|
|
exit 1
|
|
fi
|
|
size=$(stat -c%s "$f")
|
|
echo " OK: $size bytes"
|
|
done
|
|
|
|
emulator:
|
|
name: Emulator Smoke Test
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- uses: dtolnay/rust-toolchain@stable
|
|
- uses: Swatinem/rust-cache@v2
|
|
- uses: actions/setup-node@v4
|
|
with:
|
|
node-version: '22'
|
|
- name: Install Chrome runtime deps for puppeteer
|
|
# puppeteer ships its own Chrome for Testing binary via the
|
|
# package's postinstall hook, but the runner image is missing
|
|
# a handful of shared libs that Chrome needs to launch.
|
|
run: |
|
|
sudo apt-get update
|
|
sudo apt-get install -y --no-install-recommends \
|
|
libnss3 \
|
|
libatk-bridge2.0-0 \
|
|
libatk1.0-0 \
|
|
libatspi2.0-0 \
|
|
libcups2t64 \
|
|
libdbus-1-3 \
|
|
libdrm2 \
|
|
libgbm1 \
|
|
libpango-1.0-0 \
|
|
libxcomposite1 \
|
|
libxdamage1 \
|
|
libxfixes3 \
|
|
libxkbcommon0 \
|
|
libxrandr2 \
|
|
libasound2t64
|
|
- name: Build compiler
|
|
run: cargo build --release
|
|
- name: Compile all .ne examples
|
|
run: |
|
|
for f in examples/*.ne; do
|
|
echo "==> Compiling $f"
|
|
./target/release/nescript build "$f"
|
|
done
|
|
- name: Install emulator harness dependencies
|
|
working-directory: tests/emulator
|
|
run: npm ci
|
|
- name: Diff each ROM's framebuffer against its committed golden
|
|
working-directory: tests/emulator
|
|
run: node run_examples.mjs
|
|
- name: Upload actual + diff PNGs on failure
|
|
if: failure()
|
|
uses: actions/upload-artifact@v4
|
|
with:
|
|
name: emulator-diff
|
|
# `actual/` holds the run's output and red-highlighted diff
|
|
# PNGs for any ROM that didn't match its golden. `report.json`
|
|
# has the per-example pass/fail/diff counts. `goldens/` is
|
|
# included too so reviewers can compare the three side by side
|
|
# without cloning the repo.
|
|
path: |
|
|
tests/emulator/actual/
|
|
tests/emulator/goldens/
|
|
tests/emulator/report.json
|