mirror of
https://github.com/imjasonh/esp32
synced 2026-07-06 23:52:24 +00:00
Rename ota-plan.md -> ota.md; rewrite as descriptive system doc
Was structured as a multi-phase plan with each phase marked DONE; now that the system is operational, restructure as documentation describing what's built (architecture, components, partition layout, update flow, anti-bricking + USB recovery, CI signing, on-device verification). Remaining ideas moved to a 'Future work' section at the end. Dropped explicit support for legacy cosign .sig formats from the doc. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
0ebc860e6f
commit
d6d1a7ba1a
3 changed files with 272 additions and 268 deletions
|
|
@ -22,7 +22,7 @@ GHCR + cosign keyless signing. E-ink display work coming next (see
|
|||
|
||||
Push to `main` → CI builds → publish workflow pushes a signed image to
|
||||
GHCR → device picks it up on its next poll. See
|
||||
[`ota-plan.md`](ota-plan.md) for the full design.
|
||||
[`ota.md`](ota.md) for the full design.
|
||||
|
||||
## Hardware
|
||||
|
||||
|
|
@ -62,7 +62,7 @@ make publish Build, push OCI artifact to ghcr.io/imjasonh/esp32, cosign sign
|
|||
make clean cargo clean
|
||||
```
|
||||
|
||||
`make publish` requires `gh.env` (see [`ota-plan.md`](ota-plan.md) for
|
||||
`make publish` requires `gh.env` (see [`ota.md`](ota.md) for
|
||||
PAT setup) and a real cosign OIDC flow the first time per ~10min window
|
||||
— a browser pops to authenticate. CI does this automatically via the
|
||||
GitHub Actions workflow's ambient OIDC token.
|
||||
|
|
|
|||
266
ota-plan.md
266
ota-plan.md
|
|
@ -1,266 +0,0 @@
|
|||
# OTA updates via OCI artifacts — plan
|
||||
|
||||
## Goal
|
||||
|
||||
The device fetches new firmware from GHCR, verifies it, and applies it
|
||||
without USB intervention. Phase 1 is unsigned; phase 4 adds keyless
|
||||
cosign + Rekor signing and on-device verification. Single board target:
|
||||
the Inland ESP-WROOM-32 already in this project.
|
||||
|
||||
## End-to-end flow
|
||||
|
||||
```
|
||||
developer host GHCR ESP32 device
|
||||
────────────── ──── ────────────
|
||||
cargo run -p publisher ──push──▶ :latest manifest polls every 10 min
|
||||
(builds .bin, wraps + .bin layer GET manifests/latest
|
||||
as OCI artifact, pushes) ── If-None-Match ──▶
|
||||
304: do nothing
|
||||
200: stream blob
|
||||
into OTA part,
|
||||
reboot, validate,
|
||||
rollback on fail
|
||||
```
|
||||
|
||||
## Settled decisions
|
||||
|
||||
- **Registry**: GHCR. Public repo, e.g. `ghcr.io/<gh-user>/esp32-fw`.
|
||||
- **Polling interval**: default 10 min, stored in NVS so it can be
|
||||
changed at runtime without reflashing. `0` = paused.
|
||||
- **Bearer-token auth**: plumb the anonymous-token dance now (GHCR
|
||||
requires it even for public reads; same code path will work for
|
||||
private repos later).
|
||||
- **Publisher language**: Rust, in this repo, as a separate
|
||||
`tools/publisher/` cargo project (independent of the firmware crate
|
||||
so it builds for the host, not for `xtensa-esp32-espidf`).
|
||||
- **Target board**: Inland ESP-WROOM-32 only. Don't build for or test
|
||||
other targets.
|
||||
- **Anchoring**: device compares manifest **digest** (not tag) to the
|
||||
last-applied digest stored in NVS. Tags are mutable; digests are not.
|
||||
|
||||
## Anti-bricking + USB recovery
|
||||
|
||||
Two layers of protection:
|
||||
|
||||
### Layer 1 — automatic rollback (in firmware)
|
||||
|
||||
- Use ESP-IDF's "pending verify" boot state.
|
||||
`CONFIG_BOOTLOADER_APP_ROLLBACK_ENABLE=y` in `sdkconfig.defaults`.
|
||||
- After OTA reboots into the new image, only call
|
||||
`esp_ota_mark_app_valid_cancel_rollback()` once we've confirmed:
|
||||
1. Wi-Fi connected
|
||||
2. registry reachable
|
||||
3. manifest readable
|
||||
- If the new app crashes or fails to mark-valid before the next reboot,
|
||||
the bootloader reverts to the previous partition. So an OTA that
|
||||
breaks networking unbricks itself.
|
||||
|
||||
### Layer 2 — USB recovery (manual, when layer 1 fails)
|
||||
|
||||
If both partitions are bad, or the bootloader/partition table itself
|
||||
got corrupted, recover over USB:
|
||||
|
||||
1. Put the chip in serial download mode:
|
||||
- Hold the **BOOT** (IO0) button on the dev board.
|
||||
- Press and release **EN** (RESET).
|
||||
- Release BOOT.
|
||||
- (Many boards including the Inland have auto-reset; espflash will
|
||||
usually drive DTR/RTS to do this for us. Manual buttons are the
|
||||
fallback.)
|
||||
2. Reflash everything from the project root:
|
||||
```
|
||||
make flash-all
|
||||
```
|
||||
This wipes flash and writes bootloader + partition table + app, the
|
||||
same way the very first flash did. Defined in the Makefile so it's
|
||||
one command, not a sequence to remember at 1am.
|
||||
|
||||
`make flash-all` will need to know the bootloader path produced by
|
||||
`espflash save-image --bootloader`; will be added when phase 0 is done.
|
||||
|
||||
## Phased plan
|
||||
|
||||
### Phase 0 — partition table + bootstrap firmware ✅ DONE
|
||||
|
||||
Shipped:
|
||||
- `partitions.csv` with `nvs / otadata / phy_init / ota_0 / ota_1`
|
||||
(1.5MB per app slot).
|
||||
- sdkconfig: `CONFIG_PARTITION_TABLE_CUSTOM`, `CONFIG_BOOTLOADER_APP_ROLLBACK_ENABLE`,
|
||||
`CONFIG_ESPTOOLPY_FLASHSIZE_4MB`. The sdkconfig is now generated by
|
||||
the Makefile from `sdkconfig.defaults.in` so we can substitute the
|
||||
absolute path to `partitions.csv` (IDF's CMake resolves
|
||||
`CONFIG_PARTITION_TABLE_CUSTOM_FILENAME` relative to the embuild
|
||||
synthetic project under `target/`, not our repo root — see notes.txt
|
||||
for the full discovery).
|
||||
- `make flash-all` (full erase + bootloader + partition table + app).
|
||||
- `src/main.rs` logs running partition early in `main()` via
|
||||
`esp_ota_get_running_partition()`.
|
||||
- Verified: device boots into `ota_0` at offset `0x20000`, Wi-Fi +
|
||||
HTTPS still working. App is 1.15MB / 1.5MB slot (73%).
|
||||
|
||||
### Phase 1 — publisher (host-side Rust) ✅ DONE
|
||||
|
||||
Shipped:
|
||||
- `tools/publisher/` — separate cargo project (with its own
|
||||
`.cargo/config.toml` to clear the parent's `[unstable] build-std`),
|
||||
built via `--target $(HOST_TRIPLE)` so it works on any developer
|
||||
machine without hardcoded triples.
|
||||
- Crates: `oci-distribution` (with `rustls-tls`), `clap`, `anyhow`,
|
||||
`serde`, `serde_json`, `sha2`, `tokio`, `tracing`, `time`.
|
||||
- Mediatypes: `application/vnd.esp32.firmware.v1+json` (config),
|
||||
`application/vnd.esp32.firmware.bin` (layer). Config carries
|
||||
`{target_chip, idf_version, git_sha, built_at, bin_size, bin_sha256}`.
|
||||
- Annotations: `org.opencontainers.image.source`,
|
||||
`org.opencontainers.image.revision`, `org.opencontainers.image.created`.
|
||||
- Push subcommand pushes both `:latest` and `:sha-<short>`.
|
||||
- Pull-verify subcommand pulls the manifest + layer, checks descriptor
|
||||
digest matches actual blob SHA, and compares to a local file. Doubles
|
||||
as a sketch of the device-side fetch.
|
||||
- Auth via `GH_TOKEN` from gitignored `gh.env`. Classic PAT with
|
||||
`write:packages` (fine-grained PATs don't expose a Packages
|
||||
permission for user-owned packages — see notes.txt).
|
||||
- Make targets: `save-image`, `publish`, `pull-verify`.
|
||||
- Verified end-to-end against `ghcr.io/imjasonh/esp32` — pushed both
|
||||
tags, pulled both back, layer bit-for-bit identical to local .bin.
|
||||
|
||||
### Phase 2 — device-side OTA (firmware) ✅ DONE
|
||||
|
||||
Shipped:
|
||||
- `src/ota.rs` — background polling loop on a dedicated 32 KB pthread.
|
||||
Every 60s: get a Bearer token from the GHCR token endpoint
|
||||
(anonymous, scoped to `repository:<repo>:pull`), fetch the manifest,
|
||||
compare the layer digest to `last_digest` in NVS, and on mismatch
|
||||
stream the blob into the inactive OTA partition while computing
|
||||
SHA256 in parallel. Aborts the OTA update and bails on size or SHA
|
||||
mismatch. On success persists `pending_digest` to NVS and reboots.
|
||||
- `main.rs` — on every boot:
|
||||
- Reads the partition state via `esp_ota_get_state_partition()`.
|
||||
- If `PENDING_VERIFY`, runs the existing bringup checks (Wi-Fi +
|
||||
HTTPS to ipify + wttr) before calling
|
||||
`esp_ota_mark_app_valid_cancel_rollback()`. If mark-valid fails,
|
||||
reboots so the bootloader rolls back.
|
||||
- Promotes `pending_digest` → `last_digest` in NVS after mark-valid.
|
||||
- Spawns the OTA loop.
|
||||
- `FW_VERSION` is the short git SHA, baked in at compile time
|
||||
(`build.rs` reads `GIT_SHA` env var set by the Makefile) and logged
|
||||
on every boot.
|
||||
- `CONFIG_PTHREAD_TASK_STACK_SIZE_DEFAULT` raised to 16 KB; OTA thread
|
||||
takes 32 KB explicitly. 8 KB stack-overflowed during HTTPS + JSON +
|
||||
SHA work.
|
||||
- `make monitor` now auto-passes `--non-interactive` when stdout is
|
||||
not a TTY, so it works inside scripts and background tasks.
|
||||
- Verified end-to-end: published v1, flashed via USB; bumped a visible
|
||||
version string; `make publish`; the device polled, downloaded,
|
||||
SHA-verified, rebooted into `ota_1`, and marked the new image valid
|
||||
after bringup — zero USB intervention.
|
||||
|
||||
Deferred to phase 3:
|
||||
- `If-None-Match` to avoid re-fetching unchanged manifests
|
||||
- Backoff + jitter on registry errors
|
||||
- NVS-configurable poll interval / repo / tag (currently compile-time
|
||||
defaults: `ghcr.io/imjasonh/esp32:latest`, 60s)
|
||||
- Force-update GPIO button
|
||||
|
||||
### Phase 3 — operational glue ✅ DONE (partial)
|
||||
|
||||
Shipped:
|
||||
- **Exponential backoff with ±10% jitter on errors.** `ota::run`
|
||||
tracks `consecutive_failures`. Sleep grows as
|
||||
`poll_interval * 2^failures` capped at `BACKOFF_CAP` = 1h. Jitter
|
||||
is also applied on the success path so a fleet doesn't poll in
|
||||
lockstep. Randomness via `esp_random()`.
|
||||
- **NVS-readable config.** `OtaConfig::load_from_nvs()` reads
|
||||
`repo` (str), `tag` (str), `poll_secs` (u32) from the `ota`
|
||||
namespace, falling back to compile-time defaults
|
||||
(`ghcr.io/imjasonh/esp32:latest`, 60s) when keys are absent.
|
||||
- **Boot summary log.** First line emitted by the OTA thread on
|
||||
startup: `ota: boot summary fw=... repo=... tag=... poll=...s
|
||||
last_digest=...`. One grep gives you the device's full OTA state.
|
||||
- **Compose `make publish monitor`** — no fused target, just chain
|
||||
the existing two. (See `feedback_make_target_naming.md`.)
|
||||
|
||||
NVS schema as shipped:
|
||||
```
|
||||
repo str defaults to "ghcr.io/imjasonh/esp32"
|
||||
tag str defaults to "latest"
|
||||
poll_secs u32 defaults to 60
|
||||
last_digest str set after a successful mark-valid
|
||||
pending_digest str set during update, promoted on mark-valid
|
||||
```
|
||||
|
||||
Deferred (low value, or needs hardware/provisioning we don't have yet):
|
||||
- **If-None-Match** on manifest fetches — marginal savings since we
|
||||
already digest-compare locally; the manifest itself is ~500 bytes.
|
||||
- **Force-update GPIO button** — needs a button wired to a GPIO; no
|
||||
breadboard yet.
|
||||
- **NVS provisioning mechanism** — currently the NVS-configurable
|
||||
settings can only be *read* by the firmware. To actually set them
|
||||
from outside we'd need a serial console command, a small HTTP
|
||||
endpoint on the device, or a config-as-OCI-artifact channel.
|
||||
|
||||
### Phase 4 — keyless cosign signing + on-device verification
|
||||
|
||||
Split into three sub-phases because the full thing is a real subproject.
|
||||
|
||||
#### Phase 4a — sign + verify signature, no Rekor ✅ DONE
|
||||
|
||||
- **Publisher**: `cosign sign` runs after each `make publish` push.
|
||||
Keyless OIDC flow (browser the first time, cached after). Sigs are
|
||||
uploaded to GHCR as a sibling OCI artifact at tag
|
||||
`sha256-<digest>.sig`. One signing per push covers both `:latest`
|
||||
and `:sha-<short>` since they resolve to the same digest.
|
||||
- **Trust roots in firmware**: `src/trust.rs` carries:
|
||||
- `TRUSTED_IDENTITIES: &[(&str, &str)]` — allowlist of (email,
|
||||
issuer) tuples. Initial: `[("imjasonh@gmail.com",
|
||||
"https://accounts.google.com")]`. Editing requires source change
|
||||
and USB reflash; OTA cannot change this.
|
||||
- Sigstore root CA cert PEM, `include_str!`'d at build time.
|
||||
- **Device verification flow**, before download in `poll_once`:
|
||||
1. Fetch the sig artifact at `sha256-<digest>.sig`.
|
||||
2. Parse its layer's annotations: `dev.cosignproject.cosign/signature`
|
||||
(base64 ECDSA), `dev.sigstore.cosign/certificate` (PEM Fulcio cert
|
||||
chain).
|
||||
3. Fetch the layer payload blob (the JSON to-be-signed).
|
||||
4. Parse the leaf signing cert. Extract the email (SAN
|
||||
rfc822Name) and OIDC issuer (cert extension OID
|
||||
`1.3.6.1.4.1.57264.1.1` legacy or `.1.8` for v2).
|
||||
5. Reject if the (email, issuer) tuple isn't in
|
||||
`TRUSTED_IDENTITIES`.
|
||||
6. Verify the cert chains to the bundled Sigstore root.
|
||||
7. Verify the ECDSA-P256 signature over the payload bytes using
|
||||
the cert's public key.
|
||||
8. Parse the payload JSON (`type:"cosign container image
|
||||
signature"`); confirm `critical.image.docker-manifest-digest`
|
||||
matches the manifest digest we're about to apply.
|
||||
9. Only then proceed with the download/write/reboot.
|
||||
- **Defer to phase 4b**: cert-validity-window check needs a trusted
|
||||
timestamp (Rekor SET) to make sense, since Fulcio certs are 10-min
|
||||
short-lived.
|
||||
- **Crates added**: `p256` (ECDSA verify), `x509-cert` + `der` (cert
|
||||
parsing), `base64` (annotation decoding), `pem` (PEM block parsing).
|
||||
Roughly +100 KB of firmware.
|
||||
|
||||
#### Phase 4b — Rekor SET verification
|
||||
|
||||
- Add the Signed Entry Timestamp check from cosign's bundle annotation
|
||||
(or fetched from Rekor directly). Verify SET signed by Rekor's
|
||||
bundled public key. Use the SET timestamp to enforce the cert
|
||||
validity window.
|
||||
- Bundle Rekor's public key alongside the Sigstore root.
|
||||
|
||||
#### Phase 4c — operational hardening
|
||||
|
||||
- Better verification-failure logs (exact field that failed).
|
||||
- Support the multiple cosign annotation/bundle formats that exist
|
||||
across cosign versions.
|
||||
- Metrics counter for verify-pass / verify-fail / sig-not-found.
|
||||
|
||||
## Order of work
|
||||
|
||||
1. Phase 0 (partition table + flash-all). ✅
|
||||
2. Phase 1 publisher. ✅
|
||||
3. Phase 2 firmware OTA loop. ✅
|
||||
4. Phase 3 polish. ✅ (partial; some deferred)
|
||||
5. Phase 4a signing + verify (no Rekor). ✅
|
||||
6. Phase 4b Rekor SET verification.
|
||||
7. Phase 4c operational hardening.
|
||||
270
ota.md
Normal file
270
ota.md
Normal file
|
|
@ -0,0 +1,270 @@
|
|||
# OTA updates over GHCR with cosign verification
|
||||
|
||||
This document describes how OTA updates work end-to-end on this project:
|
||||
how firmware is built and signed in CI, how it's distributed via an OCI
|
||||
registry, how the device discovers and applies updates, and how the
|
||||
device protects against bad updates and unauthorized signers.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
GitHub
|
||||
Container
|
||||
Registry
|
||||
(GHCR)
|
||||
developer push │
|
||||
──────────────▶ ┌──────────────────┐ push ▼ poll
|
||||
│ GHA workflow │ ─────▶ ┌────────┐ ◀────────────
|
||||
│ publish.yml │ sign │ OCI │ pull bundle
|
||||
│ • cargo build │ ─────▶ │artifact│ verify sig
|
||||
│ • espflash │ │ + sig │ stream blob
|
||||
│ • OCI push │ └────────┘ write OTA
|
||||
│ • cosign sign │ reboot
|
||||
│ (keyless, │ │
|
||||
│ OIDC) │ ▼
|
||||
└──────────────────┘ ┌────────┐
|
||||
│ ESP32 │
|
||||
└────────┘
|
||||
```
|
||||
|
||||
Manual `make publish` from a developer machine works the same way —
|
||||
just cosigns with a different OIDC identity (a developer's email instead
|
||||
of the workflow's URI).
|
||||
|
||||
## Components
|
||||
|
||||
| Component | Lives at | Role |
|
||||
|---|---|---|
|
||||
| **Publisher** | `tools/publisher/` (Rust, host build) | Wraps the firmware `.bin` as an OCI artifact and pushes both `:latest` and `:sha-<short>` tags to GHCR. Prints the manifest digest on stdout for the next step. |
|
||||
| **Cosign** | system binary, version 3 | Signs the just-pushed digest using keyless OIDC. Stores the signature as a sibling OCI artifact (Sigstore Bundle v0.3 format). |
|
||||
| **GHA workflow** | `.github/workflows/publish.yml` | Runs `make publish` on every push to main. Uses ambient OIDC token for cosign — no secrets needed for signing. |
|
||||
| **CI workflow** | `.github/workflows/ci.yml` | Runs on PRs. Builds firmware + publisher to verify they compile. No publishing or signing. |
|
||||
| **OTA loop** | `src/ota.rs` | Background pthread, polls GHCR every 60 s with backoff + jitter, fetches manifest, compares to last applied. |
|
||||
| **Sig verifier** | `src/sig.rs` | Parses Sigstore Bundle v0.3, verifies cert chain + DSSE signature + in-toto subject digest. |
|
||||
| **Trust config** | `src/trust.rs`, `trust/*.pem` | Compile-time allowlist of `(identity, issuer)` tuples + bundled Sigstore Fulcio root + intermediate CAs. Updates require source edit + USB reflash. |
|
||||
|
||||
## Partition layout (4 MB flash)
|
||||
|
||||
```
|
||||
0x001000 bootloader (~28 KB)
|
||||
0x008000 partition table 4 KB
|
||||
0x009000 nvs 24 KB last_digest, pending_digest, runtime config
|
||||
0x00f000 otadata 8 KB which slot to boot
|
||||
0x011000 phy_init 4 KB
|
||||
0x020000 ota_0 1.75 MB app slot 0
|
||||
0x1e0000 ota_1 1.75 MB app slot 1
|
||||
0x3a0000 unused 384 KB
|
||||
```
|
||||
|
||||
The two app slots ping-pong: a new image always writes to the *inactive*
|
||||
slot, then `otadata` is updated to point at it on next boot.
|
||||
|
||||
`partitions.csv` is the source of truth. The Makefile substitutes the
|
||||
absolute project path into `sdkconfig.defaults.in` because IDF resolves
|
||||
`CONFIG_PARTITION_TABLE_CUSTOM_FILENAME` relative to embuild's synthetic
|
||||
project under `target/`, not the repo root.
|
||||
|
||||
## Update mechanism
|
||||
|
||||
### Polling loop (firmware, `src/ota.rs::run`)
|
||||
|
||||
1. Sleep `poll_interval ± 10%` jitter (default 60 s, NVS-configurable
|
||||
via `poll_secs`).
|
||||
2. Hit GHCR's anonymous token endpoint with scope
|
||||
`repository:<repo>:pull`, get a Bearer token.
|
||||
3. `GET /v2/<repo>/manifests/<tag>` with that token. Compute SHA-256
|
||||
of the response body — that's the manifest digest cosign signed.
|
||||
4. Parse manifest. Check that `layers[0].mediaType` is
|
||||
`application/vnd.esp32.firmware.bin`. Compare layer digest to
|
||||
`last_digest` from NVS.
|
||||
- Match → `NoChange`, return to (1).
|
||||
- Different → continue.
|
||||
5. **Verify the signature** (see "Device verification" below). On
|
||||
failure → log, increment `consecutive_failures`, return to (1)
|
||||
with exponential backoff (`60 s × 2^failures`, capped at 1 h).
|
||||
6. `GET /v2/<repo>/blobs/<layer-digest>`. Stream the body chunk by
|
||||
chunk into the inactive OTA partition via `EspOta::write()`,
|
||||
updating an SHA-256 hasher in parallel.
|
||||
7. After last byte: confirm `(actual size, actual SHA)` matches the
|
||||
manifest descriptor. On any mismatch → `EspOta::abort()`, fail
|
||||
loudly. On match → `EspOta::complete()` (sets boot partition).
|
||||
8. Persist `pending_digest = layer.digest` to NVS.
|
||||
9. `esp_restart()`.
|
||||
|
||||
### Post-reboot validation (firmware, `src/main.rs`)
|
||||
|
||||
1. Early in `main()`, call `is_pending_verify()` →
|
||||
reads `esp_ota_get_state_partition()`.
|
||||
2. If `ESP_OTA_IMG_PENDING_VERIFY`, run the bringup checks:
|
||||
Wi-Fi connect, then HTTPS GETs to ipify and wttr.
|
||||
3. On bringup success, call
|
||||
`esp_ota_mark_app_valid_cancel_rollback()` and promote
|
||||
`pending_digest → last_digest` in NVS.
|
||||
4. On failure, `esp_restart()` — the bootloader rolls back to the
|
||||
previous slot on the next boot because the new image was never
|
||||
marked valid.
|
||||
5. Spawn the OTA polling thread with a 48 KB stack (HTTPS + JSON +
|
||||
SHA + cert parsing + ECDSA needs the headroom).
|
||||
|
||||
## Anti-bricking + USB recovery
|
||||
|
||||
Two layers of protection:
|
||||
|
||||
### Layer 1 — bootloader rollback (always armed in firmware)
|
||||
|
||||
`CONFIG_BOOTLOADER_APP_ROLLBACK_ENABLE=y` in `sdkconfig.defaults.in`
|
||||
turns on ESP-IDF's pending-verify state. After an OTA reboot, the new
|
||||
app runs in `PENDING_VERIFY`. If `mark_app_valid` isn't called before
|
||||
the next reboot, the bootloader reverts. We gate `mark_app_valid` on
|
||||
networking actually working — so an OTA that breaks Wi-Fi or DNS or
|
||||
TLS auto-rolls back.
|
||||
|
||||
### Layer 2 — USB recovery (when both slots are bad)
|
||||
|
||||
If both OTA slots end up unbootable, or the bootloader / partition
|
||||
table itself is corrupted, recover over USB:
|
||||
|
||||
1. Put the chip in serial download mode. The Inland board has
|
||||
auto-reset; espflash usually drives DTR/RTS automatically. If
|
||||
not: hold **BOOT** (IO0), tap **EN**, release BOOT.
|
||||
2. From the project root:
|
||||
```
|
||||
make flash-all
|
||||
```
|
||||
This wipes flash and writes bootloader + partition table + a
|
||||
fresh app the same way the very first flash did.
|
||||
|
||||
## CI signing (publisher → GHCR)
|
||||
|
||||
### Publisher tool
|
||||
|
||||
`tools/publisher/` is a separate cargo project (its own
|
||||
`rust-toolchain.toml` pinning `stable`, its own `.cargo/config.toml`
|
||||
clearing the parent's xtensa target). Built for the host.
|
||||
|
||||
CLI:
|
||||
|
||||
```
|
||||
publisher push --bin <fw.bin> --repo ghcr.io/<owner>/<name> --git-sha <short>
|
||||
```
|
||||
|
||||
Produces an OCI manifest with:
|
||||
|
||||
- **Config**: `application/vnd.esp32.firmware.v1+json` — small JSON
|
||||
with `target_chip`, `idf_version`, `git_sha`, `built_at`,
|
||||
`bin_size`, `bin_sha256`.
|
||||
- **Layer**: `application/vnd.esp32.firmware.bin` — the raw firmware
|
||||
bytes (no compression).
|
||||
- **Annotations**: `org.opencontainers.image.source` so GHCR auto-links
|
||||
the package to the source repo on first push;
|
||||
`org.opencontainers.image.revision`, `created`.
|
||||
|
||||
Pushes the same artifact under both `:latest` and `:sha-<short>`. Both
|
||||
tags resolve to the same content-addressed digest, so one cosign sign
|
||||
covers both.
|
||||
|
||||
The publisher prints `digest: sha256:...` on stdout. The Makefile
|
||||
captures it and passes it to `cosign sign` as `<repo>@<digest>` rather
|
||||
than a tag — eliminates the race window between publisher push and
|
||||
cosign sign.
|
||||
|
||||
### Cosign signing
|
||||
|
||||
`make publish` runs `cosign sign --yes <repo>@<digest>` after the
|
||||
publisher push. Cosign:
|
||||
|
||||
1. Authenticates to its OIDC issuer (Sigstore's broker for interactive
|
||||
browser flow; or the workflow's ambient ID token in GHA).
|
||||
2. Fetches a short-lived (10 min) X.509 cert from Fulcio with the
|
||||
OIDC identity baked in as a SAN.
|
||||
3. Hashes the manifest payload, signs it with the cert's private key.
|
||||
4. Bundles cert + signature + (optional) Rekor entry into a Sigstore
|
||||
Bundle v0.3 (Protobuf-as-JSON).
|
||||
5. Pushes the bundle as a sibling OCI artifact at tag
|
||||
`sha256-<hex>` (no `.sig` suffix) using the OCI 1.1 referrers
|
||||
layout (image index → inner manifest → bundle blob).
|
||||
|
||||
### Identities used
|
||||
|
||||
| Source | OIDC issuer | SAN form |
|
||||
|---|---|---|
|
||||
| Manual `make publish` | `https://accounts.google.com` | `rfc822Name` (developer's email) |
|
||||
| GHA workflow on push to main | `https://token.actions.githubusercontent.com` | `URI` (`https://github.com/<owner>/<repo>/.github/workflows/publish.yml@refs/heads/main`) |
|
||||
|
||||
Both must be present in `TRUSTED_IDENTITIES` for the device to accept
|
||||
their signatures. Updating the allowlist requires editing source +
|
||||
USB reflash; OTA cannot grant new signing identities.
|
||||
|
||||
### GHA workflow specifics
|
||||
|
||||
- Trigger: `push` to `main`, plus `workflow_dispatch` for manual
|
||||
reruns.
|
||||
- Concurrency: `cancel-in-progress: true` with a single group, so a
|
||||
rapid sequence of pushes only publishes the latest commit.
|
||||
- Permissions: `packages: write` (push to GHCR), `id-token: write`
|
||||
(cosign keyless OIDC), `contents: read`.
|
||||
- Uses `astral-sh/setup-uv@v7` to provide Python 3.12 (the Makefile's
|
||||
`ensure-python-shim` symlinks `python3` to it),
|
||||
`esp-rs/xtensa-toolchain@v1.7` for the Xtensa Rust toolchain,
|
||||
`sigstore/cosign-installer@v4.1.1` for cosign 3.
|
||||
- Wi-Fi creds (`WIFI_SSID`, `WIFI_PASS`) come from repo secrets and
|
||||
get written to `wifi.env` at workflow time. They're embedded into
|
||||
the firmware at compile time via `env!()` in `src/main.rs`.
|
||||
|
||||
## Device verification (`src/sig.rs`)
|
||||
|
||||
For each candidate update, before downloading the firmware blob:
|
||||
|
||||
1. **Fetch the bundle.** Walk the OCI 1.1 referrers layout: image
|
||||
index at tag `sha256-<manifest-digest>` → inner manifest →
|
||||
`application/vnd.dev.sigstore.bundle.v0.3+json` layer blob.
|
||||
2. **Parse the leaf cert.** Decode `verificationMaterial.certificate.
|
||||
rawBytes` from base64, parse as DER X.509 with the `x509-cert`
|
||||
crate.
|
||||
3. **Identity check.** Read SAN extension. Accept either
|
||||
`Rfc822Name` (email) or `UniformResourceIdentifier` (workflow
|
||||
URI). Read OIDC issuer from extension OID
|
||||
`1.3.6.1.4.1.57264.1.1`. Reject if `(identity, issuer)` isn't in
|
||||
`TRUSTED_IDENTITIES`.
|
||||
4. **Cert chain.** Verify leaf was signed by bundled Sigstore
|
||||
intermediate (P-384 ECDSA-SHA384). Verify intermediate was signed
|
||||
by bundled Sigstore root (also P-384 ECDSA-SHA384). Both intermediate
|
||||
and root PEMs are `include_str!`'d at compile time.
|
||||
5. **DSSE signature.** Decode `dsseEnvelope.payload` (base64) and
|
||||
`signatures[0].sig` (base64). Compute the DSSE PAE
|
||||
(`"DSSEv1 <len> <payloadType> <len> <payload>"`). Verify the
|
||||
ECDSA-P256 signature using the leaf cert's public key.
|
||||
6. **In-toto binding.** Parse the DSSE payload as an in-toto
|
||||
Statement. Confirm `subject[0].digest.sha256` exactly matches the
|
||||
manifest digest we're about to install. This is the cryptographic
|
||||
binding from "what the signer attested" to "what we're about to
|
||||
apply".
|
||||
|
||||
If any step fails, the verifier returns an error. The OTA loop logs
|
||||
it (with anyhow chain), bumps `consecutive_failures` (driving
|
||||
backoff), and never touches the OTA partition.
|
||||
|
||||
## Future work
|
||||
|
||||
- **Rekor SET verification.** The Sigstore Bundle includes
|
||||
`tlogEntries[]` with Rekor's Signed Entry Timestamp. Verifying it
|
||||
on-device would give us (a) detection of a Fulcio key compromise
|
||||
that doesn't appear in the public log and (b) a trusted timestamp
|
||||
to enforce the cert validity window (Fulcio certs are 10 min, so
|
||||
without trusted time we currently skip the validity check). Bundle
|
||||
Rekor's public key, parse the SET, verify the inclusion proof.
|
||||
- **GitHub Actions OIDC trust scoping by `job_workflow_ref`.**
|
||||
Currently the GHA identity in `TRUSTED_IDENTITIES` pins to
|
||||
`publish.yml@refs/heads/main`. We could also enforce
|
||||
`1.3.6.1.4.1.57264.1.x` Fulcio extensions like `Run Invocation
|
||||
URI` for stricter trust.
|
||||
- **Force-update GPIO button.** Wire a button to a GPIO; on short
|
||||
press, skip the poll wait and trigger an immediate poll.
|
||||
- **NVS provisioning mechanism.** `OtaConfig::load_from_nvs` already
|
||||
reads `repo`/`tag`/`poll_secs` from NVS, but we have no way to
|
||||
write them from outside the firmware. Could add a tiny HTTP
|
||||
endpoint on the device, a serial-console command, or push config
|
||||
changes as a separate OCI artifact channel.
|
||||
- **`If-None-Match` on manifest fetches.** Marginal bandwidth savings
|
||||
given the manifest is ~500 bytes and we already digest-compare
|
||||
locally. Worth it only if poll interval drops well below 60 s.
|
||||
Loading…
Add table
Add a link
Reference in a new issue