Testing and verification¶

Written 2026-10-27 from the project's own docs and code.

Both projects verify with cargo test --workspace before anything is called done, then add checks that go further than ordinary unit tests. This page covers the ones that are actually in use.

A test has to be able to fail¶

The most repeated rule in both repos: prove a new test can fail. A test that passes might be checking the right thing, or it might be incapable of noticing anything wrong, and the two print the same green result. So each important test is checked by breaking the code it guards and watching the test go red.

The Crawl records these deliberate breaks next to its claims:

This kind of checking is called mutation testing: mutate the code, expect the tests to kill the mutant. (It's also where The Crawl hit the stale-build problem described in Gotchas.)

Tests named after rules¶

The Crawl's design documents number their rules (R1, R24, ...), and every rule gets a test named after its ID, so a failing test names the rule it broke:

#[test]
fn r30_b_roll_lags_one_floor_from_main_show_floor_4() { /* ... */ }

#[test]
fn r32_generic_characters_are_deleted_on_death_even_if_respawn_is_chosen() { /* ... */ }

Where the design is silent, the code marks the gap ASSUMPTION and lists it in the handoff notes for the designer to decide, rather than quietly picking an answer.

Golden-file parity against a reference implementation¶

Fboard's music engine reimplements a subset of Strudel, a JavaScript live-coding tool. To show it behaves the same, a script runs real Strudel over a list of expressions and saves every event it produces into a JSON fixture, crates/fboard_music/tests/strudel_golden.json. The Rust test parses the same expressions and compares:

#[test]
fn matches_strudel_onsets() {
    let golden: Value = serde_json::from_str(include_str!("strudel_golden.json")).unwrap();
    // for each case: parse case["code"] with our engine, collect onsets over
    // golden["cycles"] cycles, sort both lists, and compare exactly
    assert!(failures.is_empty(), "{} of {} cases differ:\n{}", /* ... */);
}

A few details that make it trustworthy:

• Times are compared as exact fractions ([1, 2] means ½), not floats with a tolerance.
• Every case runs, and every failure is collected, so one run reports all mismatches with the first differing event for each.
• include_str! compiles the fixture into the test binary, so the test can't silently run against a missing file.
• The fixture records its generator (Strudel's version and the script), so it can be regenerated when the supported subset grows.

The current fixture has 39 expressions and 438 events over 4 cycles. All match. One deliberate exception is documented: random operators are deterministic in Fboard but don't reproduce Strudel's random sequence, so they aren't in the fixture.

The pattern is reusable for any rewrite: let the original produce the answers, commit them as data, and make the new code match them exactly.

Replays with state-hash checkpoints¶

Fboard records runs as one input per 60 Hz tick. Every 300 ticks, the replay stores a hash of the player's full state:

/// FNV-1a over the player's full serialized state. RON prints floats in
/// shortest round-trip form, so any bit of drift changes the hash.
pub fn state_hash(player: &Player) -> u64 {
    let text = ron::to_string(&(&player.skater, &player.run)).expect("player serializes");
    text.bytes().fold(0xcbf29ce484222325, |h, b| (h ^ b as u64).wrapping_mul(0x100000001b3))
}

Playing a replay back and comparing hashes shows whether the simulation still does exactly what it did when the run was recorded. Per the handoff notes, headless release runs and client dev runs agree bit for bit, including grinds, manuals, vert and spins, and a one-tick edit to an input is caught. Inputs are run-length encoded ((repeat count, input)) to keep files small.

Replays double as regression tests: when Fboard's physics was retuned, replays were made to store which tuning they were recorded under, and old files default to the old tuning "and still verify."

Tests that prove a level is playable¶

When Fboard's physics moved to real-world scale, several gaps in the levels became too wide to clear. Each resized gap is "proven by a sim test": the test drives the simulation through the jump and checks that it lands. Level data lives in RON files, and catalog tests catch broken references between levels and items.

Smoke runs you look at¶

Unit tests don't show what a player sees. Both games can drive themselves through a scripted scenario and save screenshots (Fboard's FBOARD_SMOKE, The Crawl's CRAWL_SMOKE). The working rule in both repos is to look at the screenshots: "a clean exit alone proves little." Test runs point the game's data folder at a scratch directory (and, in Fboard, set master volume to zero), so nothing touches real saves or plays through the speakers.

Being explicit about what isn't verified¶

Both handoff documents keep a "not verified" list next to the "verified" one. For example, Fboard's music engine passes its parity and unit tests, but the notes say plainly that nobody has listened to it yet and the live audio path was only exercised muted. A passing test suite is reported as exactly what it covers.