Bit-Perfect Checker

What it is

Bit-Perfect Checker proves whether a playback chain delivers a file bit-perfect, with every sample unchanged. The chain is the player, the operating system and its USB audio stack.

It has two parts: the BPC1 test files, and a USB audio device that checks every sample it receives. To the host, the device is an ordinary USB audio output, like a DAC. You play a test file to it with any player on any operating system that supports USB Audio Class 2, and the device gives a verdict per file.

A pass proves that every sample arrived unchanged. A fail almost always means a real change; rarely, a USB packet the device missed fails a file the host sent intact. It does not say what changed when a file did not.

Downloads

The first release is not published yet; these links work once it is.

The test files come in four formats, one zip each. Start with WAV. A player can decode each format through a different path, so a file that passes as WAV may still fail as FLAC, AIFF or ALAC: test every format you play.

Each zip holds the PCM files at 16 and 24 bits at all ten rates from 44.1 to 768 kHz, DSD64 to DSD256 as DoP, and a folder negative/ with deliberately faulty files that must not check as bit-perfect. Each zip also has a README.txt and a SHA256SUMS of its audio files.

• bpc1-r1-wav.zip about 402 MB

  PCM WAV and DSD: DSF, DFF and DoP in WAV. Only this zip has DSF and DFF. The device takes PCM only, so play DSF and DFF files with the player’s DoP output. If a FLAC, AIFF or ALAC file does not play, the WAV file of the same name holds the same samples.

• bpc1-r1-flac.zip about 311 MB

  FLAC. The files at 705.6 and 768 kHz are outside the FLAC subset and need a decoder based on libFLAC 1.4.0 or later.

• bpc1-r1-aiff.zip about 495 MB

  AIFF, for Apple devices.

• bpc1-r1-alac.zip about 324 MB

  ALAC (Apple Lossless, .m4a), for Apple devices: the Music app takes ALAC. The files are encoded with Apple’s reference ALAC codec. Apple’s own decoders refuse ALAC above 384 kHz, so the 705.6 and 768 kHz files are for players with their own ALAC decoder. Players that decode ALAC with ffmpeg decode some of them wrongly (ffmpeg issue 25727); the WAV or AIFF file of the same name holds the same samples.

Code-sweep files, optional

The regular files test the paths of a player and a device with a low-level payload. The sweep files carry every sample code of 16 and 24 bits at full scale, and prove that every code arrives unchanged. Play them like the regular files.

They are full-scale noise for their whole length, the 24-bit ones for up to 7 minutes. Never play them on speakers or headphones.

• bpc1-r1-sweep-wav.zip about 1.38 GB
• bpc1-r1-sweep-flac.zip about 1.31 GB
• bpc1-r1-sweep-aiff.zip about 1.38 GB
• bpc1-r1-sweep-alac.zip about 1.34 GB

Checksums

SHA256SUMS lists the SHA-256 of every zip. All releases are on the releases page.

How to play

1. Connect the device to the host you want to test (see The device).
2. Select “Bit-Perfect Checker” as the audio output.
3. Set the player to bit-perfect output: exclusive mode, no volume control, no EQ and no resampling. Turn off ReplayGain, Sound Check, volume normalization, limiters and “prevent clipping” options. They change the samples, and the file fails.
4. Press the user button B1 on the board to start a new run.
5. Play the files, one after another or as a playlist.
6. Read the result on the LEDs, or in the serial log.

The files reach full scale. A player or tool that measures levels may show clipping or peaks at 0 dBFS. That is expected, and harmless when the player passes the samples unchanged.

What the LEDs mean

Blue shows the USB connection and the stream.

• Blue, fast blink: no host has set up the device yet.
• Blue, slow blink: a host has set it up; waiting for a stream.
• Blue, steady: a stream is playing.

Green and red show the whole run since B1 was pressed or the board was reset.

• Red, steady: a file is NOT_BIT_PERFECT. The file was found, but it did not arrive bit-perfect.
• Red, blinking: there is an UNKNOWN_SIGNAL result. Audio arrived that belongs to no test file; silence before, between and after the files does not count. Resampling, dither, rounding or inversion can also turn a test file into an unknown signal.
• Red and green blinking together: there is a DEVICE_OVERRUN result. The device could not judge a file. Play it again.
• Green: every result is BIT_PERFECT.
• Green and red off: nothing judged yet.

When several apply, the first in this list shows. All three LEDs on means that the firmware stopped on a fault.

Only BIT_PERFECT is a pass. Green cannot show that every file arrived, because a file that does not arrive gives no result, and neither does a file that arrives as pure silence: compare the number of BIT_PERFECT results in the serial log with the files you played (python3 -m bpcctl run --expect does this).

Long runs

To find rare faults, play the files one after another continuously, for hours if needed. Or loop one file gaplessly, so that it arrives as one continuous stream. Every repetition gets its own result.

The device

Today the device is an STM32 NUCLEO-U5A5ZJ-Q development board with the project’s firmware. It has two USB ports. Connect both: the ST-LINK USB port gives power and the serial log, and the user USB-C port goes to the host under test.

There is no ready-made firmware binary yet. Build the firmware from source and flash it, as the firmware README describes. A ready-made binary will follow.

The serial log

The device writes one line per event to the ST-LINK’s virtual COM port, at 115200 8N1, and takes commands there. Open that port to read it, or use bpcctl. The firmware README lists every line and field.

What the log shows

Each line is a time stamp, an event and its fields, mostly key=value. The main ones:

• stream start: a stream begins, with its id, rate, slot and the allow_silence setting.
• stat: every second while streaming: the average rate, the bits in use and the packets lost so far.
• result: the outcome of one file. Each stretch of the stream outside every file that is not silence also gets one, with UNKNOWN_SIGNAL.
• stream stop: the stream ended, after its last result line. outside= counts its frames outside every file, results= its result lines, and results_lost= the results the device dropped because its queue was full.

For example, a 24-bit 48 kHz file that the device could not judge (time stamps left out):

result seq=30 stream=30 file=1 verdict=DEVICE_OVERRUN kind=PCM dop=0 seed=0x42504331 nch=2 file_rate=48000 depth=24 rate=48000 alt=4 slot=3 res=24 bits=24 end=LAST diff=- lost=0 ovf=684 dur=12.458

stream stop id=30 reason=alt dur=12.649 frames=607164 pkts=101194 rate=48000.00 fpp=6-6 lost=0 missed=0 holes=0 maxhole=0 doubled=0 ovf=684 outside=0 silent=0 results=1 results_lost=0

• verdict: BIT_PERFECT, NOT_BIT_PERFECT, UNKNOWN_SIGNAL or DEVICE_OVERRUN. Only BIT_PERFECT is a pass.
• kind: the file’s kind: PCM, SWEEP, DSF, DFF or DOP, and NONE for an unknown signal.
• depth: the file’s depth: 16, 24 or 32, and 1 for DSD.
• rate, slot: the frame rate and the slot size in bytes that the host chose.
• bits: the bits in use in the received samples.
• diff: for NOT_BIT_PERFECT, where the first difference is, in seconds from the file’s start; otherwise -.
• lost, ovf: USB packets lost, and overflows of the device’s receive buffer, since the stream’s previous result line.

All fields are in the firmware README’s result lines.

Hosts pad their streams with silence; an iPad sends about 0.2 s of zeros before each file. The device accepts silence before, between and after the files by default, and the stream stop line counts it in silent=. Silence inside a file is still a difference. A file muted to silence gives no result, so count the results (bpcctl run --expect). The allow_silence setting set to false makes any silence outside a file UNKNOWN_SIGNAL.

Console commands

Send one command per line. Each gets a one-line answer. After opening the port, send an empty line first, since the line may hold noise from a reset.

• status: the USB state, rate, slot, whether a stream runs, its losses, the formats offered and allow_silence.
• version: the firmware version, the board and the USB ID.
• result: the last result line again.
• mark <text>: writes the text to the log, to tie a script’s steps to it, and starts a new run, like B1.
• formats [<list> | default]: shows the formats offered; with a list such as 24/3,16/2, offers those instead until the next reset. The device then reconnects.
• allow_silence true|false: sets the allowed silence for the next stream, until the next reset.
• help: lists the commands.

bpcctl, the host tool

bpcctl reads the log and sends commands, for test automation. It is part of the bit-perfect-checker repository: run it from the repository’s root. It needs Python 3, and on Windows also pyserial.

• python3 -m bpcctl monitor: prints the log.
• python3 -m bpcctl status, version, result: send that command and print the answer.
• python3 -m bpcctl formats [<list> | default]: shows or changes the formats; after a change it returns once a host has mounted the device again.
• python3 -m bpcctl allow-silence true|false: sets the allowed silence.
• python3 -m bpcctl run -- <player command>: marks the log, runs the player, waits for the result lines of its streams, prints them and exits with the status below.

With run --expect <n>, fewer than n results that are BIT_PERFECT or DEVICE_OVERRUN fail the run: this catches a file that did not arrive. --json prints each line as a JSON object. The port is $BPC_PORT, else /dev/ttyACM0; --port overrides both. For example, on Linux:

python3 -m bpcctl run -- \
  aplay -D hw:CARD=Checker bpc1_pcm_096k_24bit.wav

The firmware README’s From a script has the same steps by hand.

More

• The bit-perfect-checker repository on GitHub: the generator, the reference software checker and the firmware.
• The BPC1 signal format, the specification of the test files.