Bit-Perfect Checker

What it is

Bit-Perfect Checker proves whether a playback chain delivers a file bit-perfect, with every sample unchanged. The chain is the player, the operating system and its USB audio stack.

Checking that a player delivers audio bit-perfect is usually hard. A USB protocol analyzer (from about $1,300) captures what the host sends, but comparing it with the file is left to you. A digital loopback recording needs an audio interface with digital inputs, and the recording software’s own conversions can hide or cause changes. A few DACs, such as RME’s ADI-2 series, have a built-in bit test: they recognise a short pattern in their maker’s test files and show when it arrived unchanged. Bit-Perfect Checker turns an inexpensive development board (an STM32 Nucleo, about $35) into a USB audio sink that a host plays to like any DAC. It compares every sample it receives with the test signal, and shows the result on its LEDs and in a serial log.

It has two parts: the BPC1 test files, and a USB audio device that checks every sample it receives. To the host, the device is an ordinary USB audio output, like a DAC. You play a test file to it with any player on any operating system that supports USB Audio Class 2, and the device gives a verdict per file.

A pass proves that every sample arrived unchanged. A fail almost always means a real change; rarely, a USB packet the device missed fails a file the host sent intact.

Downloads

The first release is not published yet; these links work once it is.

The test files come in four formats, one zip each. Start with WAV. A player can decode each format through a different path, so a file that passes as WAV may still fail as FLAC, AIFF or ALAC: test every format you play.

Each of the four format zips holds the PCM files at 16 and 24 bits at all ten rates from 44.1 to 768 kHz and DSD64 to DSD256 as DoP, all files that should pass. Each zip also has a README.txt and a SHA256SUMS of its audio files.

• bpc1-r1-wav.zip about 381 MB

  PCM WAV and DSD: DSF, DFF and DoP in WAV. Of the four format zips, only this one has DSF and DFF. The device takes PCM only, so play DSF and DFF files with the player’s DoP output. If a FLAC, AIFF or ALAC file does not play, the WAV file of the same name holds the same samples.

• bpc1-r1-flac.zip about 286 MB

  FLAC. The format defines a streamable subset that every FLAC decoder is meant to play, and in it each frame states its own sample rate, which it can only do up to 655,350 Hz. The files at 705.6 and 768 kHz, and the DSD256 DoP file, are above that, so their rate is only in the file’s header and they are outside the subset. The reference library, libFLAC, handles such rates since version 1.4.0 (2022). Players built on an older libFLAC, or on a decoder that handles only the subset, may refuse these files; the WAV file of the same name holds the same samples.

• bpc1-r1-aiff.zip about 454 MB

  AIFF, for Apple devices.

• bpc1-r1-alac.zip about 297 MB

  ALAC (Apple Lossless, .m4a), for Apple devices: the Music app takes ALAC. The files are encoded with Apple’s reference ALAC codec. Apple’s own decoders refuse ALAC above 384 kHz, so the 705.6 and 768 kHz files are for players with their own ALAC decoder. Players that decode ALAC with ffmpeg decode some of them wrongly (ffmpeg issue 25727); the WAV or AIFF file of the same name holds the same samples.

Code-sweep files

The regular files carry quiet pseudo-random noise (about −36 dBFS at 16 bits, −48 dBFS at 24 bits). Every block starts with a full-scale burst that sets and clears every bit on its own, top bits included. Any processing that touches all samples (volume, resampling, dither, EQ, clipping) shows at once. They do not carry every sample value: a 16-bit file’s noise uses 1,024 of the 65,536 values. The sweep files carry every value of 16 and 24 bits at full scale, and prove that every value arrives unchanged.

A path is proven bit-perfect when both the regular and the sweep files pass. Play the sweep files after the regular files, in the same way.

They are full-scale noise for their whole length, the 24-bit ones for up to 7 minutes. Never play them on speakers or headphones.

• bpc1-r1-sweep-wav.zip about 1.27 GB
• bpc1-r1-sweep-flac.zip about 1.20 GB
• bpc1-r1-sweep-aiff.zip about 1.27 GB
• bpc1-r1-sweep-alac.zip about 1.23 GB

Gapless pairs, optional

These test whether your player joins two tracks without a gap. Play them only after the regular files pass in the same player and format (see Gapless playback).

• bpc1-r1-gapless.zip about 57 MB

  18 pairs in six folders: wav/, aiff/, flac/ and alac/ with PCM at 44.1 and 48 kHz, 16 and 24 bits (4 pairs each), and dsf/ and dff/ with DSD64 (1 pair each).

Negative tests, optional

Deliberately faulty files, altered or incomplete, that must not check as bit-perfect. Play them to see the device or the checker reject a file: each must give NOT_BIT_PERFECT.

The two altered sweep files in each folder are full-scale noise for their whole length. Never play them on speakers or headphones.

• bpc1-r1-negative.zip about 575 MB

  The same 8 files in each of four folders, wav/, flac/, aiff/ and alac/: PCM at 44.1 and 48 kHz 24-bit and DSD64 as DoP, each altered and incomplete, and two altered sweep files, which are full-scale noise. Its README.txt says what each must give.

Checksums

SHA256SUMS lists the SHA-256 of every zip. All releases are on the releases page.

How to play

1. Connect the device to the host you want to test (see The device).
2. Select “Bit-Perfect Checker” as the audio output.
3. Set the player to bit-perfect output: exclusive mode, no volume control, no EQ and no resampling. Turn off ReplayGain, Sound Check, volume normalization, limiters and “prevent clipping” options. They change the samples, and the file fails.
4. Press the user button B1 on the board to start a new run.
5. Play the files, one after another or as a playlist: the regular files first (a few minutes per format), then the sweep files (about 28 minutes per format).
6. Read the result on the LEDs, or in the serial log.

The files reach full scale. A player or tool that measures levels may show clipping or peaks at 0 dBFS. That is expected, and harmless when the player passes the samples unchanged.

What the LEDs mean

Blue shows the USB connection and the stream.

• Blue, fast blink: no host has set up the device yet.
• Blue, slow blink: a host has set it up; waiting for a stream.
• Blue, steady: a stream is playing.

Green and red show the whole run since B1 was pressed or the board was reset.

• Red, steady: a file is NOT_BIT_PERFECT. The file was found, but it did not arrive bit-perfect.
• Red, blinking: there is an UNKNOWN_SIGNAL result. Audio arrived that belongs to no test file; silence before, between and after the files does not count. Resampling, dither, rounding or inversion can also turn a test file into an unknown signal.
• Red and green blinking together: there is a DEVICE_OVERRUN result. The device could not judge a file. Play it again.
• Green: every result is BIT_PERFECT.
• Green and red off: nothing judged yet.

When several apply, the first in this list shows. All three LEDs on means that the firmware stopped on a fault.

Only BIT_PERFECT is a pass (for a gapless pair, with lost=0). Green cannot show that every file arrived, because a file that does not arrive gives no result, and neither does a file that arrives as pure silence: compare the number of BIT_PERFECT results in the serial log with the files you played (python3 -m bpcctl run --expect does this).

Long runs

To find rare faults, play the files one after another continuously, for hours if needed. Or loop one file gaplessly, so that it arrives as one continuous stream. Every repetition gets its own result.

Gapless playback

The gapless pairs in bpc1-r1-gapless.zip test one more thing: whether your player joins two tracks without a gap. That means no silence inserted, no samples dropped or repeated, no fade or crossfade, and the stream not closed between the tracks. Play them only after the regular files pass in the same player and format. They assume that samples already arrive unchanged.

Each pair is one test file cut in two at 7.5 s. Part 1 and part 2 are tagged as tracks 1 and 2 of one album. Play part 1, then part 2, with gapless playback on and crossfade off, for example by playing a whole folder. Read the result lines in the serial log. A pair gives:

• One BIT_PERFECT with lost=0: the player joined the parts exactly. BIT_PERFECT with lost above 0 is not a pass: the host sent no audio for a moment, which a DAC plays as a dropout. Play the pair again.
• One NOT_BIT_PERFECT: with diff=7.500, or a little less, the player changed the join. It inserted silence, dropped or repeated samples, faded or crossfaded, or started the DoP markers again. A 10 ms crossfade gives 7.490; the end of part 1 cut short also gives less.
• Two NOT_BIT_PERFECT: with diff=7.500, then 7.523 (44.1 kHz and DSD64) or 7.509 (48 kHz), the player closed the stream after part 1 and opened a new one for part 2.

Each part alone is NOT_BIT_PERFECT. The software checker in the repository judges a pair as one stream, as a gapless player joins it: python3 -m checker --join PART1 PART2.

The device

Today the device is an STM32 NUCLEO-U5A5ZJ-Q development board with the project’s firmware. It has two USB ports. Connect both: the ST-LINK USB port gives power and the serial log, and the user USB-C port goes to the host under test.

There is no ready-made firmware binary yet. Build the firmware from source and flash it, as the firmware README describes. A ready-made binary will follow.

The serial log

The device writes one line per event to the ST-LINK’s virtual COM port, at 115200 8N1, and takes commands there. Open that port to read it, or use bpcctl. The firmware README lists every line and field.

What the log shows

Each line is a time stamp, an event and its fields, mostly key=value. The main ones:

• stream start: a stream begins, with its id, rate, slot and the allow_silence setting.
• stat: every second while streaming: the average rate, the bits in use and the packets lost so far.
• result: the outcome of one file. Each stretch of the stream outside every file that is not silence also gets one, with UNKNOWN_SIGNAL.
• stream stop: the stream ended, after its last result line. outside= counts its frames outside every file, results= its result lines, and results_lost= the results the device dropped because its queue was full.

For example, a 24-bit 48 kHz file that the device could not judge (time stamps left out):

result seq=30 stream=30 file=1 verdict=DEVICE_OVERRUN kind=PCM dop=0 seed=0x42504331 nch=2 file_rate=48000 depth=24 rate=48000 alt=4 slot=3 res=24 bits=24 end=LAST diff=- lost=0 ovf=684 dur=12.458

stream stop id=30 reason=alt dur=12.649 frames=607164 pkts=101194 rate=48000.00 fpp=6-6 lost=0 missed=0 holes=0 maxhole=0 doubled=0 ovf=684 outside=0 silent=0 results=1 results_lost=0

• verdict: BIT_PERFECT, NOT_BIT_PERFECT, UNKNOWN_SIGNAL or DEVICE_OVERRUN. Only BIT_PERFECT is a pass.
• kind: the file’s kind: PCM, SWEEP, DSF, DFF or DOP, and NONE for an unknown signal.
• depth: the file’s depth: 16, 24 or 32, and 1 for DSD.
• rate, slot: the frame rate and the slot size in bytes that the host chose.
• bits: the bits in use in the received samples.
• diff: for NOT_BIT_PERFECT, where the first difference is, in seconds from the file’s start; otherwise -.
• lost, ovf: USB packets lost, and overflows of the device’s receive buffer, since the stream’s previous result line.

All fields are in the firmware README’s result lines.

Hosts pad their streams with silence; an iPad sends about 0.2 s of zeros before each file. The device accepts silence before, between and after the files by default, and the stream stop line counts it in silent=. Silence inside a file is still a difference. A file muted to silence gives no result, so count the results (bpcctl run --expect). The allow_silence setting set to false makes any silence outside a file UNKNOWN_SIGNAL.

Console commands

Send one command per line. Each gets a one-line answer. After opening the port, send an empty line first, since the line may hold noise from a reset.

• status: the USB state, rate, slot, whether a stream runs, its losses, the formats offered and allow_silence.
• version: the firmware version, the board and the USB ID.
• result: the last result line again.
• mark <text>: writes the text to the log, to tie a script’s steps to it, and starts a new run, like B1.
• formats [<list> | default]: shows the formats offered; with a list such as 24/3,16/2, offers those instead until the next reset. The device then reconnects.
• allow_silence true|false: sets the allowed silence for the next stream, until the next reset.
• help: lists the commands.

bpcctl, the host tool

bpcctl reads the log and sends commands, for test automation. It is part of the bit-perfect-checker repository: run it from the repository’s root. It needs Python 3, and on Windows also pyserial.

• python3 -m bpcctl monitor: prints the log.
• python3 -m bpcctl status, version, result: send that command and print the answer.
• python3 -m bpcctl formats [<list> | default]: shows or changes the formats; after a change it returns once a host has mounted the device again.
• python3 -m bpcctl allow-silence true|false: sets the allowed silence.
• python3 -m bpcctl run -- <player command>: marks the log, runs the player, waits for the result lines of its streams, prints them and exits with the status below.

With run --expect <n>, fewer than n results that are BIT_PERFECT or DEVICE_OVERRUN fail the run: this catches a file that did not arrive. --json prints each line as a JSON object. The port is $BPC_PORT, else /dev/ttyACM0; --port overrides both. For example, on Linux:

python3 -m bpcctl run -- \
  aplay -D hw:CARD=Checker bpc1_pcm_096k_24bit.wav

The firmware README’s From a script has the same steps by hand.

FAQ

Is this the same as RME’s Bit Test?

The idea is similar; the depth is not. RME’s Bit Test, in the ADI-2 series converters (see the ADI-2 Pro manual, section 34.21), recognises a short bit pattern of about 400 samples, repeated in its test files. It shows “Bit Test passed” with the bit depth it found. It finds common faults quickly: level changes, EQ, dither, bit reduction, polarity, channel swaps.

Bit-Perfect Checker compares every sample of every test file, at ten rates up to 768 kHz, as PCM and DoP, in WAV, FLAC, AIFF and ALAC. It gives a verdict for each file, with where the first difference lies and how many packets were lost. Its sweep files prove that every sample value of 16 and 24 bits arrives unchanged.

Do I need to test every format (WAV, AIFF, FLAC, ALAC)?

Yes, every format you play. Unless you control the whole audio chain, which is rare, you cannot be sure which parts a player uses for every format and which only for one: each format has its own decoder, and players often take a different path for each. That is why the test files come in several formats. Start with WAV: it holds plain samples, so if it fails, the other formats will too. The files of one name hold the same samples in every format, so a pass as WAV and a fail as FLAC or ALAC points at that format’s decoder. For example, ffmpeg’s ALAC decoder, used by players built on ffmpeg, decodes the files ffmpeg’s own encoder makes correctly, but some files made by Apple’s encoder wrongly, among them some of the ALAC test files at 192 kHz and above (ffmpeg issue 25727).

Do I need to test the DSD files?

Only if you play DSD. DSD takes a different path in the player than PCM: it reads the DSF or DFF file, packs the DSD bits into DoP frames with their alternating markers, keeps the channels in order and switches the output to the DoP rate. A player that passes every PCM file can still get one of these steps wrong, or convert DSD to PCM, which many players do by default. The DoP files, DSD already packed as DoP inside WAV, FLAC, AIFF or ALAC, check that the path carries DoP untouched.

Why is there no DSD sweep?

On the way to the device, DSD travels as DoP: 24-bit words at 176.4, 352.8 or 705.6 kHz. The 24-bit PCM sweep files at these rates already carry every 24-bit value, so they prove that every DoP word passes the path unchanged. What remains is the player’s packing of the DSD bits, and its faults do not depend on particular values: the regular DSD files find them.

More

• The bit-perfect-checker repository on GitHub: the generator, the reference software checker and the firmware.
• The BPC1 signal format, the specification of the test files.