Bit-Perfect Checker

What it is

Bit-Perfect Checker proves whether a playback chain delivers a file bit-perfect, with every sample unchanged. The chain is the player, the operating system and its USB audio stack.

Checking that a player delivers audio bit-perfect is usually hard. A USB protocol analyzer (from about $200) captures what the host sends, but comparing it with the file is left to you. A digital loopback recording needs an audio interface with digital inputs, and the recording software’s own conversions can hide or cause changes. A few DACs, such as RME’s ADI-2 series, have a built-in bit test: they recognise a short pattern in their maker’s test files and show when it arrived unchanged. Bit-Perfect Checker turns an inexpensive development board (an STM32 Nucleo, about $35) into a USB audio sink that a host plays to like any DAC. It compares every sample it receives with the test signal, and shows the result on its LEDs and in a serial log.

It has two parts: the BPC1 test files, and a USB audio device that checks every sample it receives. To the host, the device is an ordinary USB audio output, like a DAC. You play a test file to it with any player on any operating system that supports USB Audio Class 2, and the device gives a verdict per file.

A pass proves that every sample arrived unchanged. A fail almost always means a real change; rarely, a USB packet the device missed fails a file the host sent intact.

The device takes every rate up to 768 kHz. Whether every USB packet reaches it depends on the host: an iPad delivered every rate without losing a packet, while a Linux PC lost a few at the highest rates. A missed packet shows as lost in the result, so it is told apart from an altered sample. If a file fails at 705.6 or 768 kHz with lost above 0, play it again.

Downloads

The test files come in four formats, one zip each. Start with WAV. A player can decode each format through a different path, so a file that passes as WAV may still fail as FLAC, AIFF or ALAC: test every format you play.

Each of the four format zips holds the PCM files at 16 and 24 bits at all ten rates from 44.1 to 768 kHz and DSD64 to DSD256 as DoP, all files that should pass. Each zip also has a README.txt and a SHA256SUMS of its audio files.

Every file name starts with a five-digit number, for example 00033-bpc1_pcm_096k_24bit.wav and 00133-bpc1_pcm_096k_24bit.flac. Put the files of all zips in one folder and they play in a fixed order: regular PCM, regular DSD, sweep, gapless pairs, negative tests. Files whose names agree after the number hold the same samples. The title tag gives the same number and the format.

• bpc1-r1-wav.zip about 381 MB

  PCM WAV and DSD: DSF, DFF and DoP in WAV. Of the four format zips, only this one has DSF and DFF. The device takes PCM only, so play DSF and DFF files with the player’s DoP output. The DSF, DFF and DoP files of one DSD rate hold the same DSD bits. If a FLAC, AIFF or ALAC file does not play, the WAV file with the same name after the number holds the same samples.

• bpc1-r1-flac.zip about 286 MB

  FLAC. The format defines a streamable subset that every FLAC decoder is meant to play, and in it each frame states its own sample rate, which it can only do up to 655,350 Hz. The files at 705.6 and 768 kHz, and the DSD256 DoP file, are above that, so their rate is only in the file’s header and they are outside the subset. The reference library, libFLAC, handles such rates since version 1.4.0 (2022). Players built on an older libFLAC, or on a decoder that handles only the subset, may refuse these files; the WAV file with the same name after the number holds the same samples.

• bpc1-r1-aiff.zip about 454 MB

  AIFF, for Apple devices.

• bpc1-r1-alac.zip about 298 MB

  ALAC (Apple Lossless, .m4a), for Apple devices: the Music app takes ALAC. The files are encoded with Apple’s reference ALAC codec. Apple’s own decoders refuse ALAC above 384 kHz, so the 705.6 and 768 kHz files are for players with their own ALAC decoder. Players that decode ALAC with ffmpeg decode some of them wrongly (ffmpeg issue 25727); the WAV or AIFF file with the same name after the number holds the same samples.

Code-sweep files

The regular files carry quiet pseudo-random noise (about −36 dBFS at 16 bits, −48 dBFS at 24 bits). Every block starts with a full-scale burst that sets and clears every bit on its own, top bits included. Any processing that touches all samples (volume, resampling, dither, EQ, clipping) shows at once. They do not carry every sample value: a 16-bit file’s noise uses 1,024 of the 65,536 values. The sweep files carry every value of 16 and 24 bits at full scale, and prove that every value arrives unchanged.

A path is proven bit-perfect when both the regular and the sweep files pass. Play the sweep files after the regular files, in the same way.

They are full-scale noise for their whole length, the 24-bit ones for up to 7 minutes. Never play them on speakers or headphones.

• bpc1-r1-sweep-wav.zip about 1.27 GB
• bpc1-r1-sweep-flac.zip about 1.20 GB
• bpc1-r1-sweep-aiff.zip about 1.27 GB
• bpc1-r1-sweep-alac.zip about 1.23 GB

Gapless pairs, optional

These test whether your player joins two tracks without a gap. Play them only after the regular files pass in the same player and format (see Gapless playback).

• bpc1-r1-gapless.zip about 57 MB

  18 pairs in six folders: wav/, aiff/, flac/ and alac/ with PCM at 44.1 and 48 kHz, 16 and 24 bits (4 pairs each), and dsf/ and dff/ with DSD64 (1 pair each).

Negative tests, optional

Deliberately faulty files that must not check as bit-perfect, one per fault. They test the checker, not your player: play them to see the device or the software checker reject a file. Each must give NOT_BIT_PERFECT. The checker does not depend on the format, so they are WAV only.

The altered sweep file is full-scale noise for its whole length. Never play it on speakers or headphones.

• bpc1-r1-negative.zip about 38 MB

  Seven WAV files, numbered 90000 to 91001: PCM at 44.1 kHz 24-bit and DSD64 as DoP, each altered and incomplete; the 44.1 kHz 16-bit sweep with one value near full scale changed, which is full-scale noise; and a gapless pair with 10 ms of silence before part 2, which played gaplessly gives one NOT_BIT_PERFECT with diff=7.500. Its README.txt says what each must give.

The firmware is not among the downloads yet: see The device.

Checksums

SHA256SUMS lists the SHA-256 of every zip. All releases are on the releases page.

How to play

1. Connect the device to the host you want to test (see The device).
2. Select “Bit-Perfect Checker” as the audio output.
3. Set the player to bit-perfect output: exclusive mode, no volume control, no EQ and no resampling. Turn off ReplayGain, Sound Check, volume normalization, limiters and “prevent clipping” options. They change the samples, and the file fails.
4. Press the user button B1 on the board to start a new run.
5. Play the files, one after another or as a playlist: the regular files first (a few minutes per format), then the sweep files (about 28 minutes per format).
6. Read the result on the LEDs, or in the serial log.

The files reach full scale. A player or tool that measures levels may show clipping or peaks at 0 dBFS. That is expected, and harmless when the player passes the samples unchanged.

What the LEDs mean

Blue shows the USB connection and the stream.

• Blue, fast blink: not connected: no host has set up the device.
• Blue, slow blink: a host has set it up; waiting for a stream.
• Blue, steady: a stream is open.

Green and red show the whole run since B1 was pressed or the board was reset.

• Red, steady: a file is NOT_BIT_PERFECT. The file was found, but it did not arrive bit-perfect.
• Red, slow blink: there is an UNKNOWN_SIGNAL result. Audio arrived that belongs to no test file; silence before, between and after the files does not count. Resampling, dither, rounding or inversion can also turn a test file into an unknown signal.
• Red, fast blink: there is an OVERRUN result. The device could not judge a file. Play it again.
• Green: every result is BIT_PERFECT. No other verdict lights green.
• Green and red off: nothing judged yet.

When several apply, the first in this list shows. All three LEDs on means that the firmware stopped on a fault.

The slow and fast red blinks come with firmware 1.1.0. Firmware 1.0.0 blinks red at one rate for UNKNOWN_SIGNAL, and red and green together for an overrun.

Only BIT_PERFECT is a pass (for a gapless pair, with lost=0). Green cannot show that every file arrived, because a file that does not arrive gives no result, and neither does a file that arrives as pure silence: compare the number of BIT_PERFECT results in the serial log with the files you played (python3 -m bpcctl run --expect does this).

Long runs

To find rare faults, play the files one after another continuously, for hours if needed. Or loop one file gaplessly, so that it arrives as one continuous stream. Every repetition gets its own result.

Gapless playback

The gapless pairs in bpc1-r1-gapless.zip test one more thing: whether your player joins two tracks without a gap. That means no silence inserted, no samples dropped or repeated, no fade or crossfade, and the stream not closed between the tracks. Play them only after the regular files pass in the same player and format. They assume that samples already arrive unchanged.

Each pair is one test file cut in two at 7.5 s. Part 1 and part 2 are tagged as tracks 1 and 2 of one album. Play part 1, then part 2, with gapless playback on and crossfade off, for example by playing a whole folder. Read the result lines in the serial log. A pair gives:

• One BIT_PERFECT with lost=0: the player joined the parts exactly. BIT_PERFECT with lost above 0 is not a pass: the host sent no audio for a moment, which a DAC plays as a dropout. Play the pair again.
• One NOT_BIT_PERFECT: with diff=7.500, or a little less, the player changed the join. It inserted silence, dropped or repeated samples, faded or crossfaded, or started the DoP markers again. A 10 ms crossfade gives 7.490; the end of part 1 cut short also gives less.
• Two NOT_BIT_PERFECT: with diff=7.500, then 7.523 (44.1 kHz and DSD64) or 7.509 (48 kHz), the player closed the stream after part 1 and opened a new one for part 2.

To see what a gap looks like, play the pair with 10 ms of silence before part 2 from the negative tests the same way: it gives one NOT_BIT_PERFECT with diff=7.500.

Each part alone is NOT_BIT_PERFECT. The software checker in the repository judges a pair as one stream, as a gapless player joins it: python3 -m checker --join PART1 PART2.

The device

Today the device is an STM32 NUCLEO-U5A5ZJ-Q development board with the project’s firmware. It has two USB ports. Connect both: the ST-LINK USB port gives power and the serial log, and the user USB-C port goes to the host under test.

No compiled firmware image is published yet. Firmware 1.0.0 uses the pid.codes test USB ID, 1209:0001, which is for testing only and may not be distributed in a ready-made image, so build it from the repository (tag firmware-v1.0.0) and flash it, as the firmware README describes. Firmware 1.1.0 will use 16C0:27E0 (why) and come with a compiled image.

The serial log

The device writes one line per event to the ST-LINK’s virtual COM port, at 115200 8N1, and takes commands there. Open that port to read it, or use bpcctl. The firmware README lists every line and field.

What the log shows

Each line is a time stamp, an event and its fields, mostly key=value. The main ones:

• stream start: a stream begins, with its id, rate, slot and the allow_silence setting.
• stat: every second while streaming: the average rate, the bits in use and the packets lost so far.
• result: the outcome of one file. Each stretch of the stream outside every file that is not silence also gets one, with UNKNOWN_SIGNAL.
• stream stop: the stream ended, after its last result line. outside= counts its frames outside every file, results= its result lines, and results_lost= the results the device dropped because its queue was full.

For example, a 24-bit 48 kHz file that the device could not judge (time stamps left out):

result seq=30 stream=30 file=1 verdict=OVERRUN kind=PCM dop=0 seed=0x42504331 nch=2 file_rate=48000 depth=24 rate=48000 alt=4 slot=3 res=24 bits=24 end=LAST diff=- lost=0 ovf=684 dur=12.458

stream stop id=30 reason=alt dur=12.649 frames=607164 pkts=101194 rate=48000.00 fpp=6-6 lost=0 missed=0 holes=0 maxhole=0 doubled=0 ovf=684 outside=0 silent=0 results=1 results_lost=0

• verdict: BIT_PERFECT, NOT_BIT_PERFECT, UNKNOWN_SIGNAL or OVERRUN (firmware 1.0.0 writes DEVICE_OVERRUN). Only BIT_PERFECT is a pass.
• kind: the file’s kind: PCM, SWEEP or DSD (a DSF, DFF or DoP file alike; dop=1 when it came as DoP), and NONE for an unknown signal.
• depth: the file’s depth: 16, 24 or 32, and 1 for DSD.
• rate, slot: the frame rate and the slot size in bytes that the host chose.
• bits: the bits in use in the received samples.
• diff: for NOT_BIT_PERFECT, where the first difference is, in seconds from the file’s start; otherwise -.
• lost, ovf: USB packets lost, and overflows of the device’s receive buffer, since the stream’s previous result line.

All fields are in the firmware README’s result lines.

Hosts pad their streams with silence; an iPad sends about 0.2 s of zeros before each file. The device accepts silence before, between and after the files by default, and the stream stop line counts it in silent=. Silence inside a file is still a difference. A file muted to silence gives no result, so count the results (bpcctl run --expect). The allow_silence setting set to false makes any silence outside a file UNKNOWN_SIGNAL.

Console commands

Send one command per line. Each gets a one-line answer. After opening the port, send an empty line first, since the line may hold noise from a reset.

• status: the USB state, rate, slot, whether a stream runs, its losses, the formats offered and allow_silence.
• version: the firmware version, the board and the USB ID.
• result: the last result line again.
• mark <text>: writes the text to the log, to tie a script’s steps to it, and starts a new run, like B1.
• formats [<list> | default]: shows the formats offered; with a list such as 24/3,16/2, offers those instead until the next reset. The device then reconnects.
• allow_silence true|false: sets the allowed silence for the next stream, until the next reset.
• help: lists the commands.

bpcctl, the host tool

bpcctl reads the log and sends commands, for test automation. It is part of the bit-perfect-checker repository: run it from the repository’s root. With firmware 1.0.0, use the bpcctl of the tag firmware-v1.0.0, which knows that firmware’s verdict names. It needs Python 3, and on Windows also pyserial.

• python3 -m bpcctl monitor: prints the log.
• python3 -m bpcctl status, version, result: send that command and print the answer.
• python3 -m bpcctl formats [<list> | default]: shows or changes the formats; after a change it returns once a host has mounted the device again.
• python3 -m bpcctl allow-silence true|false: sets the allowed silence.
• python3 -m bpcctl run -- <player command>: marks the log, runs the player, waits for the result lines of its streams, prints them and exits with the status below.

With run --expect <n>, fewer than n results that are BIT_PERFECT or OVERRUN fail the run: this catches a file that did not arrive. --json prints each line as a JSON object. The port is $BPC_PORT, else /dev/ttyACM0; --port overrides both. For example, on Linux:

python3 -m bpcctl run -- \
  aplay -D hw:CARD=Checker \
  00033-bpc1_pcm_096k_24bit.wav

The firmware README’s From a script has the same steps by hand.

FAQ

Is this the same as RME’s Bit Test?

The idea is similar; the depth is not. RME’s Bit Test, in the ADI-2 series converters (see the ADI-2 Pro manual, section 34.21), recognises a short bit pattern of about 400 samples, repeated in its test files. It shows “Bit Test passed” with the bit depth it found. It finds common faults quickly: level changes, EQ, dither, bit reduction, polarity, channel swaps.

Bit-Perfect Checker compares every sample of every test file, at ten rates up to 768 kHz, as PCM and DoP, in WAV, FLAC, AIFF and ALAC. It gives a verdict for each file, with where the first difference lies and how many packets were lost. Its sweep files prove that every sample value of 16 and 24 bits arrives unchanged.

Do I need to test every format (WAV, AIFF, FLAC, ALAC)?

Yes, every format you play. Unless you control the whole audio chain, which is rare, you cannot be sure which parts a player uses for every format and which only for one: each format has its own decoder, and players often take a different path for each. That is why the test files come in several formats. Start with WAV: it holds plain samples, so if it fails, the other formats will too. The files of one name (after the number) hold the same samples in every format, so a pass as WAV and a fail as FLAC or ALAC points at that format’s decoder. For example, ffmpeg’s ALAC decoder, used by players built on ffmpeg, decodes the files ffmpeg’s own encoder makes correctly, but some files made by Apple’s encoder wrongly, among them some of the ALAC test files at 192 kHz and above (ffmpeg issue 25727).

Do I need to test the DSD files?

Only if you play DSD. DSD takes a different path in the player than PCM: it reads the DSF or DFF file, packs the DSD bits into DoP frames with their alternating markers, keeps the channels in order and switches the output to the DoP rate. A player that passes every PCM file can still get one of these steps wrong, or convert DSD to PCM, which many players do by default. The DoP files, DSD already packed as DoP inside WAV, FLAC, AIFF or ALAC, check that the path carries DoP untouched. The DSF, DFF and DoP files of a DSD rate hold the same DSD bits, so the checker gives them the same result when the path is bit-perfect.

Why is there no DSD sweep?

On the way to the device, DSD travels as DoP: 24-bit words at 176.4, 352.8 or 705.6 kHz. The 24-bit PCM sweep files at these rates already carry every 24-bit value, so they prove that every DoP word passes the path unchanged. What remains is the player’s packing of the DSD bits, and its faults do not depend on particular values: the regular DSD files find them.

Which USB ID does the device use, and why?

From firmware 1.1.0, 16C0:27E0, Objective Development’s shared VID/PID pair for audio class devices, under the serial-number rules of its USB-IDs-for-free.txt. It is free and needs no request. The pair is shared, so devices that use it tell themselves apart by their serial number. The device’s serial number is bpc.metebalci.com: followed by the chip’s unique ID in hex. Firmware 1.0.0 used the pid.codes test ID 1209:0001, which is for testing only.

The device is a standard USB Audio Class 2.0 device, so every host plays to it with its built-in class driver, whatever its ID. The ID still matters, because hosts attach vendor drivers and quirks to specific VID:PID pairs. An ID taken from another product could make a host treat the checker as that product, and a VID belongs to whoever registered it. The shared pair is reserved for audio class devices that use the host’s own class driver, so no vendor driver or quirk applies to it.

Whoever distributes a modified firmware sets the build option USB_SERIAL_PREFIX to a domain or e-mail address of their own, as the firmware README describes.

More

• The bit-perfect-checker repository on GitHub: the generator, the reference software checker and the firmware.
• The BPC1 signal format, the specification of the test files.