MP4 MP4 Analyzer by MediaOnDeck

// engineering · pixels

10-bit footage renders black: one shift amount

The same export code was byte-for-byte correct on 8-bit sources and produced a fully black frame on a 10-bit source. The cause: a hardcoded right shift of 8 for converting 16-bit samples to 8-bit, when P010LE and yuv420p10le place their 10-bit values at different positions inside the 16-bit word.

1. Symptom: a completely black frame

This happened while adding a path that lets the bundled FFmpeg wasm decode the target frame itself, for use as a preview when the browser's decoder is unavailable. The verification method was to export pixels and diff them against ffmpeg's command-line output.

The result was clean, and strange:

Test clipPixel formatResult
8-bit source yuv420p Byte-for-byte identical
10-bit source yuv420p10le Entirely black (mean difference ≈ 92 of 255)

The magnitude is the clue: a mean difference of about 92 on a 255 scale means the picture is not "slightly off" — it has collapsed into a constant. Yet there was a counter-intuitive detail: outlines, motion and glyph shapes were all still there, just crushed to the bottom of the range. Structure correct, values wrong — which rules out "the decode is broken" and points squarely at the numeric conversion during export.

2. What 10-bit pixels actually look like in memory

There is no native 10-bit byte type, so 10-bit samples live in 16-bit containers (two bytes per sample, little-endian). But which part of those 16 bits carries the 10-bit value is decided by the pixel format, and there are two conventions:

ConventionMeaningTypical formats
MSB-aligned 10-bit value occupies the high bits (15..6); low bits are padding — shift = 6 P010LE · P016LE
LSB-aligned 10-bit value sits in the low bits (9..0), usable as-is — shift = 0 yuv420p10le · yuv422p10le · yuv444p10le

In code the two are indistinguishable — both are uint16_t. The difference exists only in format metadata; you cannot see it by eyeballing the byte stream.

3. Root cause: the shift amount was hardcoded to 8

The original code converted 16-bit to 8-bit with a blanket v >> 8. For one convention that is exactly right:

  • P010LE: the value occupies bits 15..6, so shifting right by 8 yields precisely the top 8 bits → correct.
  • yuv420p10le: the value occupies bits 9..0, so shifting right by 8 keeps only the top 2 bits → nearly every sample becomes 0. 
10-bit black level     64   >> 8  →  0
10-bit mid-grey       512   >> 8  →  2
10-bit white         1023   >> 8  →  3

The whole picture's luminance is squeezed into four values, 0 through 3. That is what a black frame looks like — and where the "mean difference ≈ 92" comes from.

The unkind part: this bug is invisible on 8-bit material. The correct shift for 8-bit formats is 0 anyway, and >> 8 happens to be right for MSB-aligned formats like P010LE. One 10-bit test clip is the entire cost of catching it.

4. The correct approach: derive it from the pixel format descriptor

You do not have to work it out yourself: FFmpeg's pixel format descriptor already gives you, per plane, the two values you need.

  • depth — effective bits for that component (8 / 10 / 12 / 16)
  • shift — how far the value was left-shifted inside the container
rshift = shift + depth - 8;   // how far right to reach 8 bits
if (rshift < 0) rshift = 0;             // 0 for 8-bit formats

Checking it against every format covers the real cases exactly:

Pixel formatdepthshiftrshift
yuv420p800
yuv420p10le1002
P010LE1068
yuv420p12le1204
P016LE1608

The same code also removes the need for a per-format branch, because the loop walks planes and offsets rather than switching on a format name:

  • which memory block this component lives in;
  • byte stride between neighbours in the same plane (1 for 8-bit, 2 for 16-bit);
  • start offset within the plane; NV12's interleaved UV is expressed with this plus a stride of 2.

The result: yuv420p, yuv422p, yuv444p, nv12, p010le and yuv420p10le share one packing routine, gated on whether the format is planar rather than on format names.

Two details that are easy to miss: big-endian variants (such as yuv420p10be) reverse the byte order, so 16-bit reads must consult the descriptor's big-endian flag; and this path only covers planar and semi-planar YUV — packed RGB formats need their own branch.

5. How the fix was proven

Choosing the right evidence mattered more than the fix itself. Here you must not compare RGBA — RGBA already contains a chroma upsampling policy, which blends "wrong numeric mapping" together with "different upsampling strategy". That deserves its own note: why pixel export has to be verified on YUV planes.

Compare planes directly. Generate the reference with ffmpeg:

ffmpeg -i input.mp4 \
  -vf "select='eq(n\,<display index>)'" -vsync 0 \
  -pix_fmt yuv420p -f rawvideo ref.yuv
Test clipCriterionResult
8-bit source Byte-for-byte identical 3110400 / 3110400 bytes identical
10-bit source Bytes differing by more than 1 after the new shift Zero

The 10-bit case tolerates a difference of 1 because the source is 10-bit: ffmpeg rounds once when producing the 8-bit reference, and our path rounds once too. Two roundings in opposite directions differ by one. That is not an error.

One more step: replay the chunked feed as well. Same data, chunk sizes 1 / 3 / 8 / 1000 — the output must be identical every time. If changing the chunk size changes the pixels, that would be a real bug.

6. Transferable lessons

  • "The same-looking uint16" is not the same semantics. Bit alignment, byte order and effective depth are all properties of the format. Hardcoding any of them breaks on some class of material — usually not the file you happen to have open.
  • The magnitude of the symptom hints at the class of cause. "Mean difference ≈ 92" says the frame collapsed to a constant; "mean difference 1.4 but individual pixels differ by 97" says a policy difference. Tell those two apart and you can locate the cause in one step.
  • When adding a pixel format, test both 8-bit and 10-bit. This check is nearly free and skipping it is expensive. Once the checklist exists, it pays off on every future change to the pixel path.

Related: verifying pixel export on YUV planes · when the browser decoder rejects a stream · Open the analyzer