MP4 MP4 Analyzer by MediaOnDeck

// engineering · memory

Fitting a GB-scale file into a 2 GB wasm heap

A 1.7 GB film has to be analysed block by block inside WebAssembly linear memory — a heap capped at 2 GB that only ever grows. The approach is to load only the byte window covering the target GOP, measured at 0.46 MB. The hard part was not the idea; it was the four traps around it that actually crashed the program.

1. The constraint: a heap that only grows

ModuleInitial memoryMaximum
Container parsing64 MB2 GB
Bitstream analysis512 MB2 GB

WebAssembly linear memory has two properties that shape everything below: it only grows, and it grows in step-wise multiples. Space that has been used is never returned to the operating system. So any design that reads the whole file in is out of the running on GB-scale input within the first second — even if it barely fits, it consumes every remaining byte of headroom.

Phones are tighter still: a single-process budget of roughly 1 GB. Files above 1 GB are therefore physically infeasible on mobile, and the tool shows a confirmation prompt at 900 MB. That is a boundary, not an optimisation to be made later.

2. The approach: load only the byte window covering the target GOP

The key realisation: block analysis does not need the file. It needs the contiguous bytes from the nearest IDR up to the target frame. The other 99.9% of the file is useless for that one request. 

load [winBase, winEnd)  ⊂  the file
where winBase = start offset of the IDR frame opening that GOP
      winEnd  = end offset of the target frame

feeding a frame, as a relative offset: frame.offset - winBase
with length:                           frame.size

Measured on a 1.7 GB film, the window came to 0.46 MB — three orders of magnitude smaller than the file.

One detail makes the implementation simpler: the offset the C side accepts is relative to the loaded buffer (it adds it to the buffer base internally), not an absolute file offset. Since a windowed load requires relative offsets anyway, the two line up and no extra copying is needed.

Windowed loading is an equivalent transform: the same bytes are fed, just relocated in memory. We did not assume that, though — an A/B on a 675 MB file ran every frame through both the windowed and whole-file paths, and the results were byte-for-byte identical by hash.

3. Trap one: teardown frees the file buffer

When a job finishes, the decoder frees the loaded file buffer and nulls the pointer. That is correct resource management, but it also kills an optimisation that looks entirely natural: "load each file only once".

The old code had exactly that cache, and the symptom was: a second block analysis of the same file always returns empty, with no error. The cache hit meant no reload happened, but the buffer had already been freed by the previous teardown — so feeding frames used a null pointer.

Hence a hard rule: do not cache a resource that frees itself. A window is a few megabytes and reloading costs about a millisecond — cheaper and safer than any cache.

4. Trap two: the heap only grows, so "small file, no risk" is false

The natural assumption is that small files cannot run into trouble. Measurement killed it.

When the C side runs away — typically a loop that should terminate but does not — heap consumption is unrelated to input size. It doubles its way upward until it crosses 2 GB and calls abort(). Measured: an 8.6 MB file consumed 1 GB and then crashed.

  • "The file is small" is not a safety argument. The real defences are bounded loops and a byte-level guard — not a judgement call about the input.
  • You cannot see wasm heap usage from JavaScript. The technique is to load the wasm in Node and sample the heap at stages; and you must print e.stack, because only then does the crash trace contain wasm function names — otherwise you get a list of addresses.

5. Trap three: a safety valve measured in frames is the wrong shape

The walk-back to an IDR needs an upper bound, otherwise a misjudged file will eat all the memory. The first bound was 240 frames.

That produced a permanent false "GOP too long" on one film. Its IDR interval turned out to be 250 frames, so the samples sitting 241 to 249 frames from an IDR — about 3.6% of them — fell outside the bound no matter how healthy the file was.

The underlying mistake: GOP length is a property of the content, not of the code. A frame-count threshold encodes an unverified assumption about the material, and no file can guarantee that assumption for you.

Switching to a byte budget (128 MB) makes the threshold express the real constraint — memory — without silently excluding valid frames.

Relaxing the frame limit is safe for three reasons:

  • the decoder reuses a single frame object across frames, so memory does not accumulate with the number of frames fed;
  • the DPB (decoded picture buffer) is bounded by max_dec_frame_buffering in the SPS, typically no more than 16;
  • so peak memory for "decode 250 frames" and "decode 25 frames" is nearly the same — the difference is time (see below).

One valve was deliberately kept: the cap on frames retained (64). It bounds latency, not memory. Do not treat it as the same kind of thing as the walk-back bound.

6. Trap four: reading past the end of a bitstream is silent

A bitstream reader that runs past the end throws nothing. It enters a silent regime: readBit() returns 0 forever and readUE() returns -1. The program has no idea it is reading memory that does not exist.

So every "loop until you hit the sentinel" construct needs both guards, with an explicit negative break:

while (!bs.eof() && n < LIMIT) {
    const idc = bs.readUE();
    if (idc < 0) break;        // sentinel read: must be rejected explicitly
    if (idc > 2) break;
    ...
}

This is not theoretical. The real case: the loop's only validity check was idc > 2, and -1 > 2 is false — so -1 passed as valid, the loop never exited, the heap doubled upward, and the process aborted.

7. The cost: saved memory, not saved time

Windowed loading solves memory. It does nothing for time. To display the frame 250 frames after an IDR, all 250 frames have to be decoded — inter prediction leaves no shortcut.

MetricValue
Software decode per frame≈ 9.0 ms
Worst case (250-frame IDR interval)≈ 2.2 s
IDR interval itselfA property of the content; not controllable

Which is why operations like this ship with a progress bar and cancellation instead of freezing the interface — and why that 64-frame retention cap exists as a latency ceiling.

8. Transferable lessons

  • Never cache a resource that frees itself. A cache hit must agree with the lifetime of the cached object, or failure will be silent.
  • Set thresholds on the resource, not on the shape of the content. "At most 240 frames" sounds conservative but smuggles in an assumption about the material; "at most 128 MB" speaks directly to the constraint.
  • For any loop that stops on bad data, write the upper bound first — especially when the underlying reader reports nothing and just returns sentinel values.
  • Verify equivalent transforms; do not assume them. Hashing windowed output against whole-file output is cheap and buys a claim you can actually write down.

Related: 623 keyframes, zero detected · when the browser decoder rejects a stream · How it works · Open the analyzer