// engineering · performance

为了读 4 个字节，搬了 92% 的文件

打开一部 1.7 GB 的影片，只是要读出每一帧是 I / P / B。为此搬运了 1506 MB—— 文件的 92%，外加 15.5 万次内存分配，耗时 6.9 秒。改造之后读取量降到 0.84%（13.7 MB），耗时 460 毫秒，帧类型逐字节不变。

一、症状：打开文件要等好几秒

文件明明只在本地，也只是列个帧表，却要等 6.9 秒才出结果。这类「纯读元数据却很慢」的现象，原因通常不是算法复杂，而是读得太多。

当时读帧类型的做法是：定位到样本的起点与长度，然后把这个样本整块读出来，再从里面解析 slice 头，取出 slice_type 这一个字段。

指标	改造前	改造后
读取字节数	1506 MB（92%）	13.7 MB（0.84%）
分析耗时	约 6.9 s	约 460 ms
内存分配次数	约 15.5 万	大幅下降
帧类型结果	逐字节相同

二、先量，再判断

这里最值得记下来的不是优化本身，而是我们最初是怎么判断错的。

当时的推理是这样的：「媒体数据盒（mdat）不是容器型盒子，里面没有子盒子结构，所以读盒子树的时候不会碰到媒体数据——因此这条路径不会读大文件。」

这个推理听起来完全合理，但结论是错的。原因很简单：「读盒子树时不会碰媒体数据」和「整条流程都不会碰媒体数据」是两件事——读帧类型是另一条路径，它按样本偏移去取数据，与盒子树无关。

正确的第一步是加一个计数器：在这条路径真正读数据的地方累加字节数，跑一部大文件，把数字打出来。实测 92% 的结果一出来，方向立刻就明确了。 不要从数据结构推断 I/O 量，直接量。

三、现方案：递增窗口

真正需要的字节其实很少：解析一个 slice 头，通常几十字节就够。所以改成按需放大的窗口，而不是一次读满：

窗口起点 = 样本起点
窗口大小 = 64 B

循环:
  在窗口内扫 NAL 链
  找到承载数据的 NAL（slice NAL）？
     └─ 是 → 在当前窗口里解析 slice 头，结束
     └─ 否 → 窗口大小 ×4，重试（有次数上限）

之所以能「截断了也照解析」，是因为解析需要的只是 slice 头，而它总在 NAL 的开头部分。所以窗口边界取 min(NAL 结束位置, 已读数据末尾)，并额外留 8 字节余量即可。

四、两个容易写错的边界

情况	正确做法	写错的后果
窗口截断了一个 slice NAL	照常解析（只需开头部分）	无——这是可以容忍的截断
窗口截断了一个非 slice NAL	必须放大重试	该样本后面可能还有 slice NAL，被当成「没有 slice」→ 帧类型判错

第二行是这次真正踩到的坑。以 SEI 开头的样本很常见（[SEI, SEI, IDR]），如果窗口在 SEI 中间就被截断，而代码没有放大重试，就会得出「这个样本里没有 slice」的错误结论—— 帧类型于是错掉，而且看起来完全正常。

区分点很清楚：截断的 slice 是「读够了」，截断的非 slice 是「还没读完」。两者必须走不同的分支。

五、必须逐条对拍，而不是抽查

改读法一定会让人担心「结果是不是变了」。这里的验证方式是逐帧逐字节对拍：把改造前后的帧类型序列完整导出，做全量 diff。

要求是：

素材必须覆盖 H.264 与 H.265 各一份（两种码流的 slice 头结构不同）；
尺寸上覆盖 大文件与小文件（小文件才会暴露窗口放大逻辑的边界）；
比较对象是全量帧类型序列，不是「随便挑几帧看看」。

理由是这次的改动性质：它理论上是等价变换（少读的那部分本来就没用到），但「理论上等价」必须被证明。而证明它只需要一次全量对拍——比事后排查一个「某类素材帧类型偏掉」便宜太多。

六、可迁移的经验

I/O 量要量，不要推。从数据结构推断「会不会读大文件」会得出完全相反的结论。加个字节计数器，跑一次大文件，比任何推理都可靠。
想「流式化」，先找最小必要窗口。绝大多数解析只用到某个结构的开头几十字节。先确定「最少需要多少」，再决定读多少——而不是反过来，先读满再截。
把「截断」当成一等公民。区分「读够了但被截断」与「还没读完」，前者可以直接解析，后者必须重试。这两条路径如果混在一起，会产生静默错误。
等价变换必须全量证明。「少读的部分用不到」是可用于设计的推理，但不是可用于交付的证据。全量对拍一次，把推理变成事实。

// engineering · performance

Moving 92% of a file to read four bytes

Opening a 1.7 GB film, all we needed was each frame's I / P / B type. Doing so moved 1506 MB — 92% of the file — plus about 155,000 allocations, in 6.9 seconds. After the rework: 0.84% (13.7 MB) and 460 ms, with byte-identical frame types.

1. The symptom: opening a file takes seconds

The file is local, and all we produce is a frame table — yet it took 6.9 seconds. "Reading metadata is slow" almost never means the algorithm is expensive; it means you are reading too much.

Frame-type detection used to locate the sample's start and length, then read the whole sample, purely to parse a slice header and pick out one field: slice_type.

Metric	Before	After
Bytes read	1506 MB (92%)	13.7 MB (0.84%)
Analysis time	≈ 6.9 s	≈ 460 ms
Allocations	≈ 155,000	Sharply down
Frame types	Byte-for-byte identical

2. Measure first, then judge

The valuable part here is not the optimisation but how we first got it wrong.

The reasoning was: "the media data box (mdat) is not a container box — it holds no child boxes — so walking the box tree never touches media data, and therefore this path does not read large files."

That sounds entirely reasonable, and the conclusion was wrong. "Walking the box tree does not touch media data" and "the whole pipeline never touches media data" are two different claims. Frame type detection is a separate path: it seeks by sample offset and knows nothing about the box tree.

The right first move is a counter: accumulate bytes at every point where this path actually reads, run it on a large file, print the number. The moment 92% appeared, the direction was settled. Do not infer I/O volume from data structures — measure it.

3. The approach: a growing window

Very few bytes are actually needed: parsing a slice header usually takes a few dozen. So the read became a grow-on-demand window instead of a full read:

window start = sample start
window size  = 64 B

loop:
  scan the NAL chain inside the window
  did we find the NAL that carries picture data (a slice NAL)?
     └─ yes → parse the slice header in the current window; done
     └─ no  → window size ×4, retry (with an attempt cap)

Parsing can proceed even from a truncated window because everything needed lives at the very start of the NAL. So the window bound is min(end of NAL, end of data read) with an extra 8-byte margin.

4. Two boundaries that are easy to get wrong

Case	Correct behaviour	Consequence of getting it wrong
The window truncates a slice NAL	Parse it as usual (only the start is needed)	None — this truncation is harmless
The window truncates a non-slice NAL	Must grow the window and retry	A slice NAL later in the sample is missed → wrong frame type

The second row is the trap actually hit here. Samples beginning with SEI are common ([SEI, SEI, IDR]). If the window is cut in the middle of that SEI and the code does not grow and retry, it concludes "this sample contains no slice" — the frame type goes wrong, and it looks entirely normal.

The distinction is clean: a truncated slice means "read enough"; a truncated non-slice means "not finished reading". The two must take different branches.

5. Verify by full diff, not by sampling

Changing how bytes are read always raises the question "did the results change?". The answer here came from a frame-by-frame, byte-by-byte diff: export the entire frame-type sequence before and after, and compare all of it.

The requirements:

material must cover H.264 and H.265 (their slice headers differ);
and both large and small files (small files expose the window-growth boundaries);
the comparison is the full frame-type sequence, not "a few frames look fine".

The reason is the nature of this change: it is supposed to be an equivalent transform (the bytes no longer read were never used). But "supposed to be equivalent" has to be demonstrated — and one full diff is far cheaper than later chasing "frame types are wrong on some material".

6. Transferable lessons

Measure I/O, never infer it. Reasoning about "does this touch large files" from data structures produced exactly the wrong answer. A byte counter and one large file beat any amount of inference.
To stream something, find the minimum window first. Most parsing needs only the first few dozen bytes of a structure. Establish "how little do we need" before deciding how much to read — not the other way round.
Treat truncation as a first-class case. Separate "read enough, but truncated" from "not finished reading". Conflating them produces silent errors.
Equivalent transforms need full proof. "The bytes we skip are unused" is good enough for design, not for delivery. One full diff turns reasoning into fact.