Apple's FastFrames Reaches 36 ms per Video Frame, With Research-Only Weights
Apple released FastFrames-S, a 1.3-billion-parameter model, and FastFrames-L, a 14-billion-parameter model. The team reports 36 ms and 45 ms per frame, respectively, for 480p video in its test configurations. The goal is responsive generation, where an input can change the next frames without waiting for a complete clip.
How it reduces latency
FastFrames generates frames autoregressively. The researchers identified VAE decoding and updating the attention KV cache as substantial costs in an ordinary frame-by-frame pipeline. They redesigned those parts of the inference path instead of treating them as fixed overhead.
Hardware and output limits
The smaller model runs on one GPU in the reported setup. The 14B timing uses eight NVIDIA B200 GPUs and should be read with that hardware configuration in mind. The paper describes video quality as competitive with the baselines it tested, with output still limited to 480p.
Who should try it
The release is useful for interactive-video and streaming research. The released weights are for research use. Check the model card and license before planning a product integration, and measure end-to-end input latency on the hardware you actually plan to use.