Why frame-by-frame video analysis cannot scale
Sending one frame per second to a frontier vision model is roughly $200 a month per heavy user. Most of those frames cannot answer the question you are asking, so the cost is unscalable as a product.
Frame-by-frame video analysis cannot scale if you extract one frame per second and send each image to a frontier model. A 60-second clip becomes 60 vision completions, plus tokens spent describing bathrooms and lighting. Do that for every asset in a monitor and the bill lands around $200 a month per heavy user before you have answered a useful question. The pipeline that survives is triage: prepare frames against the specific visual question, drop near-duplicates, and only pay for frames that can change the record.
The frame-rate budget burn
Scripts, “Claude Code” loops, and weekend prototypes all converge on this pattern because it is the obvious one. Either you hide the cost by undersampling frames — and miss the SKU that is on screen for 400 milliseconds — or you stop calling it a platform price.
Naive pipeline versus dynamic frame analysis
Cost shape from internal R&D on the naive 1 fps loop| Approach | What gets billed | What you miss |
|---|
| 1 fps into a frontier model | Every frame, including the bathroom wall | Budget; or you downsample and miss short pack-shots |
| Undersample to survive the bill | Fewer frames | SKUs on screen for a few hundred milliseconds |
| Dynamic frame analysis | Frames that can answer the VQA task | Near-duplicates and frames that cannot change the record |
Dynamic frame analysis
- Classify the task: pack-shot, logo, claims, wear-test.
- Drop near-duplicate frames.
- Keep frames that can answer that task.
- Only then send those frames to a large model.
Coverage is then a function of the task, not frames-per-second vanity. You still see the Water Bronzer. You do not pay to caption the same bathroom wall 47 times.
How we know this
These notes come from building Noodle4’s review and listening pipelines, not from a published academic sample. Cost curves, intent splits, and ASR collisions are from internal R&D and client-shaped tests. They will not match every category. We include them because the failure mode is structural — not because one campaign is universal.
Alex Gray is the founder of Noodle4 and a former technical AI product owner. He works with agencies to apply multimodal AI to creator screening, draft approval, and post-publication monitoring.
Frequently asked questions
Why is AI video analysis so expensive?
Naive pipelines extract one frame per second and send each image to a frontier model. At campaign volume that cost is unscalable. Most of those frames cannot answer the question you are actually asking.
How does Noodle4 analyse video without frame-by-frame LLM calls?
Dynamic frame analysis triages frames for the specific visual question first, drops near-duplicates, and only sends the frames that can change the record.