Extracting frames from a video with ffmpeg very slow if not using jpeg
Extracting frames from a video with ffmpeg very slow if not using jpeg
I'm trying to extract the frames of a video as individual images but it's really slow, except when I'm using jpeg. The obvious issue with jpegs is the data loss from the compression, I want the images to be lossless. Extracting them as jpegs manages about 50-70 fps but as pngs it's only 4 fps and it seems to continue getting slower, after 1 minute of the 11 minute video it's only 3.5 fps.
I suspect it's because I'm doing this on an external 5tb hard drive, connected over USB 3.0 and the write speed can't keep up. So my idea was to use a different image format. I tried lossless jpeg xl and lossless webp but both of them are even slower, only managing to extract at about 0.5 fps or something. I have no idea why that's so slow, the files are a lot smaller than png, so it can't be because of the write speed.
I would appreciate it if anyone could help me with this.
Honestly I don't know, but it seems to me like extracting every single frame of a video as a lossless PNG is only really something that's necessary if you're trying to archive something or do frame by frame restoration. Either way, it is something that you hopefully aren't doing every day, so why not just let it run overnight & move on?
Otherwise ask yourself if you can settle with just extracting a single clip/section, or what's actually wrong with lossy jpeg with a low -qscale:v (high quality) - start around 5 and work down until you visually can't see any difference
I'm doing this to upscale and interpolate the video and I want the best quality possible, since the source is using h.264 and I'm exporting to AV1. I was using jpeg with qscale:v 0 and 100% quality but you could still see compression artifacts, which is why I want to use a lossless format now. The upscaling and interpolation also takes quite a lot of time, so I'm also trying to minimize the time each step takes, if possible, since I'll be doing this with multiple videos and I'll probably use these scripts I made in the future a few more times.
It probably becomes CPU limited with those other compression algorithms.
You could use something like
atopto find the bottleneck.Yeah, that's the probably the case for those. I looked at CPU usage when using webp and one CPU core was always at 100%. Even tough it seems to not be able to use multiple cores, that's still really slow, no? Or is that normal?
Also, my CPU is a Ryzen 5 3600, just to get an idea of what performance would be expected.
My first thought was similar - there might be some hardware acceleration happening for the jpgs that isn't for the other formats, resulting in a CPU bottleneck. A modern harddrive over USB3.0 should be capable of hundreds of megabits to several gigabits per second. It seems unlikely that's your bottleneck (though you can feel free to share stats and correct the assumption if this is incorrect - if your pngs are in the 40 megabyte range, your 3.5 per second would be pretty taxing).
If you are seeing only 1 CPU core at 100%, perhaps you could split the video clip, and process multiple clips in parallel?
PNG is a good format for graphics, lettering, logos... not photography so unless your video is some cartoons you're using png compression for something is not meant for.
I agree that you're not really leveraging any features of PNG like you would using JPEG or RAW here, but saying it's not meant for this use is an odd way to phrase it. There's nothing inherently wrong with wanting lossless compression on an image...
PNG is a rather slow algorithm based on the DEFLATE compression from zip/gzip. You could extract to bmp or some other uncompressed format. First, to ensure it is lossless, make sure it supports the video's pix_fmt without needing conversion.
I'll bet with mpeg to jpeg it doesn't have to re-encode the image, which it's doing with the other formats.
h.264 (the compression algorithm the video uses) and jpeg are entirely different, so it does have to re-encode
Actually they both use Discrete Cosign Transform!
PNGs use DEFLATE which is a generic compression standard that exhaustively searches for smaller ways to compact the data.
I would recommend comparing the quality of images of different formats against eachother to see if there is noticeable lossyness.
If the PNGs are indeed better, try to set the initial compression of the PNGs to "zero" and come back later to "crush" them smaller.