How many frames are in a movie?

In IDL 8.2.3, we introduced video read capabilities, though the IDLffVideoRead class and the READ_VIDEO function, to complement the video write capabilities available since IDL 8.1. I'm still new to using video as a data format, so I thought I'd post an example of something interesting that I learned recently. We've included a video of a coronal mass ejection viewed from NASA's SDO and SOHO spacecraft in the IDL distribution:

IDL> video_file = file_which('CME.mp4')

How many frames are in this video file? The answer isn't as simple I'd expected. Start with QUERY_VIDEO, which can return a structure of information about a video file:

IDL> !null = query_video(video_file, video_info)
IDL> print, video_info.num_frames
         574

There are 574 frames in the file. But wait, let's try to read the entire file into IDL with READ_VIDEO:

IDL> movie = read_video(video_file, /all)
IDL> help, movie
MOVIE           BYTE      = Array[3, 512, 288, 564]

READ_VIDEO returns frames as pixel-interleaved RGB images. So I guess there are only 564 frames in the file? Let's turn to the lower-level API exposed in IDLffVideoRead to check whether it provides different information. The GetStreams method gives information about the single video stream in the file:

IDL> v = idlffvideoread(video_file)
IDL> print, (v.getstreams()).count
         574

If you look at the source code for QUERY_VIDEO, you'll see that it uses this technique for returning the frame count. But what about iterating through the file, reading frame by frame, until the end is reached?  This code block:

   i = 0
   repeat begin
      data = v.getnext(type=t)
      ++i
   endrep until t eq -1

does so. The result:

IDL> print, --i
         564

OK, why are there two different values for the number of frames in the video file? I asked an engineer on the IDL team, Andrew Magill, who is far more knowledgeable about video than I am, about this. He offered a pair of possibilities:

It's possible FFmpeg doesn't actually know ahead of time how many frames there are. The number out of ::GetStreams might be an estimate based on video length, framerate, file size, etc. Or maybe there are actually 574 frames, but the last 9 can't be decoded.

Andrew also gave some technical details that I haven't included, and suggested that these may not be the only possibilities. Further, I thought his summary was enlightening:

Unfortunately, video technology is full of these little technical gotchas, and seems to be full of questions that can only be answered with "well, it depends".  FFmpeg can seem really inconsistent sometimes, but I think they've done a heroic job of making all these different standards work almost exactly the same.

I hope that through Andrew's work, and the power of FFmpeg, we can make video processing a straightforward task in IDL. I'll post other examples of working with video as I learn more about it!