Parameters involving frame count in detection algorithms

[SAGNIKMJR]

Hi, are the default values for parameters like min_scene_len in multiple detection algorithms and window_width in AdaptiveDetector assume a certain fps (like videos at 30 fps or something )? If not, I was wondering if these paratmeres should be tied to temporal intervals in seconds, instead of frame count.

[wjs018]

If you are using the cli, then the default min_scene_len should be defined as a time already, specifically 0.6 seconds (docs).

Using the python api, the min_scene_len parameter is defined for each detector you add. By default (and by convention up to this point) the value for min_scene_len is 15 frames for all the detectors. The value passed to a detector via the python api needs to be an int. So, passing a string like 0.6s would error out due to being an incorrect type. You would need to include logic prior to the detector construction to calculate how many frames should be used for min_scene_len based on the desired time interval and framerate of the video being analyzed.

As for window width, that parameter currently only accepts integer numbers of frames rather than a timecode.

---

That is how things work now. This does seem like something that could be harmonized in a future version though. I can imagine having the detector __init__ functions be a bit more flexible in what types they will accept as a min_scene_len parameter (window_width as well). Ideally, you would probably want the detector __init__ functions to accept the same inputs as the cli currently accepts.

[SAGNIKMJR]

Hi @wjs018, thanks for your response!

I am using the python api and understand the types of different parameters. My question/comment was more about the logic going behind setting the parameter values. My understanding is that the parameter values should be based on the temporal duration rather than the frame count. Does that sound right?

Since the default min_scene_len in the api is 15 frames and CLI is 0.6 s, does it mean that the assumption is that 15 frames is equivalent to 0.6 s of the video?

[wjs018]

I can't speak for @Breakthrough as for the rationale behind 15 frames being the default. If I remember correctly, the cli used to only use number of frames as well but was updated to also accept timecode arguments in later versions. So, in that respect, I suspect that the 15 frame default came first and then the 0.6 seconds came later.

For reference, most film content is ~24 fps. With that framerate, 15 frames corresponds to 0.625 seconds which pretty closely aligns and I suspect is where the 0.6 seconds cli default comes from.

[Breakthrough]

One shortcoming of the existing API is that detectors work in frames (#168), so you need to handle different framerates when dealing with times. You can also use FrameTimecode objects.

There is no assumption that 15 frames is equivalent to 0.6s of the video. With the current API (0.6.x), the min_scene_len param for any detector is in frames.

This is something I want to address in the future in the next major release. Ideally min_scene_len should be enforced by SceneManager and not individual detectors, so this work will take some time to overhaul the API. It is planned to fix things like this up for a final 1.0 release in the future however.