FFmpeg can open any CCTV video file?

I've heard people say "Can't FFMpeg open anything?" - unfortunately the answer is a firm 'no'.

Although FFMpeg has improved lately, and can open quite a number of tricky formats - it is quite far from being universal. Yesterday I did a test on the latest build of FFMpeg (v3.4) to see how many CCTV formats it can successfully open and convert.

For test data, I used the Kinesense collection of CCTV formats which we have collected over the years from donations from our partners. In the test, FFMpeg successfully converted only 16.5% of CCTV video file formats.

For some background on the issues surrounding weird and wonderful CCTV file formats, take a look at the eForensic's Article I wrote on this topic.

Interpretation of image and video evidence

The UK's Forensic Science Regulator has published a short useful document on Forensic image comparison and interpretation evidence. Much of what is says should be no surprise to law enforcement and forensic experts who understand video - but its still a useful document as it sets out some points and useful reminders.

Some Highlights:

If the subject to be identified is a substantial distance from the camera or only occupies a small proportion of the image, then the number of pixels that make up the subject may be low. As a result, the level of detail will be very limited and may not be suitable for reliable interpretation. Enlarging an image beyond its resolution will not add additional detail, but will only duplicate existing pixels resulting in a blocky or blurry appearance, depending upon the method of enlargement. The potential for enhancement of low resolution imagery is generally very limited.

There is a great example of how recorder compression artifacts plus 'enhancement' can lead to bad recovery. Based on the enhanced license plate below you may be quite sure the number plate reads DUN 150J, but the real plate is OUM T5GJ.

 Image enhancement should be approached with caution. If you want to know more about what enhancement or clarification can do, check out Kinesense CEO's post on "Video Clarfication: Fact or Fiction". 

Licence Plate Recovery

The good people at VideoCleaner.com have set a 'license plate challenge', saying:

(Note: Doug asked me to block out the actual number plate in this blog post)

Unprecedented... Hmmm... Spready from Amped solved this challenge easily so I thought, I'm sure Kinesense can do this too using our Video Clarification toolkit.

Sure enough, a couple of clicks and the license plate is recovered:

To recover the plate, I used a histogram edit filter (to crop out some of the added noise), temporal median filter to recombine frames, and rotate to correct the orientation. Then I clicked 'histogram equalization' to make it a bit clearer'. About 5 clicks from start to finish.

However, this is clearly not a 'real' license recovery video. By that I mean, the noise 'mode' is artificial and manually added to create the 'license plate challenge', and would never happen in a real video job faced by law enforcement. That said a more realistic video would be solved using the same clarification filters. 

I dug out this example from my archives. It is a read covert surveillance video, in low light with an older analog bullet-cam connected to a commonly used brand of covert surveillance recorder. The issue was caused by the analog camera's auto-gain going too high in low light, and thus adding a huge amount of noise. Running the 'temporal median filter' across the video recovered license plates of any vehicle parked for more than a few minutes in view, or in this screenshot, it recovers the logo on the skip/dumpster.

VideoCleaner set a neat challenge. Recovery under these conditions is not too difficult with the right tools, and its great to see issues like this discussed and highlighted by the community.

H264 is a standard, right?

I've written before about the massive issues around the lack of a true video file standard for CCTV (and the lack of any enforcement of existing standards)

I've often heard claims that H264 is a standard format, and you might see this claim from DVR manufacturers. Unfortunately, most DVR makers seem to be endlessly, er, creative, in how they structure their file formats, even while claiming to use 'H264'.

Our PlayerManager application has 94 different players from DVR manufacturers who label their formats as '264' or 'h264'.

Case in point is Swann, who have at least 3 different mutually incompatible versions of 'h264' files. With some effort, FFMpeg can be made to play most of them. The files often include non-standard headers, mystery extra packets, etc, which can trip up even FFMpeg.

Its been a long hope that the CCTV format issue would eventually 'go away' because manufacturers are moving to more standard encoding formats. Unfortunately, this does not appear to be the case. While more and more DVRs will be labelled 'h264' it seems few are rigorously applying this standard, leaving us investigators back at square one.

London Met fail to play CCTV 118,000 times a year

In 2013 the London Metropolitan Police collected CCTV video files in 118,000 criminal cases and were unable to play the video. For robberies, they were only able to play the video collected 13% of the time.

Most police forces don't keep statistics on this - but I am certain the numbers are similar around the world. The real-life result of the chaos in CCTV video file formats, and the lack of standards is a lot of failed convictions.

London Evening Standard: Met failed to check CCTV footage on 118,000 crimes"Met failed to check CCTV footage on 118,000 crimes", 11/12/2013

CCTV Formats - The Mystery Filetypes

Our Online Vid-ID Tool has been running now for over 2 years and has been used thousands of times by police around the world. Every now and again, someone enters a file type that the system doesn't have data on. When this happens, I get an email and usually a few minutes googling or checking forums like DME Resources or the FVA mailing lists can uncover information - I then add this to the Vid-ID database. Sometimes though I draw a blank. Here are some of the filetypes I have been unable to identify:

Unknown DVR video file extensions:

.tbl	.tho	.ctarc	.stream	.bu	.mp6
.stw	.sdv	.bsk	.rms	.sv4	.ifo
.vsf4	.pvf	.nsf	.rms	.dat5	.def
.t1	.cvi	.lrxh	.iri	.cdt	.nplx
.ifv	.tag	.ivf	.eflv	.nmd	.svc

If you can help identify anything about these video file formats (DVR manufacturer, make/model, player software, or any other clues), you'll be helping the whole law enforcement community. Please email TechSupport @ Kinesense-vca.com or leave a comment below.

H264 Player - what a pain!

This is a very annoying player. Just called "H264 Player", there doesn't seem to be an about box (probably the manufacturer is too embarrassed to claim it). It looks like this:

If you right click on the H264Player.exe file and click properties, you get this:

Not a lot to go on really. The player plays files with ".264". The odd part, is it only plays on some machines; on others it shows a blank black window (as in the image above). It took me a very long time to understand why, but here is the fix: This player can't deal with dual monitors. To solve the issue, you can disconnect your second monitor (physically unplug it as turning it off wont help) or you can use the windows settings to "Duplicate these displays".

1. Right click on your desktop and click the "Screen Resolution" option.

 2. Under the "Multiple Displays" dropdown, pick "Duplicate these Displays". Click "Apply" and accept changes. (Note, to undo this, change back to "Extend these Displays"

3. Restart the H264 Player.

Alternatively, you can try converting the .264 files to AVI using ffmpeg, but I found this doesn't always work for these files. Out of 20 files I tried, at least 3 didn't convert correctly.

This is a command you can try which re-wraps the .264 files without re-encoding them. (More details can be found here: http://spreadys.wordpress.com/2013/01/17/264-files-and-ffmpeg/)

ffmpeg -f h264 -i originalfile.264 -f avi -r 25 -vcodec copy -vsync drop -debug_ts -loglevel verbose -report originalfile-rewrap.avi

The problem I found is that in several files, the resulting avi doesn't play correctly in either VLC or Windows Media player. The issue is that after about 15 minutes of playback, the image breaks up and the screen goes gray. I suspect this is due to some non-standard encoding or dropped frames in the .264 file. For this reason, screen capture from the H264 Player is the best option for converting these files.