Video: A Forensic Approach to Video

Unplayable media is everyone’s nightmare, made all the worse if it could be key evidence in a crimnial case. This is daily fight that Gareth Harbord from the Metropolitan Police has as he tries to render old CCTV footage and files from crashed dash cams playable, files from damaged SD cards and hard drives readable and recover video from old tape formats which have been obselete for years.

In terms of data recovery, there are two main elments: Getting the data off the device and then fixing the data to make it playable. Getting the data off a device tends to be difficult because either the device is damaged and/or connecting to the device requires some proprietary hardware/software which simply isn’t available any more. Pioneers in a field often have to come up with their own way of interfacing which, when the market becomes bigger, is often then improved by a standard way of doing things. Take, as an example, mobile phone cables. They used to be all sorts of shapes and sizes but are now much more uniform with 3 main types. The same was initially true with hard drives, however the first hard drives were so long ago that osolecence is much more of an issue.

Once you have the data on your own system, it’s then time to start analysing it to see why it won’t play. It may play because the data itself is of an old or proprietary format, which Gareth says is very common with CCTV manufacturers. While there are some poular formats, there are many variations from different companies including putting all, say, 4 cameras onto one image or into one file, running the data for the four cameras in parallel. After a while, you start to be able to get a feel for the formats but not without many hours of previous trial and error.

Gareth starts his talk explaining that he works in the download and data receovery function which is different from the people who make the evidence ready for presentation in a trial. Their job is to find the best way to show the relevant parts both in terms of presentation but also technically making sure it is easy to play for the technically uninitiated in court and that it is robust and reliable. Presentation covers the effort behind combining multiple sources of video evidence into one timeline and ensuring the correct chronology. Other teams also deal with enhancing the video and Gareth shows examples of deblurring an image and also using frame averaging to enhance the intelligability of the picture.

Gareth spends some time discussing CCTV where he calls the result of the lack of standardisation “a myriad of madness.” He says it’s not uncommon to have 15-year-old systems which are brought in but, since the hard drives have been spinning for one and half decades, don’t start again when they are repowered. On the otherhand the newer IP cameras are more complicated whereby each camera is generating its own time-stampped video going into a networked video recorder which also has a timestamp. What happens when all of the timestamps disagree?

Mobile devices cause problems due to variable frame rates which are used to deal with dim scenes, non-conformance with standards and who can forget the fun of CMOS videos where the CMOS sensors lead to wobbling of the image when the phone is panned left or right. Gareth highlights a few of the tools he and his colleagues use such as the ever-informative MediaInfo and FFProbe before discussing the formats that they transode to in order to share the videos internally.

Gareth walks us through an example file looking at the how data can be lined up to start understanding the structure and start to decode it. This can lead to the need to write some simple code in C#, or similar, to rework the data. When it’s not possible to get hold of the data in a partiular format to be playable in VLC, or similar, a proprietary player may be the only way forward. When this is the case, often a capture of the computer screen is the only way to excerpt the clip. Gareth looks at the pros and cons of this method.

Watch now!
Speakers

Gareth Harbord Gareth Harbord
Senior Digital Forensic Specialist (Video)
Metropolitan Police Service

Video: LCEVC – The Latest MPEG Standard

Video is so pervasive in our world that we need to move past thinking of codecs and compression being about reducing bitrate. That will always be a major consideration, but speed of compression and the computation needed can also be deal breakers. Millions of embedded devices need to encode video which don’t have the grunt available to the live AV1-encoding clusters in the cloud. Further more, the structure of the final data itself can be important for later processing and decoding. So we can see how use-cases can arise out needs of various industries, far beyond broadcast, which mean that codecs need to do more than make files small.

This year LCEVC from MPEG will be standardised. Called Low Complexity Enhancement Video Coding, this codec provides compression both where computing is constrained and where it is plentiful. Guido Meardi, CEO of V-Nova, talks us through what LCEVC is starting with a chart showing how computation has increased vastly as compression has improved. It’s this trend that this codec intends to put an end to by adding, Guido explains, an enhancement layer over some lower-resolution video. By encoding a lower-resolution, computational processing is minimised. When displayed, an enhancement layer allows this low resolution video to be sharpened again to bring it back to the original.

After demonstrating the business benefits, we see the block diagram of the encoder and decoder which helps visualise how this enhancement might be calculated and work. Guido then shows us what the enhancement layer looks like – a fairy flat image with lots of thin edges on it but, importantly, it also captures a lot of almost random detail which can’t be guessed by upsamplers. This, of course, is the point. If it were possible to upscale the low-resolution video and guess/infer all the data, then we would always do that. Rather, downscaling and upscaling is a lossy process. Here, that loss is worth it because of the computational gains and because the enhancement layer will put back much of what was once lost.

In order to demonstrate LCEVC’s ability, Guido shows graphs comparing LCEVC at UHD for x264 showing improvements of between 20 and 45% and image examples of artefacts which are avoided using LCEVC. We then see that when applied to AVC, HEVC and VVC it speeds up encodes at least two fold. Guido finishes this presentation showing how you can test out the encoder and decoder yourself.

The last segment of this video, Tarek Amara from Twitch sits down to talk with Guido about the codec and the background behind it. Their talk covers V-Nova’s approach to open source, licensing, LCEVC’s gradual improvements as it went through the proving process as part of MPEG standardisation plus questions from the floor.

Watch now!
Speakers

Guido Meardi Guido Meardi
CEO & Co-Founder,
V-Nova
Tarek Amara Tarek Amara
Principal Video Specialist,
Twitch

Video: AV1 at Netflix

Netflix have continually been pushing forward video compression and analysis because their assets are played so many times that every bit saved is real money saved. VMAF is a great example of Netflix’s desire to push the state of the art forward. Developed by Netflix and two universities, this new objective metric allowed them to better evaluate the quality of videos using computer analysis and has continued to be the foundation of their work since.

One use of VMAF has been to verify the results of Netflix’s Per-Shot Encoding method which alters encoding parameters for each shot of the film rather than using a fixed set of parameters for the whole film. The Broadcast Knowledge has featured talks on their previous technique, per-title encoding (among others).

AV1, however must be the most famous innovation that Neflix is behind. A founding member of the Alliance for Open Media (AoM), Netflix saw a need a for a better codec and by making an open one, which also played to the needs of other internet giants such as Google, was a good way to create a vibrant community around it driving submissions to the codec itself but also, it is hoped, in the implementation and adoption.

In this two-part talk, LiWei Guo starts off by explaining the ways in which AV1 will be used by Netflix. Since this talk took place, Netflix has started streaming in AV1 to Android clients. LiWei points out that AV1 supports 10-bit video as standard – a notable difference from other codecs like AVC and HEVC. This allows Netflix to use 10-bit without worrying about decoder compatibility and he shows examples of skies and water which are significantly by the use of 10-bit.

Another feature of AV1 is the Film Grain synthesis which seeks to improve encoding efficiency by removing the random film grain of the source during the encode process then inserting a similar random noise on top to recreate the same look and feel. As anything random can’t be predicted, noise such of this is very wasteful for a codec to try and encode, therefore it’s not <a surprise that this can result in as much as a 30% reduction in bitrate. Before concluding, LiWei briefly explains per-shot encoding then shows data showing the overall improvements.

Andrey Norkin, also from Netflix explains their work with Intel on the SVT-AV1 software video encoder which leverages Intel’s SVT technology, a framework optimised for Xeon chips for video encoding and analysis. Netflix’s motivations are to further increase adoption by delivering a data centre-ready, optimised encoder and to create an AV1 encoder they can use to support their own internal research activities (did someone say AV2?). SVT allows for parallelisation, important for any computer nowadays with so many cores available.

Finishing up, Andrey points us to the Github repository, lets us know the development statement (as of November 2019) and looks at the speed increases that have taken off, comparing SVT-AV1 against the reference libaom encoder.

Watch now!
Speakers

Andrey Norkin Andrey Norkin
Senior Research Scientist,
Netflix
LiWei Guo LiWei Guo
Senior Software Engineer,
Netflix

Video: MPEG-5 EVC

The MPEG-5: Essential Video Codec (EVC) promises to do what no MPEG standard has done before, deliver great improvements in compression and give assurances over patents. With a novel standardisation process, EVC provides a royalty-free base layer plus licensing details are provided upfront.

SMPTE 2019 saw Jonatan Samuelson take us through the details. Founder of Divideon and an editor of the evolving standard. Jonatan starts by explaining the codec landscape in terms of the new and recent codecs coming online showing how EVC differs including from it’s sister codec, VVC in parallel with which EVC is being developed.

Jonatan explains how the patents are being dealt with, comparing to HEVC, he shows that there is a much more simplified range of patent holders. But importantly, the codec has very granular tools to turn on and off separate tools so that you can exclude any that you don’t wish to use for licensing reasons. This is the first time this level of control has been possible. Along with the royalty-free base layer, this codec hopes to provide companies the control they need in order to safely use the codec with predictable costs and without legal challenges.

Target applications for EVC are realtime encoding, video conferencing but also newer ’emerging’ video formats such as 8K with HDR & WCG. To do this, Jonatan explains the different blocks that create the codec itself ahead of walking us through the results.

Watch now!
Speaker

Jonatan Samuelsson Jonatan Samuelsson
Founder,
Divideon