Mux

Video: Delivering Better Manifests with Effective VMAF

Posted on by Russell Trafford-Jones

Measuring video quality is done daily around the world between two video assets. But what happens when you want to take the aggregate quality of a whole manifest? With VMAF being a well regarded metric, how can we use that in an automatic way to get the overview we need?

In this talk, Nick Chadwick from Mux shares the examples and scripts he’s been using to analyse videos. Starting with an example where everything is equal other than quality, he explains the difficulties in choosing the ‘better’ option when the variables are much less correlated. For instance, Nick also examines the situations where a video is clearly better, but where the benefit is outweighed by the minimal quality benefit and the disproportionately high bitrate requirement.

So with all of this complexity, it feels like comparing manifests may be a complexity too far, particularly where one manifest has 5 renditions, the other only 4. The question being, how do you create an aggregate video quality metric and determine whether that missing rendition is a detriment or a benefit?

Before unveiling the final solution, Nick makes the point of looking at how people are going to be using the service. Depending on the demographic and the devices people tend to use for that service, you will find different consumption ratios for the various parts of the ABR ladder. For instance, some services may see very high usage on 2nd screens which, in this case, may take low-resolution video and also lot of ‘TV’ size renditions at 1080p50 or above with little in between. Similarly other services may seldom ever see the highest resolutions being used, percentage-wise. This shows us that it’s important not only to look at the quality of each rendition but how likely it is to be seen.

To bring these thoughts together into a coherent conclusion, Nick unveils an open-source analyser which takes into account not only the VMAF score and the resolution but also the likely viewership such that we can now start to compare, for a given service, the relative merits of different ABR ladders.

The talk ends with Nick answering questions on the tendency to see jumps between different resolutions – for instance if we over-optimise and only have two renditions, it would be easy to see the switch – how to compare videos of different resolutions and also on his example user data.

Watch now!
Speakers

Nick Chadwick Nick Chadwick
Software Engineer,
Mux

Video: Open Source Streaming

Posted on by Russell Trafford-Jones

Open source software can be found powering streaming solutions everywhere. Veterans of the industry on this panel at Streaming Media West, give us their views on how to successfully use open source in on-air projects whilst minimising risk.

The Streaming Video Alliance’s Jason Thibeault starts by finding out how much the panelists and their companies use open source in their work and expands upon that to ask how much the support model matters. After all, some projects have paid support but based on free software whereas others have free community-provided support. The feeling is that it really depends on the community; is it large and is it active? Not least of the considerations is that, in a corporate setting, if the community is quick to accuse, is it right to ask your staff to go through layers of ‘your a newbie’ and other types of pushback each time they need to get an answer?

Another key question is whether we give should back to the open source community and, if so, how. The panels discusses the difficulties in contributing code but also covers the importance of other ways of contributing – particularly when the maintainer is one individual. Contribution of money is an obvious, but often forgotten way to help but writing documentation is also really helpful as is contributing on the support forums. This all makes for a vibrant community and increases the chances that other companies will adopt the project into their workflows…which then makes the community all the stronger.

With turn-key proprietary solutions ready to to be deployed, Jason asks whether open source actually saves money on the occasions that you can, indeed, find a proprietary solution that fits your requirements.

Lastly, the panel talks about the difficulty in balancing adherence to the standards compared with the speed at which open source communities can move. They can easily deliver the full extent of the standard to date and then move on to fixing the remaining problems so far not addressed by the developing standard. Whilst this is good, they risk implementing in ways which may cause issues in the future when the standard finally catches up.

The panel session finishes with questions from the audience.

Watch now!
Speakers

Steve Heffernan Steve Heffernan
Head of Product
Mux
Yuriy Reznik Yuriy Reznik
Head of Research,
Brightcove
Rob Dillon Rob Dillon
Dillon Media Ventures
Rema Morgan-Aluko Rema Morgan-Aluko
Engineering Dango
FandangoNOW
Jason Thibeault Jason Thibeault
Executive Director,
Streaming Video Alliance

Video: What to do after per-title encoding

Posted on by Russell Trafford-Jones

Per-title encoding is a common method of optimising quality and compression by changing the encoding options on a file-by-file basis. Although some would say the start of per-scene encoding is the death knell for per-title encoding, either is much better than the more traditional plan of applying exactly the same settings to each video.

This talk with Mux’s Nick Chadwick and Ben Dodson looks at what per-title encoding is and how to go about doing it. The initial work involves doing many encodes of the same video and analysing each for quality. This allows you to out which resolutions and bitrates to encode at and how to deliver the best video.

Ben Dodson explains the way they implemented this at Mux using machine learning. This was done by getting computers to ‘watch’ videos and extract metadata. That metadata can then be used to inform the encoding parameters without the computer watching the whole of a new video.

Nick takes some time to explain MUX’s ‘convex hulls’ which give a shape to the content’s performance at different bitrates and helps visualise the optimum encoding parameters the content. Moreover, we see that using this technique, we can explore how to change resolution to create the best encode. This doesn’t always mean reducing the resolution; there are some surprising circumstances when it makes sense to start at high resolutions, even for low bitrates.

The next stage after per-title encoding is to segment the video and encode each segment differently which Nick explores and explains how to deliver different resolutions throughout the stream seamlessly switching between them. Ben takes over and explains how this can be implemented and how to chose the segment boundaries correctly, again, using a machine learning approach to analysis and decision making.

Watch now!
Speakers

Nick Chadwick Nick Chadwick
Software Engineer,
Mux
Ben Dodson Ben Dodson
Data Scientist,
Mux

Video: A Standard for Video QoE Metrics

Posted on by Russell Trafford-Jones

A standard in progress for quality of experience networks, rebufereing time etc. Under the CTA standards body wanting to create a standard around these metrics. The goal of the group is to come up with a standard set of player events, metrics & terminology around QoE streaming. Concurrent viewers, isn’t that easy to define? If the user is paused, are they concurrently viewing the video? Buffer underruns is called rebuffering, stalling, waiting. Intentionally focussing on what the viewers actually see and experience. QoS is a measurement of how well the platform is performing, not necessarily the same as what they are experiencing.

The standard has ideas of different levels. There are player properties and events which are standardised ways of signalling that certain things are happening. Also Session Metrics are defined which then can feed into Aggregate Metrics. The first set of metrics include things such as playback failure percentage, average playback stalled rate, average startup time and playback rate with the aim of setting up a baseline and to start to get feedback from companies as they implement these, seemingly simple, metrics.

This first release can be found on github.

Watch now!
Speaker

Steve Heffernan Steve Heffernan
Co-Founder, Head of Product,
Mux