Video: Best Practices for End-to-End Workflow and Server-Side Ad Insertion Monitoring

This video from the Streaming Video Alliance, presented at Mile High Video 2020 looks at the result of recent projects document the best practice for two important activities: server-side ad insertion (SSAI) and end-to-end (E2E) workflow monitoring. First off, is E2E monitoring which defines a multi-faceted approach to making sure you’re delivering good-quality content well.

The this part of the talk is given by Christopher Kulbakas who introduces us to the document published by the Streaming Video Alliance covering monitoring best practices. The advice surrounds three principles: Creating a framework, deciding on metrics, and correlation. Christopher explains the importance of monitoring video quality after a transcode or encode since it’s easy to take a sea of green from your transport layer to indicate that viewers are happy. If your encode looks bad, viewers won’t be happy just because the DASH segments were delivered impeccably.

The guidance helps your monitor your workflow. ‘End to end’ doesn’t imply the whole delivery chain, only how to ensure the part you are responsible for is adequately monitored.

Christopher unveils the principles behind the modular monitoring across the workflow and tech stack:
1) Establish monitoring scope
Clearly delineate your responsibility from that of other parties. Define exactly how and to what standard data will be handled between the parties.

2) Partition workflow with monitoring points
Now your scope is clear, you can select monitoring points before and after key components such as the transcoder.

3) Decompose tech stage
Here, think of each point in the workflow to be monitored as a single point in a stack of technology. There will be content needing a perceptual quality monitor, Quality of Service (QoS) and auxiliary layers such as player events, logs and APIs which can be monitored.

4) Describe Methodology
This stage calls for documenting the what, where, how and why of your choices, for instance explaining that you would like to check the manifest and chunks on the output of the packager. You’d do this with HTTP-GET requests for the manifest and chunks for all rungs of the ladder. After you have finished, you will have a whole set of reasoned monitoring points which you can document and also share with third parties.

5) Correlate results
The last stage is bringing together this data, typically by using an asset identifier. This way, all alarms for an asset can be grouped together and understood as a whole workflow.

End-to-End Server-Side Ad Monitoring

The last part of this talk is from Mourad Kioumgi from Sky who walks us through a common scenario and how to avoid it. An Ad Buyer complains their ad didn’t make it to air. Talking to every point in the chain, everyone checks their own logs and says that their function was working, from the schedulers to the broadcast team inserting the SCTE markers. The reality is that if you can’t get to the bottom of this, you’ll lose money as you lose business and give refunds.

The Streaming Video Alliance considered how to address this through better monitoring and are creating a blueprint and architecture to monitor SSAI systems.

Mourad outlines these possible issues that can be found in SSAI systems:
1) Duration of content is different to the ad duration.
2) Chunks/manifest are not available or poorly hosted
3) The SCTE marker fails to reach downstream systems
4) Ad campaigns are not fulfilled despite being scheduled
5) Ad splicing components fail to create personalised manifests
6) Over-compression of the advert.

Problems 2,3, 5 and 6 are able to be caught by the monitoring proposed which revolves around adding the Creative ID and AdID into the manifest file. This way, problems can be correlated which particularly improves the telemetry back from the player which can deliver a problem report and specify which asset was affected. Other monitoring probes are added to monitor the manifests and automatic audio and video quality metrics. Sky successfully implemented this as a proof of concept with two vendors working together resulting in a much better overview of their system.

Mourad finishes his talk looking at the future creating an ad monitoring framework to distribute an agreed framework document for. best practices.

Watch now!
Speakers

Christopher Kulbakas Christopher Kulbakas
Project Lead, Senior Systems Designer, Media Technology & infrastructure,
CBC/Radio Canada
Mourad Kioumgi Mourad Kioumgi
VOD Solutions Architect.
Sky

Video: Broadcast in the cloud!

Milan Video Tech’s back with a three takes on putting broadcast into the cloud. So often we see the cloud as ‘for streaming’. That’s not today’s topic; we’re talking ingest and live transmissions in the cloud. Andrea Fassina from videodeveloper.io introduces the three speakers who share their tips for doing cloud well by using KPIs, using the cloud to be efficient, agile & scale and, finally, running your live linear channels through the cloud as part of their transmission path.

First up is Christopher Brähler from Akamai who looks at a how they helped a customer becomes more efficient, be agile and scale. His first example shows how using a cloud workflow in AWS, including many AWS services such as Lambda, the customer was able to reduce human interaction with a piece of content during ingest by 80%. The problem was that every piece of content took two hours to ingest, mainly due to people having to watch for problems. Christopher shows how this process was automated. He highlights some easy wins by front-loading the process with MediaInfo which could easily detect obvious problems like incorrect duration, codec etc. Christopher then shows how the rest of the workflow used AWS components and Lamda to choose to transcode/rewrap files if needed and then pass them on to a whole QC process. The reduction was profound and whilst this could have been achieved with similar MAM-style processing on-premise, being in the cloud allows the next two benefits.

The next example is how the same customer was able to quickly adjust to a new demand on the workflow when they found that some files were arriving and weren’t compatible with their ingest process due to a bug in a certain vendor’s software which was going to take months to fix. Using this same workflow they were able to branch out, using MediaInfo to determine if this vendor’s software was involved. If it was it would be sent down a newly-created path in the workflow that worked around the problem. The benefit of this being in the cloud touches on the third example – scalability. Being in the cloud, it didn’t really matter how much or little this new branch was used. When it wasn’t being used, the cost would be nothing. If it was needed a lot, it would scale up.

The third example is when this customer merged with another large broadcaster, The cloud-based workflow meant that they were able to easily scale up and put a massive library of content through ingest in a matter of two or three months, rather than a year or more than otherwise would be the case on dedicated equipment.

Next up is Luca Moglia from Akamai who’s sharing with his experience on getting great value out of cloud infrastructure. Security should be the basis of any project whether it’s on the internet or not, so it’s no surprise that Luca starts with the mandate to ‘Secure all connections’. Whilst he focuses on the streaming use case, his points can be generalised to programme contribution. He splits up the chain into ‘first mile’ (origin/DC to cloud/CDN), ‘middle mile’ (cloud/CDN to edge) and last mile which is the delivery from the edge to the viewer. Luca looks at options to secure these segments such as ‘AWS Connect’ and other services for Azure & GCP. He looks at using private network interconnections (PNIs) for CDNs and then examines options for the last mile.

His other pieces of advice are to offload as mich ‘origin’ as you can, meaning to reduce the load on your origin server by using an Origin Gateway but also a Multi-CDN strategy. Similarly, he suggests offloading as much logic to the edge as is practical. After all, the viewer’s ping to the edge (RTT) is the lowest practical, so having two-way traffic is best there than deeper into the CDN as the edge is usually in the same ISP.

Another plea is to remember that CMAF is not just there to reduce latency, Luca emphasises all the other benefits which aren’t only important for low-latency use cases such as being able to use the same segments for delivering HLS and DASH streams. Being able to share the same segments allows CDNs to cache better which is a win for everyone. It also reduces storage costs and brings all DRM under CENC, a single mechanism supporting several different DRM methods.

Luca finishes his presentation suggesting looking at the benefits of using HTTP/2 and HTTP/3 to reduce round trips and, in theory, speed up delivery. Similarly, he talks about the TCP algorithm BBR which should improve throughput.

Last to speak is Davide Maggioni from Sky Italia who shows us how they quickly transitioned to a cloud workflow for NOWTV and SKYGO when asked to move to HD, maintain costs and make the transition quickly. They developed a plan to move the metadata enrichement, encryption, encoding and DRM into the cloud. This helped them reduce procurement overhead and allowed them to reduce deployment time.

Key to the project was taking an ‘infrastructure as code’ approach whereby everything is configured by API, run from automated code. This reduces mistakes, increases repeatability and also allowed them to, more easily, deploy popup channels.

Davide takes us through the diagrams and ways in which they are able to deploy permanent and temporary channels showing ‘mezzanine’ encoding on-premise, manipulation done in the cloud, and then a return to on premise ahead of transmission to the CDN.

Watch now!
Speakers

Christopher Brähler Christopher Brähler
Director of Product Management,
SDVI Corporation
Davide Maggioni Davide Maggioni
OTT & Cloud Process and Delivery,
Sky Italia
Luca Moglia Luca Moglia
Media Solutions Engineer,
Akamai
Andrea Fassina Andrea Fassina
Freelance Developer,
https://videodeveloper.io

Video: TV Sport Innovation – Staying Ahead of the Game

Sports has always led innovation in many areas of broadcast, but during COVID not only did they have to adapt nearly every workflow and redeploy staff, but they then had to brace to deliver 100 games in 40 days. Gordon Roxburgh sums it up: “I’ve been at Sky twenty years, and I think [these have] been the most challenging six months…we’ve faced.”

In this session from the DTG’s Future Vision 2020 conference, Carl Hibbert from Futuresource Consulting talks to Sky, Arsenal TV and Facebook to find how their businesses have adapted. Melissa Lawton from Facebook explains how their live streaming, both for user-generated footage and produced sport have adapted to the changing needs. When COVID hit, Facebook lost some very valuable content. Their response was to double down on fan engagement, with challenges to fans to create content and also staging events which were produced and commentated as real sports events, but all shots were people at home exercising but being brought into the narrative of an Iron Man competition. Facebook have also invested in their user-facing tools and dashboards to help expose and monitor contribution via live streaming.

Gordon Roxburgh from Sky explains the seachange he’s seen in production. “The first thing was to keep channels on air…and keep staff safe.” They moved rapidly from a fully staffed office to just three or four people on-site and a presenter. In order to mix, they created a Virtual Production suite which allowed people to create content in the cloud.

For content, Gordon says that watch-alongs proved very popular where key sports personalities talk through what they were thinking during key sporting moments. This was just one of the many content ideas that keep programming going until “Project restart” commenced where the whole sports ecosystem asked itself ‘How can we deliver 100 games in 40 days?’ Once they knew the season would start, Gordon says, this opened up a 3-week build period during which BT Media and Broadcast, NEP, NEP Connect and multiple internal departments collaborated to produce rapid turnarounds.

“As an industry, we came together.” The working practices developed at Sky were shared with other major broadcasters who also shared their best practice – always putting staff first. Sky even went to the extent of building a technical space in a large studio floor to keep people apart and co-opted a set of training rooms to become a self-contained graphics unit. These ideas kept graphics operators together but not mixing with the rest of the production.

The view from Arsenal TV is explained by John Dollin. They worked quickly very early on and were able to be back in the office from February. Whilst Arsenal TV doesn’t have the rights to stream live, they produce their programmes live for transmission later. This used to be done in a crowded room but was soon transferred to a virtual mixer in the cloud with remote editors. John highlights the challenge of involving freelancers into the system and providing them with appropriate supervision. More importantly, he feels that their current ability to maintain the pre-covid production quality is due to the continued dedication of certain personnel who are putting in long hours which is not a sustainable situation to be in.

Watch now!
Free Registration
Speakers

Gordon Roxburgh Gordon Roxburgh
Technical Manager,
Sky Sports
Melissa Lawton Melissa Lawton
Live Sports Production Strategy,
Facebook
John Dollin John Dollin
Senior Product & Engineering Manager,
Arsenal Footballl Club
Carl Hibbert Carl Hibbert
Head of Consumer Media & Technology,
Futuresource Consulting

Video: The future of addressable TV advertising in the UK

When it comes to advertising, there’s a lot of value in understanding who you’re talking to. This video examines the practicality of that within the UK and the relative value that brings. Nigel Walley from Decipher looks at how the landscape is changing both in the ability to address the TV externally and the information available within the home.

Nigel starts by looking at how the broadcast TV companies and the online streaming companies are able to target and concludes that broadcast can often fine tune to the region and and include dayparting whereas though we assume streaming companies can target by individuals, in reality Nigel asserts, they typically target by household. He goes further to explain that almost 50% of viewing is still linear TV with YouTube taking up 12.4% of the 50% which remains of the 4hours and 42 minutes of average viewing time per day.

Nigel makes the point that with HbbTV and many of the streaming services being available on the ‘big screen’, it makes the idea of ‘broadcast’ vs ‘streaming’ a nonsense as they are already converged. The big difference is in how we can provide the ads to these platforms. The Virgin and Sky closed platforms comprise nearly 13 million viewers with Freeview plus others making up nearly 16 million. Nigel highlights that 30% of viewing is with the BBC and hence no advertising, although trailers may be delivered using addressable technologies.

Nigel explains that Sky’s Adsmart has been extended to Virgin cable. Then explains how YouView and other channels move up to the big screen – the TV. The important issue for publishers is how the Sky and Virgin platforms end up as controlling influences. Nigel explains the Linear Addressability of the platforms showing that YouView is the next potential area this will happen. There’s also the opportunity for smart TVs themselves to help in delivering these ads. “What can a broadcaster do alone” asks Nigel which he answers by saying ‘very little’ unless they are Sky or Virgin in the UK. They can deliver addressable TV into apps and computers, however.

Nigel finishes with a call to action to the broadcasters to change their focus from individual apps to the ways they and agencies can work together to reach more, and more targeted viewers.

Watch now!
Speaker

Nigel Walley Nigel Walley
Managing Director,
Decipher