The Broadcast Knowledge

Video: CDNs: Delivering a Seamless and Engaging Viewing Experience

Posted on 31st March 2021 by Russell Trafford-Jones

This video brings together broadcasters, telcos and CDNs to talk about the challenges of delivering a perfect streaming experience to large audiences. Eric Klein from Disney+ addresses the issues along with Fastly’s Gonzalo de la Vega, Jorge Hernandez from Telefonica, Adriaan Bloem with Shahid moderated by Robert Ambrose.

Eric starts by talking from the perspective of Disney+. Robert asked if scaling up quickly enough to meet Disney+’s extreme growth has been a challenge. Eric replies that scale is built by having multiple routes to markets using multiple CDNs so the main challenge is making sure they can quickly move to the next new market as they are announced. Before launching, they do a lot of research to work out which bitrates are likely to be streamed and on what devices for the market and will consider offering ABR ladders to match. They work with ISPs and CDNs using Open Caching. Eric has spoken previously about open caching which is a specification from the Streaming Video Alliance to standardise the API between for CDNs and ISPs. Disney+ uses 7-8 different provers currently and never rely on only one method to get content to the CDN. Eric and his team have built their own equipment to manage cache fill.

Adriaan serves the MENA market and whilst the gulf is fairly easy to address, north Africa is very difficult as internet bandwidths are low and telcos don’t peer except in Marseille. Adriaan feels that streaming in Europe and North America as ‘a commodity’ as, relatively, it’s so much easier compared to north Africa. They have had to build their own CDN to reach their markets but because they are not in competition with the telcos, unlike CDNs, they find it relatively easy to strike the deals needed for the CDN. Shahid has a very large library so getting assets in the right place can be difficult. They see an irony that their AVOD services are very popular and get many hits for a lot of the popular content meaning it is well cached. Their SVOD content has a very long tail meaning that despite viewers paying for the service, they risk getting a worse service because most of the content isn’t being cached.

Jorge presents his view as both a streaming provider, Movistar, and a telco, Telefonica which services Spain and South America. With over 100 POPs, Telefonica provides a lot of IPTV infrastructure for streaming but also over the internet. They have their own CDN, TCDN, which delivers most of their traffic, bursting to commercial CDNs when necessary. Telefonica also supports Open Caching.

Eric explains that the benefit of Open Caching is that, because certain markets are hard to reach, you’re going to need a variety of approaches to get to these markets. This means you’ll have a lot of different companies involved but to have stability in your platform you need to be interfacing with them in the same way. With Open Caching, one command for purge can be sent to everyone at once. For Adriaan, this is “almost like a dream” as he has 6 different dashboards and is living through the antithesis of Open Caching. He says it can be very difficult to track the different failovers on the CDNs and react.

Gonzalo points out how far CDNs like Fastly have come. Recently they had 48 hours’ notice to enable resources for 1-million concurrent views which is the same size as the whole of the Fastly CDN some years go. Fastly are happy to be part of customers’ multi-CDN solutions and when their customers do live video, Fastly recommend that they have more than one simply for protection against major problems. Thinking about live video, Eric says that everything at Disney+ is designed ‘live first’ because if it works for live, it will work for VoD.

The panel finishes by answering questions from the audience.

Watch now!
Free registration required

Speakers

	Eric Klein Director, Media Distribution, CDN Technology, Disney+
	Jorge Hernandez Head of CDN Development and Deployment, Telefonica/Movistar
	Adriaan Bloem Head of Infrastructure, Shahid
	Gonzalo de la Vega VP Strategic Projects, Fastly
	Robert Ambrose Co-Founder and Research Director, Caretta Research

Video: Standardising Microservices on the Basis of MCMA

Posted on 30th March 2021 by Russell Trafford-Jones

Microservices are a way of splitting up large programs and systems into many, many, smaller parts. Building up complex workflows from these single-function modules makes has many benefits including simplifying programming and testing, upgrading your system seamlessly with no downtime, scalability and the ability to run on the cloud. Microservices were featured last week on The Broadcast Knowledge. Microservices do present challenges, such as orchestrating hundreds of processes into a coherent media workflow.

The EBU is working with SMPTE and the Open Services Alliance for Media on a cloud-agnostic open source project called MCMA, Media Cloud Microservice Architecture. The MCMA project isn’t a specification, rather it a set of software providing tools to enable a move to microservices. We hear from Alexandre Rouxel from the EBU and Loïc Barbou from Bloomberg that this project started out of a need from some broadcasters to create a scalable infrastructure that could sit on a variety of cloud infrastructure.

Direct link

What is a service? Created a standard idea of a service that contains standard operations. Part of the project is a set of libraries that work with NodeJS and .net which deal with the code needed time and time again such as logging, handling data repositories, security etc. Joost Rovers explains how the Job Processor and Service Registry work together to orchestrate the media workflows and ensure there’s a list of every microservice available, and how to communicate with it. MCMA places shims in front of cloud services on GCP, AWS, Azure etc in order that each service looks the same. Joost outlines the libraries and modules available for MCMA and how they could be used.

Watch now!
Speakers

	Loïc Barbou Consultant, Bloomberg
	Alexandre Rouxel Data Scientist & Project Coordinator, EBU
	Joost Rovers Managing Director, Rovers IT

Video: ST 2110 The Future of Live Remote Production

Posted on 29th March 2021 by Russell Trafford-Jones

Trying to apply the SMPTE ST 2110 hype to the reality of your equipment? This video is here to help. There are many ‘benefits’ of IP which are banded about yet it’s almost impossible to realise them all in one company. For the early adopters, there’s usually one benefit that has been the deal-breaker with other benefits helping boost confidence. Smaller broadcast companies, however, can struggle to get the scale needed for cost savings, don’t require as much flexibility and can’t justify the scalability. But as switches get cheaper and ST 2110 support continues to mature, it’s clear that we’re beyond the early adopter phase.

This panel gives context to ST 2110 and advises on ways to ‘get started’ and skill up. Moderated by Ken Kerschbaumer from the Sports Video Group, Leader’s Steve Holmes, Prinyar Boon from Phabrix join the panel with Arista colleagues Gerard Phillips and Robert Welch and Bridge Technologies’ Chairman Simen Frostad.

The panel quickly starts giving advice. Under the mantra ‘no packet left behind’, Gerard explains that, to him, COTS (Commercial Off The Shelf) means a move to enterprise-grade switches ‘if you want to sleep at night’. Compared to SDI, the move to IT can bring cost savings but don’t skimp on your switch infrastructure if you want a good quality product. Simen was pleased to welcome 2110 as he appreciated the almost instant transmission that analogue gave. The move to digital added a lot of latency, even in the SDI portions of the chain thanks to frame syncs. ST 2110, he says, allows us to get back, most of the way, to no-latency production. He’s also pleased to bid good-bye to embedded data.

It is possible to start small, is the reassuring message next from the panel. The trick here is to start with an island of 2110 and do your learning there. Prinyar lifts up a tote bag saying he has a 2110 system he can fit in there which takes just 10 minutes to get up and running. With two switches, a couple of PTP grandmasters and some 2110 sources, you have what you need to start a small system. There is free software that can help you learn about it, Easy NMOS is a quick-to-deploy NMOS repository that will give you the basics to get your system up and running. You can test NMOS APIs for free with AMWA’s testing tool. The EBU’s LIST project is a suite of software tools that help to inspect, measure and visualize the state of IP-based networks and the high-bitrate media traffic they carry and there’s is also SDPoker which lets you test ST 2110 SDP files. So whilst there are some upfront costs, to get the learning, experience and understanding you need to make decisions on your ST 2110 trajectory, it’s cost-effective and can form part of your staging/test system should you decide to proceed with a project.

The key here is to find your island project. For larger broadcasters or OB companies, a great island is to build an IP OB truck. IP has some big benefits for OB Trucks as we heard in this webinar, such as weight reduction, integration with remote production workflows and scalability to ‘any size’ of event. Few other ‘islands’ are able to benefit in so many ways, but a new self-op studio or small control room may be just the project for learning how to design, install, troubleshoot and maintain a 2110 system. Prinyar cautions that 2110 shouldn’t be just about moving an SDI workflow into IP. The justification should be about improving workflows.

Remote control is big motivator for the move to ST 2110. Far before the pandemic, Discovery chose 2110 for their Eurosport production infrastructure allowing them to centralise into two European locations all equipment controlled in production centres in countries around Europe. During the pandemic, we’ve seen the ability to create new connections without having to physically install new SDI is incredibly useful. Off the back of remote control of resources, some companies are finding they are able to use operators from locations where the hourly rate is low.

Before a Q&A, the panel addresses training. From one quarter we hear that ensuring your home networking knowledge is sound (DHCP, basic IP address details) is a great start and that you can get across the knowledge needed very little time. Prinyar says that he took advantage of a SMPTE Virtual Classroom course teaching the CCNA, whilst Robert from Arista says that there’s a lot in the CCNA that’s not very relevant. The Q&A covers 2110 over WAN, security, hardware life cycles and the reducing carbon footprint of production.

Watch now!
Speakers

	Steve Holmes Applications Engineer, Leader
	Prinyar Boon Product Manager, PHABRIX
	Gerard Phillips Systems Engineer, Arista
	Simen Frostad Chairman, Bridge Technologies
	Robert Welch Technical Solutions Lead, Arista
	Moderator: Ken Kerschbaumer Chair & Editorial Directo, Sports Video Group

Video: Best Practices for End-to-End Workflow and Server-Side Ad Insertion Monitoring

Posted on 26th March 2021 by Russell Trafford-Jones

This video from the Streaming Video Alliance, presented at Mile High Video 2020 looks at the result of recent projects document the best practice for two important activities: server-side ad insertion (SSAI) and end-to-end (E2E) workflow monitoring. First off, is E2E monitoring which defines a multi-faceted approach to making sure you’re delivering good-quality content well.

The this part of the talk is given by Christopher Kulbakas who introduces us to the document published by the Streaming Video Alliance covering monitoring best practices. The advice surrounds three principles: Creating a framework, deciding on metrics, and correlation. Christopher explains the importance of monitoring video quality after a transcode or encode since it’s easy to take a sea of green from your transport layer to indicate that viewers are happy. If your encode looks bad, viewers won’t be happy just because the DASH segments were delivered impeccably.

The guidance helps your monitor your workflow. ‘End to end’ doesn’t imply the whole delivery chain, only how to ensure the part you are responsible for is adequately monitored.

Christopher unveils the principles behind the modular monitoring across the workflow and tech stack:
1) Establish monitoring scope
Clearly delineate your responsibility from that of other parties. Define exactly how and to what standard data will be handled between the parties.

2) Partition workflow with monitoring points
Now your scope is clear, you can select monitoring points before and after key components such as the transcoder.

3) Decompose tech stage
Here, think of each point in the workflow to be monitored as a single point in a stack of technology. There will be content needing a perceptual quality monitor, Quality of Service (QoS) and auxiliary layers such as player events, logs and APIs which can be monitored.

4) Describe Methodology
This stage calls for documenting the what, where, how and why of your choices, for instance explaining that you would like to check the manifest and chunks on the output of the packager. You’d do this with HTTP-GET requests for the manifest and chunks for all rungs of the ladder. After you have finished, you will have a whole set of reasoned monitoring points which you can document and also share with third parties.

5) Correlate results
The last stage is bringing together this data, typically by using an asset identifier. This way, all alarms for an asset can be grouped together and understood as a whole workflow.

End-to-End Server-Side Ad Monitoring

The last part of this talk is from Mourad Kioumgi from Sky who walks us through a common scenario and how to avoid it. An Ad Buyer complains their ad didn’t make it to air. Talking to every point in the chain, everyone checks their own logs and says that their function was working, from the schedulers to the broadcast team inserting the SCTE markers. The reality is that if you can’t get to the bottom of this, you’ll lose money as you lose business and give refunds.

The Streaming Video Alliance considered how to address this through better monitoring and are creating a blueprint and architecture to monitor SSAI systems.

Mourad outlines these possible issues that can be found in SSAI systems:
1) Duration of content is different to the ad duration.
2) Chunks/manifest are not available or poorly hosted
3) The SCTE marker fails to reach downstream systems
4) Ad campaigns are not fulfilled despite being scheduled
5) Ad splicing components fail to create personalised manifests
6) Over-compression of the advert.

Problems 2,3, 5 and 6 are able to be caught by the monitoring proposed which revolves around adding the Creative ID and AdID into the manifest file. This way, problems can be correlated which particularly improves the telemetry back from the player which can deliver a problem report and specify which asset was affected. Other monitoring probes are added to monitor the manifests and automatic audio and video quality metrics. Sky successfully implemented this as a proof of concept with two vendors working together resulting in a much better overview of their system.

Mourad finishes his talk looking at the future creating an ad monitoring framework to distribute an agreed framework document for. best practices.

Watch now!
Speakers

	Christopher Kulbakas Project Lead, Senior Systems Designer, Media Technology & infrastructure, CBC/Radio Canada
	Mourad Kioumgi VOD Solutions Architect. Sky

Subscribe to get daily updates