Video: How CBS Sports Digital streams live events at scale

Delivering high scale in streaming really exposes the weaknesses of every point of your workflow, so even those of us who are not streaming at maximum scale, there are many lessons to be learnt. CBS Sports Digital delivered the Super Bowl using the principles of ‘practice, practice, practice’, keeping the solution as simple as possible and making mitigation of problems primary to solving them.

Taylor Busch tells walks us through their solution explaining how it supported their key principles and highlighting the technology used. Starting with Acquisition, he covers the SDI fibre delivery to a backup facility as well as the AWS Direct Connect links for their Elemental Live encoders. The origin servers were in two different regions and both received data from both sets of encoders.

CBS used ‘Output locking’ which ensures that the TS segments are all aligned even across different encoders which is done by respecting the timecode in the SDI and helps in encoder failover situations. QVBR encoding is a method of encoding up to a quality level rather than simply saying ‘7000 kbps’. QVBR provides a more efficient use a bandwidth since in the situations where a scene doesn’t require a lot of bandwidth, it won’t be sent. This variability, even if you run in capped mode to limit the bandwidth of particularly complex scenes, can look like a failing encoder to some systems, so the fact this is now in ‘VBR’ mode, needs to be understood by all the departments and companies who monitor your feed.

Advertising is famously important for the Super Bowl, so Taylor gives an overview of how they used the CableLabs ESAM protocol and SCTE to receive information about and trigger the adverts. This combined SCTE-104, ESAM and SCTE-35 as we’ll as allowing clients to use VAST for tracking. Extra caching was provided by Fastly’s Media Shield which tests for problems with manifests, origin servers and encoders. This fed a Multi-CDN setup using 4 CDNs which could be switched between. There is a decision point for requests to determine which CDN should answer.

Taylor then looks at the tools, such as Mux’s dashboard, which they used to spot problems in the system; both NOC-style tools and multiviewers. They set up three war rooms which looked at different aspects of the system, connectivity, APIs etc. This allowed them to focus on what should be communicated keeping ‘noise’ down to give people the space they needed to do their work at the same time as providing the information required. Taylor then opens up to questions from the floor.

Watch now!

Taylor Busch, Sr. Taylor Busch
Senior Director Engineering,
CBS Sports Digital

Video: Harness SSAI’s Superpowers

Server-side Ad Insertion (SSAI) is a great option for streaming services delivering video to a wide variety of devices and for those who need to avoid ad blockers. Whilst ad insertion can happen in the player, this mechanism can be interfered with allowing users to avoid ads. Whilst client-side ad insertion can much more easily create a unique stream for each client, dynamic SSAI can now do the same with a better user experience.

This panel from the OTT Leadership Summit at Streaming Media West 2019 brings together Disney, WarnerMerdia and Crunchyroll to share their experiences with SSAI. They discuss beaconing, ad standards, scaling, SCTE and more.

Beaconing goes hand in hand with ad playback providing metrics on what happened. When you perform certain actions, the player will reach out to a URL. This can be used to indicate such things as users skipping or pausing a video. The beacon information can then be used to verify how much of which ads were seen by whom and charge advertisers accordingly.

The panel moves on to discussing scaling using live sports as an example and cover questions to ask vendors to ensure you and they are ready for maximum scale. Bandwidth, is declared the biggest challenge, but a less obvious problem is that your upstream ad providers can’t always scale well. If you rely on calls from your server to others, then it’s vital to understand their scaling capacity and strategy. They discuss issues with losing beacons when operating at scale and the need for detailed logging and debugging in order to spot errors and reconcile the results.

Some time is next spent on VPAID and VAST 4 which are both messaging specifications to allow ad servers to tell applications which ads to play. The panel discusses the pros and cons in their use for SSAI where the stitcher needs to reach out to and ad server in real time to find out which ads to play.

At the end of the discussion, the panel takes questions from the floor but not before discussing SCTE Markers and ‘content conditioning’ which surrounds taking care of your source videos and encoder such that the two assets fit together properly at I-frame boundaries.

Watch now!

Robert Jameson Robert Jameson
Technical Director, Media Enablement
Turner | WarnerMedia
Stephen Gray Stephen Gray
Director, Ad Tech Systems
Walt Disney Direct-to-Consumer & International
Michael Dale Michael Dale
VP Engineering,
Nadine Krefetz Nadine Krefetz
Consultant, Reality Software
Contributing Editor, Streaming Media

Webinar: Out of Band SCTE 35

A great look at SCTE35 and how it’s used from Roger Franklin and Alan Young, CEO and COO at Crystal given at the SMPTE Conference 2017.Watch Now.

SCTE 35 – “Digital Program Insertion Cueing Message for Cable” – is routinely used to identify the location and composition of programming content and advertising breaks in linear television for OTT providers and has been for a long time.
SCTE 35 specifies metadata that can be inserted into the MPEG-2 Transport Stream carrying the compressed content. SCTE 35 contains the precise frame of the beginning and end of video segments, content identifiers and rights-related information. However, the real-world implementation of SCTE 35 by content providers is inconsistent despite SCTE 67 – “Recommended Practice for SCTE 35 Digital Program Insertion Cueing Message for Cable”. Worse, the ever-increasing complexity of distribution and transcoding for delivery to multiple devices has taken its toll on SCTE 35. It rarely survives delivery to the OTT provider without being corrupted. This is obviously a problem for both the OTT providers and the content providers not only because it limits their ability to monetize the content but also because it makes it much harder to effectively automate the implementation of the complex rights associated with online content in an auditable manner.
This webinar describes a method of delivering SCTE 35 out of band using temporal fingerprints to re-synchronize the SCTE metadata with the video at each receiving point. This not only solves the core problem but provides many side benefits including automatic lip sync error correction, enabling broadcast adverts to become ‘clickable’ and enabling graphics to become customizable and user selectable.

Watch Now!

Video: Automating Digital Asset Production with SCTE Messages

From the SMPTE Technical conference, a talk on SCTE 104 and SCTE 35 which allow for live insertion of ads.

Content production networks produce up to 70 different variants of each program asset to meet obligations to a growing number of distribution partners. An improved vocabulary for SCTE 104 and SCTE 35 messages enables in-band description and instructions for processing content segments of live streams. With this information, the production of live and on-demand digital assets can be automated to extract, format and brand assets to meet the various requirements.

This talk describes the critical elements of designing and implementing systems that automate the creating the digital assets. We see a modular, highly scalable architecture and and end-to-end technology for automation. And we finish with looking at how to define structure and timing in the schemas.

Presented By: Jim Duval and Eric Openshaw

Watch now!