Video: Broadcasting WebRTC over Low Latency Dash


Using sub-second WebRTC with the scalability of CMAF: Allowing panelists and presenters to chat in real-time is really important to foster fluid conversations, but broadcasting that out to thousands of people scales more easily with CMAF based on MPEG DASH. In this talk, Mux’s Dylan Jhaveri (formerly CTO, Crowdcast.io) explains how they’ve combined WebRTC and CMAF to keep latencies low for everyone.

Speaking at the San Francisco VidDev meetup, Dylan explains that the Crowdspace webpage allows you to watch a number of participants talk in real-time as a live stream with live chat down the side of the screen. The live chat, naturally, feeds into the live conversation so latency needs to be low for the viewers as much as the on-camera participants. For them, WebRTC is used as this is one of the very few options that provides reliable sub-second streaming. To keep the interactivity between the chat and the participants, Crowdcast decided to look at ultra-low-latency CMAF which can deliver between 1 and 5 second latency depending on your risk threshold for rebuffering. So the task became to convert a WebRTC call into a low-latency stream that could easily be received by thousands of viewers.

 

 
Dylan points out that they were already taking WebRTC into the browser as that’s how people were using the platform. Therefore, using headless Chrome should allow you to pipe the video from the browser into ffmpeg and create an encode without having to composite individual streams whilst giving Crowdcast full layout control.

After a few months of tweaking, Dylan and his colleagues had Chrome going into ffmpeg then into a nodejs server which delivers CMAF chunks and manifests (click to learn more about how CMAF works). In order to scale this, Dylan explains the logic implemented in a CDN to use the nodejs server running in a docker container as an origin server. Using HLS they have a 95% cache hit rate and achieve 15 seconds latency. The tests at the time of the talks, Dylan explains, show that the CMAF implementation hits 3 seconds of latency and was working as expected.

The talk ends with a Q&A covering how they get the video out of the headerless Chrome, whether CMAF latency could be improved and why there are so many docker containers.

Watch now!
Speaker

Dylan Jhaveri Dylan Jhaveri
Senior Software Engineer, Mux
Formerly CTO & Co-founder, Crowdcast.io

Video: SMPTE Presents – MovieLabs 2030

Workflows changed when we moved from tapes to files, so it makes sense they will change as they increasingly move to the cloud. What if they changed to work in the cloud, but also benefited on-prem workflows at the same time? The work at MovieLabs is doing just that. With the cloud, infrastructure can be dynamic and needs to work between companies so data needs to be able to flow between companies easily and in an automated way. If this can be achieved, the amount of human interaction can be reduced and focused on the creative parts of content making.

This panel from at this SMPTE Hollywood section meeting discusses the work underway moderated by Greg Ciaccio at ETC. First off, Jim Helman from MoveLabs gives an overview of the 2030 vision. Kicked off at IBC 2019 with the publication of ‘Evolution of Media Creation‘, 10 principles were laid out for building the future of production covering topics like realtime engines, remote working, cloud deployments, security access and software-defined workflows (SDWs). This was followed up by a white paper covering ‘The Evolution of Production Security‘ which laid out the need for a new, zero-trust approach to security and then, in 2020, the jointly run industry lab released ‘The Evolution of Production workflows‘ which talked about SDWs.

“A software-defined workflow (SDW) uses a highly configurable set of tools and processes to support creative tasks by connecting them through software-mediated collaboration and automation.”

This SDW thread of the MovieLabs 2030 vision aims to standardise workflows at the data-model level, and, in the future, API level to allow for data to be easily exchanged and understood. Annie Chang from Universal Pictures explains that the ‘Evolution of Production Workflows’ publication deals with Tasks, Assets, Relationships and participants. If you can define your work with these four areas, you have enough information to get computers to understand the workflows and external data.

This leads to the idea of building each scene in an object model showing relationships between them. The data would describe key props and stage elements (for instance, if they were important for VFX), actors and their metadata and technical information such as colour correction, camera files and production staff.

Once modelled, a production can be viewed in many ways. For instance, the head of a studio may just be interested in high-level stories, people involved and distribution routes. Whereas a VFX producer would have a different perspective needing more detail about the scene. A VFX asset curator, on the other hand, would just need to know about the shots down to the filenames and storage locations. This work promises to allow all of these views of the same, dynamic data. So this not only improves workflows’ portability between vendor systems but is also a way of better organising any workflow irrespective of automation. Dreamworks is currently using this method of working with the aim of trying it out on live-action projects soon.

Annie finishes by explaining that there are efficiencies to be had in better organising assets. It will help reduce duplicates both by uncovering duplicate files but also stopping duplicate assets being produced. AI and similar technology will be able to sift through the information to create clips, uncover trivia and, with other types of data mining, create better outputs for inclusion in viewer content.

Sony Pictures’ Daniel De La Rosa then talks about the Avid platform in the cloud that they build in response to the Covid crisis and how cloud infrastructure was built in order of need and, often, based on existing solutions which were scaled up. Daniel makes the point the working in the cloud is different because it’s “bringing the workflows to the data” as opposed to the ‘old way’ where the data was brought to the machine. In fact, cloud or otherwise, with the globalisation of production there isn’t any way of doing things the ‘old way’ any more.

This reliance on the cloud – and to be clear, Daniel talks of multi-cloud working within the same production – does prompt a change in the security model employed. Previously a security perimeter would be set up around a location, say a building or department, to keep the assets held within safe. They could then be securely transferred to another party who had their own perimeter. Now, when assets are in the cloud, they may be accessed by multiple parties. Although this may not always happen simultaneously, through the life of the asset, this will be true. Security perimeters can be made to work in the cloud, but they don’t offer the fine-grained control that’s really needed where you really need to be able to restrict the type of access as well as who can access on a file-by-file basis. Moreover, as workflows are flexible, these security controls need to be modified throughout the project and, often, by the software-defined workflows themselves without human intervention. There is plenty of work to do to make this vision a reality.

The Q&A section covers exactly feedback from the industry into these proposals and specifications, the fact they will be openly accessible, and a question on costs for moving into the cloud. On the latter topic, Daniel said that although costs do increase they are offset when you drop on-premise costs such as rent and utilities. Tiered storage costs in the cloud will be managed by the workflows themselves just like MAMs manage asset distribution between online, near-line and LTO storage currently.

The session finishes talking about how SDWs will help automation and spotting problems, current gaps in cloud workflow tech (12-bit colour grading & review to name but one) and VFX workflows.

Watch now!
Speakers

Jim Helman Jim Helman
MovieLabs CTO
Daniel De La Rosa Daniel De La Rosa
Sony Pictures’ Vice President of Post Production, Technology Development
Annie Chang
Universal Pictures Vice President of Creative Technologies
Greg Ciaccio Moderator: Greg Ciaccio
ASC Motion Imaging Technology Council Workflow Chair
EP and Head of Production Technology & Post, ETC at University Southern California

Video: Cloud Encoding – Overview & Best Practices

There are so many ways to work in the cloud. You can use a monolithic solution which does everything for you which is almost guaranteed by its nature to under-deliver on features in one way or another for any non-trivial workflow. Or you could pick best-of-breed functional elements and plumb them together yourself. With the former, you have a fast time to market and in-built simplicity along with some known limitations. With the latter, you may have exactly what you need, to the standard you wanted but there’s a lot of work to implement and test the system.

Tom Kuppinen from Bitmovin joins Christopher Olekas from SSIMWAVE and host of this Kirchner Waterloo Video Tech talk on cloud encoding. After the initial introduction to ‘middle-aged’ startup, Bitmovin, Tom talks about what ‘agility in the cloud’ means being cloud-agnostic. This is the, yet unmentioned, elephant in the room for broadcasters who are so used to having extreme redundancy. Whether it’s the BBC’s “no closer than 70m” requirement for separation of circuits or the standard deployment methodology for systems using SMPTE’s ST 2110 which will have two totally independent networks, putting everything into one cloud provider really isn’t in the same ballpark. AWS has availability zones, of course, which is one of a number of great ways of reducing the blast radius of problems. But surely there’s no better way of reducing the impact of an AWS problem than having part of your infrastructure in another cloud provider.

Bitmovin have implementations in Azure, Google Cloud and AWS along with other cloud providers. In this author’s opinion, it’s a sign of the maturity of the market that this is being thought about, but few companies are truly using multiple cloud providers in an agnostic way; this will surely change over the next 5 years. For reliable and repeatable deployments, API control is your best bet. For detailed monitoring, you will need to use APIs. For connecting together solutions from different vendors, you’ll need APIs. It’s no surprise that Bitmovin say they program ‘API First’; it’s a really important element to any medium to large deployment.

 

 

When it comes to the encoding itself, per-title encoding helps reduce bitrates and storage. Tom explains how it analyses each video and chooses the best combination parameters for the title. In the Q&A, Tom confirms they are working on implementing per-scene encoding which promises more savings still.

To add to the complexity of a best-of-breed encoding solution, using best-of-breed codecs is part and parcel of the value. Bitmovin were early with AV1 and they support VP9 and HEVC. They can also distribute the encoding so that it’s encoded in parallel by as many cores as needed. This was their initial offering for AV1 encoding which was spread over more than 200 cores.

Tom talks about how the cloud-based codecs can integrate into workflows and reveals that HDR conversion, instance pre-warming, advanced subtitling support and AV1 improvements are on the roadmap while leads on to the Q&A. Questions include whether it’s difficult to deploy on multiple clouds, which HDR standards are likely to become the favourites, what the pain points are about live streaming and how to handle metadata.

Watch now!
Speakers

Tom Kuppinen Tom Kuppinen
Senior Sales Engineer,
Bitmovin
Moderator: Christopher Olekas
Senior Software Engineer,
SSIMWAVE Inc.

Video: As Time Goes by…Precision Time Protocol in the Emerging Broadcast Networks

How much timing do you need? PTP can get you timing in the nanoseconds, but is that needed, how can you transport it and how does it work? These questions and more are under the microscope in this video from RTS Thames Valley.

SMPTE Standards Vice President, Bruce Devlin introduces the two main speakers by reminding us why we need timing and how we dealt with it in the past. Looking back to the genesis of television, points out Bruce, everything was analogue and it was almost impossible to delay a signal at all. An 8cm, tightly wound coil of copper would give you only 450 nanoseconds of delay alternatively quartz crystals could be used to create delays. In the analogue world, these delays were used to time signals and since little could be delayed, only small adjustments were necessary. Bruce’s point is that we’ve swapped around now. Delays are everywhere because IP signals need to be buffered at every interface. It’s easy to find buffers that you didn’t know about and even small ones really add up. Whereas analogue TV got us from camera to TV within microseconds, it’s now a struggle to get below two seconds.

Hand in hand with this change is the change from metadata and control data being embedded in the video signal – and hence synchronised with the video signal – to all data being sent separately. This is where PTP, Precision Time Protocol, comes in. An IP-based timing mechanism that can keep time despite the buffers and allow signals to be synchronised.

 

 

Next to speak is Richard Hoptroff whose company works with broadcasters and financial services to provide accurate time derived from 4 satellite services (GPS, GLONASS etc) and the Swedish time authority RiSE. They have been working on the problem of delivering time to people who can’t put up antennas either because they are operating in an AWS datacentre or broadcasting from an underground car park. Delivering time by a wired network, Richard points out, is much more practical as it’s not susceptible to jamming and spoofing, unlike GPS.

Richard outlines SMPTE’s ST 2059-2 standard which says that a local system should maintain accuracy to within 1 microsecond. the JT-NM TR1001-1 specification calls for a maximum of 100ms between facilities, however Richard points out that, in practice, 1ms or even 10 microseconds is highly desired. And in tests, he shows that with layer 2, PTP unicast looping around western Europe was able to adhere to 1 microsecond, layer 3 within 10 microseconds. Over the internet, with a VPN Richard says he’s seen around 40 microseconds which would then feed into a boundary clock at the receiving site.

Summing up Richard points out that delivering PTP over a wired network can deliver great timing without needing timing hardware on an OPEX budget. On top of that, you can use it to add resilience to any existing GPS timing.

Gerard Philips from Arista speaks next to explain some of the basics about how PTP works. If you are interested in digging deeper, please check out this talk on PTP from Arista’s Robert Welch.

Already in use by many industries including finance, power and telecoms, PTP is base on IEEE-1588 allowing synchronisation down to 10s of nanoseconds. Just sending out a timestamp to the network would be a problem because jitter is inherent in networks; it’s part and parcel of how switches work. Dealing with the timing variations as smaller packets wait for larger packets to get out of the way is part of the job of PTP.

To do this, the main clock – called the grandmaster – sends out the time to everyone 8 times a second. This means that all the devices on the network, known as endpoints, will know what time it was when the message was sent. They still won’t know the actual time because they don’t know how long the message took to get to them. To determine this, each endpoint has to send a message back to the grandmaster. This is called a delay request. All that happens here is that the grandmaster replies with the time it received the message.

PTP Primary-Secondary Message Exchange.
Source: Meinberg [link]

This gives us 4 points in time. The first (t1) is when the grandmaster sent out the first message. The second (t2) is when the device received it. t3 is when the endpoint sent out its delay request and t4 is the time when the master clock received that request. The difference between t2 and t1 indicates how long the original message took to get there. Similarly, t4-t3 gives that information in the other direction. These can be combined to derive the time. For more info either check out Arista’s talk on the topic or this talk from RAVENNA and Meinberg from which the figure above comes.

Gerard briefly gives an overview of Boundary Clock which act as secondary time sources taking pressure off the main grandmaster(s) so they don’t have to deal with thousands of delay requests, but they also solve a problem with jitter of signals being passed through switches as it’s usually the switch itself which is the boundary clock. Alternatively, Transparent Clock switches simply pass on the PTP messages but they update the timestamps to take account of how long the message took to travel through the switch. Gerard recommends only using one type in a single system.

Referring back to Bruce’s opening, Gerard highlights the need to monitor the PTP system. Black and burst timing didn’t need monitoring. As long as the main clock was happy, the DA’s downstream just did their job and on occasion needed replacing. PTP is a system with bidirectional communication and it changes depending on network conditions. Gerard makes a plea to build a monitoring system as part of your solution to provide visibility into how it’s working because as soon as there’s a problem with PTP, there could quickly be major problems. Network switches themselves can provide a lot of telemetry on this showing you delay values and allowing you to see grandmaster changes.

Gerard’s ‘Lessons Learnt’ list features locking down PTP so only a few ports are actually allowed to provide time information to the network, dealing carefully with audio protocols like Dante which need PTP version 1 domains, and making sure all switches are PTP-aware.

The video finishes with Q&A after a quick summary of SMPTE RP 2059-15 which is aiming to standardise telemetry reporting on PTP and associated information. Questions from the audience include asking how easy it is to do inter-continental PTP, whether the internet is prone to asymmetrical paths and how to deal with PTP in the cloud.

Watch now!
Speakers

Bruce Devlin Bruce Devlin
Standards Vice President,
SMPTE
Gerard Phillips Gerard Phillips
Systems Engineer,
Arista
Richard Hoptroff Richard Hoptroff
Founder and CTO
Hoptroff London Ltd