What’s implementing SMPTE ST-2110 like in real life? How would you design your network and what were the problems? In this case study Ammar Latif from Cisco Systems presents the architecture, best practices and lessons learned they gleaned in this live IP broadcast production facility project designed for a major US broadcaster. Based on SMPTE ST-2110 standard, it spanned five studios and two control rooms. The central part of this project was a dual Spine-Leaf IP fabric with bandwidth equivalent of a 10,000 x 10,000 HD SDI router with a fully non-blocking multicast architecture. The routing system was based on Grass Valley Convergent broadcast controller and a Cisco DCNM media controller.
As the project was commissioned in 2018, the AMWA IS-04 and IS-05 specifications providing an inter-operable mechanism for routing media around SMPTE 2110 network were not yet available. Multicast flow subscription was based on a combination of IGMP (Internet Group Management Protocol) and PIM (Protocol Independent Multicast) protocols. While PIM is very efficient and mature, it lacks the ability to use bandwidth as a parameter when setting up a flow path. Ammar explains how Non-Blocking Multicast (NBM) developed by Cisco brings bandwidth awareness to PIM by signalling a type of data (video, audio or metadata).
The talk continues by discussing PTP distribution & monitoring, SMPTE 2022-7 seamless protection switching and remote site production. Ammar also lets us see how the user interfaces on the Cisco DCNM media controller were designed which include a visualisation of multicast flow, network topology and link saturation of ports.
The broadcast industry is still producing many new hardware-based products with FPGAs and encoding ASICs still ruling the roost for many companies when it comes to fitting video products into small, power efficient spaces. But the battle continues as software-based products continue to ramp up, server-based products continue to improve and the need to be able to virtualise or place functions into the cloud drives the desire for software-based solutions.
We all know that hardware isn’t dead and that the interest of the topic is where we are today, what is possible and why people are choosing this route and that’s what Broadcast Solutions’ panel discusses in this video. Often called COTS – commercial off-the shelf – hardware, the idea is that you can buy the same server that any other industry does and run your broadcast-related functions on it. When it’s in the cloud, you’re not even selecting the hardware as much as saying how many CPUs and other resources you’d like.
The first comments made come from Marcel Koustaal from Grass Valley who feels that the industry doesn’t entirely appreciate the value software as it’s less tangible than hardware but Pierre Mestrez from Simplylive makes the point that creating products quickly in a modular way is an important part of that company’s success. Zero Density makes the point that they can work quickly as they can build their software on top of other software, Unreal Engine, for example.
Troubleshooting changes for those who run of the systems, we hear from Laurent Petit from EVS. It takes a different set of thinking and processes compared to the idea of swapping a card. The transition to IP, adds Marcel, creates a training opportunity where the technology and the workflows are changing at the same time.
Kuban Altan compares the ability with audio to be processed in real time, easily, by CPUs, by consumer laptops with the future of video processing. Whilst now it’s not so easy to process video with CPUs at the moment, this will change over the coming decade as CPUs improve significantly. Moreover, Kuban looks towards a day where IO is reduced between devices and rather stays within the same CPU/GPU.
The move to software is a global trend, states Laurent, partly because of the imperative to work quickly and efficiently in our small industry whereby we can benefit by building on software developed for similar uses in other industries. The move will take time, however explains Marcel, and will take longer than bringing online the technology itself.
The video ends with a discussion of how clearly hardware-bound devices such as cameras can still embrace software in order, in the future, to create lighter, more flexible cameras which will improve the range of what you can do with each camera and, ultimately, enhance the creative options available to programme makers.
It’s being closely watched throughout the industry, a long-in-the-making project to deploy SMPTE ST 2110 throughout a fully green-field development. Its failure would be a big setback for the push to a completely network-based broadcast workflow.
The BBC Cardiff Central Square project is nearing completion now and is a great example of the early-adopter approach to bringing cutting-edge, complex, large-scale projects to market. They chose a single principle vendor so that they could work closely in partnership at a time when the market for ST 2110 was very sparse. This gave them leverage over the product roadmap and allowed to the for the tight integration which would be required to bring this project to market.
Nowadays, the market for ST 2110 products continues to mature and whilst it has still quite a way to go, it has also come a long way in the past four years. Companies embarking similar projects now have a better choice of products and some may now feel they can start to pick ‘best of breed’ rather than taking the BBC approach. Whichever approach is taken there is still a lot to be gained by following and learning from the mistakes and successes of others. Fortunately, Mark Patrick, Lead Architect on the project is here to provide an update on the project.
Mark starts by giving and overview of the project, its scale and its aims. He presents the opportunities and challenges it presents and the key achievements and milestones passed to date.
Live IP has benefits and risks. Mark takes some time to explain the benefits of the flexibility and increasingly lower cost of the infrastructure and weighs them agains the the risks which include the continually developing standards and skills challenges
The progress overview names Grass Vally as the main vendor, control via BNCS having being designed and virtualised, ST 2110 network topology deployed and now the final commissioning and acceptance testing is in progress.
The media topology for the system uses an principal of an A and a B network plus a separate control network. It’s fundamentally a leaf and spine network and Mark shows how this links in to both the Grass Valley equipment but also the audio equipment via Dante and AES67. Mark takes some time to discuss the separate networks they’ve deployed for the audio part of the project, driven by compatibility issues but also within the constraints of this project, it was better to separate the networks rather than address the changes necessary to force them together.
PTP timing is discussed with a nod to the fact that PTP design can be difficult and that it can be expensive too. NMOS issues are also actively being worked on and remains an outstanding issue in terms of getting enough vendors to support it, but also having compatible systems once an implementation is deployed. This has driven the BBC to use NMOS in a more limited way than desired and creating fall-back systems.
From this we can deduce, if it wasn’t already understood, that interoperability testing is a vital aspect of the project, but Mark explains that formalised testing (i.e. IT-style automated) is really important in creating a uniform way of ensuring problems have been fully addressed and there are no regressions. ST 2110 systems are complex and fault finding can be similarly complex and time consuming.
Mark leaves us by explaining what keeps him awake at night which includes items such as lack of available test equipment, lack of single-stream UHD support and NMOS which leads him to a few comments on ST 2110 readiness such as the need for vendors to put much more effort into configuration and management tools.
Anyone with an interest in IP in broadcast will be very grateful at Mark’s, and the BBC’s, willingness to share the project’s successes and challenges in such a constructive way.
The most complex part of this solution is Broadcast Centre built for very large premium UHD productions (routing capabilities of 2000×2000 UHD IP feeds, 4 vision mixers). Such large productions take place only a few time a year, so for all the other times the same hardware can be reconfigured into smaller flypacks that can do multiple independent productions at different places around the world. All devices in Broadcast Centre are installed in mobile racks, so you can simply wheel them in and out of different sports venues.
These flypacks can also be used to extend capabilities of IP OB vans – the only limit is the number of ports available on the switches. A truck can be put in any location and connected to multiple IP systems, creating fully scalable and large broadcast system – the kind that you would only previously find in a fixed studio set up.
The case study covers lessons learned from this COTS based system which leverages SMPTE ST 2110, SMPTE 2059, and adaptive FPGA based edge processing. Maurice Snell focuses on advantages of ST 2110 IP design (massive simplification of wiring, use of COTS equipment, audio breakaway possibility, signal agnostic capabilities, flexibility, scalability) and describes the challenges (operators shouldn’t need to know or care if they are routing SDI, IP or a hybrid mixture of the two, importance of unified facility monitoring and configuration and a new approach to fault finding for engineers).