IEEE 1588

Video: How to Successfully Commission a SMPTE ST 2059/PTP System

Posted on by Russell Trafford-Jones

PTP is the beating heart behind video- and audio-over-IP installations. As critical as black and burst reference, it pays to get it right. But PTP is a system, not a monolithic signal distributed around the facility. Unlike genlock, it’s a two-way conversation over networked infrastructure and whilst that brings great benefits, it changes how we deal with it. The system should be monitored, both at the ST 2059 layer and network layer. But before we even get to that point, implementation requires care particularly as the industry is still in the early phases of developing tools and best practices for project deployments.

Leigh Whitcomb from Imagine Communications has stepped up to bring us his experiences and best practices as part of the Broadcast Engineering and IT Conference at NAB. This talk assumes an existing level of knowledge of PTP. If you would like to start at the beginning, then please look at this talk from Meinberg and this from Tektronix.

Leigh starts by explaining that, typically, the best architecture is to have a red and a blue network. A grand master would then be on both networks and both would be set to lock to GPS. He explains how do deal with prioritisation and preventing other devices from becoming grand masters. He also explains some of the basic PTP parameter values such as setting the Announcement time outs. Other good design practices he discusses are where to use Boundary Clocks, avoiding PTP Domain numbers of 0 and 127 plus using QoS and DSCP.

As part of the commissioning piece, Leigh goes through some frequently-seen problems such as locking up slowly due to an incorrect Delay Request setting or the Grand Master announce rate being the same as the timeout. To understand when your system isn’t working properly, Leigh makes the point that it’s vital to understand in detail how you expect the system to behave. Use checklists to ensure all parameters and configuration have been applied correctly but also to verify the PTP packets themselves leaving the GM. Leigh then highlights checklists for other parts of the network such as the switches and Media Nodes.

There are a number of tools available for faultfinding and checking compliance. As part of commissioning, the first port of call is the device’s GUI and API which will obviously give most of the parameters needed but often will go further and help with fault finding. WireShark can help verifying the fields in the packets, the timing and message rates. Whilst Meinberg’s Track Hound is a free program which allows you to verify the PTP protocol and Grand Masters. The EBU List project also covers PTP/ST 2059. Helpfully, Leigh talks through how to use Wireshark to verify fields and message rates.

In terms of Testing, Leigh suggests running a packet capture (PCap) for 48 hours after commissioning to verify any issues. He then highlights the need for redundancy testing. This is where understanding how you intend the network to work is important as redundancy testing should also be combined with network testing where you deliberately pull down part of your network and see the GMs change as intended. This changeover will be managed by the Best Master Clock Algorithm (BMCA). When troubleshooting, you should use your monitoring system to help you visualise what’s happening. A good system should enable you to see the devices on the network and their status. Many companies would want to test how successfully the system recovers from a full failure as this will represent the maximum traffic load on the PTP system.

How to watch
1) Click on ‘Add to favourites’
2) Register for free – or log in if you are already part of NAB Express

3) You will then see the video on the left of the screen.

Watch now!
Speakers

Leigh Whitcomb Leigh Whitcomb
Architect,
Imagine Communications

Video: Benefits of IP Systems for Sporting Venues

Posted on by Russell Trafford-Jones

As you walk around any exhibitions there seems to be a myriad of ‘benefits’ of IP working, many of which don’t resonate for particular use cases. Only the most extraordinary businesses need all of the benefits, so in this talk, Imagine Communication’s John Mailhot discusses how IP helps sports venues.

John sets the scene by separating out the function of OB trucks and the ‘inside production’ facilities which have a whole host of non-TV production to do including driving scoreboards, displays inside the venue, replays and importantly has to deal with over 250 events a year, not all of which will have an OB truck.

We see that the scale that IP can work at is a great benefit as many signals can fit down one fibre and 2022-7 seamless switching can easily provide full redundancy for every fibre and SFP. This is a level of redundancy which is simply not seen in SDI systems. With stadia being very large, necessitating cable runs of over 500m, the fact that IP needs fewer cables overall is a great benefit.

John shows an example of an Arista switch only 7U in height which provides 144x 100G ports meaning it could support over 4000 inputs and 4000 outputs. Such density is unprecedented and for OB trucks can be a dealbreaker. For sports venues, this can also be a big motivator but also allow more flexibility in distributing the solution rather than relying on a massive central interconnect with a 1100×1100 SDI router in a central CTA.

TV is nothing without audio and the benefits to audio in 2110 are non trivial since with the audio being split off from the video, we are no longer limited to dealing with just 16 channels per video and de-embedding from a video frame any time we want to touch it.

Timing is an interesting benefit. I say this because, whilst PTP can end up being quite complex compared to black and burst, it has some big benefits. First off, it can live in the same cables as your data where as black and burst requires a whole separate cable infrastructure. PTP also allows you to timestamp all essences which helps with lip-sync throughout your workflow.

John leads us through some examples of how this works for different areas finishing by summing up the relevant benefits such as scalability, multi-format, space efficient, and timing amongst others.

Watch now!
Download the slides
Speakers

John Mailhot John Mailhot
CTO, Networking & Infrastructure,
Imagine Communications

Video: PTP in Virtualized Media Environment

Posted on by Russell Trafford-Jones

How do we reconcile the tension between the continual move towards virtualisation, microservices and docker-like deployments and the requirements of SMPTE 2110 to have highly precise timing so it can synchronise the video, audio and other essence streams? Virtualisation adds fluidity in to computing so it can share a single set of resources amongst many virtual computers yet PTP, the Precision Time Protocol a successor to NTP, requires close to nano-second precision in its timestamps.

Alex Vainman from Mellanox explains how to make PTP work in these cases and brings along a case study to boot. Starting with a little overview and a glossary, Alex explains the parts of the virtual machine and the environment in which it sits. There’s the physical layer, the hypervisor as well as the virtual machines themselves – each virtual machine being it’s own self-contained computer sitting on a shared computer. Hardware must be shared between, often, many different computers. However some devices aren’t intended to be shared. Take, for instance, a dongle that contains a licence for software. This should clearly be only owned by one computer therefore there is a ‘direct’ mode which means that it is only seen by one computer. Alex goes on to explain the different virtualisation I/O modes which allow devices which can be shared, a printer, storage or CPU for instance need to have access computers may need to wait until they have access to the device to enable sharing. Waiting, of course, is not good for a precision time protocol.

In order to understand the impact that virtualisation might have, Alex details the accuracy and other requirements necessary to have PTP working well enough to support SMPTE 2110 workflows. Although PTP is an IEEE standard, this is just a standard to define how to establish accurate time. It doesn’t help us understand how to phase and bring together media signals without SMPTE ST 2059-1 and -2 which provide the standard of the incoming PTP signal and the way by which we can compare timing and media signals (more info here.) All important is to understand how PTP can actually determine the accurate time given that we know every single message has an unknown propagation delay. By exchanging messages, Alex shows, it is quite practical to measure the delays involved and bring them into the time calculation.

We now have enough information to see why the increased jitter of VM-based systems is going to cause a problem as there are non-deterministic factors such as contention and traffic load to consider as well as factors such as software overhead. Alex takes us through the different options of getting PTP well synchronised in a variety of different VM architectures. The first takes the host clock and ensures that is synchronised to PTP. Using a dedicated PTP library within the VM, this then speaks to the host clock and synchronises the VM OS clock providing very accurate timing. Another, where hardware support in the VM’s hardware for PTP isn’t present, is to have NICs with dedicated PTP support which can directly support the VM OSes maintaining their own PTP clock. The major downside here is that each OS will have to make their own PTP calls creating more load on the PTP system as opposed to the previous architecture whereby the host clock of the VM was the only clock synchronising to the system PTP and therefore there was only ever one set of PTP messages no matter how many VMs were being supported.

To finish off, Alex explains how Windows VMs can be supported – for now through third-party software – and summarises the ways in which we can, in fact, create PTP ecosystems that incorporate virtual machines.

Watch now!
Download the slides
Speakers

Alex Vainman Alex Vainman
Senior Staff Engineer,
Mellanox Technologies

Video: Timing Tails & Buffers

Posted on by Russell Trafford-Jones

Timing and synchronisation have always been a fundamental aspect of TV and as we move to IP, we see that timing is just as important. Whilst there are digital workflows that don’t need to be synchronised against each other, many do such as studio productions. However, as we see in this talk from The Broadcast Bridge’s Tony Orme, IP networks make timing all the more variable and accounting for this is key to success.

To start with Tony looks at the way the OBs, also known as REMIs, are moving to IP and need a timing plane across all of the different parts of production. We see how traditionally synchronisation is needed and the effect of timing problems not only in missed data but also with all essences being sent separately synchronisation problems between them can easily creep in.

When it comes to IP timing itself, Tony explains how PTP is used to record the capture time of the media/essences and distribute through the system. Looking at the data on the wire and the interval between that and the last will show a distribution of, hopefully, a few microseconds variation. This variation gives rise to jitter which is a varying delay in data arrival. The larger the spread, the more difficult it will be to recover data. To examine this more closely, Tony looks at the reasons for and the impacts of congestion, jitter, reordering of data.

Bursting, to make one of these as an example, is a much overlooked issue on networks. While it can occur in many scenarios without any undue problems, microbusting can be a major issue and one that you need to look for to find. This surrounds the issue of how you decide that a data flow is, say, 500Mbps. If you had an encoder which sent data at 1Gbps for 5 minutes and no data for 5 minutes, then over the 10 minute window, the average bitrate would have been 500Mbps. This clearly isn’t a 500Mbps encoder, but how narrow do you need to have your measurement window to be happy it is, indeed, 500Mbps by all reasonable definitions? Do you need to measure it over 1 second, 1 millisecond? Behind microbursting is the tendency of computers to send whatever data they have as quickly as possible; if a computer has a 10Gbe NIC, then it will send at 10Gbps. What video receivers actually need is well spaced packets which always come a set time apart.

Buffers a necessary for IP transmission, in fact within a computer there are many buffers. So using and understanding buffers is very important. Tony takes us through the thought process of considering what buffers are and why we need them. With this groundwork laid, understanding their use and potential problems is easier and well illustrated in this talk. For instance, since there are buffers in many parts of the chain to send data from an application to a NIC and have it arrive at the destination, the best way to maximise the chances of having a deterministic delay in the Tx path is to insert PTP information almost at the point of egress in the NIC rather than in the application itself.

The talk concludes by looking at buffer fill models and the problems that come with streaming using TCP/IP rather then UDP/IP (or RTP). The latter being the most common.

Watch now!
Download the presentations!

Speakers

Tony Orme Tony Orme
Editor,
The Broadcast Bridge