The Broadcast Knowledge
Menu Close
  • Diversity in Broadcast
    • Diversity in Broadcast – The Problems and Opportunities
    • Rise – Women in Broadcast
    • Diversity Index
  • Twitter
  • LinkedIn
  • Facebook
  • Email
  • Submissions

metadata

Video: Audio Metadata over IP

Posted on 6th August 2020 by Russell Trafford-Jones

Next-Generation Audio is gradually becoming this generation’s audio as new technologies seep into the mainstream. Dolby Atmos is one example of a technology which is being added to more and more services and which goes way beyond stereo and even 5.1 surround sound. But these technologies don’t just rely on audio, they need data, too to allow the decoders to understand the sound so they can apply the needed processing. It’s essential that this data, called metadata, keeps in step with the audio and, indeed, that it gets there in the first place.

Dolby have long used metadata along with surround sound to maintain the context in which the recording was mastered. There’s no way for the receiver to know what maximum audio level the recording was mixed to without being told, for instance. With NGA, the metadata needed can be much more complex. With Dolby Atmos, for example, the audio objects need position information along with the mastering information needed for surround sound.

Kent Terry from Dolby laboratories joins us to discuss the methods, both current and future, that we can use to convey metadata from point to point in the broadcast chain. He starts by looking at the tried and trusted methods of carrying data within the audio of SDI. This is the way that Dolby E and Dolby D are carried, as data within what appears to be an AES 3 stream. There are two SMPTE standards for this in a sample-accurate fashion, ST 2109 and ST 2116.

SMPTE 2109 allows for metadata to be carried over an AES 3 channel using SMPTE ST 337, 337 being the standard which defines how to put compressed audio over AES 3 which would normally expect PCM audio data. This allows for any metadata at all to be carried. SMPTE ST 2116, similarly, defines metadata transport over AES3 but specifically for ITU-R BS.2125 and BS.2076 which define how to carry the Audio Definition Model.

The motivation for these standards is to enable live workflows which don’t have a great way of delivering live metadata. There are a few types of metadata which are worth considering. Static metadata, which doesn’t change during the programme such as the number of channels or the sample rate. Dynamic metadata such as spacial location and dialogue levels. And importantly optional metadata and required metadata, the latter being essential for the functioning of the technology.

Kent says that live productions are held back in their choice of NGA technologies by the limitations of metadata carriage and this is one reason that work is being done in the IP space to create similar standards for all-IP programme production.

For IP there are two approaches. The first is to define a way to send metadata separately to the AES67 audio which is found within SMPTE ST 2110-30, which is done with the new AES standard AES-X242. The other way being developed is using SMPTE 2110-41 which allows for any metadata (not solely ST 292) to be carried in a time-synchronised way with the other 2110 essences. Both of these methods, Kent explains are actively being developed and are open to input from users.

Watch now!
Speaker

Kent Terry Kent Terry
Snr. Manager, Sound Technology, Office of the CTO,
Dolby Laboratories
Video AES-X242, Dolby, Dolby Laboratories, Kent Terry, metadata, Next Generation Audio, NGA, ST 2109, ST 2110-41, ST 2116, VSF

Video: ST 2110-41 Fast Metadata – Under the Hood and Applications

Posted on 1st July 2020 by Russell Trafford-Jones

Why would you decode 12Gbps of data in order to monitor for an occasional SCTE switching signal or closed captions? SMPTE 2110 separates video, audio and metadata freeing up workflows to be more flexible. But so far, SMPTE 2110 only defines how to take the metadata used in SDI under SMPTE 291. The point of freeing up workflows is to allow innovation, so this talk looks at SMPTE ST 2110-41 which allows for more types of data to be carried, time synchronised with the other essences.

Paul Briscoe joins us at the podium to explain why we need to extend SMPTE ST 2110-40 beyond SDI-compatible metadata. He starts by explaining how ST 2110 was intended to work well with SDI. Since SDI already has a robust system for Ancillary Data (ANC Data), it made sense for 2110 to support that first in a 100% compatible way. One of the key characteristics of metadata is that it’s synchronised to the other media and we expect the decoders to be selective over what it acts on, explains Paul. The problem that he sees, however is that if you wanted to send some XML, JSON or other data which has never been included in the SDI signal, there is no way to send that within 2110 and keep the synchronisation. To prove his point, Paul puts up the structure of ST-291M in 2110-40 which still has references to lines and horizontal offsets. Furthermore, he points out that future 2110 workflows aren’t going to need to be tied to SDI-compatible metadata.

The Fast Metadata (FMX) proposal is to allow arbitrary metadata streams which can be time-aligned with a media stream. It would provide a standardised encoding method, be extensible, minimise IP address use and retain the option of being interoperable with ST 291 if needed.

Having chosen the KLV data structure, which is well known and scalable, SMPTE provides for this to be delivered with a timestamp and even early so that processing can be done ahead of time. This opens the door to carrying XMl, JSON and other data structure.

Paul explains how this has been implemented as an RTP stream hence using the RTP timestamps. Within the stream there are blobs of data. Paul explains the structure of a blob and how payloads which are smaller than a blob, as well as those which are larger, can be dealt with. Buffering and SDPs need to be supported, after all, this is a 2110 stream.

After doing into the detail of 2110-41, Paul explains there is a 2110-42 standard which can carry technical metadata about the stream. It provides in-band messaging of important stream details but is not intended to replace IS-04 or SDPs.

Find out more and Watch now!
Download the presentation
Speakers

Paul Briscoe Paul Briscoe
Televisionary Consulting
Video Evertz, IPShowcase, metadata, Paul Briscoe, SMPTE, ST 2110-40, ST 2110-41, ST 2110-42

Video: AI – more ways it will revolutionise our industry

Posted on 12th June 2020 by Russell Trafford-Jones

Artificial Intelligence and Machine Learning (ML) dominate many discussions and for good reason, they usually reduce time and reduce costs. In the broadcast industry their are some obvious areas where it will, an already does, help. But what’s the time table? Where are we now? And what are we trying to achieve with the technology?

Edmundo Hoyle from TV Globo explains how they have managed to transform the thumbnail selection for their OTT service from a manual process taking an editor 15 minutes per video to an automated process using machine learning. A good thumbnail is relevant, it is a clear picture and has no nudity or weapons in it. Edmundo explains that they tackled this in a three-step process. The first step uses NLP analysis of the episode summary to understand what’s relevant and to match that with the subtitles (closed captions). Doing this identifies times int he video which should be examined more closely for thumbnails.

The durations identified by the process are then analysed for blur-free frames (amongst other metrics to detect clear videography) which gives them candidate pictures which may contain problematic imagery. the AWS service Rekognition which returns information regarding whether faces, guns or nudity are present in the frame. Edmundo finishes by showing the results which are, in general very positive. Final choice of thumbnails is still moderated by editors, but the process is much more streamlined because they are much less likely to have to find an image manually since the process selects 4 options. Edmundo finishes by explaining some of the chief causes of rejecting an image which are all relatively easy to improve upon and tend to be related to a person looking down or away from the camera.

We’ve seen before on The Broadcast Knowledge the idea of super-resolution which involves up-scaling images/video using machine learning. The result is better than using standard linear filters like lanczos. This is has been covered in a talk from Mux’s Nick Chadwick about LCEVC. Yiannis Andreopoulos from iSize talks next about the machine learning they use to improve video which uses some of these same principles to pre-treat, or as they call it ‘pre-code’ video before it’s encoded using a standard MPEG encoder (whether that be AVC, HEVC or the upcoming VVC). Yiannis explains how they are able to understand the best resolutions to encode at and scale the image intelligently appropriately. This delivers significant gains across all the metrics leading to bandwidth reduction. Furthermore he outlines a system which feeds back to maintain both the structure of the video which avoids it becoming too blurry which can be a consequence of being to subservient to the drive to reduce bitrate and thus simplifying the picture. It can also, though, protect itself from going too far down the sharpness path and only chasing metrics gains. He concludes by outlining future plans.

Grant Franklin Totten then steps up to explain how Al Jazeera have used AI/machine learning to help automate editorial compliance processes. He introduces the idea of ‘Contextual Video Metadata’ which ads a level of context to what would otherwise be stand-alone metadata. To understand this, we need to learn more about what Al Jazeera is trying to achieve.

As a news organisation, Al Jazeera has many aspects of reporting to balance. They are particularly on the look out for bias, good fact-checking & fake news. In order to support this, they are using AI and machine learning. They have both textual and video-based methods of detecting fake news. As an example of their search for bias, they have implemented voice detection and analysed MP’s speech time in Ireland. Irish law requires equal speaking time, yet Al Jazeera can easily show that some MPs get far more time than others. Another challenge is detecting incorrect on-screen text with the example given of naming Trump as Obama by accident on a lower-third graphic. Using OCR, NLP and Face recognition, they can flag issues with the hope the they can be corrected before Tx. In terms of understanding, for example, who is president, Al Jazeera is in the process of refining the Knowledge graph to capture the information they need to check against.

AI and machine learning (ML) aren’t going anywhere. This talk shines a light on two areas where it’s particularly helpful in broadcast. You can count on hearing significant improvements in AI and ML’s effectiveness in the next few years and it’s march into other parts of the workflow.
Watch now!
Speakers

Edmundo Hoyle Edmundo Hoyle
TV System Researcher
TV Globo
Yiannis Andreopoulos Yiannis Andreopoulos
Technical Director,
iSize Technologies
Grant Franklin Totten Grant Franklin Totten
Head of Media & Emerging Platforms,
Al Jazeera Media Networks

Edmundo Hoyle (GLOBO), Yiannis Andreopoulos (iSize Technologies) and Grant Totten (Al Jazeera Media Network).

Video AI, Al Jazeera, Edmundo Hoyle, Globo, Grant Franklin Totten, iSize Technologies, Machine Learning, metadata, V-Nova, Yiannis Andreopoulos

Webinar: Why data-driven OTT will overtake the Broadcast TV model – The TV Data Revolution

Posted on 27th April 2020 by Russell Trafford-Jones

‘On demand’ must be the top reason people love streaming services. But for streaming providers, the motivation to deliver these services goes deeper than meeting market demand; The data that can be gathered as people watch is revolutionising business models.

Eyevinn Technology, the Swedish specialist consultancy firm focused on video and distribution brings this data into the spotlight tomorrow, Tuesday 28th, to discuss how different segments of the industry are using the data and why you should have a data strategy. This virtual panel brings together AWS, Braze, Edgeware, Eyevinn Technology, Jump TV, and Kaltura that will shed light on the
benefits you can get having a compelling data strategy.

“I believe we have gathered a very interesting combination of speakers and I can promise an insightful hour with a lot of good takeaways”

Magnus Svensson, VP Sales and Business Development at Eyevinn Technology

We see the evidence of the power of this data in the EULAs for Smart TVs which make it clear that they are watching you. Famously, North American TV Manufacturer Vizio was caught collecting “as many as 100 billion data points each day from millions of TVs,” according to the US FTC. This resulted in a class action suit and a judgement against Vizio.

This virtual panel looks at collecting data in the right way, discussing what data to collect and how to motivate subscribers to share their data. Importantly in today’s global society which needs to serve very privacy-conscious countries, there will be discussion about how to use the data that has been collected. They’ll be looking at how can data be used to understand how to scale your OTT solution, how can data be used to make the experience better and how can data be used to increase engagement and reduce churn.

Register now!

 

Speakers

Johan Bolin Johan Bolin
Chief Product and Technology Officer,
Edgeware
Gideon Gilboa Gideon Gilboa
Marketing and Solutions – Media and Telecom,
Kaltura
Jerónimo Macanás

 

Jerónimo Macanás
CEO,
Jump Data-Driven Video
Warrick Godfrey Warrick Godfrey
VP, Industry Solutions,
Braze
Simon Frost Simon Frost
Head of EMEA Media Marketing,
Amazon Web Services (AWS)
Magnus Svensson Magnus Svensson
Media Solution Specialist,
Eyevinn
Webinar Amazon Web Services, AWS, Braze, Edgeware, Eyevinn Technology, Gideon Gilboa, Jerónimo Macanás, Johan Bolin, JUMP-Data-Driven Video, Kaltura, Magnus Svensson, metadata, Online Streaming, OTT, Simon Frost, streaming, Warrick Godfrey

Post navigation

Older Articles

Get Updates by Email

Put in your email address to receive new posts by email.

We'll send you an email each time there is a new article and may occasionally send you other emails. By signing up, you agree to our Privacy Policy

Check your inbox or spam folder to confirm your subscription.

Popular Posts

  • Video: The Fundamentals of Virtualisation
  • Video: How to Build an SRT Streaming Flow from Encoder to Edge
  • Video: Where can SMPTE 2110 and NDI co-exist?
  • Video: The New Video Codec Landscape - VVC, EVC, HEVC, LC-EVC, AV1 and more
  • Video: Decoder Complexity Aware AV1 Encoding Optimization

Diversity in Broadcast

Find out about Rise, an organisation which promotes gender diversity for women in technical roles throughout the industry.

Search

About This Site

The Broadcast Knowledge aggregates all the Broadcast industry’s free, educational webinars into one place with daily updates.

Recent Posts

  • Video: Scaling Video with AV1!
    19th January 2021
  • Video: The Fundamentals of Virtualisation
    18th January 2021
  • Video: Where The Puck Is Going: What’s Next for Esports & Sports Streaming
    15th January 2021
  • Video: AES67/ST 2110-30 over WAN
    14th January 2021
  • Video: Decoder Complexity Aware AV1 Encoding Optimization
    13th January 2021
  • Video: LCEVC, The Compression Enhancement Standard
    12th January 2021

Tags

AES67 (23) AI (26) AV1 (54) AVC (35) AWS (28) Bitmovin (37) CDN (21) Cloud (43) CMAF (35) codecs (50) DASH (24) DVB (22) Encoding (32) fundamentals (48) HDR (47) HEVC (54) HLS (30) IBC (38) IBC365 (24) IP (43) IPShowcase (55) IS-04 (21) Live Streaming (104) Low latency (51) MPEG DASH (22) Mux (22) NMOS (26) OTT (153) PTP (55) Remote Production (27) RIST (21) SMPTE (93) SRT (26) ST 2022-6 (21) ST 2059 (21) ST 2110 (121) ST 2110-30 (21) streaming (84) streaming media (47) UHD (27) Video Quality (22) VoD (21) VSF (30) WebRTC (23) Workflow (41)

Subscribe to get daily updates

The Broadcast Knowledge links to free educational videos & webinars focused on the Broadcast Industry.

We'll send you an email each time there is a new article and may occasionally send you other emails. By signing up, you agree to our Privacy Policy.

Thanks! Check your inbox or spam folder to confirm your subscription.

Views and opinions expressed on this website are those of the author(s) and do not necessarily reflect those of SMPTE or SMPTE Members. This website is presented for informational purposes only. Any reference to specific companies, products or services does not represent promotion, recommendation, or endorsement by SMPTE Powered by SMPTE
© 2021 The Broadcast Knowledge. All rights reserved.
Hiero by aThemes