Corrupted data is a fact of life. Yet LTO tape Systems, SAN, NAS, Object Store, RAM, and WAN Optimizers all have configurations available to protect the fidelity of image contents in the workflow. Most serious is an archive scenario where content may sit untouched for a long duration and any corruption remains undetected for extended periods of time.
This talk from SMPTE Technical Conference 2017 by Keith Hogan covers the problems with hashes, looks at where errors can get introduced and ways to mitigate problems.
Depending on the path a video frame takes through the workflow, it will be treated to a varying set of protection technologies, like RAID, erasure coding, ECC Memory, and parity checking. Even on the network, errors can be introduced but checksums don’t always work.
To overcome the uncertainty of associated with how these methods ensure fidelity, the industry employs failure detection at each stage of the workflow (generally MD5 checksums). Keith discusses the protection mechanisms provided or employed by each workflow element and how frame corruption can occur, even when all of the protection technologies are working as designed. Finishing with a method for providing protection to images at the frame level using Forward Error Correction, such that there is uniformity of protection for images applied throughout the workflow, Keith shows that media errors may be recovered in most cases without having to access a backup copy.
