iSIZE has developed BitClear, an AI-based video processing technology that removes compression artifacts (like blurring and blocking artifacts) from user-generated (or heavily-compressed) content.
Video content that has been through multiple transcoding iterations can be revived to the maximum possible quality without affecting the artistic intent of the original creators.
The process can also allow for video upscaling, all with as little as 5ms processing latency on GPUs or high-performance CPUs.
“The huge traffic in shared video content today passes through several platforms and users before potentially reaching a large audience or big influencers,” explained Sergio Grce, CEO and founder of iSIZE.
“Quite often, UGC that is viewed by millions of people over multiple social media platforms has been re-shared and re-uploaded numerous times. This process tends to make it very degraded or even unwatchable due to multiple transcoding iterations.
“For social media or UGC distribution and streaming companies, it is imperative that video content is presented in as high quality as possible to retain audiences,” he continued.
“BitClear is designed to achieve the maximum quality improvement by alleviating the effect of compression artifacts. Its AI nature and implementation efficiency means it can be deployed at scale without the need for human inspection or tuning.”
Other applications of BitClear include real-time de-noising before transcoding and broadcasting of live game streams, live sports, entertainment, and virtual reality feeds.
Through bespoke neural network designs that optimally learn to disentangle the noise from the data manifolds, BitClear is able to remove compression noise and retain or recover the original content features.
While it learns the noise signatures of the various encoding standards, BitClear does not need to know the history of the specific asset: it can process any highly-compressed content and produced a higher-quality output that improves the value of the asset.
Unlike previous noise removal technologies that aim towards specific noise patterns like film grain or blur, BitClear specifically targets compression artifacts of typical MPEG or AOMedia encoders, with particular focus on highly-compressed or highly-distorted content.
On an Intel Xeon CPU, the fastest BitClear models achieve 30 frames per second for 1080p content.
Its scalable neural network architecture means it can scale up to high volumes of content and is ideal for cloud as well as on-premise deployment on CPUs, GPUs or custom hardware that supports neural network inference.
“BitClear is currently in evaluation trials with major UGC distribution platforms,” said Grce.
“While the current focus is on removal of compression artifacts from heavily-compressed user generated content, our unique AI-powered technology can also be trained for other denoising tasks or to optimize delivery where limited by infrastructures, for example in mobile streaming and IoT security.”