Fremont (CA) - A company called ArcSoft Inc. has developed a graphical upscaling software application. The program takes lesser quality video and converts it (upscales it) to HD video using advanced upscaling algorithms. These algorithms allow for realistic or life-like 1080p presentations from significantly lesser sources, such as those captured by cell phones. The extensive compute needs of the algorithms have been applied to the massively parallel GPU and Nvidia's CUDA software engine, allowing for real-time conversion and playback with an 80% lower CPU requirement.
BEFORE AND AFTER
Left is an MPEG-2 720 x 480 4.95 Mbps video frame showing the original video. Right is a software upscaling to 1920 x 1080 at 24 fps with only 2% of CPU utilization, having the bulk of the compute load handled by the GPU and CUDA software engine. The SimHD software is a $19.95 plug-in extension to ArcSoft's TotalMedia Theatre 3 ($89.99) or TotalMedia Extreme ($119.99) software tools for HD video editing.
Nvidia and ArcSoft announced today the availability of a GPU-accelerated upscaling plug-in for full-motion HD video editing software, called TotalMedia. The application performs extremely complex, compute-intensive algorithms to extrapolate lost or ignored data by the original encoder.
For example, an original MPEG-2 720 x 480 video captured at 24fps or greater may produce individual frames which appear of significantly lower quality compared to the uncompressed original. The loss of quality results in a significant reduction in the media's storage requirement, allowing for much more video content to be captured. The down side is the loss of quality per frame. To compensate for the reduction in quality and depending on the encoder software, individual frames are rendered in such a way that each one loses certain types of data, such as color, resolution, contrast or other feature. When 24 or more frames are presented in sequence, the human eye gathers the larger amount of information and presents it to the brain as a video of notably higher quality than individual frames might otherwise indicate.
ArcSoft's upscaling algorithms take data that's present across multiple frames in the encoded (and compressed) video, and extrapolates it to reconstruct more of the original image data in each frame. In this way, the algorithm is able to reconstruct data that may have been lost or ignored on one frame, but may have been present on another, and then apply it to the surrounding frames.
This makes colors more vibrant, pixelated data less pixelated, contrasts greater, and it is able to remove a lot of the JPEG-like square-box artifacts which inhabit most compressed video forms. The end result is an upscaled video that is not a literal pixel-by-pixel rendering, but rather one which includes more image data per pixel per frame than the original video itself would've exposed using only its decoder and a scale-to-new-image-size algorithm.
Because of the massive compute requirements of this type of algorithm, searching forward and backward from every given frame for additional color data, ignoring noise and filtering through to real exposed data, on a traditional CPU the algorithm takes considerably longer and uses nearly all of the CPU power. By moving the workload away from the CPU and onto the GPU courtesy of Nvidia's CUDA software library, large compute jobes can be carried out with a minimal impact on the overall system performance, allowing for a single machine and user to be more productive on a more responsive machine.
Upscale DVD to HD in real-time
The software is designed to replace the need of having a Blu-ray player, or of having to do anything with the DVD disc itself, rather than just using it. To operate, start up the software and begin playing the DVD disc with settings specific to your taste. The software reads the DVD in real-time, upscales it to 1080p (or other resolution) output, applying the SimHD algorithms. This can be done via CPU-only, or GPU-acceleration.
To upscale in real-time, ArcSoft reports 80% CPU utilization on an Intel Core 2 Duo E4500 at 2.2 GHz and an Nvidia GeForce 9600 video card with 2 GB of memory when upsampling a DVD to HD using their algorithms in CPU-only mode. When switched to GPU-accelerated, the CPU load dropped to 15%.
Using this software with the appropriate video card adapter, a 1080p HDTV could be powered with a traditional DVD collection to near-HD quality in real-time from the PC, according to ArcSoft.
READ ON NEXT PAGE: Pricing, Terascale and massive compute...
UPSCALING IMPROVES QUALITY AND PATTERNS
This sample shows how an analysis of several frames can reconstruct the original image far more accurately. On the left, data from several frames is used to reveal greater detail in the stands supports and seats, where on the right the pattern image algorithms used by the encoder create pronounced lines which in the seats. In the real original image, those lines would not appear as pronounced as they are the result of a form of integer approximation or an aliasing of pixel position by the encoder.
The software sells for $89.99 to $119.99, with the SimHD GPU-accelerated plug-in costing an additional $19.95. ArcSoft claims the algorithm is sufficient to upscale most traditional DVDs into full- or near-HD quality through software alone -- meaning this $110 to $140 investment could turn your DVD library into a near-Blu ray library without having to buy all new discs or equipment.
In 2007, I had the opportunity to sit down with Jerry Bautista of Intel's Terascale project. Terascale is a massively parallel CPU that contains 80 cores, capable of over 1 Teraflop of performance in under 60 watts of power. While the CPU was nowhere near ready for commercial use, its proof-of-concept silicon form demonstrated the amazing power of parallel compute CPU abilities. Bautista and his team were able to discuss with me the significance of having such a massively parallel compute engine at their disposal, and one of the subjects discussed was this very thing -- upscaling of video data. Bautista suggested that even shaky, low-color, cell-phone quality videos could be processed through a compute engine like Terascale, and a software algorithm like SimHD, to create a real-time presentation where the cell phone video looks more like a regular TV video.
Massive compute revealed
To put it mildly, the power of parallel computing is amazing. And this type of software realization and real-world application of it is very exciting to see. Engines like CUDA and OpenCL, which expose compute abilities in easy-to-wield forms, abstracting away hardware considerations and allowing the developer to deal in data concepts, will change the way many of our devices operate in the years to come. This is true of mobile, handheld devices, as well as our desktop powerhouses that, thanks to these kinds of technologies, will only continue to become more and more powerful despite Moore's Law-like limitations -- at least for most software algorithms.