Enhance! Google researchers detail new method for upscaling low-resolution images with impressive results

Cascaded generation of unconditional 1024x1024 faces.

Thanks to CSI, as well as plenty of other crime dramas, the phrase ‘Enhance’ has taken on a life of its own as a tongue-in-cheek way of attempting to digitally extract extra information from low-resolution images that simply isn’t feasible in real-world situations. Or is it? A new blog post on the Google AI Blog showcases a new technology its developed to upscale low-resolution images with incredible results.

The blog post, titled ‘High Fidelity Image Generation Using Diffusion Models,’ explains how Google researchers have developed a pair of AI technologies that can take a low-resolution image and steadily increase resolution through selective destruction and reconstruction of the original input image.

The before and after of increasing the resolution of the portraits from 64 x 64 pixels to 1024 x 1024 pixels.

The first component of the process is Super-Resolution via Repeated Refinements (SR3), ‘a super-resolution diffusion model that takes as input a low-resolution image, and builds a corresponding high-resolution image from pure noise.’ In essence, this model applies pure Gaussian noise to a low-resolution image before using noise-reduction technologies to effectively reconstruct a nearly noise-less image that’s four times the resolution of the input.

The researchers then use Cascaded Diffusion Models (CDM) to intelligently apply Gaussian noise and blur to the output image before repeating the process again. This technique, which Google calls ‘conditioning augmentation,’ improves the image quality to the point that it surpasses current AI upscaling methods, which include BigGAN-deep and VQ-VAE-2.

According to Google, this new technology ‘achieves strong benchmark results on the super-resolution task for face and natural images when scaling to resolutions 4x–8x that of the input low-resolution image.’ As visible from the above illustration, this means a 64 x 64 pixel image can output an impressively clear 1024 x 1024 pixel image.

Super Resolution results: (Above) 64x64 > 512x512 face super-resolution, (Below) 64x64 -> 256x256 natural image super-resolution.

Google researchers say the technology ‘[pushes] the performance of diffusion models to state-of-the-art on super-resolution and class-conditional ImageNet generation benchmarks’ and notes they’re ‘excited to further test the limits of diffusion models for a wide variety of generative modeling problems.’

You can read the entire article on the Google AI blog.