Preface
D5 Render has rolled out a new feature "D5 SR Image Rendering" in the latest version 2.5. "SR" stands for "Super Resolution", a D5's self-developed tech that transforms low-res images into high-res ones.
Based on neural network super-resolution image generation algorithm, D5 SR is combined with multi-channel rendering feature maps to significantly speed up image rendering.
The Super Resolution tech can be used in a wide range of applications. For instance, it can enhance the details and resolution in video rendering, and in real-time rendering, it can increase the frame rate to improve user experience.
But let's be honest, amplifying the resolution of rendered images without distortion has always been a tough nut to crack.
Highlights of D5 SR
More Realistic Reconstruction
Real-esrgan is a blind super-resolution algorithm that has not been specifically trained for D5 Render images. Therefore, when doubling the resolution of rendered images, real-esrgan may have some obvious artifacts such as unnatural materials.
Yet, this problem is well tackled with D5 SR, as you can see in the following test images. That's because D5 SR added the geometry, material, texture and other info of this object to improve the quality of reconstructed high-res images.
Zoom in to check the details. You can see that the SR algorithm without training leads to serious distortions such as blurring, smearing, or abnormal textures, while D5 SR doesn't.
Optimized Reflections
When testing D5 SR at first, the twice-expanded images lost some reflection details.
After importing the info of reflection channels and materials, D5 SR can better reproduce the reflections.
You can see the difference shown in these two scenes.
Improved Rendering Efficiency
After enabling D5 SR, you will witness a boosted rendering speed for large-resolution images.
Take this scene as an example.
If rendering an image of 16K resolution (15360X8640), you will need 1h17m13s when D5 SR is off and only 27m23s after it's turned on, saving 65% of the total time.
How to Use D5 SR
Go to Menu > Preference > Widget and switch on 'D5 SR Image Rendering beta'. It will automatically work when D5 renders images with no need of any other settings.
Please note that the D5 SR algorithm is activated only on images with a size larger than 1440 x 1440 resolution.
How D5 SR was born
D5 team trains the SR algorithm with a vast collection of images generated by D5. The neural network can effectively make use of the prior information learned from this training data to give us a better high-resolution reconstruction.
However, at first we did not have enough information of 3D scenes such as geometry, materials, and textures, leading to a gap between the reconstruction and the physical rendering results. Therefore, we introduced these 3D information as input features to produce a better result.
Technically, the D5 R & D team has done the following:
First, we use high-resolution albedo and normal information to reconstruct texture and geometric features.
Then we bring in reflection, transparency, metal, and roughness info to optimize reflection and highlight effects.
To further improve the reconstruction result, we used a new algorithm. It improves the original ESRGAN neural network and uses a dual-stream network to process low-resolution original images and high-resolution channel information graphs respectively. We used relative L1 loss to enhance the supervision on pixel error and perceptual loss, and introduce adversarial loss to improve the super-resolution result.
Finally, we used U-shaped network structure to replace some Residual Dense Blocks in the ESRGAN, so as to expand the sensing field range of feature extraction of high-resolution channel information maps and further improve the inference speed.
