SNU Professor Lee Kyoung Mu Selected as “This Month’s Scientist”

- Development of Deep Learning Based Super-Resolution Algorithm
- Broadly Applicable to Areas like Scientific Investigation, Traffic Enforcement, and Medical Treatment


▲ Professor Lee Kyoung Mu Discussing about Deep Learning Video Analysis with Researchers

 
SNU College of Engineering (Dean Cha Kook-Heon) announces that Professor Lee Kyoung Mu from SNU Department of Electrical and Computer Engineering has been nominated as February’s winner of “This Month’s Scientist” – an award established by the Ministry of Science and ICT and co-sponsored by the National Research Foundation of Korea and the Seoul Economic Daily.
 
Professor Lee received this nomination for his development of deep learning based super-resolution algorithm that allows a lossless image enhancement up to 8 times the original. This new super-resolution image reconstruction method is a breakthrough technology that prevents the loss of resolution due to upsize.
 
The key to this deep learning super-resolution algorithm is the use of the VDSR (Very Deep Network for Super-Resolution) system that implements 20 Convolutional Neural Network (CNN) layers and applies the principles of residual connection.
 
Convolutional Neural Network (CNN) is comprised of concatenating layers – the deeper the layers are, the broader the use of input image context, thus a more valuable image feature extraction. Through residual learning, the network learns the difference between the input and the output layers. This has allowed a faster and a more stable deep learning. Enhanced Deep Residual Networks (EDSR) and Multi-scale Deep Super Resolution System (MDSR) have also increased the speed and accuracy of the overall image enhancement process.
 
This technology can potentially enhance the resolution of photos produced by low-price cameras. It may also be able to restore defocused and blurred images caused by camera shake.
 
In fact, super-resolution imaging can be stretched to any areas that require an image enhancement. For instance, this technique can generate high resolution image from compressed, low resolution video frames of common security surveillance videos for face or car recognition. Videos taken in dark are often found to be low in clarity and degraded with noises, but now face and car license plate may be recognized easier. This includes videos taken by automobile black boxes. Super-resolution imaging is already an essential in the forensic field and shall be used at the National Forensic Field, Supreme Prosecutor’s Office, and police agencies.
 
Professor Lee explains, “Although there are many surveillance cameras installed on streets and buildings, recognition gets difficult due to low resolution when the object is only 10~20m away. But then again, cost becomes an issue when we turn to professional installation and maintenance of high definition cameras. This is when deep learning algorithm software can come into play for image enhancement.” He adds, “Video compression degrades the image resolution, but using this software, resolution recovery close to the original video may be possible.”
 
In addition, this algorithm can also be applied to medical image like the magnetic resonance imaging (MRI) and computed tomography (CT) scan for precise analysis. It can also be used for high resolution image transformation for today’s ultra-high-definition (UHD) television, image enhancement on smart phones, defect inspection for metal parts and semiconductors, enlargement of car image taken by satellite, etc. Professor Lee states, “Costly equipment is required to read high-quality satellite pictures and an expensive UHD camera is required to match up with the UHD television. However, these pricey equipments can be easily replaced with this cost-effective software that converts low-quality image to high definition image.”
 
This technology is expected to become a core part of drone, intelligent robot, and autonomous vehicle video analysis – some major technology breakthroughs that characterize the Fourth Industrial Revolution. In order to embed this software into surveillance cameras and autonomous vehicles, development of chip is mandatory. Professor Lee plans to devote to developing strategies for successful “technology transfer of algorithm software to various industrial fields.”
 

▲ Lossless Picture Enlargement through Deep Learning Based Super-Resolution Algorithm