Acquiring visually pleasing images and videos under insufficient lighting has been an essential task for human perception and computer-based scene understanding, yet it remains to be challenging because of the fundamental restrictions of insufficient photon counts and complex noises implied by consumer-grade image sensors. In past decades, a great variety of techniques, on the basis of filtering, sparsity, and so on, for image denoising and enhancement have been proposed, which are proven less applicable to severely degraded images with a variety of intertwined noises. A breakthrough was achieved by the study of Chen et al. (Learning to See in the Dark, CVPR2018), which reveals that supervised deep learning techniques are capable of enhancing low-light images in the RAW domain, as long as sufficient amount of paired dark/bright images are provided.

A typical way of collecting paired dark/bright images with short exposure and long exposure is applicable to static scenes only, and the scale and variation of such a dataset are usually limited. To resolve these limitations, some novel optical and mechanical systems have recently been developed, which enable the capturing of paired dark/bright videos, rather than static images. Another promising thread of research is to model, either fully or partially, the distribution of major types of noise in the photon-electron-digit transition process, and then to synthesize realistic low-light images from clean bright images. It has been verified that models trained on such synthesized data can even surpass those trained on captured data.

This half-day tutorial will cover the latest advances in the broad theme of Optics for Better AI, with a specific focus on how to capture and synthesize realistic data for training low-light enhancement deep models. In this tutorial, we will first present the overall pipeline and effects of using realistic data, including (i) Low-light Image Enhancement using Synthesized Data; (ii) Low-light Video Enhancement using Captured Data. Then, we show detailed instructions on noise calibration and construction of optical imaging systems, including (iii) How to Calibrate the Noise Model of a Specific Camera; (iv) How to Construct a Co-axial Imaging System.

Considering that realistic data are critical for many other image restoration tasks, like deblurring, super-resolution, frame interpolation, dynamic range extension, the presented techniques in noise modeling and calibration, as well as the construction details of co-axial imaging system, will be valuable to all scholars in scene enhancement and understanding with adversarial imaging factors.

Detailed schedule will be updated soon.