Two Papers Accepted at AAAI 2020

The following two papers have been accepted in AAAI 2020 conference. Congratulations!

AAAI 2020 had a record number of over 8,800 submissions this year, 7,737 were reviewed, and 1,591 papers were accepted, yielding an acceptance rate of 20.6%. There was especially stiff competition this year because of the high number of submissions.

Title: FISR: Deep Joint Frame Interpolation and Super-Resolution with A Multi-scale Temporal Loss

Authors: Soo Ye Kim*, Jihyong Oh*, and Munchurl Kim (*equal contribution)

Abstract:

Super-resolution (SR) has been widely used to convert low resolution legacy videos to high-resolution (HR) ones, to suit the increasing resolution of displays (e.g. UHD TVs). However, it becomes easier for humans to notice motion artifacts (e.g. motion judder) in HR videos being rendered on larger sized display devices. Thus, broadcasting standards support higher frame rates for UHD (Ultra High Definition) videos (4K@60fps, 8K@120fps), meaning that applying SR only is insufficient to produce genuine high quality videos. Hence, to up-convert legacy videos for realistic applications, not only SR but also video frame interpolation (VFI) is necessitated. In this paper, we first propose a joint VFI-SR framework for upscaling the spatio-temporal resolution of videos from 2K 30fps to 4K 60fps. For this, we propose a novel training scheme with a multi-scale temporal loss that imposes temporal regularization on the input video sequence, which can be applied to any general video-related task. The proposed structure is analyzed in depth with extensive experiments.

Title: JSI-GAN: GAN-Based Joint Super-Resolution and Inverse Tone-Mapping with Pixel-wise Details- and Contrast-aware Filters for UHD HDR Video

Authors: Soo Ye Kim*, Jihyong Oh* and Munchurl Kim (*equal contribution)

Abstract:

Joint learning of super-resolution (SR) and inverse tonemapping (ITM) has been explored recently, to convert legacy low resolution (LR), standard dynamic range (SDR) videos to high resolution (HR) high dynamic range (HDR) videos for the growing need of UHD HDR TV/broadcasting and streaming applications. However, previous CNN-based methods directly reconstruct the HR HDR frames from LR SDR frames, and are only trained with a simple L2 loss. In this paper, we take a divide-and-conquer approach to designing a novel GAN-based joint SR-ITM network, called JSI-GAN, which is composed of three task-specific subnets: an image reconstruction subnet, a detail restoration (DR) subnet and a local contrast enhancement (LCE) subnet. We delicately design these subnets so that they are appropriately trained for the intended purpose, learning a pair of pixel-wise 1D separable filters via the DR subnet for details restoration and a pixel-wise 2D local filter by the LCE subnet for contrast enhancement. The pixel-wise 1D separable and 2D local filters are applied for detail layer and base layer components of each input frame to reconstruct HR HDR frames. To train the JSI-GAN, we adopt two adversarial losses including a typical GAN loss and a novel detail GAN loss that helps restoring fine details for HR HDR reconstruction. When all subnets are jointly trained well, the reconstructed HR HDR results of higher quality are obtained with at least 0.41 dB gain in PSNR over those generated by the previous methods.