Kwanwoong Song

Il-Lyong Jung

Jun-Seong Kim

Chul Lee

Tae-Young Chung

Jae-Kyun Ahn

Dae-Youn Lee

Jin-Hwan Kim

Chulwoo Lee

Hansang Kim

Yeong Jun Koh

Won-Dong Jang

Lee Se-Ho

Jun-Tae Lee

Han-Ul Kim

Kyung-Rae Kim

Jae-Han Lee

Keunsoo Ko

Hwan Choi

Dongkwon Jin

Junheum Park

Yuk Heo

Seon-Ho Lee

Ji-Soo Kim

Nyeong-Ho Shin

Jinyoung Jun

Htoo Maung

Hendro Baskoro

Marcelo Duarte Esperidiao

Sergey Chalkov

Leonardo Juniti Nomoto

Nikolay Akatyev

Sang Hwan Kim

Seong-Gyun Jeong

Kim Miae

Jae-Moon Lim

Youngbae Kim

Dae-Jung Han

Kyungsun Lim

Juhyeok Mun

Minhyeok Heo

Won-Hee Park

Dahyun Kim

The Media Communications Laboratory conducts research in the field of media and communication technologies.

미디어 통신 연구실은 미디어 및 통신 기술 분야의 연구를 수행한다.

Blind Image Quality Assessment Based on Geometric Order Learning

Unsupervised Order Learning

Geometric Order Learning for Rank Estimation

Order Learning Using Partially Ordered Data via Chainization

Moving Window Regression: A Novel Approach to Ordinal Regression

Deep Repulsive Clustering of Ordered Data Based on Order-Identity Decomposition

Order Learning and Its Application to Age Estimation

Forbes: Face Obfuscation Rendering via Backpropagation Refinement Scheme

Context-based Trit-Plane Coding for Progressive Image Compression

DPICT: Deep Progressive Image Compression Using Trit-Planes

OMR: Occlusion-Aware Memory-Based Refinement for Video Lane Detection

Recursive Video Lane Detection

Eigenlanes: Data-Driven Lane Descriptors for Structurally Diverse Lanes

Eigencontours: Novel Contour Descriptors Based on Low-Rank

BiFormer: Learning Bilateral Motion Estimation via Bilateral Transformer for 4K Video Frame Interpolation

Asymmetric Bilateral Motion Estimation for Video Frame Interpolation

BMBC: Bilateral Motion Estimation with Bilateral Cost Volume for Video Interpolation

Motion-compensated frame interpolation based on multihypothesis motion estimation and texture optimization

Continuously Masked Transformer for Image Inpainting

Light Field Super-Resolution via Adaptive Feature Remixing

Masked Spatial Propagation Network for Sparsity-Adaptive Depth Refinement

Depth Map Decomposition for Monocular Depth Estimation

Multi-Loss Rebalancing Algorithm for Monocular Depth Estimation

Monocular Depth Estimation Using Relative Depth Maps

Single-Image Depth Estimation Based on Fourier Domain Analysis

Global and Local Enhancement Networks For Paired and Unpaired Image Enhancement

PieNet: Personalized Image Enhancement

Optimized brightness compensation and contrast enhancement for transmissive liquid crystal displays

Contrast enhancement based on layered difference representation of 2D histograms

Power-constrained contrast enhancement for emissive displays based on histogram equalization

MFP: Making Full Use of Probability Maps for Interactive Image Segmentation

Guided Interactive Video Object Segmentation Using Reliability-Based Attention Maps

Interactive Video Object Segmentation Using Global and Local Transfer Modules

Temporal Superpixels Based on Proximity-Weighted Patch Matching

Online Video Object Segmentation via Convolutional Trident Network

Primary Object Segmentation in Videos Based on Region Augmentation and Reduction

Contour-Constrained Superpixels for Image and Video Processing

Streaming Video Segmentation via Short-Term Hierarchical Segmentation and Frame-by-Frame Markov Random Field Optimization

Semi-supervised Video Object Segmentation Using Multiple Random Walkers

Primary Object Segmentation in Videos via Alternate Convex Optimization of Foreground and Background Distributions

Compressed Domain Video Saliency Detection Using Global and Local Spatiotemporal Features

Spatiotemporal saliency detection for video sequences based on random walk with restart

Multiple Random Walkers and Their Application to Image Cosegmentation

Multiscale saliency detection using random walk with restart

Instance-Level Future Motion Estimation in a Single Image Based on Ordinal Regression

Unsupervised Primary Object Discovery in Videos Based on Evolutionary Primary Object Modeling with Reliable Object Proposals

Locator-Checker-Scaler Object Tracking Using Spatially Ordered and Weighted Patch Descriptor

CDT: Cooperative Detection and Tracking for Tracing Multiple Objects in Video Sequences

POD: Discovering Primary Objects in Videos Based on Evolutionary Refinement of Object Recurrence, Background, and Primary Object Models

SOWP: Spatially Ordered and Weighted Patch Descriptor for Visual Tracking

Multihypothesis Trajectory Analysis for Robust Visual Tracking

Visual tracking using pertinent patch selection and masking

Multiscale Feature Extractors for Stereo Matching Cost Computation

Consistent stereo matching under varying radiometric conditions

Video deraining and desnowing using temporal correlation and low-rank matrix completion

Optimized contrast enhancement for real-time image and video dehazing

Inter-Frame Compression of 3D Point Cloud Sequences

Video Stabilization Based on Feature Trajectory Augmentation and Selection and Robust Mesh Grid Warping

Contour-based object forecasting for autonomous driving

Error-mask-adaptive dynamic filtering for image inpainting

Local and global mixture network for image inpainting

Image cropping based on order learning

Versatile depth estimator based on common relative depth estimation and camera-specific relative-to-metric depth conversion

RPF: Reference-Based Progressive Face Super-Resolution Without Losing Details and Identity

Video Frame Interpolation Based on Symmetric and Asymmetric Motions

Edge-Aware Interactive Contrast Enhancement

Local Memory Read-and-Comparator for Video Object Segmentation

Single-Image Depth Estimation using Relative Depths

Blind and Compact Denoising Network Based on Noise Order Learning

Gluing Reference Patches Together for Face Super-Resolution

IceNet for Interactive Contrast Enhancement

Monocular Human Depth Estimation via Pose Estimation

Multi-Scale Warping for Video Frame Interpolation

SAF-Nets: Shape-Adaptive Filter Networks for 3D point cloud processing

Subpixel Rendering for Diamond-Shaped PenTile Displays Using Patch-Based Adaptive Filters

Online Multiple Object Tracking Based on Open-Set Few-Shot Learning

Instance-level future motion estimation in a single image based on ordinal regression and semi-supervised domain adaptation

Three-dimensional convolutional neural network for prostate MRI segmentation and comparison of prostate volume measurements by use of artificial neural network and ellipsoid formula

Superpixels for image and video processing based on proximity-weighted patch matching

ELF-Nets: Deep Learning on Point Clouds Using Extended Laplacian Filter

Property-Specific Aesthetic Assessment With Unsupervised Aesthetic Property Discovery

Object tracking under large motion: Combining coarse-to-fine search with superpixels

Tracking-by-Segmentation Using Superpixel-Wise Neural Network

Photographic composition classification and dominant geometric element detection for outdoor scenes

Multiscale feature extractors for stereo matching cost computation

Reflection removal under fast forward camera motion

Unsupervised primary object discovery in videos based on evolutionary primary object modeling with reliable object proposals

Locator-checker-scaler object tracking using spatially ordered and weighted patch descriptor

Contrast enhancement of noisy low-light images based on structure-texture-noise decomposition

Compressed domain video saliency detection using global and local spatiotemporal features

Video stabilization based on feature trajectory augmentation and selection and robust mesh grid warping

FDQM: fast quality metric for depth maps without view synthesis

Large-scale 3D point cloud compression using adaptive radial distance prediction in hybrid coordinate domains

Progressive 3D mesh compression using MOG-based Bayesian entropy coding and gradual prediction

Bit allocation algorithm with novel view synthesis distortion model for multi-view video plus depth coding

Technologies for 3D mesh compression: A survey

Motion-compensated frame interpolation using bilateral motion estimation and adaptive overlapped block motion compensation

Single-image deraining using an adaptive nonlocal means filter

Interactive image segmentation via backpropagating refinement scheme

Power-constrained contrast enhancement for emissive displays based on histogram equalization.

Primary object segmentation in videos based on region augmentation and reduction

Spatial and temporal error concealment techniques for video transmission over noisy channels

Monocular depth estimation using relative depth maps

Single image dehazing based on contrast enhancement

Representative Color Transform for Image Enhancement

Sang-Uk Lee

Jae-Young Sim

C.-C. Jay Kuo

Ioannis Katsavounidis

Hanul Kim

Young Yoon Lee

Juntae Lee

IL-Lyong Jung

Chang-Su Kim

A novel contrast enhancement algorithm based on the layered difference representation of 2D histograms is proposed in this paper. We attempt to enhance image contrast by amplifying the gray-level differences between adjacent pixels. To this end, we obtain the 2D histogram h(k, k+l) from an input image, which counts the pairs of adjacent pixels with gray-levels k and k+l, and represent the gray-level differences in a tree-like layered structure. Then, we formulate a constrained optimization problem based on the observation that the gray-level differences, occurring more frequently in the input image, should be more emphasized in the output image. We first solve the optimization problem to derive the transformation function at each layer. We then combine the transformation functions at all layers into the unified transformation function, which is used to map input gray-levels to output gray-levels. Experimental results …

A fast and optimized dehazing algorithm for hazy images and videos is proposed in this work. Based on the observation that a hazy image exhibits low contrast in general, we restore the hazy image by enhancing its contrast. However, the overcompensation of the degraded contrast may truncate pixel values and cause information loss. Therefore, we formulate a cost function that consists of the contrast term and the information loss term. By minimizing the cost function, the proposed algorithm enhances the contrast and preserves the information optimally. Moreover, we extend the static image dehazing algorithm to real-time video dehazing. We reduce flickering artifacts in a dehazed video sequence by making transmission values temporally coherent. Experimental results show that the proposed algorithm effectively removes haze and is sufficiently fast for real-time dehazing applications.

Three-dimensional (3D) meshes have been widely used in graphic applications for the representation of 3D objects. They often require a huge amount of data for storage and/or transmission in the raw data format. Since most applications demand compact storage, fast transmission, and efficient processing of 3D meshes, many algorithms have been proposed to compress 3D meshes efficiently since early 1990s. In this survey paper, we examine 3D mesh compression technologies developed over the last decade, with the main focus on triangular mesh compression technologies. In this effort, we classify various algorithms into classes, describe main ideas behind each class, and compare the advantages and shortcomings of the algorithms in each class. Finally, we address some trends in the 3D mesh compression technology development.

In this work, we develop a new motion-compe (MC) interpolation algorithm to enhance the temporal resolution of video sequences. First, we propose the bilateral motion estimation scheme to obtain the motion field of an interpolated frame without yielding the hole and overlapping problems. Then, we partition a frame into several object regions by clustering motion vectors. We apply the variable-size block MC (VS-BMC) algorithm to object boundaries in order to reconstruct edge information with a higher quality. Finally, we use the adaptive overlapped block MC (OBMC), which adjusts the coefficients of overlapped windows based on the reliabilities of neighboring motion vectors. The adaptive OBMC (AOBMC) can overcome the limitations of the conventional OBMC, such as over-smoothing and poor de-blocking. Experimental results show that the proposed algorithm provides a better image quality than conventional …

An adaptive rain streak removal algorithm for a single image is proposed in this work. We observe that a typical rain streak has an elongated elliptical shape with a vertical orientation. Thus, we first detect rain streak regions by analyzing the rotation angle and the aspect ratio of the elliptical kernel at each pixel location. We then perform the nonlocal means filtering on the detected rain streak regions by selecting nonlocal neighbor pixels and their weights adaptively. Experimental results demonstrate that the proposed algorithm removes rain streaks more efficiently and provides higher restored image qualities than conventional algorithms.

An interactive image segmentation algorithm, which accepts user-annotations about a target object and the background, is proposed in this work. We convert user-annotations into interaction maps by measuring distances of each pixel to the annotated locations. Then, we perform the forward pass in a convolutional neural network, which outputs an initial segmentation map. However, the user-annotated locations can be mislabeled in the initial result. Therefore, we develop the backpropagating refinement scheme (BRS), which corrects the mislabeled pixels. Experimental results demonstrate that the proposed algorithm outperforms the conventional algorithms on four challenging datasets. Furthermore, we demonstrate the generality and applicability of BRS in other computer vision tasks, by transforming existing convolutional neural networks into user-interactive ones.

Video interpolation increases the temporal resolution of a video sequence by synthesizing intermediate frames between two consecutive frames. We propose a novel deep-learning-based video interpolation algorithm based on bilateral motion estimation. First, we develop the bilateral motion network with the bilateral cost volume to estimate bilateral motions accurately. Then, we approximate bi-directional motions to predict a different kind of bilateral motions. We then warp the two input frames using the estimated bilateral motions. Next, we develop the dynamic filter generation network to yield dynamic blending filters. Finally, we combine the warped frames using the dynamic blending filters to generate intermediate frames. Experimental results show that the proposed algorithm outperforms the state-of-the-art video interpolation algorithms on several benchmark datasets. The source codes and pre-trained models …

A novel algorithm to remove rain or snow streaks from a video sequence using temporal correlation and low-rank matrix completion is proposed in this paper. Based on the observation that rain streaks are too small and move too fast to affect the optical flow estimation between consecutive frames, we obtain an initial rain map by subtracting temporally warped frames from a current frame. Then, we decompose the initial rain map into basis vectors based on the sparse representation, and classify those basis vectors into rain streak ones and outliers with a support vector machine. We then refine the rain map by excluding the outliers. Finally, we remove the detected rain streaks by employing a low-rank matrix completion technique. Furthermore, we extend the proposed algorithm to stereo video deraining. Experimental results demonstrate that the proposed algorithm detects and removes rain or snow streaks efficiently …

A power-constrained contrast-enhancement algorithm for emissive displays based on histogram equalization (HE) is proposed in this paper. We first propose a log-based histogram modification scheme to reduce overstretching artifacts of the conventional HE technique. Then, we develop a power-consumption model for emissive displays and formulate an objective function that consists of the histogram-equalizing term and the power term. By minimizing the objective function based on the convex optimization theory, the proposed algorithm achieves contrast enhancement and power saving simultaneously. Moreover, we extend the proposed algorithm to enhance video sequences, as well as still images. Simulation results demonstrate that the proposed algorithm can reduce power consumption significantly while improving image contrast and perceptual quality.

We propose a novel video frame interpolation algorithm based on asymmetric bilateral motion estimation (ABME), which synthesizes an intermediate frame between two input frames. First, we predict symmetric bilateral motion fields to interpolate an anchor frame. Second, we estimate asymmetric bilateral motions fields from the anchor frame to the input frames. Third, we use the asymmetric fields to warp the input frames backward and reconstruct the intermediate frame. Last, to refine the intermediate frame, we develop a new synthesis network that generates a set of dynamic filters and a residual frame using local and global information. Experimental results show that the proposed algorithm achieves excellent performance on various datasets. The source codes and pretrained models are available at https://github. com/JunHeum/ABME.

A novel algorithm to segment a primary object in a video sequence is proposed in this work. First, we generate candidate regions for the primary object using both color and motion edges. Second, we estimate initial primary object regions, by exploiting the recurrence property of the primary object. Third, we augment the initial regions with missing parts or reducing them by excluding noisy parts repeatedly. This augmentation and reduction process (ARP) identifies the primary object region in each frame. Experimental results demonstrate that the proposed algorithm significantly outperforms the state-of-the-art conventional algorithms on recent benchmark datasets.

We propose a deep learning algorithm for single-image depth estimation based on the Fourier frequency domain analysis. First, we develop a convolutional neural network structure and propose a new loss function, called depth-balanced Euclidean loss, to train the network reliably for a wide range of depths. Then, we generate multiple depth map candidates by cropping input images with various cropping ratios. In general, a cropped image with a small ratio yields depth details more faithfully, while that with a large ratio provides the overall depth distribution more reliably. To take advantage of these complementary properties, we combine the multiple candidates in the frequency domain. Experimental results demonstrate that proposed algorithm provides the state-of-art performance. Furthermore, through the frequency domain analysis, we validate the efficacy of the proposed algorithm in most frequency bands.

Two novel error concealment techniques are proposed for video transmission over noisy channels in this work. First, we present a spatial error concealment method to compensate a lost macroblock in intra-coded frames, in which no useful temporal information is available. Based on selective directional interpolation, our method can recover both smooth and edge areas efficiently. Second, we examine a dynamic mode-weighted error concealment method for replenishing missing pixels in a lost macroblock of inter-coded frames. Our method adopts a decoder-based error tracking model and combines several concealment modes adaptively to minimize the mean square error of each pixel. The method is capable of concealing lost packets as well as reducing the error propagation effect. Extensive simulations have been performed to demonstrate the performance of the proposed methods in error-prone environments

We propose a novel algorithm for monocular depth estimation using relative depth maps. First, using a convolutional neural network, we estimate relative depths between pairs of regions, as well as ordinary depths, at various scales. Second, we restore relative depth maps from selectively estimated data based on the rank-1 property of pairwise comparison matrices. Third, we decompose ordinary and relative depth maps into components and recombine them optimally to reconstruct a final depth map. Experimental results show that the proposed algorithm provides the state-of-art depth estimation performance.

A novel saliency detection algorithm for video sequences based on the random walk with restart (RWR) is proposed in this paper. We adopt RWR to detect spatially and temporally salient regions. More specifically, we first find a temporal saliency distribution using the features of motion distinctiveness, temporal consistency, and abrupt change. Among them, the motion distinctiveness is derived by comparing the motion profiles of image patches. Then, we employ the temporal saliency distribution as a restarting distribution of the random walker. In addition, we design the transition probability matrix for the walker using the spatial features of intensity, color, and compactness. Finally, we estimate the spatiotemporal saliency distribution by finding the steady-state distribution of the walker. The proposed algorithm detects foreground salient objects faithfully, while suppressing cluttered backgrounds effectively, by …

A semi-supervised online video object segmentation algorithm, which accepts user annotations about a target object at the first frame, is proposed in this work. We propagate the segmentation labels at the previous frame to the current frame using optical flow vectors. However, the propagation is error-prone. Therefore, we develop the convolutional trident network (CTN), which has three decoding branches: separative, definite foreground, and definite background decoders. Then, we perform Markov random field optimization based on outputs of the three decoders. We sequentially carry out these processes from the second to the last frames to extract a segment track of the target object. Experimental results demonstrate that the proposed algorithm significantly outperforms the state-of-the-art conventional algorithms on the DAVIS benchmark dataset.

A simple yet effective object descriptor for visual tracking is proposed in this paper. We first decompose the bounding box of a target object into multiple patches, which are described by color and gradient histograms. Then, we concatenate the features of the spatially ordered patches to represent the object appearance. Moreover, to alleviate the impacts of background information possibly included in the bounding box, we determine patch weights using random walk with restart (RWR) simulations. The patch weights represent the importance of each patch in the description of foreground information, and are used to construct an object descriptor, called spatially ordered and weighted patch (SOWP) descriptor. We incorporate the proposed SOWP descriptor into the structured output tracking framework. Experimental results demonstrate that the proposed algorithm yields significantly better performance than the state-of-the-art trackers on a recent benchmark dataset, and also excels in another recent benchmark dataset.

A simple and adaptive single image dehazing algorithm is proposed in this work. Based on the observation that a hazy image has low contrast in general, we attempt to restore the original image by enhancing the contrast. First, the proposed algorithm estimates the airlight in a given hazy image based on the quad-tree subdivision. Then, the proposed algorithm estimates the transmission map to maximize the contrast of the output image. To measure the contrast, we develop a cost function, which consists of a standard deviation term and a histogram uniformness term. Experimental results demonstrate that the proposed algorithm can remove haze efficiently and reconstruct fine details in original scenes clearly.

A novel ordinal regression algorithm, called moving window regression (MWR), is proposed in this paper. First, we propose the notion of relative rank (rho-rank), which is a new order representation scheme for input and reference instances. Second, we develop global and local relative regressors (rho-regressors) to predict rho-ranks within entire and specific rank ranges, respectively. Third, we refine an initial rank estimate iteratively by selecting two reference instances to form a search window and then estimating the rho-rank within the window. Extensive experiments results show that the proposed algorithm achieves the state-of-the-art performances on various benchmark datasets for facial age estimation and historical color image classification. The codes are available at https://github. com/nhshin-mcl/MWR.

Recently, the encoder-decoder and intensity transformation approaches lead to impressive progress in image enhancement. However, the encoder-decoder often loses details in input images during down-sampling and up-sampling processes. Also, the intensity transformation has a limited capacity to cover color transformation between low-quality and high-quality images. In this paper, we propose a novel approach, called representative color transform (RCT), to tackle these issues in existing methods. RCT determines different representative colors specialized in input images and estimates transformed colors for the representative colors. It then determines enhanced colors using these transformed colors based on the similarity between input and representative colors. Extensive experiments demonstrate that the proposed algorithm outperforms recent state-of-the-art algorithms on various image enhancement problems.

MCL - Media Communications Lab.

교수님 소개

김창수

연구실 소개

연구실 키워드

알럼나이가 간 회사

연구실 지원 방법

대우 조건

연도별 논문 인용수

저널 & 학회 논문 출판 수

연구실 논문 목록

Frequency-domain Manipulation for Face Obfuscation

CanonCGT: Reference-Based Color Grading via Canonical Pivot Representation

GATE: Gaussian-Attentive Transformer for Uncertainty-Aware Age Estimation

연구실 보유 특허

대학별 입시끈

우리 연구실 모집