MinYeong Seo

CheolWoo Lee

RoUn Lee

JaeYoung Seon

MiSong Kim

You need to enable JavaScript to run this app.

Edit-As-Act: Goal-Regressive Planning for Open-Vocabulary 3D Indoor Scene Editing

From Assumptions to Actions: Turning LLM Reasoning into Uncertainty-Aware Planning for Embodied Agents

Event-T2M: Event-level Conditioning for Complex Text-to-Motion Synthesis

ORACLE: Orchestrate NPC Daily Activities using Contrastive Learning with Transformer-CVAE

BiPO: Bidirectional Partial Occlusion Network for Text-to-Motion Synthesis

How Does a Virtual Agent Decide Where to Look? Symbolic Cognitive Reasoning for Embodied Head Rotation

ForceGrip: Reference-Free Curriculum Learning for Realistic Grip Force Control in VR Hand Manipulation

Visual Interfaces to Mitigate Eye Problems in a Virtual Environment via Triggering Eye Blinking and Movement

Shaping the Future of VR Hand Interactions: Lessons Learned from
Modern Methods

REVECA: Adaptive Planning and Trajectory-based Validation in Cooperative Language Agents Using Information Relevance and Relative Proximity

P-Hologen: An End-to-End Generative Framework for Phase-Only Holograms

3DStyleGLIP: Part-Tailored Text-Guided 3D Neural Stylization

DAMO: A Deep Solver for Arbitrary Marker Configuration in Optical Motion Capture

인지 기반 에이전트의 시선에 따른 머리 회전 결정 방법 및 이를 수행하는 시스템

METHOD AND APPARATUS FOR TORQUE CONTROL OF VIRTUAL REALITY HAND MODEL BASED ON CURRICULUM LEARNING WITH PROGRESSIVE DIFFICULTY CONTROL

METHOD AND APPARATUS FOR AGENT CONTROL THROUGH COOPERATIVE PLAN VALIDATION BASED ON INFORMATION RELEVANCE AND RELATIVE PROXIMITY USING LLM IN MULTI-AGENT COOPERATIVE ENVIRONMENT BASED ON VIRTUAL REALITY

가상 에이전트의 일일 활동 계획 생성 시스템 및 이의 동작 방법

Device and method for generating generative model-based phase holograms

A method of reconstructing human pose from unordered sparse point cloud, and a computer program for executing the same

AVATAR-HUMAN MOTION REMAPPING METHOD AND A COMPUTER PROGRAM FOR EXECUTING THE SAME

Apparatus for virtual hand motion generation based on controller and method for thereof

Virtual Reality Service Providing User Device, Method and System

A user study on view-sharing techniques for one-to-many mixed reality collaborations

Multi-resolution terrain rendering with GPU tessellation

Obstacle detection and alert system for smartphone ar users

Visual manipulation for underwater drag force perception in immersive virtual environments

SafeXR: alerting walking persons to obstacles in mobile XR environments

Flotation Simulation in a Cable-driven Virtual Environment--A Study with Parasailing

Jumping further: Forward jumps in a gravity-reduced immersive virtual environment

Terrain rendering with unlimited detail and resolution

Interactive learning in the classroom: A mobile augmented reality assistance application for learning

Reveca: Adaptive planning and trajectory-based validation in cooperative language agents using information relevance and relative proximity

Vr-handnet: A visually and physically plausible hand manipulation system in virtual reality

VRCAT: VR collision alarming technique for user safety

Toward virtual stair walking

Damo: A deep solver for arbitrary marker configuration in optical motion capture

Multi-resolution Perlin Noise Decomposition and Procedural Texture Synthesis by Example

3DStyleGLIP: Part-tailored text-guided 3D neural stylization

Feature-preserving procedural texture

Bipo: Bidirectional partial occlusion network for text-to-motion synthesis

P‐Hologen: An End‐to‐End Generative Framework for Phase‐Only Holograms

Gravity control-based data augmentation technique for improving VR user activity recognition

JungHyun Han

Geonsun Lee

Seung-Hwan Baek

Seokhee Jeon

Waseem Hassan

HyeongYeop Kang

In a one-to-many mixed reality collaboration environment, where multiple local users wearing AR headsets are supervised by a remote expert wearing a VR HMD, we evaluated three view-sharing techniques: 2D video, 360 video, and 3D model augmented with 2D video. Through a pilot test, the weaknesses of the techniques were identified, and additional features were integrated into them. Then, their performances were compared in two different collaboration scenarios based on search and assembling. In the first scenario, a local user performed both search and assembling. In the second scenario, two local users had dedicated roles, one for search and the other for assembling. The experiment results showed that the 3D model augmented with 2D video was time-efficient, usable, less demanding and most preferred in one-to-many mixed reality collaborations.

GPU tessellation is very efficient and is reshaping the terrain-rendering paradigm. We present a novel terrain-rendering algorithm based on GPU tessellation. The planar domain of the terrain is partitioned into a set of tiles, and a coarse-grained quadtree is constructed for each tile using a screen-space error metric. Then, each node of the quadtree is input to the GPU pipeline together with its own tessellation factors. The nodes are tessellated and the vertices of the tessellated mesh are displaced by filtering the displacement maps. The multi-resolution scheme is designed to optimize the use of GPU tessellation. Further, it accepts not only height maps but also geometry images, which displace more vertices toward the higher curvature feature parts of the terrain surface such that the surface detail can be well reconstructed with a small number of vertices. The efficiency of the proposed method is proven through …

This paper presents an obstacle detection and alert system for the pedestrians who use smartphone AR applications. The system analyzes the input camera image to extract feature points and determines whether the feature points come from obstacles ahead in the path. With the obstacle detector, two experiments were made. The first investigated the obstacle alert interfaces, and the second investigated the orientation guide interfaces that instruct users to hold their smartphones with some angles/orientations appropriate to capture the environment. Then, the best interfaces identified from the experiments were integrated and tested to examine their usability and user experiences.

In this paper, we propose to reproduce drag forces in a virtual underwater environment. To this end, we first compute the drag forces to be exerted on human limbs in a physically correct way. Adopting a pseudo-haptic approach that generates visual discrepancies between the real and virtual limb motions, we compute the extent of drag forces that are applied to the virtual limbs and can be naturally perceived. Through two tests, our drag force simulation method is compared with others. The results show that our method is effective in reproducing the sense of being immersed in water. Our study can be utilized for various types of virtual underwater applications such as scuba diving training and aquatic therapy.

The increasing performance of smartphones has made it possible for people to enjoy various mobile XR applications as they walk. Due to the limit of human capability of multitasking, however, it may compromise mobile users’ safety. In this paper, we propose a safety assistance system for walking smartphone users, which utilizes only the smartphone’s built-in sensors. Our solution detects obstacles by analyzing the feature points extracted from the input camera images and then alerts the users to the danger ahead. In typical smartphone game, AR game, and mobile VR game environments, we made two experiments. The first experiment identifies the desirable and preferred distances to alert the obstacles, and the second one identifies what kinds of interfaces are appropriate for alerting walking persons while least distracting them from gaming. The results obtained from the experiments are integrated into a system …

This paper presents flotation simulation in a cable-driven virtual environment. For this, a virtual parasailing system was developed, where the visual stimulus was provided through a VR headset and the physical stimulus was given by wires. In order to prevent the user from moving out of the limited workspace of the cable-driven system, the visual acceleration was washout-filtered to produce the physical acceleration. In the parasailing trajectory, we focused on the stages of vertical acceleration/deceleration and conducted an experiment to identify how much gain can be applied to the visual acceleration, which makes the user feel the natural self-motion when integrated with physical stimulus. Then, the results were tested using several types of full-course virtual parasailing. The results showed that fairly large differences between visual and physical stimuli would be accepted and different gains could be assigned …

In a cable-driven suspension system developed to simulate the reduced gravity of lunar or Martian surfaces, we propose to manipu-late/reduce the physical cues of forward jumps so as to overcome the limited workspace problem. The physical cues should be manipulated in a way that the discrepancy from the visual cues provided through the HMD is not noticeable by users. We identified the extent to which forward jumps can be manipulated naturally. We combined it with visual gains, which can scale visual cues without being noticed by users. The test results obtained in a prototype application show that we can use both trajectory manipulation and visual gains to overcome the spatial limit. We also investigated the user experiences when making significantly high and far jumps. The results will be helpful in designing astronaut-training systems and various VR entertainment content.

The most commonly used model for terrain rendering is height map that stores elevation data on a regular grid. Whereas the distant views of a height map may look natural, the close views usually reveal its polygonal nature. As a solution to this problem, we propose to add a procedural detail and present a noise-by-example method to generate the procedural detail which reflects the geometric characteristics of the height map. It is then enhanced by what we call stamped detail. The procedural and stamped details have inherently an infinite extent. Neatly integrated with GPU tessellation, the detail-added surface can be rendered in unlimited resolutions at real-time. The efficiency of the proposed approach is proven through experiments on real data of the lunar surface.

The rapid evolution of augmented reality (AR) technology has presented new opportunities in the domain of education. Acting as a bridge between the virtual and real worlds, AR technology overcomes the physical limitations of our classrooms at a low cost and provides an interactive learning experience. In this study, we present a smartphone AR application, named the AR‐E‐Helper, which assists the learning of students in higher education lectures. Our goal is to provide an AR‐enhanced learning experience for students. To validate the effectiveness of the AR‐E‐Helper, we conducted an experiment that compares three classes: AR‐enhanced, smartphone‐enhanced, and nontechnology‐enhanced classes. Through the experiment, we observed that our application was helpful in maintaining student's focus in class, promoting their interest, and increasing their satisfaction. Furthermore, we also found how to …

We address the challenge of multi-agent cooperation, where agents achieve a common goal by cooperating with decentralized agents under complex partial observations. Existing cooperative agent systems often struggle with efficiently processing continuously accumulating information, managing globally suboptimal planning due to lack of consideration of collaborators, and addressing false planning caused by environmental changes introduced by other collaborators. To overcome these challenges, we propose the RElevance, Proximity, and Validation-Enhanced Cooperative Language Agent (REVECA), a novel cognitive architecture powered by GPT-4o-mini. REVECA enables efficient memory management, optimal planning, and cost-effective prevention of false planning by leveraging Relevance Estimation, Adaptive Planning, and Trajectory-based Validation. Extensive experimental results demonstrate REVECA's superiority over existing methods across various benchmarks, while a user study reveals its potential for achieving trustworthy human-AI cooperation.

This study aims to allow users to perform dexterous hand manipulation of objects in virtual environments with hand-held VR controllers. To this end, the VR controller is mapped to the virtual hand and the hand motions are dynamically synthesized when the virtual hand approaches an object. At each frame, given the information about the virtual hand, VR controller input, and hand-object spatial relations, the deep neural network determines the desired joint orientations of the virtual hand model in the next frame. The desired orientations are then converted into a set of torques acting on hand joints and applied to a physics simulation to determine the hand pose at the next frame. The deep neural network, named VR-HandNet, is trained with a reinforcement learning-based approach. Therefore, it can produce physically plausible hand motion since the trial-and-error training process can learn how the interaction …

The rapid advancement of virtual reality (VR) head-mounted displays (HMDs) has made it possible to experience immersive VR at home. However, such immersion inevitably disconnects users from reality and puts their safety at risk. We suggest a VR Collision Alarming Technique (VRCAT) to overcome this problem. The fundamental idea is to use an RGB camera to identify physical obstacles around VR users and warn them about potential collisions via the HMD. We built VRCAT on a smartphone platform that is readily available to the general public to reduce learning expenses and boost accessibility to our system. To validate whether the VRCAT improves user safety and is easy to use, we ran an evaluation test and an application test. The evaluation test reveals that VRCAT can be installed by novice users in about a minute. It also demonstrates that VRCAT can estimate the 3D positions of the user and …

This paper presents a motion remapping-based locomotion technique. Our technique can provide a realistic sensation of climbing and descending stairs when users navigate the virtual environment on foot. The main contribution is to provide users a realistic experience of walking up and down virtual stairs while in reality, they are walking on a flat surface. When a user lifts their real foot, our technique controls the position of virtual foot in order to match the timing of real foot touching the floor with that of virtual foot touching the stairs. The avatar’s head and waist are also controlled to mimic the height change movements of stair walking. To achieve this, we collected the actual motion data beforehand and then designed our locomotion technique using the data. Then, we conducted an experiment and an application test. In the experiment, we identified how much visual gain should be applied to foot motion to induce a …

Marker-based optical motion capture (mocap) systems are increasingly utilized for acquiring 3D human motion, offering advantages in capturing the subtle nuances of human movement, style consistency, and ease of obtaining desired motion. Motion data acquisition via mocap typically requires laborious marker labeling and motion reconstruction, recent deep-learning solutions have aimed to automate the process. However, such solutions generally presuppose a fixed marker configuration to reduce learning complexity, thereby limiting flexibility. To overcome the limitation, we introduce DAMO, an end-to-end deep solver, proficiently inferring arbitrary marker configurations and optimizing pose reconstruction. DAMO outperforms state-of-the-art like SOMA and MoCap-Solver in scenarios with significant noise and unknown marker configurations. We expect that DAMO will meet various practical demands such as …

Procedural noise has many advantages such as memory compactness, non-periodicity, etc. However, it is hard to obtain appropriate textures through direct controlling of parameters. Procedural noise by example has been presented as a solution. In this paper, we describe a texture decomposition method, called difference map, for the analysis of given example generated by multi-resolution Perlin noise. We illustrate the relationship between difference map with corresponding Perlin band noise and explain how the appropriate parameters can be estimated. Finally, we validate our method through the experimental results and show the compatibility to real-time interactive systems.

3D stylization, the application of specific styles to three-dimensional objects, offers substantial commercial potential by enabling the creation of uniquely styled 3D objects tailored to diverse scenes. Recent advancements in artificial intelligence and text-driven manipulation methods have made the stylization process increasingly intuitive and automated. While these methods reduce human costs by minimizing reliance on manual labor and expertise, they predominantly focus on holistic stylization, neglecting the application of desired styles to individual components of a 3D object. This limitation restricts the fine-grained controllability. To address this gap, we introduce 3DStyleGLIP, a novel framework specifically designed for text-driven, part-tailored 3D stylization. Given a 3D mesh and a text prompt, 3DStyleGLIP utilizes the vision-language embedding space of the Grounded Language-Image Pre-training (GLIP) model to localize individual parts of the 3D mesh and modify their appearance to match the styles specified in the text prompt. 3DStyleGLIP effectively integrates part localization and stylization guidance within GLIP's shared embedding space through an end-to-end process, enabled by part-level style loss and two complementary learning techniques. This neural methodology meets the user's need for fine-grained style editing and delivers high-quality part-specific stylization results, opening new possibilities for customization and flexibility in 3D content creation. Our code and results are available at https://github.com/sj978/3DStyleGLIP.

This paper presents how to synthesize a texture in a procedural way that preserves the features of the input exemplar. The exemplar is analyzed in both spatial and frequency domains to be decomposed into feature and non-feature parts. Then, the non-feature parts are reproduced as a procedural noise, whereas the features are independently synthesized. They are combined to output a non-repetitive texture that also preserves the exemplar’s features. The proposed method allows the user to control the extent of extracted features and also enables a texture to edited quite effectively.

Text-to-motion generation allows language-driven animation, yet current models struggle to deliver long-range coherence and fine-grained limb coordination. A competitive system must (i) preserve temporal consistency across hundreds of frames,(ii) synchronize limb motions, and (iii) align nuanced sentences with a spectrum of plausible trajectories. We introduce BiPO, the first part-based bidirectional autoregressive network trained with a lightweight Partial Occlusion regulariser. Each limb attends to both past and future frames for anticipatory coordination, while stochastic masking weakens spurious cross-part dependencies and encourages varied solutions. On HumanML3D and KIT-ML, BiPO lowers FID by 15-30% relative to MoMask and BAMM, secures the highest human-perceived realism scores, and sets new state-of-the-art results on motion-editing tasks requiring infill from partial sequences. These findings demonstrate that bidirectional reasoning coupled with Partial Occlusion yields a length-agnostic, high-fidelity framework for expressive, language-conditioned motion synthesis.

Holography stands at the forefront of visual technology, offering immersive, three‐dimensional visualizations through the manipulation of light wave amplitude and phase. Although generative models have been extensively explored in the image domain, their application to holograms remains relatively underexplored due to the inherent complexity of phase learning. Exploiting generative models for holograms offers exciting opportunities for advancing innovation and creativity, such as semantic‐aware hologram generation and editing. Currently, the most viable approach for utilizing generative models in the hologram domain involves integrating an image‐based generative model with an image‐to‐hologram conversion model, which comes at the cost of increased computational complexity and inefficiency. To tackle this problem, we introduce P‐Hologen, the first end‐to‐end generative framework designed for …

The neural-network-based human activity recognition (HAR) technique is being increasingly used for activity recognition in virtual reality (VR) users. The major issue of a such technique is the collection large-scale training datasets which are key for deriving a robust recognition model. However, collecting large-scale data is a costly and time-consuming process. Furthermore, increasing the number of activities to be classified will require a much larger number of training datasets. Since training the model with a sparse dataset can only provide limited features to recognition models, it can cause problems such as overfitting and suboptimal results. In this paper, we present a data augmentation technique named gravity control-based augmentation (GCDA) to alleviate the sparse data problem by generating new training data based on the existing data. The benefits of the symmetrical structure of the data are that it increased the number of data while preserving the properties of the data. The core concept of GCDA is two-fold: (1) decomposing the acceleration data obtained from the inertial measurement unit (IMU) into zero-gravity acceleration and gravitational acceleration, and augmenting them separately, and (2) exploiting gravity as a directional feature and controlling it to augment training datasets. Through the comparative evaluations, we validated that the application of GCDA to training datasets showed a larger improvement in classification accuracy (96.39%) compared to the typical data augmentation methods (92.29%) applied and those that did not apply the augmentation method (85.21%).

IIIXR LAB

교수님 소개

강형엽

연구실 소개

연구실 키워드

알럼나이가 간 회사

연구실 지원 방법

대우 조건

연도별 논문 인용수

저널 & 학회 논문 출판 수

연구실 논문 목록

Edit-As-Act: Goal-Regressive Planning for Open-Vocabulary 3D Indoor Scene Editing

From Assumptions to Actions: Turning LLM Reasoning into Uncertainty-Aware Planning for Embodied Agents

Event-T2M: Event-level Conditioning for Complex Text-to-Motion Synthesis

연구실 보유 특허

대학별 입시끈

우리 연구실 모집

교수님 소개

강형엽

연구실 키워드

알럼나이가 간 회사

연구실 지원 방법

대우 조건

연도별 논문 인용수

저널 & 학회 논문 출판 수

연구실 논문 목록

Edit-As-Act: Goal-Regressive Planning for Open-Vocabulary 3D Indoor Scene Editing

From Assumptions to Actions: Turning LLM Reasoning into Uncertainty-Aware Planning for Embodied Agents

Event-T2M: Event-level Conditioning for Complex Text-to-Motion Synthesis

연구실 보유 특허

최근 3년간 연구실적 높은 연구실

대학별 입시끈

우리 연구실 모집