The Robotics and Intelligence Lab (RILAB) focuses on motion retargeting for humanoids and quadruped robots, physically plausible motion generation of legged robots and neuromuscular models, human-robot interaction with a focus on companion robots, embodied AI utilizing multimodal large language models, and shared autonomy for robotic tasks.

로봇 지능 연구실은 인간형 로봇과 4족 로봇의 모션 리타겟팅, 다족 로봇과 신경근 모델의 물리적으로 실현 가능한 모션 생성, 동반 로봇에 중점을 둔 인간-로봇 상호 작용, 다중 모드 대규모 언어 모델을 활용한 Embodied AI, 로봇 작업을 위한 공유 자율성에 중점을 둔다.

Spatio-Temporal Motion Retargeting for Quadruped Robots

Towards Embedding Dynamic Personas in Interactive Robots: Masquerading Animated Social Kinematics (MASK)

Kinematics-Informed Neural Networks: Enhancing Generalization Performance of Soft Robot Model Identification

CLARA: Classifying and Disambiguating User Commands for Reliable Interactive Robotic Agents

Elucidating robust learning with uncertainty-aware corruption pattern estimation

Conditional motion in-betweening

Cross-Domain Motion Transfer Via Safety-Aware Shared Latent Space Modeling

Robust Learning From Demonstrations With Mixed Qualities Using Leveraged Gaussian Processes

Sparse Markov Decision Processes With Causal Sparse Tsallis Entropy Regularization for Reinforcement Learning

Structured Kernel Subspace Learning for Autonomous Robot Navigation

Interactive Text2Pickup Networks for Natural Language Based Human-Robot Collaboration

Online Learning to Approach a Person with No-Regret

Real-Time Nonparametric Reactive Navigation of Mobile Robots in Dynamic Environments

Distributed Gaussian Process Regression Under Localization Uncertainty

Vision-Based Coordinated Localization for Mobile Sensor Networks

CoRe: A Hybrid Approach of Contact-aware Optimization and Learning for Humanoid Robot Motions

A Unified Framework for Motion Reasoning and Generation in Human Interaction

Robust and Expressive Humanoid Motion Retargeting via Optimization-Based Rig Unification

High DOF Tendon-Driven Soft Hand: A Modular System for Versatile and Dexterous Manipulation

Learning-based Dynamic Robot-to-Human Handover

SOCRATES: Text-based Human Search and Approach using a Robot Dog

Visual Preference Inference: An Image Sequence-Based Preference Reasoning in Tabletop Object Manipulation

Generating Realistic Sound with Prosthetic Hand: A Reinforcement Learning Approach

Towards Natural Prosthetic Hand Gestures: A Common-Rig and Diffusion Inpainting Pipeline

SPOTS: Stable Placement of Objects with Reasoning in Semi-Autonomous Teleoperation Systems

Score-based Generative Modeling through Stochastic Evolution Equation in Hilbert Spaces

Zero-shot active visual search (ZAVIS): Intelligent object search for robotic assistants

FLAME: Free-form Language-based motion synthesis and editing

Towards Defensive Autonomous Driving: Collecting and Probing Driving Demonstrations of Mixed Qualities

Learning-based Motion Stabilizer Leveraging Offline Temporal Optimization

Semi-Autonomous Teleoperation via Learning Non-Prehensile Manipulation Skills

Self-Supervised Motion Retargeting with Safety Guarantee

Realistic and Interactive Robot Gaze

Nonparametric Motion Retargeting for Humanoid Robots on Shared Latent Space

Generalized Tsallis Entropy Reinforcement Learning and Its Application to Soft Mobile Robots

Task Agnostic Robust Learning on Corrupt Outputs by Correlation-Guided Mixture Density Network

Towards a Natural Motion Generator: a Pipeline to Control a Humanoid based on Motion Data

Trajectory-based Probabilistic Policy Gradient for Learning Locomotion Behaviors

Maximum Causal Tsallis Entropy Imitation Learning

Unsupervised Holistic Image Generation from Key Local Patches

Uncertainty-Aware Learning from Demonstration Using Mixture Density Networks with Sampling-Free Variance Modeling

A Nonparametric Motion Flow Model for Human Robot Cooperation

Sparse Markov Decision Processes with Causal Sparse Tsallis Entropy Regularization for Reinforcement Learning

Scalable Robust Learning from Demonstration with Leveraged Deep Neural Networks

A Multi-Agent Coverage Algorithm with Connectivity Maintenance

Self-Correcting Online Navigation via Leveraged Gaussian Processes

Robust Modeling and Prediction in Dynamic Environments Using Recurrent Flow Networks

Gaussian Random Paths for RealTime Motion Planning

Inverse Reinforcement Learning with Leveraged Gaussian Processes

Robust Learning From Demonstration Using Leveraged Gaussian Processes and Sparse Constrained Optimization

Structured Low-Rank Matrix Approximation in Gaussian Process Regression for Autonomous Robot Navigation

Leveraged Non-Stationary Gaussian Process Regression for Autonomous Robot Navigation

Chance-Constrained Target Tracking for Mobile Robots

A Robust Autoregressive Gaussian Process Motion Model Using l 1 -Norm Based Low-Rank Kernel Matrix Approximation

Real-Time Navigation in Crowded Dynamic Environments Using Gaussian Process Motion Control

Distributed Gaussian Process Regression for Mobile Sensor Networks Under Localization Uncertainty

Human Behavior Prediction for Smart Homes Using Deep Learning

Text-to-Motion

Motion Retargeting for Quadrupeds

Uncertainty in LLMs

Persona-based HRI

Combining LLM and Physics Simulator

Visual Preference Inference

Model Identification and Control of a Soft Hand

Hand Motion Inpainting

Active Visual Search

Autonomous Navigation of Mobile Manipulator

RAG-based Code Completion

Sungjoon Choi

Ph.D. degree in Electrical Computer Engineering

Bachelor's degree in Electrical Engineering and Computer Sciences

Associate Professor

Swad: Domain generalization by seeking flat minima

Domain generalization by mutual-information regularization with pre-trained models

Uncertainty-aware learning from demonstration using mixture density networks with sampling-free variance modeling

Sparse markov decision processes with causal sparse tsallis entropy regularization for reinforcement learning

Monte carlo tree search in continuous spaces using voronoi optimistic optimization with regret bounds

Distributional deep reinforcement learning with a mixture of gaussians

Robust learning from demonstrations with mixed qualities using leveraged gaussian processes

Neural markov controlled sde: Stochastic optimization for continuous-time data

Score-based generative modeling through stochastic evolution equations in hilbert spaces

Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards

Robust learning from demonstration using leveraged Gaussian processes and sparse-constrained optimization

Learning to walk a tripod mobile robot using nonlinear soft vibration actuators with entropy adaptive reinforcement learning

Real-time nonparametric reactive navigation of mobile robots in dynamic environments

MixGAIL: Autonomous driving using demonstrations with mixed qualities

Trust region-based safe distributional reinforcement learning for multiple constraints

Maximum causal tsallis entropy imitation learning

Sequential preference ranking for efficient reinforcement learning from human feedback

Inverse reinforcement learning with leveraged Gaussian processes

Gaussian random paths for real-time motion planning

Songhwai Oh

Lim, Sungbin (임성빈, 林聖彬)

Sanghyuk Chun

Sungrae Park

Junbum Cha

Dohyeong Kim

Seunghyun Park

Han Cheol Cho

Yunsung Lee

Eunwoo Kim

Taehyun Cho

Yunho Choi

Seungyub Han

Mineui Hong

Gunmin Lee

Yong-Lae Park

Jae In Kim

Tomas Lozano-Perez

Beomjoon Kim

Kyungjae Lee

Domain generalization (DG) methods aim to achieve generalizability to an unseen target domain by using only training data from the source domains. Although a variety of DG methods have been proposed, a recent study shows that under a fair evaluation protocol, called DomainBed, the simple empirical risk minimization (ERM) approach works comparable to or even outperforms previous methods. Unfortunately, simply solving ERM on a complex, non-convex loss function can easily lead to sub-optimal generalizability by seeking sharp minima. In this paper, we theoretically show that finding flat minima results in a smaller domain generalization gap. We also propose a simple yet effective method, named Stochastic Weight Averaging Densely (SWAD), to find flat minima. SWAD finds flatter minima and suffers less from overfitting than does the vanilla SWA by a dense and overfit-aware stochastic weight sampling strategy. SWAD shows state-of-the-art performances on five DG benchmarks, namely PACS, VLCS, OfficeHome, TerraIncognita, and DomainNet, with consistent and large margins of+ 1.6% averagely on out-of-domain accuracy. We also compare SWAD with conventional generalization methods, such as data augmentation and consistency regularization methods, to verify that the remarkable performance improvements are originated from by seeking flat minima, not from better in-domain generalizability. Last but not least, SWAD is readily adaptable to existing DG methods without modification; the combination of SWAD and an existing DG method further improves DG performances. Source code is available at https://github. com/khanrc …

Domain generalization (DG) aims to learn a generalized model to an unseen target domain using only limited source domains. Previous attempts to DG fail to learn domain-invariant representations only from the source domains due to the significant domain shifts between training and test domains. Instead, we re-formulate the DG objective using mutual information with the oracle model, a model generalized to any possible domain. We derive a tractable variational lower bound via approximating the oracle model by a pre-trained model, called Mutual Information Regularization with Oracle (MIRO). Our extensive experiments show that MIRO significantly improves the out-of-distribution performance. Furthermore, our scaling experiments show that the larger the scale of the pre-trained model, the greater the performance improvement of MIRO. Code is available at https://github.com/kakaobrain/miro.

In this paper, we propose an uncertainty-aware learning from demonstration method by presenting a novel uncertainty estimation method utilizing a mixture density network appropriate for modeling complex and noisy human behaviors. The proposed uncertainty acquisition can be done with a single forward path without Monte Carlo sampling and is suitable for real-time robotics applications. Then, we show that it can be decomposed into explained variance and unexplained variance where the connections between aleatoric and epistemic uncertainties are addressed. The properties of the proposed uncertainty measure are analyzed through three different synthetic examples, absence of data, heavy measurement noise, and composition of functions scenarios. We show that each case can be distinguished using the proposed uncertainty measure and presented an uncertainty-aware learning from demonstration …

In this letter, a sparse Markov decision process (MDP) with novel causal sparse Tsallis entropy regularization is proposed. The proposed policy regularization induces a sparse and multimodal optimal policy distribution of a sparse MDP. The full mathematical analysis of the proposed sparse MDP is provided. We first analyze the optimality condition of a sparse MDP. Then, we propose a sparse value iteration method that solves a sparse MDP and then prove the convergence and optimality of sparse value iteration using the Banach fixed-point theorem. The proposed sparse MDP is compared to soft MDPs that utilize causal entropy regularization. We show that the performance error of a sparse MDP has a constant bound, while the error of a soft MDP increases logarithmically with respect to the number of actions, where this performance error is caused by the introduced regularization term. In experiments, we apply …

Many important applications, including robotics, data-center management, and process control, require planning action sequences in domains with continuous state and action spaces and discontinuous objective functions. Monte Carlo tree search (MCTS) is an effective strategy for planning in discrete action spaces. We provide a novel MCTS algorithm (voot) for deterministic environments with continuous action spaces, which, in turn, is based on a novel black-box function-optimization algorithm (voo) to efficiently sample actions. The voo algorithm uses Voronoi partitioning to guide sampling, and is particularly efficient in high-dimensional spaces. The voot algorithm has an instance of voo at each node in the tree. We provide regret bounds for both algorithms and demonstrate their empirical effectiveness in several high-dimensional problems including two difficult robotics planning problems.

In this paper, we present a new class of entropyregularized Markov decision processes (MDPs), which will be referred to as Tsallis MDPs. that inherently generalize wellknown maximum entropy reinforcement learning (RL) by introducing an additional real-valued parameter called an entropic index. Our theoretical result enables us to derive and analyze different types of optimal policies with interesting properties relate to the stochasticity of the optimal policy by controlling the entropic index. To handle complex and model-free problems, such as learning a controller for a soft mobile robot, we propose a Tsallis actor-critic (TAC) method. We first observe that different RL problems have different desirable entropic indices where using proper entropic index results in superior performance compared to the state-of-the-art actor-critic methods. To mitigate the exhaustive search of the entropic index, we propose a quickand-dirty curriculum method of gradually increasing the entropic index which will be referred to as TAC with Curricula (TAC2). TAC2 shows comparable performance to TAC with the optimal entropic index. Finally, We apply TAC2 to learn a controller of a soft mobile robot where TAC2 outperforms existing actor-critic methods in terms of both convergence speed and utility.

In this paper, we propose a novel distributional reinforcement learning (RL) method which models the distribution of the sum of rewards using a mixture density network. Recently, it has been shown that modeling the randomness of the return distribution leads to better performance in Atari games and control tasks. Despite the success of the prior work, it has limitations which come from the use of a discrete distribution. First, it needs a projection step and softmax parametrization for the distribution, since it minimizes the KL divergence loss. Secondly, its performance depends on discretization hyperparameters such as the number of atoms and bounds of the support which require domain knowledge. We mitigate these problems with the proposed parameterization, a mixture of Gaussians. Furthermore, we propose a new distance metric called the Jensen-Tsallis distance, which allows the computation of the distance …

In this paper, we focus on the problem of learning from demonstration (LfD) where demonstrations with different proficiencies are provided without labeling. To this end, we model multiple policies with different qualities as correlated Gaussian processes and present a leverage optimization method that estimates the leverage of each policy where the difference between two leverages defines the correlation between the corresponding policies. To recover a single policy function of an expert, we present a sparsity constraint on the leverage parameters. We first show that the proposed leverage optimization method can recover the correlations between sensory fields where the fields are realized from correlated Gaussian processes and sensor measurements are collected from the fields. Furthermore, we applied the proposed method to autonomous driving experiments, where demonstrations are collected from three …

We propose a novel probabilistic framework for modeling stochastic dynamics with the rigorous use of stochastic optimal control theory. The proposed model called the neural Markov controlled stochastic differential equation (CSDE) overcomes the fundamental and structural limitations of conventional dynamical models by introducing the following two components: (1) Markov dynamic programming to efficiently train the proposed CSDE and (2) multi-conditional forward-backward losses to provide rich information for accurate inference and to assure theoretical optimality. We demonstrate that our dynamical model efficiently generates a complex time series in the data space without extra networks while showing comparable performance against existing model-based methods on several datasets.

Continuous-time score-based generative models consist of a pair of stochastic differential equations (SDEs)—a forward SDE that smoothly transitions data into a noise space and a reverse SDE that incrementally eliminates noise from a Gaussian prior distribution to generate data distribution samples—are intrinsically connected by the time-reversal theory on diffusion processes. In this paper, we investigate the use of stochastic evolution equations in Hilbert spaces, which expand the applicability of SDEs in two aspects: sample space and evolution operator, so they enable encompassing recent variations of diffusion models, such as generating functional data or replacing drift coefficients with image transformation. To this end, we derive a generalized time-reversal formula to build a bridge between probabilistic diffusion models and stochastic evolution equations and propose a score-based generative model called …

In this paper, we consider stochastic multi-armed bandits (MABs) with heavy-tailed rewards, whose p-th moment is bounded by a constant nu_p for 1

In this paper, we propose a novel method for robust learning from demonstration using leveraged Gaussian process regression. While existing learning from demonstration (LfD) algorithms assume that demonstrations are given from skillful experts, the proposed method alleviates such assumption by allowing demonstrations from casual or novice users. To learn from demonstrations of mixed quality, we present a sparse-constrained leveraged optimization algorithm using proximal linearized minimization. The proposed sparse constrained leverage optimization algorithm is successfully applied to sensory field reconstruction and direct policy learning for planar navigation problems. In experiments, the proposed sparse-constrained method outperforms existing LfD methods.

Soft mobile robots have shown great potential in unstructured and confined environments by taking advantage of their excellent adaptability and high dexterity. However, there are several issues to be addressed, such as actuating speeds and controllability, in soft robots. In this letter, a new vibration actuator is proposed using the nonlinear stiffness characteristic of a hyperelastic material, which creates continuous vibration of the actuator. By integrating three proposed actuators, we also present an advanced soft mobile robot with high degrees of freedom of movement. However, since the dynamic model of the soft mobile robot is generally hard to obtain(intractable), it is difficult to design a controller for the robot. In this regard, we present a method to train a controller, using a novel reinforcement learning (RL) algorithm called adaptive soft actor-critic (ASAC). ASAC gradually reduces a parameter called an entropy …

In this paper, we propose a nonparametric motion controller using Gaussian process regression for autonomous navigation in a dynamic environment. Particularly, we focus on its applicability to low-cost mobile robot platforms with low-performance processors. The proposed motion controller predicts future trajectories of pedestrians using the partially-observable egocentric view of a robot and controls a robot using both observed and predicted trajectories. Furthermore, a hierarchical motion controller is proposed by dividing the controller into multiple sub-controllers using a mixture-of-experts framework to further alleviate the computational cost. We also derive an efficient method to approximate the upper bound of the learning curve of Gaussian process regression, which can be used to determine the required number of training samples for the desired performance. The performance of the proposed method is …

In this paper, we consider autonomous driving of a vehicle using imitation learning. Generative adversarial imitation learning (GAIL) is a widely used algorithm for imitation learning. This algorithm leverages positive demonstrations to imitate the behavior of an expert. In this paper, we propose a novel method, called mixed generative adversarial imitation learning (MixGAIL), which incorporates both of expert demonstrations and negative demonstrations, such as vehicle collisions. To this end, the proposed method utilizes an occupancy measure and a constraint function. The occupancy measure is used to follow expert demonstrations and provides a positive feedback. On the other hand, the constraint function is used for negative demonstrations to assert a negative feedback. Experimental results show that the proposed algorithm converges faster than the other baseline methods. Also, hardware experiments using a …

In safety-critical robotic tasks, potential failures must be reduced, and multiple constraints must be met, such as avoiding collisions, limiting energy consumption, and maintaining balance. Thus, applying safe reinforcement learning (RL) in such robotic tasks requires to handle multiple constraints and use risk-averse constraints rather than risk-neutral constraints. To this end, we propose a trust region-based safe RL algorithm for multiple constraints called a safe distributional actor-critic (SDAC). Our main contributions are as follows: 1) introducing a gradient integration method to manage infeasibility issues in multi-constrained problems, ensuring theoretical convergence, and 2) developing a TD ( ) target distribution to estimate risk-averse constraints with low biases. We evaluate SDAC through extensive experiments involving multi-and single-constrained robotic tasks. While maintaining high scores, SDAC shows 1.93 times fewer steps to satisfy all constraints in multi-constrained tasks and 1.78 times fewer constraint violations in single-constrained tasks compared to safe RL baselines. Code is available at: https://github. com/rllab-snu/Safe-Distributional-Actor-Critic.

In this paper, we propose a novel maximum causal Tsallis entropy (MCTE) framework for imitation learning which can efficiently learn a sparse multi-modal policy distribution from demonstrations. We provide the full mathematical analysis of the proposed framework. First, the optimal solution of an MCTE problem is shown to be a sparsemax distribution, whose supporting set can be adjusted. The proposed method has advantages over a softmax distribution in that it can exclude unnecessary actions by assigning zero probability. Second, we prove that an MCTE problem is equivalent to robust Bayes estimation in the sense of the Brier score. Third, we propose a maximum causal Tsallis entropy imitation learning (MCTEIL) algorithm with a sparse mixture density network (sparse MDN) by modeling mixture weights using a sparsemax distribution. In particular, we show that the causal Tsallis entropy of an MDN encourages exploration and efficient mixture utilization while Boltzmann Gibbs entropy is less effective. We validate the proposed method in two simulation studies and MCTEIL outperforms existing imitation learning methods in terms of average returns and learning multi-modal policies.

Reinforcement learning from human feedback (RLHF) alleviates the problem of designing a task-specific reward function in reinforcement learning by learning it from human preference. However, existing RLHF models are considered inefficient as they produce only a single preference data from each human feedback. To tackle this problem, we propose a novel RLHF framework called SeqRank, that uses sequential preference ranking to enhance the feedback efficiency. Our method samples trajectories in a sequential manner by iteratively selecting a defender from the set of previously chosen trajectories and a challenger from the set of unchosen trajectories , where is the replay buffer. We propose two trajectory comparison methods with different defender sampling strategies:(1) sequential pairwise comparison that selects the most recent trajectory and (2) root pairwise comparison that selects the most preferred trajectory from . We construct a data structure and rank trajectories by preference to augment additional queries. The proposed method results in at least 39.2% higher average feedback efficiency than the baseline and also achieves a balance between feedback efficiency and data dependency. We examine the convergence of the empirical risk and the generalization bound of the reward model with Rademacher complexity. While both trajectory comparison methods outperform conventional pairwise comparison, root pairwise comparison improves the average reward in locomotion tasks and the average success rate in manipulation tasks by 29.0% and 25.0%, respectively. The source code and the videos are provided in …

In this paper, we propose a novel inverse reinforcement learning algorithm with leveraged Gaussian processes that can learn from both positive and negative demonstrations. While most existing inverse reinforcement learning (IRL) methods suffer from the lack of information near low reward regions, the proposed method alleviates this issue by incorporating (negative) demonstrations of what not to do. To mathematically formulate negative demonstrations, we introduce a novel generative model which can generate both positive and negative demonstrations using a parameter, called proficiency. Moreover, since we represent a reward function using a leveraged Gaussian process which can model a nonlinear function, the proposed method can effectively estimate the structure of a nonlinear reward function.

In this paper, we propose Gaussian random paths by defining a probability distribution over continuous paths interpolating a finite set of anchoring points using Gaussian process regression. By utilizing the generative property of Gaussian random paths, a Gaussian random path planner is developed to safely steer a robot to a goal position. The Gaussian random path planner can be used in a number of applications, including local path planning for a mobile robot and trajectory optimization for whole body motion planning. We have conducted an extensive set of simulations and experiments, showing that the proposed planner outperforms look-ahead planners which use a pre-defined subset of egocentric trajectories in terms of collision rates and trajectory lengths. Furthermore, we apply the proposed method to existing trajectory optimization methods as an initialization step and demonstrate that it can help produce …

Robot Intelligence Lab

교수님 소개

최성준

연구실 소개

연구실 키워드

알럼나이가 간 회사

연구실 지원 방법

대우 조건

연도별 논문 인용수

저널 & 학회 논문 출판 수

연구실 논문 목록

Spatio-Temporal Motion Retargeting for Quadruped Robots

Towards Embedding Dynamic Personas in Interactive Robots: Masquerading Animated Social Kinematics (MASK)

CoRe: A Hybrid Approach of Contact-aware Optimization and Learning for Humanoid Robot Motions

연구실 보유 특허

대학별 입시끈

우리 연구실 모집

교수님 소개

최성준

연구실 키워드

알럼나이가 간 회사

연구실 지원 방법

대우 조건

연도별 논문 인용수

저널 & 학회 논문 출판 수

연구실 논문 목록

Spatio-Temporal Motion Retargeting for Quadruped Robots

Towards Embedding Dynamic Personas in Interactive Robots: Masquerading Animated Social Kinematics (MASK)

CoRe: A Hybrid Approach of Contact-aware Optimization and Learning for Humanoid Robot Motions

연구실 보유 특허

최근 3년간 연구실적 높은 연구실

대학별 입시끈

우리 연구실 모집