Dilara Derya

Taeho Kang

Hyeongsuk Ryu

Hyeokmook Kang

Yiyu Chen

Uijong Ju

June Kang

Sonyoung Choi

Haemy Lee Masson

Jaehwan Kang

Yoongi Chung

Younbyoung Chae

Dongcheol Hur

Björn Browatzki

Christian Herdtweck

Kathrin Kaulard

Manfred Nusseck

Nina Gaissert

Lisa Dopjans (Whittingstall)

Christoph Dahl

Theresa Cooke

Hyunseung Lee

Junseok Seo

Bora Kim

Haenah Lee

Eunsoo Choi

Ahyoung Shin

Christina Guschelbauer

Rita Grießer

Alexander Brendel

Sandra Moritz

In the Cognitive Systems Lab, we address these questions aiming to enhance the understanding of the algorithms employed by the human cognitive system. For this, we make use of cutting-edge methods from machine learning (AI) and mixed reality coupled with neuroimaging and psychophysics experiments.

인지 시스템 연구실에서는 인간 인지 시스템에서 사용되는 알고리즘에 대한 이해를 향상시키는 것을 목표로 이러한 질문을 해결합니다. 이를 위해 머신 러닝(AI) 및 혼합 현실의 최첨단 방법과 신경 영상 및 정신 물리학 실험을 결합하여 사용합니다.

Seeing What Matters: Attentional (Mis-) Alignment Between Humans and AI in VR-Simulated Prediction of Driving Accidents

Charting decodability of dynamic facial expressions in young and old adults: similarities and differences

Can impersonal touch replace interpersonal touch? An investigation using the rubber hand illusion

Predicting emergency decision-making in conditionally automated driving from personality traits and heart rate

Impact of Intergroup Dialogue on LGBTQ+ and Non-LGBTQ+ Individuals in South Korea

Decoding the dynamic perception of risk and speed using naturalistic stimuli: A multivariate, whole‐brain analysis

Decoding visual fatigue in a visual search task selectively manipulated via myopia-correcting lenses

An EEG Dataset of Neural Signatures in a Competitive Two-Player Game Encouraging Deceptive Behavior

I see artifacts: ICA-based EEG artifact removal does not improve deep network decoding across three BCI tasks

Resting State Network Connectivity Patterns in Early Aging: Late Middle-age Adults Contrasted with Young Adults

Relationship between virtual reality balloon analogue risk task and risky decision-making

Neurorobotics explores the human senses

Dynamic measurements of speed and risk perception during driving: Evidence of speed misestimation from continuous ratings and video analysis

Visuo-haptic processing of unfamiliar shapes: Comparing children and adults

Investigating Dynamics of Subjective Anxiety and Behavior due to Personal Space Violations and COVID-19-related Stressors in a Social VR Simulation

Seeing What Matters: Attentional (Mis-)Alignment Between Humans and AI in VR-Simulated Prediction of Driving Accidents (invited also into special issue of IEEE TVCG)

Predicting Future Driving Decisions in an Accident Situation From Videos: A Combined Behavioral, Eye Gaze, and Computational Analysis

Investigating Subjective Anxiety Dynamics due to Personal Space Violations and COVID-19-related Stressors in a Social VR Simulation

Whom to Save? A Novel, Realistic Paradigm for Studying Human Decision-Making in Moral Dilemmas

Single-View, Video-Based Diagnosis of Parkinson’s Disease via Margin of Stability Gait Analysis

Humans versus machines: Distinguishing Korean, Chinese, and Japanese faces via internal and external features

The importance of external features for categorizing ethnicity: Can Koreans identify Korean, Japanese, and Chinese faces?

Decoding deceipt: EEG signatures of lying behavior under spontaneous versus instructed lying and truth-telling in a two-player game

Gotta Go Fast: Measuring Input/Output Latencies of Virtual Reality 3D Engines for Cognitive Experiments

Testing intuitive decision-making in VR: personality traits predict decisions in an accident situation

To brake or not to brake? Personality traits predict decision-making in an accident situation

You or Me? Personality Traits Predict Sacrificial Decisions in an Accident Situation

Pop or not? EEG-correlates of risk-taking behavior in the balloon analogue risk task

EEG-Based prediction of successful memory formation during vocabulary learning

Recognition with Local Features: the Kernel Recipe

Categorizing art: Comparing humans and computers

Toward Auvers Period: Evolution of van Gogh's Style

Beyond the horizon: perceptual and computational estimates of horizon position

Estimation of the horizon in photographed outdoor scenes by human and machine

We remember what we like?: Aesthetic value and memorability for photos and artworks - a combined behavioral and computational study

Measuring the discernability of virtual objects in conventional and stylized augmented reality

Use of non- photorealistic rendering and photometric stereo in making bas-reliefs from photographs

3fabrec: Fast few-shot face alignment by reconstruction

Label quality in AffectNet: results of crowd-based re-annotation

The contribution of different facial regions to the recognition of conversational expressions

Temporal information for the recognition of conversational expressions

Humans and macaques employ similar face-processing strategies

The thatcher illusion in humans and monkeys

A morphable 3d-model of korean faces

The MPI facial expression database - a validated database of emotional and conversational facial expressions

Cardiff conversation database (CCDB): A database of natural dyadic conversations

Touching on face space: Comparing visual and haptic processing of face shapes

Visual experience is necessary for efficient haptic face recognition

Cannot avert the eyes: Reduced attentional blink toward others’ emotional expressions in empathic people

Facial Expression Processing Is Not Affected by Parkinson’s Disease, But by Age-Related Factors

Predominance of eyes and surface information for face race categorization

Active In-Hand Object Recognition on a Humanoid Robot

Learning to recognize face shapes through serial exploration

Similarity and categorization: From vision to touch

Exploiting object constancy: effects of active exploration and shape morphing on similarity judgments of novel objects

Visual and Haptic Shape Processing in the Human Brain: Unisensory Processing, Multisensory Convergence, and Top-Down Influences

Can touch this: cross-modal shape categorization performance is associated with micro-structural characteristics of white matter association pathways

Putting vision and touch into conflict: results from a multimodal mixed reality setup

Recognition with local features: the kernel recipe

Inserting needles into the body: a meta-analysis of brain activity associated with acupuncture needle stimulation

The MPI facial expression database—a validated database of emotional and conversational facial expressions

Dynamic information for the recognition of conversational expressions

Experimental design: From user studies to psychophysics

Visual and haptic perceptual spaces show high similarity in humans

Multimodal similarity and categorization of novel, three-dimensional objects

Evaluating the perceptual realism of animated facial expressions

Abstract representations of associated emotions in the human brain

Biologically Motivated Computer Vision: Second International Workshop, BMCV 2002, Tübingen, Germany, November 22-24, 2002, Proceedings

Going into depth: Evaluating 2D and 3D cues for object classification on a new, large-scale object dataset

Active object recognition on a humanoid robot

Perception-motivated interpolation of image sequences

The Role of Perception for Computer Graphics.

The components of conversational facial expressions

Visual and haptic shape processing in the human brain: unisensory processing, multisensory convergence, and top-down influences

Heinrich H. Bülthoff

Douglas w. Cunningham

Adrian Schwaninger

In-Seon Lee

Hi-Joon Park

Isabelle Bülthoff

Johannes Schultz

Ye-Seul Lee

Christoph D. Dahl

TOMASO POGGIO

Arnulf B.A. Graf

Nikos Logothetis

Sung-Phil Kim

Mateu Sbert

Miquel Feixas

Christian Wallraven

Recent developments in computer vision have shown that local features can provide efficient representations suitable for robust object recognition. Support vector machines have been established as powerful learning algorithms with good generalization capabilities. We combine these two approaches and propose a general kernel method for recognition with local features. We show that the proposed kernel satisfies the Mercer condition and that it is, suitable for many established local feature frameworks. Large-scale recognition results are presented on three different databases, which demonstrate that SVMs with the proposed kernel perform better than standard matching techniques on local features. In addition, experiments on noisy and occluded images show that local feature representations significantly outperform global approaches.

Acupuncture is a therapeutic treatment that is defined as the insertion of needles into the body at specific points (ie, acupoints). Advances in functional neuroimaging have made it possible to study brain responses to acupuncture; however, previous studies have mainly concentrated on acupoint specificity. We wanted to focus on the functional brain responses that occur because of needle insertion into the body. An activation likelihood estimation meta-analysis was carried out to investigate common characteristics of brain responses to acupuncture needle stimulation compared to tactile stimulation. A total of 28 functional magnetic resonance imaging studies, which consisted of 51 acupuncture and 10 tactile stimulation experiments, were selected for the meta-analysis. Following acupuncture needle stimulation, activation in the sensorimotor cortical network, including the insula, thalamus, anterior cingulate cortex …

The ability to communicate is one of the core aspects of human life. For this, we use not only verbal but also nonverbal signals of remarkable complexity. Among the latter, facial expressions belong to the most important information channels. Despite the large variety of facial expressions we use in daily life, research on facial expressions has so far mostly focused on the emotional aspect. Consequently, most databases of facial expressions available to the research community also include only emotional expressions, neglecting the largely unexplored aspect of conversational expressions. To fill this gap, we present the MPI facial expression database, which contains a large variety of natural emotional and conversational expressions. The database contains 55 different facial expressions performed by 19 German participants. Expressions were elicited with the help of a method-acting protocol, which guarantees both well-defined and natural facial expressions. The method-acting protocol was based on every-day scenarios, which are used to define the necessary context information for each expression. All facial expressions are available in three repetitions, in two intensities, as well as from three different camera angles. A detailed frame annotation is provided, from which a dynamic and a static version of the database have been created. In addition to describing the database in detail, we also present the results of an experiment with two conditions that serve to validate the context scenarios as well as the naturalness and recognizability of the video sequences. Our results provide clear evidence that conversational expressions can be recognized …

The human face is an important and complex communication channel. Humans can, however, easily read in a face not only identity information but also facial expressions with high accuracy. Here, we present the results of four psychophysical experiments in which we systematically manipulated certain facial areas in video sequences of nine conversational expressions to investigate recognition performance and its dependency on the motions of different facial parts. The results help to demonstrate what information is perceptually necessary and sufficient to recognize the different facial expressions. Subsequent analyses of the facial movements and correlation with recognition performance show that, for some expressions, one individual facial region can represent the whole expression. In other cases, the interaction of more than one facial area is needed to clarify the expression. The full set of results is used to develop a systematic description of the roles of different facial parts in the visual perception of conversational facial expressions.

Communication is critical for normal, everyday life. During a conversation, information is conveyed in a number of ways, including through body, head, and facial changes. While much research has examined these latter forms of communication, the majority of it has focused on static representations of a few, supposedly universal expressions. Normal conversations, however, contain a very wide variety of expressions and are rarely, if ever, static. Here, we report several experiments that show that expressions that use head, eye, and internal facial motion are recognized more easily and accurately than static versions of those expressions. Moreover, we demonstrate conclusively that this dynamic advantage is due to information that is only available over time, and that the temporal integration window for this information is at least 100 ms long.

As computers proliferate and as the field of computer graphics matures, it has become increasingly important for computer scientists to understand how users perceive and interpret computer graphics. Experimental Design: From User Studies to Psychophysics is an accessible introduction to psychological experiments and experimental design, covering the major components in the design, execution, and analysis of perceptual studies. The book begins with an introduction to the concepts central to designing and understanding experiments, including developing a research question, setting conditions and controls, and balancing specificity with generality. The book then explores in detail a number of types of experimental tasks: free description, rating scales, forced-choice, specialized multiple choice, and real-world tasks as well as physiological studies. It discusses the advantages and disadvantages of each type and provides examples of that type of experiment from the authors’ own work. The book also covers stimulus-related issues, including popular stimulus resources. It concludes with a thorough examination of statistical techniques for analyzing results, including methods specific to individual tasks.

Primates developed the ability to recognize and individuate their conspecifics by the face. Despite numerous electrophysiological studies in monkeys [1–3], little is known about the face-processing strategies that monkeys employ. In contrast, face perception in humans has been the subject of many studies [4–6] providing evidence for specific face processing that evolves with perceptual expertise [7]. Importantly, humans process faces holistically, here defined as the processing of faces as wholes, rather than as collections of independent features (part-based processing) [8]. The question remains to what extent humans and monkeys share these face-processing mechanisms. By using the same experimental design and stimuli for both monkey and human behavioral experiments, we show that face processing is influenced by the species affiliation of the observed face stimulus (human versus macaque face …

In this study, we show that humans form highly similar perceptual spaces when they explore complex objects from a parametrically defined object space in the visual and haptic domains. For this, a three-dimensional parameter space of well-defined, shell-like objects was generated. Participants either explored two-dimensional pictures or three-dimensional, interactive virtual models of these objects visually, or they explored three-dimensional plastic models haptically. In all cases, the task was to rate the similarity between two objects. Using these similarity ratings and multidimensional scaling (MDS) analyses, the perceptual spaces of the different modalities were then analyzed. Looking at planar configurations within this three-dimensional object space, we found that active visual exploration led to a highly similar perceptual space compared to passive exploration, showing that participants were able to reconstruct the complex parameter space already from two-dimensional pictures alone. Furthermore, we found that visual and haptic perceptual spaces had virtually identical topology compared to that of the physical stimulus space. Surprisingly, the haptic modality even slightly exceeded the visual modality in recovering the topology of the complex object space when the whole three-dimensional space was explored. Our findings point to a close connection between visual and haptic object representations and demonstrate the great degree of fidelity with which haptic shape processing occurs.

Similarity has been proposed as a fundamental principle underlying mental object representations and capable of supporting cognitive-level tasks such as categorization. However, much of the research has considered connections between similarity and categorization for tasks performed using a single perceptual modality. Considering similarity and categorization within a multimodal context opens up a number of important questions: Are the similarities between objects the same when they are perceived using different modalities or using more than one modality at a time? Is similarity still able to explain categorization performance when objects are experienced multimodally? In this study, we addressed these questions by having subjects explore novel, 3D objects which varied parametrically in shape and texture using vision alone, touch alone, or touch and vision together. Subjects then performed a pair-wise …

The human face is capable of producing an astonishing variety of expressions—expressions for which sometimes the smallest difference changes the perceived meaning considerably. Producing realistic-looking facial animations that are able to transmit this degree of complexity continues to be a challenging research topic in computer graphics. One important question that remains to be answered is: When are facial animations good enough? Here we present an integrated framework in which psychophysical experiments are used in a first step to systematically evaluate the perceptual quality of several different computer-generated animations with respect to real-world video sequences. The first experiment provides an evaluation of several animation techniques, exposing specific animation parameters that are important to achieve perceptual fidelity. In a second experiment, we then use these benchmarked …

Emotions can be aroused by various kinds of stimulus modalities. Recent neuroimaging studies indicate that several brain regions represent emotions at an abstract level, i.e., independently from the sensory cues from which they are perceived (e.g., face, body, or voice stimuli). If emotions are indeed represented at such an abstract level, then these abstract representations should also be activated by the memory of an emotional event. We tested this hypothesis by asking human participants to learn associations between emotional stimuli (videos of faces or bodies) and non-emotional stimuli (fractals). After successful learning, fMRI signals were recorded during the presentations of emotional stimuli and emotion-associated fractals. We tested whether emotions could be decoded from fMRI signals evoked by the fractal stimuli using a classifier trained on the responses to the emotional stimuli (and vice versa). This …

It was our great pleasure to organize and host the Second International Works-hop on Biologically Motivated Computer Vision (BMCV 2002), which followed the highly successful BMCV 2000 in Seoul. We welcomed biologists, computer scientists, mathematicians, neuroscientists, physicists, and psychologists to share their views of how the brain solves the ill-posed problems of vision. Nature is the best existence proof that there is a solution of the most fundamental vision problems, and we hope to learn from nature the way to build artificial vision systems which can adapt to different environments and tasks as easily and reliably as we do. We enjoyed a lively discussion of vision topics spanning early vision, mid-level vision, attention, recognition, robotics and cognitive vision systems. Even though the decision to host the workshop in Tübingen came very late (March 2002), and therefore the deadlines were rather tight, we received a total of 97 papers by the end of June. Each of the papers was thoroughly reviewed by at least two members of the program committee and in addition by the local organizing committee. In this context, we especially want to thank the program committee and additional referees for the time and effort that went into the reviews. In the end, 22 papers were accepted for oral presentation and 37 for poster presentation. The selected papers span the whole range of vision from neuronal models of vision to psychophysical investigations of human recognition performance. Correspondingly, the workshop was divided into seven sessions, proceeding (roughly) from topics concerning low-level early vision to high-level cognitive …

Current supervised methods for facial landmark detection require a large amount of training data and may suffer from overfitting to specific datasets due to the massive number of parameters. We introduce a semi-supervised method in which the crucial idea is to first generate implicit face knowledge from the large amounts of unlabeled images of faces available today. In a first, completely unsupervised stage, we train an adversarial autoencoder to reconstruct faces via a low-dimensional face embedding. In a second, supervised stage, we interleave the decoder with transfer layers to retask the generation of color images to the prediction of landmark heatmaps. Our framework (3FabRec) achieves state-of-the-art performance on several common benchmarks and, most importantly, is able to maintain impressive accuracy on extremely small training sets down to as few as 10 images. As the interleaved layers only add a low amount of parameters to the decoder, inference runs at several hundred FPS on a GPU.

Categorization of objects solely based on shape and appearance is still a largely unresolved issue. With the advent of new sensor technologies, such as consumer-level range sensors, new possibilities for shape processing have become available for a range of new application domains. In the first part of this paper, we introduce a novel, large dataset containing 18 categories of objects found in typical household and office environments-we envision this dataset to be useful in many applications ranging from robotics to computer vision. The second part of the paper presents computational experiments on object categorization with classifiers exploiting both two-dimensional and three-dimensional information. We evaluate categorization performance for both modalities in separate and combined representations and demonstrate the advantages of using range data for object and shape processing skills.

Interaction with its environment is a key requisite for a humanoid robot. Especially the ability to recognize and manipulate unknown objects is crucial to successfully work in natural environments. Visual object recognition, however, still remains a challenging problem, as three-dimensional objects often give rise to ambiguous, two-dimensional views. Here, we propose a perception-driven, multisensory exploration and recognition scheme to actively resolve ambiguities that emerge at certain viewpoints. We define an efficient method to acquire two-dimensional views in an object-centered task space and sample characteristic views on a view sphere. Information is accumulated during the recognition process and used to select actions expected to be most beneficial in discriminating similar objects. Besides visual information we take into account proprioceptive information to create more reliable hypotheses. Simulation …

We present a method for image interpolation that is able to create high-quality, perceptually convincing transitions between recorded images. By implementing concepts derived from human vision, the problem of a physically correct image interpolation is relaxed to that of image interpolation which is perceived as visually correct by human observers. We find that it suffices to focus on exact edge correspondences, homogeneous regions and coherent motion to compute convincing results. A user study confirms the visual quality of the proposed image interpolation approach. We show how each aspect of our approach increases perceived quality of the result. We compare the results to other methods and assess achievable quality for different types of scenes.

The categorization of art (paintings, literature) into distinct styles such as Expressionism, or Surrealism has had a profound influence on how art is presented, marketed, analyzed, and historicized. Here, we present results from human and computational experiments with the goal of determining to which degree such categories can be explained by simple, low-level appearance information in the image. Following experimental methods from perceptual psychology on category formation, naive, non-expert participants were first asked to sort printouts of artworks from different art periods into categories. Converting these data into similarity data and running a multi-dimensional scaling (MDS) analysis, we found distinct categories which corresponded sometimes surprisingly well to canonical art periods. The result was cross-validated on two complementary sets of artworks for two different groups of participants showing …

Traditionally, computer graphics strived to achieve the technically best representation of the scenario or scene. For rendering, this lead to the preeminence of representations based on the physics of light interacting with different media and materials. Research in virtual reality has focussed on interactivity and therefore on real-time rendering techniques that improve the immersion of users in the virtual environments. In contrast, visualization has focused on representations that that maximizes the information content. In most cases, such representations are not physically-based, requiring instead more abstract approaches. Recently, the increasing integration of the extensive knowledge and methods from perception research into computer graphics has fundamentally altered both fields, offering not only new research questions, but also new ways of solving existing issues.

Conversing with others is one of the most central of human behaviours. In any conversation, humans use facial motion to help modify what is said, to control the flow of a dialog, or to convey complex intentions without saying a word. Here, we employ a custom, image-based, stereo motion-tracking algorithm to track and selectively "freeze" portions of an actor or actress's face in video recordings in order to determine the necessary and sufficient facial motions for nine conversational expressions. The results show that most expressions rely primarily on a single facial area to convey meaning, with different expressions using different facial areas. The results also show that the combination of rigid head, eye, eyebrow, and mouth motion is sufficient to produce versions of these expressions that are as easy to recognize as the original recordings. Finally, the results show that the manipulation technique introduced few …

Humans are highly adept at multisensory processing of object shape in both vision and touch. Previous studies have mostly focused on where visually perceived object-shape information can be decoded, with haptic shape processing receiving less attention. Here, we investigate visuo-haptic shape processing in the human brain using multivoxel correlation analyses. Importantly, we use tangible, parametrically defined novel objects as stimuli. Twogroups of participants first performed either a visual or haptic similarity-judgment task. The resulting perceptual object-shape spaces were highly similar and matched the physical parameter space. In a subsequent fMRI experiment, objects were first compared within the learned modality and then in the other modality in a one-back task. When correlating neural similarity spaces with perceptual spaces, visually perceived shape was decoded well in the occipital lobe along …

affiliated with the departments of Artificial Intelligence and Brain and Cognitive Engineering

Cognitive Systems Lab

교수님 소개

크리스티안 월레이븐

연구실 소개

연구실 키워드

알럼나이가 간 회사

연구실 지원 방법

대우 조건

연도별 논문 인용수

저널 & 학회 논문 출판 수

연구실 논문 목록

Seeing What Matters: Attentional (Mis-) Alignment Between Humans and AI in VR-Simulated Prediction of Driving Accidents

Charting decodability of dynamic facial expressions in young and old adults: similarities and differences

Can impersonal touch replace interpersonal touch? An investigation using the rubber hand illusion

연구실 보유 특허

대학별 입시끈

우리 연구실 모집

교수님 소개

크리스티안 월레이븐

연구실 키워드

알럼나이가 간 회사

연구실 지원 방법

대우 조건

연도별 논문 인용수

저널 & 학회 논문 출판 수

연구실 논문 목록

Seeing What Matters: Attentional (Mis-) Alignment Between Humans and AI in VR-Simulated Prediction of Driving Accidents

Charting decodability of dynamic facial expressions in young and old adults: similarities and differences

Can impersonal touch replace interpersonal touch? An investigation using the rubber hand illusion

연구실 보유 특허

최근 3년간 연구실적 높은 연구실

대학별 입시끈

우리 연구실 모집