Natasia Wulansari

Yun, Youdong

Yu, Wonhui

Ji, Hyesung

Jo, Jaechoon

Yang, Yeongwook

Choi, Soonyoung

Andrew Matteson

Lee, Dongyub

Park, Jeongbae

Whang, Taesun

Yoon, Yeochan

So, Aram

Jwa, heejung

Han, Seunggyu

Yi, Eunseon

Lee, Seolhwa

Lee, Chanhee

Park, Jeiyoon

Park, Sungjin

Aiyanyo Imatitikua Danielle

Lee, Hyunhee

Hur, Yuna

Kim, Kuekyeng

Kim, Gyeongmin

Oh, Dongsuk

Park, Chanjun

Lee, Jeongwoo

Lim, Jungwoo

Kim, Jeongwook

Lee, Seungjun

Jang, Yoonna

Seo, Jaehyung

Eo, Sugyeong

Moon, Hyeonseok

Kim, Dongjun

Chun, Yongchan

The Natural Language Processing (NLP) Laboratory at Korea University has been selected as one of the top five elite research groups in the 'Independent AI Foundation Model' development project sponsored by the Ministry of Science and ICT and has been awarded the K-AI title.

고려대학교 자연어처리 연구실은 과학기술정보통신부 주관 '독자 AI 파운데이션 모델' 개발 사업의 K-AI 타이틀이 부여된 5대 정예 연구실 중 하나로 선정되었다.

News currently spreads rapidly through the internet. Because fake news stories are designed to attract readers, they tend to spread faster. For most readers, detecting fake news can be challenging and such readers usually end up believing that the fake news story is fact. Because fake news can be socially problematic, a model that automatically detects such fake news is required. In this paper, we focus on data-driven automatic fake news detection methods. We first apply the Bidirectional Encoder Representations from Transformers model (BERT) model to detect fake news by analyzing the relationship between the headline and the body text of news. To further improve performance, additional news data are gathered and used to pre-train this model. We determine that the deep-contextualizing nature of BERT is best suited for this task and improves the 0.14 F-score over older state-of-the-art models.

exbake: Automatic fake news detection model based on bidirectional encoder representations from transformers (bert)

Several studies have reported that adaptivity and personalization in educational computer games facilitate reaching their full educational potential. However, there is little effort to develop adaptive educational computer games for promoting students' computational thinking (CT). In this study, an adaptive computer game is introduced, called AutoThinking, that not only promotes both CT skills and conceptual knowledge, but also provides adaptivity in both game-play and learning processes. To evaluate the possible effects of the game, an experimental study was carried out with 79 students in an elementary school in Estonia. AutoThinking and a conventional technology-enhanced learning approach were used for teaching CT to the experimental and control group, respectively. Our results reveal that AutoThinking improved students’ CT skills and conceptual knowledge better than the conventional approach. It was …

An adaptive educational computer game: Effects on students' knowledge and learning attitude in computational thinking

Educational games have been increasingly used to improve students’ computational thinking. However, most existing games have focused on the theoretical knowledge of computational thinking, ignoring the development of computational thinking skills. Moreover, there is a lack of integration of adaptivity into educational computer games for computational thinking, which is crucial to addressing individual needs in developing computational thinking skills. In this study, we present an adaptive educational computer game, called AutoThinking, for developing students’ computational thinking skills in addition to their conceptual knowledge. To evaluate the effects of the game, we conducted an experimental study with 79 elementary school students in Estonia, where the experimental group learned with AutoThinking, while the control group used a traditional technology-enhanced learning approach. Our findings show …

From gaming to computational thinking: An adaptive educational computer game-based learning approach

Intelligent Tutoring Systems (ITSs) have been producing consistent learning gains for decades. However, a common problem with these systems is maintaining student engagement without reducing the learning benefits. In order to balance the learning benefits of ITSs with the motivational benefits of games, an online formative assessment game called tic-tac-toe quiz for single-player (TRIS-Q-SP) is incorporated in a Flowchart-based Intelligent Tutoring System (FITS), benefiting from Bayesian networks for the process of decision making, for learning computer programming. This assessment game combines tic-tac-toe with online assessment, and revises the rule of tic-tac-toe for stimulating students to use online formative assessment actively. Finally, an empirical investigation carried out to evaluate the performance of FITS indicating that considerable gains for the experimental group over the control group. The …

Applying an online game-based formative assessment in a flowchart-based intelligent tutoring system for improving problem-solving skills

Dementia is the most prevalent degenerative disease in seniors in which progression can be prevented or delayed by early diagnosis. In this study, we proposed a two-layer model inspired by the method used in dementia support centers for the early diagnosis of dementia and using machine learning techniques. Data were collected from patients who received dementia screening from 2008 to 2013 at the Gangbuk-Gu center for dementia in the Republic of Korea. The data consisted of the patient’s gender, age, education, the Mini-Mental State Examination in the Korean version of the CERAD Assessment Packet (MMSE-KC) for dementia screening test, and the Korean version of the Consortium to Establish a Registry for Alzheimer’s Disease (CERAD-K) for the dementia precise test. In the proposed model, MMSE-KC data are initially classified into normal and abnormal. In the second stage, CERAD-K data are used to classify dementia and mild cognitive impairment. The performance of each algorithm is compared with that of Naive Bayes, Bayes Network, Begging, Logistic Regression, Random Forest, Support Vector Machine (SVM) and Multilayer Perceptron (MLP) using Precision, Recall and F-measure. Comparing the F-measure values of normal, mild cognitive impairment (MCI), and dementia, the MLP was the highest in the F-measure values of normal with 0.97, while the SVM appear to be the highest in MCI and dementia with 0.739. Using the proposed early diagnosis model for dementia reduces the time and economic burden and can help simplify the diagnosis method for dementia.

Early diagnosis of dementia from clinical data by machine learning techniques

Modeling and predicting player behavior is of the utmost importance in developing games. Experience has proven that, while theory-driven approaches are able to comprehend and justify a model's choices, such models frequently fail to encompass necessary features because of a lack of insight of the model builders. In contrast, data-driven approaches rely much less on expertise, and thus offer certain potential advantages. Hence, this study conducts a systematic review of the extant research on data-driven approaches to game player modeling. To this end, we have assessed experimental studies of such approaches over a nine-year period, from 2008 to 2016; this survey yielded 46 research studies of significance. We found that these studies pertained to three main areas of focus concerning the uses of data-driven approaches in game player modeling. One research area involved the objectives of data-driven …

Data-driven approaches to game player modeling: a systematic literature review

Though naive Bayes text classifiers are widely used because of its simplicity, the techniques for improving performances of these classifiers have been rarely studied. In this paper, we propose and evaluate some general and effective techniques for improving performance of the naive Bayes text classifier. We suggest document model based parameter estimation and document length normalization to alleviate the problems in the traditional multinomial approach for text classification. In addition, Mutual-Information-weighted naive Bayes text classifier is proposed to increase the effect of highly informative words. Our techniques are evaluated on the Reuters21578 and 20 Newsgroups collections, and significant improvements are obtained over the existing multinomial naive Bayes approach.

Effective methods for improving naive bayes text classifiers

We focus on multi-turn response selection in a retrieval-based dialog system. In this paper, we utilize the powerful pre-trained language model Bi-directional Encoder Representations from Transformer (BERT) for a multi-turn dialog system and propose a highly effective post-training method on domain-specific corpus. Although BERT is easily adopted to various NLP tasks and outperforms previous baselines of each task, it still has limitations if a task corpus is too focused on a certain domain. Post-training on domain-specific corpus (e.g., Ubuntu Corpus) helps the model to train contextualized representations and words that do not appear in general corpus (e.g., English Wikipedia). Experimental results show that our approach achieves new state-of-the-art on two response selection benchmarks (i.e., Ubuntu Corpus V1, Advising Corpus) performance improvement by 5.9% and 6% on R@1.

An effective domain adaptive post-training method for bert in response selection

The success of Transformer architecture has seen increased interest in machine translation (MT). The translation quality of neural network-based MT transcends that of translations derived using statistical methods. This growth in MT research has entailed the development of accurate automatic evaluation metrics that allow us to track the performance of MT. However, automatically evaluating and comparing MT systems is a challenging task. Several studies have shown that traditional metrics (e.g., BLEU, TER) show poor performance in capturing semantic similarity between MT outputs and human reference translations. To date, to improve performance, various evaluation metrics have been proposed using the Transformer architecture. However, a systematic and comprehensive literature review on these metrics is still missing. Therefore, it is necessary to survey the existing automatic evaluation metrics of MT to enable both established and new researchers to quickly understand the trend of MT evaluation over the past few years. In this survey, we present the trend of automatic evaluation metrics. To better understand the developments in the field, we provide the taxonomy of the automatic evaluation metrics. Then, we explain the key contributions and shortcomings of the metrics. In addition, we select the representative metrics from the taxonomy, and conduct experiments to analyze related problems. Finally, we discuss the limitation of the current automatic metric studies through the experimentation and our suggestions for further research to improve the automatic evaluation metrics.

A survey on evaluation metrics for machine translation

Recent years have seen growing interest in open-ended interactive educational tools such as games. One of the most crucial aspects of developing games lies in modeling and predicting individual behavior, the study of computational models of players in games. Although model-based approaches have been considered standard for this purpose, their application is often extremely difficult due to the huge space of actions that can be created by educational games. For this reason, data-driven approaches have shown promise, in part because they are not completely reliant on expert knowledge. This study seeks to systematically review the existing research on the use of data-driven approaches in player modeling of educational games. The primary objectives of this study are to identify, classify, and bring together the relevant approaches. We have carefully surveyed a 10-year sample (2008–2017) of research …

A systematic review of data-driven approaches in player modeling of educational games

This is a systematic review of over one hundred research papers about machine learning methods applied to defensive and offensive cybersecurity. In contrast to previous reviews, which focused on several fragments of research topics in this area, this paper systematically and comprehensively combines domain knowledge into a single review. Ultimately, this paper seeks to provide a base for researchers that wish to delve into the field of machine learning for cybersecurity. Our findings identify the frequently used machine learning methods within supervised, unsupervised, and semi-supervised machine learning, the most useful data sets for evaluating intrusion detection methods within supervised learning, and methods from machine learning that have shown promise in tackling various threats in defensive and offensive cybersecurity.

A systematic review of defensive and offensive cybersecurity with machine learning

This study analyzes the effectiveness of adding educational gaming elements into the online lecture system of the flipped classroom as a method to increase participation and interest in online preparation before class. This study held an “understanding sequence” for 20 classes during the 7‐week automation equipment class with 30 s year high school students in Incheon's specialized technical high school as target. After the class, the learning attitude of learners was measured through surveys and in‐depth interviews. As a result, first, it was found that the degree of preparation participation in flip learning using gaming elements had a statistically significant increase from 65.56% to 78.89%, compared with the traditional flip learning using YouTube. Second, comparing the academic achievements showed a diagnostic assessment of 57.44 before applying the gaming elements and 20.17 after the application …

A comparative study on gamification of the flipped classroom in engineering education to enhance the effects of learning

Countless cyber threat intelligence (CTI) reports are used by companies around the world on a daily basis for security reasons. To secure critical cybersecurity information, analysts and individuals should accordingly analyze information on threats and vulnerabilities. However, analyzing such overwhelming volumes of reports requires considerable time and effort. In this study, we propose a novel approach that automatically extracts core information from CTI reports using a named entity recognition (NER) system. During the process of constructing our proposed NER system, we defined meaningful keywords in the security domain as entities, including malware, domain/URL, IP address, Hash, and Common Vulnerabilities and Exposures . Furthermore, we linked these keywords with the words extracted from the text data of the report. To achieve a higher performance, we utilized the character-level feature vector as …

Automatic extraction of named entities of cyber threats using a deep Bi-LSTM-CRF network

Programming mistakes frequently waste software developers’ time and may lead to the introduction of bugs into their software, causing serious risks for their customers. Using the correlation between various software process metrics and defects, earlier work has traditionally attempted to spot such bug risks. However, this study departs from previous works in examining a more direct method of using psycho-physiological sensors data to detect the difficulty of program comprehension tasks and programmer level of expertise. By conducting a study with 38 expert and novice programmers, we investigated how well an electroencephalography and an eye-tracker can be utilized in predicting programmer expertise (novice/expert) and task difficulty (easy/difficult). Using data from both sensors, we could predict task difficulty and programmer level of expertise with 64.9 and 97.7% precision and 68.6 and 96.4% recall …

Mining biometric data to predict programmer expertise and task difficulty

For programming language comprehension, high cognitive skills (e.g., reading, writing, working memory, etc.) and information processing are required. However, there are few papers that approach this from a neuroscientific perspective. In this paper, we examine program comprehension neuroscientifically and also observe the differences between novice and expert programmers. We designed an EEG (electroencephalogram) experiment and observed 18 participants during a series of program comprehension tasks. We found clear differences in program comprehension ability between novice and expert programmers. Experts exhibited higher brainwave activation than novices in electrodes F3 and P8. These results indicate that experts have outstanding program comprehension-associated abilities such as digit encoding, coarse coding, short-term memory, and subsequent memory effect. Our findings can serve …

Comparing programming language comprehension between novice and expert programmers using eeg analysis

Automated content generation for educational games has become an emerging research problem, as manual authoring is often time consuming and costly. In this article, we present a procedural content generation framework that intends to produce educational game content from the viewpoint of both designer and user. This framework generates content by means of genetic algorithm, and thereby offers designers the ability to control the process of content generation for various learning goals according to their preferences. It further takes into consideration how the content can adapt according to the skill of the users. We demonstrate effectiveness of the framework by way of an empirical study of human players in an educational language learning game aiming at developing early English reading skills of young children. The results of our study confirm that users’ performance measurably improves when game …

A procedural content generation-based framework for educational games: Toward a tailored data-driven game for developing early English reading skills

A significant amount of educational data mining (EDM) research consider students’ past performance or non-academic factors to build predictive models, paying less attention to students’ activity data. While procrastination has been found as a crucial indicator which negatively affects performance of students, no research has investigated this underlying factor in predicting achievement of students in online courses. In this study, we aim to predict students’ course achievement in Moodle through their procrastination behaviour using their homework submission data. We first build feature vectors of students’ procrastination tendencies by considering their active, inactive, and spare time for homework, along with homework grades. Accordingly, we then use clustering and classification methods to optimally sort and put students into various categories of course achievement. We use a Moodle course from the University of …

Predicting course achievement of university students based on their procrastination behaviour on Moodle

The most commonly used algorithm in recommendation systems is collaborative filtering. However, despite its wide use, the prediction accuracy of this algorithm is unexceptional. Furthermore, whether quantitative data such as product rating or purchase history reflect users’ actual taste is questionable. In this article, we propose a method to utilise user review data extracted with opinion mining for product recommendation systems. To evaluate the proposed method, we perform product recommendation test on Amazon product data, with and without the additional opinion mining result on Amazon purchase review data. The performances of these two variants are compared by means of precision, recall, true positive recommendation (TPR) and false positive recommendation (FPR). In this comparison, a large improvement in prediction accuracy was observed when the opinion mining data were taken into account …

Developing a hybrid collaborative filtering recommendation system with opinion mining on purchase review

OBJECTIVE In recent years, several machine learning approaches have been applied to modeling the specificity of the human immunodeficiency virus type 1 (HIV-1) protease cleavage domain. However, the high dimensional domain dataset contains a small number of samples, which could misguide classification modeling and its interpretation. Appropriate feature selection can alleviate the problem by eliminating irrelevant and redundant features, and thus improve prediction performance. METHODS We introduce a new feature subset selection method, FS-MLP, that selects relevant features using multi-layered perceptron (MLP) learning. The method includes MLP learning with a training dataset and then feature subset selection using decompositional approach to analyze the trained MLP. Our method is able to select a subset of relevant features in high dimensional, multi-variate and non-linear domains …

An MLP-based feature subset selection for HIV-1 protease cleavage site analysis

Although game‐based learning has been increasingly promoted in education, there is a need to adapt game content to individual needs for personalized learning. Procedural content generation (PCG) offers a solution for difficulty in developing game contents automatically by algorithmic means as it can generate individually customizable game contents applicable to various objectives. In this paper, we advanced a data‐driven PCG approach benefiting from a genetic algorithm and support vector machines to automatically generate educational‐game contents tailored to individuals' abilities. In contrast to other content generation approaches, the proposed method is not dependent on designer's intuition in applying game contents to fit a player's abilities. We assessed this data‐driven PCG approach at length and showed its effectiveness by conducting an empirical study of children who played an educational …

A data‐driven procedural‐content‐generation approach for educational games

I Know, but I Don’t Know! How Persona Conflict Undermines Instruction Adherence in Large Language Models

Evaluating Over-Empathizing in Emotional Support Conversations: A User-Centered Framework

Improving Semantic Proximity in English-Centric Information Retrieval through Cross-Lingual Alignment

Evidential Transformation Network: Turning Pretrained Models into Evidential Models for Uncertainty Estimation

M3DocDep: Multi-modal, Multi-page, Multi-document Dependency Chunking with Large Vision-Language Models

The Impact of Negated Text on Hallucination with Large Language Models

MultiDocFusion : Hierarchical and Multimodal Chunking Pipeline for Enhanced
RAG
on Long Industrial Documents

Mixture-of-Clustered-Experts: Advancing Expert Specialization and Generalization in Instruction Tuning

TORSO: Template-Oriented Reasoning Towards General Tasks

Benchmark Profiling: Mechanistic Diagnosis of
LLM
Benchmarks

Semantic Inversion, Identical Replies: Revisiting Negation Blindness in Large Language Models

Metric Calculating Benchmark: Code-Verifiable Complicate Instruction Following Benchmark for Large Language Models

KoLEG: On-the-Fly
Korean Legal Knowledge Editing with Continuous Retrieval

StepKE: Stepwise Knowledge Editing for Multi-Hop Question Answering

HAWK: Highlighting Entity-aware Knowledge for Alleviating Information Sparsity in Long Contexts

LimaCost: Data Valuation for Instruction Tuning of Large Language Models

Call for Rigor in Reporting Quality of Instruction Tuning Data

Cross-Lingual Optimization for Language Transfer in Large Language Models

Semantic Aware Linear Transfer by Recycling Pre-trained Language Models for Cross-lingual Transfer

Enhancing Automatic Term Extraction with Large Language Models via Syntactic Retrieval

REVISE: A Framework for Revising
OCRed text in Practical Information Systems with Data Contamination Strategy

K-HALU: Multiple Answer Korean Hallucination Benchmark for Large Language Models

CoME: An Unlearning-based Approach to Conflict-free Model Editing

Efficient Long-form Context and Query Dependent Modeling in
LLMs

Find the Intention of Instruction: Comprehensive Evaluation of Instruction Understanding for Large Language Models

FLEX: A Benchmark for Evaluating Robustness of Fairness in Large Language Models

MIRAGE: A Metric-Intensive Benchmark for Retrieval-Augmented Generation Evaluation

Open
Ko-LLM
Leaderboard2: Bridging Foundational and Practical Evaluation for
Korean
LLMs

Understanding
LLM
Development Through Longitudinal Study: Insights from the Open
Ko-LLM
Leaderboard

CharacterGPT: A Persona Reconstruction Framework for Role-Playing Agents

Dataverse: Open-Source
ETL
(Extract, Transform, Load) Pipeline for Large Language Model

MIGRATE: Cross-Lingual Adaptation of Domain-Specific
LLMs through Code-Switching and Embedding Transfer

Representing the Under-Represented: Cultural and Core Capability Benchmarks for Developing
Thai Large Language Models

sDPO: Don’t Use Your Data All at Once

An analysis on language transfer of pre-trained language model with cross-lingual post-training

Where am
I? Large Language Models Wandering between Semantics and Structures in Long Contexts

PANDA: Persona Attributes Navigation for Detecting and Alleviating Overuse Problem in Large Language Models

Translation of Multifaceted Data without Re-Training of Machine Translation Systems

Search if you don’t know! Knowledge-Augmented
Korean Grammatical Error Correction with Large Language Models

Intelligent Predictive Maintenance
RAG
framework for Power Plants: Enhancing
QA
with
StyleDFS
and Domain Specific Instruction Tuning

SAAS: Solving Ability Amplification Strategy for Enhanced Mathematical Reasoning in Large Language Models

Evalverse: Unified and Accessible Library for Large Language Model Evaluation

Revisiting Under-Represented Knowledge of Latin American Literature in Large Language Models

Open
Ko-LLM
Leaderboard: Evaluating Large Language Models in
Korean with
Ko-H5 Benchmark

KoCommonGEN
v2: A Benchmark for Navigating
Korean Commonsense Reasoning Challenges in Large Language Models

Length-aware Byte Pair Encoding for Mitigating Over-segmentation in
Korean Machine Translation

Towards Precise Localization of Critical Errors in Machine Translation

Explainable
CED: A Dataset for Explainable Critical Error Detection in Machine Translation

Exploring Inherent Biases in
LLMs within
Korean Social Context: A Comparative Analysis of
ChatGPT
and
GPT-4

Leveraging Pre-existing Resources for Data-Efficient Counter-Narrative Generation in
Korean

Generative Interpretation: Toward Human-Like Evaluation for Educational Question-Answer Pair Generation

Hyper-BTS
Dataset: Scalability and Enhanced Analysis of Back
TranScription (BTS) for
ASR
Post-Processing

Ask, Assess, and Refine: Rectifying Factual Consistency and Hallucination in
LLMs with Metric-Guided Feedback Learning

CHEF
in the Language Kitchen: A Generative Data Augmentation Leveraging
Korean Morpheme Ingredients

Explore the Way: Exploring Reasoning Path by Bridging Entities for Effective Cross-Document Relation Extraction

KEBAP:
Korean Error Explainable Benchmark Dataset for
ASR
and Post-processing

Post-hoc Utterance Refining Method by Entity Mining for Faithful Knowledge Grounded Conversations

Beyond Candidates : Adaptive Dialogue Agent Utilizing Persona and Knowledge

CReTIHC: Designing Causal Reasoning Tasks about Temporal Interventions and Hallucinated Confoundings

Alternative Speech: Complementary Method to Counter-Narrative for Better Discourse

Informative Evidence-guided Prompt-based Fine-tuning for
English-Korean Critical Error Detection

Improving Formality-Sensitive Machine Translation using Data-Centric Approaches and Prompt Engineering

Towards Practical Automatic Speech Recognition and Post-Processing: a Call for Explainable Error Benchmark Guideline

Knowledge Graph-Augmented Korean Generative Commonsense Reasoning

Synthetic Alone: Exploring the Dark Side of Synthetic Data for Grammatical Error Correction

Data-Driven Approach for Formality-Sensitive Machine Translation: Language-Specific Handling and Synthetic Data Generation

Towards Diverse and Effective Question-Answer Pair Generation from Children Storybooks

PEEP-Talk: A Situational Dialogue-based Chatbot for
English Education

Adaptive Multi-Domain Dialogue State Tracking on Spoken Conversations

Doubts on the reliability of parallel corpus filtering

PicTalky: Augmentative and Alternative Communication for Language Developmental Disabilities

NLP&AI Lab

교수님 소개

임희석

연구실 소개

연구실 키워드

알럼나이가 간 회사

연구실 지원 방법

대우 조건

연도별 논문 인용수

저널 & 학회 논문 출판 수

연구실 논문 목록

I Know, but I Don’t Know! How Persona Conflict Undermines Instruction Adherence in Large Language Models

The Impact of Negated Text on Hallucination with Large Language Models

MultiDocFusion : Hierarchical and Multimodal Chunking Pipeline for Enhanced RAG on Long Industrial Documents

연구실 보유 특허

대학별 입시끈

우리 연구실 모집

교수님 소개

임희석

연구실 키워드

알럼나이가 간 회사

연구실 지원 방법

대우 조건

연도별 논문 인용수

저널 & 학회 논문 출판 수

연구실 논문 목록

I Know, but I Don’t Know! How Persona Conflict Undermines Instruction Adherence in Large Language Models

The Impact of Negated Text on Hallucination with Large Language Models

MultiDocFusion : Hierarchical and Multimodal Chunking Pipeline for Enhanced RAG on Long Industrial Documents

연구실 보유 특허

최근 3년간 연구실적 높은 연구실

대학별 입시끈

우리 연구실 모집