🎓 Affiliation: Pattern Recognition and Machine Learning Lab in Korea University (2020~)

I'm researching efficient and effective in-context augmentation for multimodal large language models (MLLMs).

Research direction: face generation (vision) $\rightarrow$ dialogue generation (NLP) $\rightarrow$ neural network debugging (XAI) $\rightarrow$ 🌞 language modeling for multimodalities (NLP)

• [code ✅] "Enhancing Multimodal Query Representation via Visual Dialogues for End-to-End Knowledge Retrieval", arXiv (Under review in AAAI 2025), 2024.

  Authors: Y.-J. Ju, H.-J. Kim, and S.-W. Lee

• [code ✅] "Neuron-based Debugging Framework for Neural Networks Using Class-conditional Visualizations", arXiv, 2023.

  Authors: Y.-J. Ju, J.-H. Park, and S.-W. Lee

  History -> NeurIPS 2023: rejected - 3 borderline accept and 1 reject

• "뉴런 기반 디버깅을 위한 현실적인 이미지 공간에서의 클래스 조건부 특징 시각화(Class-conditional Feature Visualizations in Realistic Image Space for Neuron-based Debugging)", KCC XAI Workshop, 2024 (oral).

  Authors: Y.-J. Ju, J.-H. Park, and S.-W. Lee

  Oral speaker: Yeong-Joon Ju

• [code ✅] "Complete Face Recovery GAN: Unsupervised Joint Face Rotation and De-Occlusion from a Single-View Image", WACV, 2022.

  Authors: Y.-J. Ju, G.-H. Lee, J.-H. Hong, and S.-W. Lee

• "CPR: Mitigating Large Language Model Hallucinations with Curative Prompt Refinement", SMC, 2024.

  Authors: J.-W. Shim, Y.-J. Ju, J.-H. Park, and S.-W. Lee

• "CIRF: Importance of Related Features for Plausible Counterfactual Explanations", Information Sciences, 2024.

  Authors: H.-D. Kim, Y.-J. Ju, J.-H. Hong, and S.-W. Lee

• [code ✅] "Explaining Generative Diffusion Models via Visual Analysis for Interpretable Decision-Making Process", Expert Systems with Applications, 2024.

  Authors: J.-H. Park, Y.-J. Ju, and S.-W. Lee

• "Enhancing Robustness of Over-Parameterized Models via Feature Reweighting Using Logit-Wise Mixup", ICPRAI, 2024 (oral).

  Authors: W.-S. Jo, Y.-J. Ju, and S.-W. Lee

  Oral speaker: Yeong-Joon Ju

• [code ✅] "Compensatory Debiasing for Gender Imbalances in Language Models", ICASSP, 2023 (oral).

  Authors: T.-J. Woo, W-J. Nam, Y.-J. Ju, and S.-W. Lee

  Oral speaker: Yeong-Joon Ju

• Development of Core Technologies for Artificial General Intelligence based on Task-free Generalized Self-Awareness and Self-Learning (태스크에 무관한 자가 인지 및 학습 기반 범용 인공지능 핵심 기술 개발)

  • Institution: Institute of Information & communications Technology Planning & Evaluation (IITP, 정보통신기획평가원), Republic of Korea
  • Duration: Jul. 2024 ~ present
  • Role: Designing long-term research and development goals and content for AGI, and preparing a research and development plan for the project (과제 제안서 작성) / Research of self-improving multimodal agents

• Research on AI-based large-scale language modeling techniques for military-related topic (hidden)

  • Institution: Agency for Defense Development (ADD, 국방과학연구소), Republic of Korea
  • Duration: Aug. 2023 ~ Nov. 2023
  • Role: Team leader (practitioner)

• Development of an AI assistant for elementary education teacher based on a large language model (대규모 언어 모델 기반 초등 교육 교사 어시스턴트 개발)

  • Institution: i-Scream Edu (아이스크림에듀), Republic of Korea
  • Duration: May 2023 ~ Jan. 2024
  • Role: Team leader (practitioner) / Development of LLM-based assistant for primary education teachers and elementary students

• Development of Artificial Intelligence Technology for Personalized Plug-and-Play Explanation and Verification of Explanation (플러그앤플레이 방식으로 설명가능성을 제공하는 인공지능 기술 개발 및 인공지능 시스템에 대한 설명 제공 검증)

  • Institution: Institute of Information & communications Technology Planning & Evaluation (IITP, 정보통신기획평가원), Republic of Korea
  • Duration: Jun. 2022 ~ Dec. 2023
  • Role: Team leader (practitioner) / Research of plug-and-play debugging for the decision-making process of neural networks

• Development of facial skin change analysis and visualization techniques (얼굴 피부 변화 분석 및 시각화 기술 개발)

  • Institution: Samsung Electronics (삼성전자), Republic of Korea
  • Duration: Jun. 2020 ~ Jun. 2021
  • Role: Research of facial dense alignment technique based on optical flow

• Reviewer at IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024

• Grand Prize in The 6th SW·AI Hackerton, Sejong University, Korea (Jul. 2019)

  Project: Android application that manages seat status for discussion lounge / Development of seat status detection model and API based on Flask (Jul. 2019 ~ Jul. 2019)

• Grand Prize in The 7th Creative Design Competition, Sejong University, Korea (Jun. 2019)

  Project: Web application that automatically blurs out license plates and faces / Development of web application based on React.js, detection models, and API. (Mar. 2019 ~ Jun. 2019)