Sehyun Choi

I have a huge interest in Machine Learning (ML) and Artificial Intelligence (AI), and especially Natural Language Processing (NLP) and Multi-Modal AI, working with Language Models (LMs). Other orthogonal fields of interest include Trustworthy AI, such as eXplainable AI (XAI) and robustness. I am also interested in more engineering focused works, such as large-scale LLM pretraining engineering.

Research Questions:

Language Model +

1. Inference-time optimization
2. Knowledge Grounding
3. compositional generalization
4. World Model

In more details:

First topic is something I call “inference-time optimization”. Language Models have come a long way, and they are extremely strong nowadays. However, their behaviors are strongly dependent on the training dataset’s property. It is impossible to design dataset that is “perfect”, so they will always exhibit undesired attributes. I believe this should be addressed with inference-time adaptation method which includes controllable generation or constrained modeling for given constraints / desiderata (objectives).

Second, I am also interested in “Knowledge Grounding”. Hallucination, or confabulation (credit to Dr. Hinton), has become the hot potato in the era of generative AI. I believe reference-grounded generation is an important direction of solving this problem in the safe AI direction.

Third topic is reasoning ability, or more precisely, “Compositional generalization”. Dr. Chomsky states that human language has “infinite uses of finite means”, which suggests that humans compose finite set of functions to create infinite different possibilities. There are evidences that current SOTA AI (arguably LLMs) still may lack such ability.

The final topic is “World Model”, which is related to understanding commonsense about how the world works. I have a few publications about commonsense reasoning and building commonsense knowledge bases (CSKBs).

📄 Resume ←

(Last Update: Nov 16, 2023)

My another Passion: Photography

During my free time, I love doing street photography. You can check out my photos on my Instagram @sync.hpoto (More Anything) & @sync.hcut (More Street). I’m also big into analog film photography; which is exclusively showcased in my personal gallery.