01
Video Language Models
Building Pegasus - a foundational video language model.
Machine learning research · South Korea
Sehyun Choi
ML Research Scientist at TwelveLabs
I work on LLM and VLMs (Video!), with a focus on making their capabilities more grounded, reliable, and useful.
Currently
Researching language and multimodal models at TwelveLabs. I’m interested in inference-time methods, grounding, compositional generalization, and world models.
Questions I work on
Research interests
02
Knowledge grounding
Connecting generated answers to evidence and reducing unsupported claims.
03
Training Efficiency
From efficient transfer learning across architecture to distributed GPU training optimization.
A few selected works
Research
- 2025
- 2024
Cross-Architecture Transfer Learning for Linear-Cost Inference Transformers
SelectedarXiv preprint
- 2023
KCTS: Knowledge-Constrained Tree Search Decoding with Token-Level Hallucination Detection
SelectedEMNLP 2023
Abstract
KCTS guides a frozen language model to generate text aligned with reference knowledge using knowledge-classifier scores and Monte Carlo Tree Search. It also introduces RIPA, a token-level hallucination detection method.
- 2021
Places and people
Experience
TwelveLabs
ML Research Scientist · Building Pegasus, a video language model.
Nucleus AI
Research Intern · Foundational language model pretraining.
Naver Papago
Research Intern
Recent updates
News
I joined TwelveLabs as a full-time ML Research Scientist, working on Pegasus, a foundational video language model. TwelveLabs ↗
My work on Cross-Architecture Transfer Learning for Linear-Cost Inference Transformers (XATL) was released as a preprint. Read the paper ↗
My first-author paper on Knowledge-Constrained Decoding was accepted to EMNLP 2023 main conference. KCTS paper ↗
Won second place in the Naver Clova AI Rush 2022 Unknown Document Detection task. AI Rush ↗
Notes from the lab and beyond
Writing
Compositional Generalization
What is compositional generalization problem and why is this interesting?
Big NLP Trends from 2016 to 2022
Recent trends in NLP from Attention to ChatGPT
AINize + Gradio: Easiest Automatic ML model deployment
Serving a simple demo page (along with REST API) for you ML model on the web.
A little more context
Education
Master of Philosophy
The Hong Kong University of Science and Technology
CGA: 4.12 / 4.30
Bachelor's Degree
The Hong Kong University of Science and Technology
GGA: 4.01 / 4.30 · Academic Achievement Medal · First Class Honours · Dean’s List