publications

2024

1. NAACL 2024 Findings

   AbsPyramid: Benchmarking the Abstraction Ability of Language Models with a Unified Entailment Graph

   Zhaowei Wang, Haochen Shi, Weiqi Wang, and 5 more authors

   In Findings of the Association for Computational Linguistics: NAACL 2024, Jun 2024

   Abs

   Cognitive research indicates that abstraction ability is essential in human intelligence, which remains under-explored in language models. In this paper, we present AbsPyramid, a unified entailment graph of 221K textual descriptions of abstraction knowledge. While existing resources only touch nouns or verbs within simplified events or specific domains, AbsPyramid collects abstract knowledge for three components of diverse events to comprehensively evaluate the abstraction ability of language models in the open domain. Experimental results demonstrate that current LLMs face challenges comprehending abstraction knowledge in zero-shot and few-shot settings. By training on our rich abstraction knowledge, we find LLMs can acquire basic abstraction abilities and generalize to unseen events. In the meantime, we empirically show that our benchmark is comprehensive to enhance LLMs across two previous abstraction tasks.

2. ACL 2024

   AbsInstruct: Eliciting Abstraction Ability from LLMs through Explanation Tuning with Plausibility Estimation

   Zhaowei Wang, Wei Fan, Qing Zong, and 7 more authors

   In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Aug 2024

   Abs

   Abstraction ability is crucial in human intelligence, which can also benefit various tasks in NLP study. Existing work shows that LLMs are deficient in abstract ability, and how to improve it remains unexplored. In this work, we design the framework AbsInstruct to enhance LLMs’ abstraction ability through instruction tuning. The framework builds instructions with in-depth explanations to assist LLMs in capturing the underlying rationale of abstraction. Meanwhile, we introduce a plausibility estimator to select instructions that are more consistent with the abstraction knowledge of LLMs to be aligned. Then, our framework combines abstraction instructions with general-purpose ones to build a hybrid dataset. Extensive experiments and analyses demonstrate that our framework can considerably enhance LLMs’ abstraction ability with strong generalization performance while maintaining their general instruction-following abilities.

3. arXiv

   Cross-Architecture Transfer Learning for Linear-Cost Inference Transformers

   Sehyun Choi

   Aug 2024

   Bib

   @misc{choi2024crossarchitecture,
     journal = {arXiv preprint},
     title = {Cross-Architecture Transfer Learning for Linear-Cost Inference Transformers},
     author = {Choi, Sehyun},
     year = {2024},
     eprint = {2404.02684},
     archiveprefix = {arXiv},
     primaryclass = {cs.CL},
   }

2023

1. EMNLP 2023

   KCTS: Knowledge-Constrained Tree Search Decoding with Token-Level Hallucination Detection

   Sehyun Choi, Tianqing Fang, Zhaowei Wang, and 1 more author

   In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Dec 2023

   Abs Bib Code Poster Slides

   Large Language Models (LLMs) have demonstrated remarkable human-level natural language generation capabilities. However, their potential to generate misinformation, often called the *hallucination* problem, poses a significant risk to their deployment. A common approach to address this issue is to retrieve relevant knowledge and fine-tune the LLM with the knowledge in its input. Unfortunately, this method incurs high training costs and may cause catastrophic forgetting for multi-tasking models. To overcome these limitations, we propose a knowledge-constrained decoding method called KCTS (Knowledge-Constrained Tree Search), which guides a frozen LM to generate text aligned with the reference knowledge at each decoding step using a knowledge classifier score and MCTS (Monte-Carlo Tree Search). To adapt the sequence-level knowledge classifier to token-level guidance, we also propose a novel token-level hallucination detection method called RIPA (Reward Inflection Point Approximation). Our empirical results on knowledge-grounded dialogue and abstractive summarization demonstrate the strength of KCTS as a plug-and-play, model-agnostic decoding method that can effectively reduce hallucinations in natural language generation.

   @inproceedings{choi-etal-2023-kcts,
     title = {{KCTS}: Knowledge-Constrained Tree Search Decoding with Token-Level Hallucination Detection},
     author = {Choi, Sehyun and Fang, Tianqing and Wang, Zhaowei and Song, Yangqiu},
     editor = {Bouamor, Houda and Pino, Juan and Bali, Kalika},
     booktitle = {Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing},
     month = dec,
     year = {2023},
     address = {Singapore},
     journal = {EMNLP 2023},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/2023.emnlp-main.867/},
     doi = {10.18653/v1/2023.emnlp-main.867},
     pages = {14035--14053}
   }

2. arXiv

   CKBP v2: An Expert-Annotated Evaluation Set for Commonsense Knowledge Base Population

   Tianqing Fang, Quyet V. Do, Sehyun Choi, and 2 more authors

   arXiv preprint, Apr 2023

   arXiv Bib Code

   @article{fang2023ckbp,
     journal = {arXiv preprint},
     title = {CKBP v2: An Expert-Annotated Evaluation Set for Commonsense Knowledge Base Population},
     author = {Fang, Tianqing and Do, Quyet V. and Choi, Sehyun and Wang, Weiqi and Song, Yangqiu},
     year = {2023},
     month = apr,
     eprint = {2304.10392},
     archiveprefix = {arXiv},
     primaryclass = {cs.CL},
   }

2021

1. Fine-grained Pneumonia Detection from Chest X-Rays Using Complementary Model Training

   Duhyeuk Chang, Sungbin Lee, and Sehyun Choi

   한국정보과학회 학술발표논문집, Jun 2021

   Bib PDF

   @article{chang_lee_choi_2021,
     title = {Fine-grained Pneumonia Detection from Chest X-Rays Using Complementary Model Training},
     journal = {한국정보과학회 학술발표논문집},
     author = {Chang, Duhyeuk and Lee, Sungbin and Choi, Sehyun},
     year = {2021},
     month = jun,
     pages = {1004–1006},
   }

2. EMNLP 2021

   Benchmarking Commonsense Knowledge Base Population with an Effective Evaluation Dataset

   Tianqing Fang, Weiqi Wang, Sehyun Choi, and 4 more authors

   EMNLP 2021, Sep 2021

   Bib PDF Code

   @article{fang_choi_2021,
     title = {Benchmarking Commonsense Knowledge Base Population with an Effective Evaluation Dataset},
     journal = {EMNLP 2021},
     author = {Fang, Tianqing and Wang, Weiqi and Choi, Sehyun and Hao, Shibo and Zhang, Hongming and Song, Yangqiu and He, Bin},
     year = {2021},
     month = sep,
   }