Thunder-Tok: Minimizing Tokens per Word in Tokenizing Korean Texts for Generative Language Models.

Yeonkyoun So

Gyeongje Cho Yeonkyoun So Chanwoo Park Sangmin Lee Sungmok Jung Jaejin Lee Thunder-Tok: Minimizing Tokens per Word in Tokenizing Korean Texts for Generative Language Models. 2025 June abs/2506.15138 CoRR https://doi.org/10.48550/arXiv.2506.15138 db/journals/corr/corr2506.html#abs-2506-15138 streams/journals/corr

Jinpyo Kim Gyeongje Cho Chanwoo Park Jongwon Park Jongmin Kim Yeonkyoun So Jaejin Lee Thunder-LLM: Efficiently Adapting LLMs to Korean with Minimal Resources. 2025 June abs/2506.21595 CoRR https://doi.org/10.48550/arXiv.2506.21595 db/journals/corr/corr2506.html#abs-2506-21595 streams/journals/corr

Gyeongje Cho Sungmok Jung Jinpyo Kim Jongmin Kim Jaejin Lee Sangmin Lee Chanwoo Park Jongwon Park