S.Korean LLM war intensifies to fend off ChatGPT

South Korean technology companies, both big and small, are rushing to advance Korean large language models (LLM) to fend off the pioneering chatbot service OpenAI’s ChatGPT, demand for which has grown rapidly at home and abroad.

Korean startup Upstage on Monday said that it has launched “1T Club” to build a Korean LLM ecosystem where a massive volume of Korean language data can be collected and shared to advance the country’s AI sector.

The startup is in talks with about 20 news media, private companies and academia to form partnerships to amass more than 40 million Korean words on which an LLM can train to process and generate human language, the company added.

Upstage's new AI model scored 72.3 on the latest HuggingFace Open LLM Leaderboard, a standard for open-source LLM evaluation of over 500 LLMs. earlier this month.

Its move to speed up its Korean LLM advancement comes as demand for OpenAI’s ChatGPT-4 is rapidly growing in Korea since its rollout in March thanks to its excellent Korean language ability, which is better than the predecessor ChatGPT-3.5’s English ability.

The growing popularity of ChatGPT in Korea has led another global tech behemoth Google also to focus on the advancement of its chatbot Bard’s Korean ability.

LLM, A CORE AI TECHNOLOGY IN CHATBOTS 

An LLM is a deep-learning algorithm that can mimic human intelligence using a very large language dataset.

This is a core AI technology, which is at the center of generative AIs like ChatGPT or Bard, and it is a must to build competitive local LLMs to take the lead in the AI market.

As a majority of Korean generative AIs use OpenAI’s ChatGPT services, concerns are also growing that foreign AI technology would rule the country’s AI market.

To prevent it, major Korean technology companies are also accelerating the upgrade of their own Korean AI LLMs.

Naver Corp. is currently developing HyperCLOVA X, an upgraded version of its previous LLM HyperCLOVA.

The newer LLM is known to have trained on Korean language data from local news stories published over the past 50 years and blog posts on Naver for nine years.

The Korean internet giant is due to unveil HyperCLOVA X and its generative AI service using the upgraded LLM on Aug. 24.

Its archrival Kakao Corp. also plans to launch KoGPT, its own Korean LLM with more advanced Korean language ability, in the second half of this year. 

The Korean telecom giant KT Corp. is also developing its own AI language model MI:DEUM.

Write to Joo-Wan Kim at kjwan@hankyung.com

Sookyung Seo edited this article.