Currently, DeepSeek stands as one of the few prominent AI companies in China that operates independently of financial support from tech behemoths like Baidu, Alibaba, or ByteDance.
A Young Collective of Innovators Eager to Make Their Mark
Liang shared that when he assembled the research team at DeepSeek, his goal was not to recruit seasoned engineers to develop a consumer-oriented product. Instead, he sought PhD candidates from China’s leading universities, such as Peking University and Tsinghua University, who were eager to showcase their capabilities. Many of these individuals had been published in prestigious journals and received accolades at international conferences, but still lacked practical experience, according to the Chinese tech outlet QBitAI.
“Our primary technical roles are mainly occupied by individuals who graduated this year or within the last couple of years,” Liang told 36Kr in 2023. This hiring approach has fostered a collaborative environment where team members can freely utilize abundant computing resources to engage in unconventional research endeavors. This contrasts sharply with established internet firms in China, where teams often vie for limited resources. (A recent illustration: ByteDance accused a former intern—an esteemed award winner, no less—of undermining his colleagues’ projects to seize more computing power for his own team.)
Liang believes that students may be better suited for high-risk, low-reward research settings. “When young, many individuals can fully commit themselves to a cause without self-serving motivations,” he elucidated. His message to potential recruits is that DeepSeek was founded to tackle “the most challenging questions facing the world.”
Experts suggest that the nearly exclusive education of these young researchers within China contributes to their motivation. “This younger generation also embodies a sense of nationalism, especially as they navigate US restrictions and challenges in critical hardware and software technologies,” explains Zhang. “Their resolve to transcend these obstacles reflects not only personal ambition but also a broader dedication to elevating China’s status as a global leader in innovation.”
Innovation Arising from Adversity
In October 2022, the US government began instituting export controls that significantly limited Chinese AI firms’ access to advanced chips like Nvidia’s H100. This posed a substantial challenge for DeepSeek. Although the company had initially secured a stockpile of 10,000 H100s, it required additional resources to compete with entities like OpenAI and Meta. “Our challenge has never been funding; it has been the export restrictions on advanced chips,” Liang told 36Kr in a follow-up interview in 2024.
DeepSeek was compelled to devise more efficient methods for training its models. “They fine-tuned their model architecture through a series of engineering strategies—custom communication schemes between chips, minimizing field sizes to conserve memory, and innovative application of the mix-of-models approach,” remarks Wendy Chang, a software engineer who transitioned to a policy analyst role at the Mercator Institute for China Studies. “While many of these tactics are not novel, adeptly merging them to create a state-of-the-art model is an impressive accomplishment.”
DeepSeek has also made notable advancements in Multi-head Latent Attention (MLA) and Mixture-of-Experts, two technical frameworks that enhance the cost-effectiveness of DeepSeek’s models by necessitating fewer computing resources for training. In fact, their latest model is so effective that it utilized just one-tenth of the computational power needed to train Meta’s corresponding Llama 3.1 model, according to the research organization Epoch AI.
DeepSeek’s openness in sharing these innovations with the public has garnered significant goodwill within the global AI research community. For many Chinese AI enterprises, developing open-source models represents a viable strategy to catch up with their Western peers, as it attracts a broader user base and contributors, aiding in the models’ development. “They have successfully shown that cutting-edge models can be constructed with less, albeit still significant, investment and that the prevailing standards of model development leave substantial room for refinement,” Chang expresses. “We are certain to witness more initiatives along these lines in the near future.”
This trend could pose challenges for the existing US export controls aimed at creating bottlenecks in computing resources. “Current estimates of China’s AI computing capacity, and what they can accomplish with it, could be significantly altered,” Chang notes.
