Some analysts have recently said that the West can either technologically become the follower of China, or slowly decline into the status of a colony, or if it wants to avoid these scenarios, take urgent actions to leapfrog the current status of lethargy and denial. Read this article on how the approach to AI differs between China and the West, and why Europe has to take action now.
Large Language Models vs Large Industrial Models
We have all seen the hype around ChatGPT and got a glimpse of how Large Language Models can revolutionize the life and society. We are using it to ask questions, learn many new things without having to “google” them, and many have recently started developing prompts to apply ChatGPT and other LLMs in the business situations.
But what do Chinese do about LLMs ?
First, we know that we need a lot of data to train Large Language Models. As Chinese internet is closed for data harvesting by big US digital platforms, Chinese companies and universities are here in absolute advantage. Chinese consumers on WeChat, DouYin, TaoBao and other platforms generate huge amounts of data, which is being used by these platforms to develop language and behavioral models. We can add here also surveillance cameras, B2B systems, government and smart city projects. All that is part of the sovereign Chinese system where Chinese companies have exclusive access and rights for use of data.
Development of microprocessors industry. According to these MERICS reports one and two, China has invested significantly in the microprocessors value chain, from design, to wafer fabrication and packaging, and is largely capable of covering its own needs to supply the industry with low- and mid-end microchips. They are not at par with Taiwanese 7nm or smaller chips, but some are sufficient for training LLMs. Also, American companies like NVIDIA and ARM don’t want to lose Chinese market, which provides 20% of their revenues (and less revenue would also mean less budget for R&D), so their advanced GPUs V100 and A100 are also used for different LLM projects in China. The microprocessors are maybe the most critical piece in the whole value chain, and I will explore this situation in one of the future articles.
Next, massive big data needs to be transferred quickly, and we are aware of 5G technologies where China is in absolute dominance in the world both in terms of quantity and quality. According to this article, as of end 2022 there are worldwide around 3 million 5G stations, out of which 2.3 million are installed in China, and this is still expanding. Most of the 5G network systems are produced and installed by Huawei, and this despite the US sanctions and latest EU intentions to ban Huawei equipment. There were the reports on disappointment in 5G in America, but I conclude this can rather be attributed to the inferior performance of Western equipment comparing to Huawei technology, and the lobbying of Cisco and other large WiFi manufacturers. The importance of this communication protocol is mainly for the industrial and mobile use: logistic centers, self-driving cars, smart cities, remote factories with moving machines etc, and different use cases of augmented reality.
Next, China is determined from top down to become global superpower in AI. Xi Jinping in his famous speech during the 19th CPC Congress in 2017 exposed the vision of China in 2035 as “global leader in innovation, whose economic and technological strength will increase significantly“. And after 2035, when China has achieved modernization, it will become become a “global leader in terms of composite national strength and international influence“. This is epitomized in the 14th Five-Year Plan as of January 2022 that specifically refers to digital economy and artificial intelligence as key goals of Chinese strategic development. The scope of use for AI are key industries, agriculture, smart cities, in enterprises and public institutions alike. Xi’s speech during the 20th CPC Congress in 2022 is more balanced and less refers explicitly to technology, but highlights the major past achievements like lunar and Mars exploration, supercomputers, quantum information, nuclear power technology and wide-body airplane manufacturing, and boasts that the nationwide R&D spending has increased from 1 trillion yuan to 2.8 trillion yuan in the past decade: “China is now home to the largest cohort of R&D personnel in the world“.
China has also pioneered AI Governance initiative, by being the first country to issue the law regulating generative AI. The law enforces the adherence to the core values of socialism and leading role of Chinese Communist Party, and prohibits the development of AI that would incite ethnic hatred, discrimination, violence, obscenity, or spread of fake news. The law stipulates that the measures must be taken “during the algorithm design, training data selection, model generation and optimization, and service provision, taking effective measures to prevent discrimination based on ethnicity, belief, country, region, gender, age, occupation, health, etc“. Providers of generative AI must “respect intellectual property rights and business ethics, and not use algorithms, data, platforms, and other advantages to implement monopoly and unfair competition, and must not not infringe on the reputation, personal information and privacy rights of others“. This law has been integrated in already large legal framework covering cybersecurity, personal information protection and protection of critical information infrastructure in China.
The implementation of these policies and ambitions has already reached lowest levels of Chinese governments. For example, Xuhui district in Shanghai is not the prime location for the AI research and development, but it boasts to be the home of several companies developing Large Language Models. The goal of this district is to develop two poles for AI development and deployment, giving subsidies to AI companies from 5 to 15 million CNY. The objective of this local project is to “support the breakthroughs in AI, development of public services platforms for AI, integration of AI in the existing value chains, use of AI in all domains of life, better protection of intellectual property used for AI, and strengthening international collaboration in the domain of AI“. Of course, Shanghai has another cluster dedicated to the development of AI, called AISland, which is actually not the largest pole for the Large Language Model development in China, the most important being Beijing Academy of Artificial Intelligence, which acts also as management agency for the Ethical AI program focused on achieving Sustainable Development Goals.
Having all this in mind, it is no wonder that Reuters in May 2023 reported that China has launched 79 LLMs since 2020 on par with the US, and as per 2023, “China has been in the lead with 19 LLMs to U.S.’ 18“.
But this is not all. What is the most important here is actually what China is doing with these LLMs and in general with AI. As we know, the size of the industrial base of China is something like the whole West taken together. If we study the structure of national GDPs, we will see that the one of China ($18T) consists around 50% in agriculture, industry and mining, and GDPs of developed Western nations like US consist around 70-80% of services. The economies that rely heavily on services will naturally see more benefit and impact through NLP-based AI, but economies that are oriented towards goods production and manufacturing will benefit more from the AI-powered 4th Industrial Revolution, which China is actually focusing on. This is reflected in the PWC report issued already in 2017 about the impact of AI on leading countries’ economies. China is expected to benefit the most, up to 26% of its GDP from AI.
To illustrate the focus of Chinese AI on industrial processes, I will cite here CEO of Huawei Cloud, Zhang Pingan, who in his speech on July 7 said: “Our new Large Industrial Model Pangu 3.0 (盘古) doesn’t write poetry because it is too busy accelerating and automating industrial processes”. The concept is admirable, for AI should be used to enable value creation in sense of automation and innovation. The architecture of Huawei Large Industrial Model-based solution has three layers, where the whole system is modular and configurable:
-
-
- Layer 0 is a set of foundation models for LLM, Vision, Graph-based networks and Multi-modal knowledge representation. Each of these models is available in different flavors to satisfy the requirements for speed, cost and accuracy. The largest one has 207 billions parameters, more than 15% larger than GPT-3.5.
- Next Layer 1 is industrial and consists of additional AI models trained on publicly available data for particular industry like insurance, manufacturing, agriculture, banking, tourism etc. This is where Huawei clients can train their own models based on their own proprietary data.
- Top Layer 3 is use-case specific (examples are shopping assistant, conveyor belt monitoring, weather forecasts), and is mostly based on the customer’s training data set.
-
With this industry-focused AI, travel industry can for example provide intelligent agent services that can answer questions of its customers, help them prepare detailed travel plans, book hotels and transportation, act as call center agent and perform back-office tasks fully automatically.
Conclusions and recommendations
Many believe that AI-powered industrial revolution v4.0 will bring new global winners and losers, and the geopolitical changes of such a scale, which can lead to a new sort of colonialism. Even though we are not seeing many Chinese AI solutions and applications right around us here in Europe, I believe this snapshot of what is happening is China should serve as an eye-opening experience for the professionals and citizens in Switzerland and European Union.
So, I suggest to European (and Swiss) decision makers to consider urgently these policies and solutions being implemented in China. It is completely within reach to develop the ecosystem around AI, based on Large Industrial Models for business use cases to power Industrial Revolution 4.0. This requires to stop wasting time on writing pseudo-poetry, over-hyping generative chatbots, and moronizing the population with entertainment use cases, which is often done by the dominant B2C digital platforms. ChatGPT demonstrated the capabilities of generative AI, but we should not stop short of AI-powering the economy. These Large Industrial Models can help re-industrialize Europe and contribute more strongly in sustainable development. And what is needed here and now is the serious intention of policy-makers and the right set of strategies to enable and grow new generation of European AI players.
July 2023
Sasha Lazarevic