Among the numerous innovations unveiled at Google I/O 2024, the release of the sixth-generation tensor processing unit (TPU) named Trillium stood out. With its high energy efficiency and significantly higher peak compute performance, the new chip intensifies competition in the AI accelerator market.
The technology is experiencing robust growth, spurred by the surge in generative AI applications, and is seen as a transformative technology with long-term implications.
According to Technology Foresights, despite the market being dominated by major players, startups have begun to challenge these incumbents,.
TPUs, specialised hardware designed to expedite the processing of AI and machine learning models, were initially developed by Google for its TensorFlow framework. Now, TPUs encompass a wider category of AI accelerator chips used for running neural networks and other machine learning models in complex operations, from autonomous driving to drug discovery.
GlobalData's Technology Foresights highlights several key insights about TPUs. The innovation landscape for TPUs is becoming increasingly dynamic, with 17 new companies entering the market and filing new patents in the past year.
Notably, innovation is shifting towards efficiently managing AI processes on edge devices, with 24 per cent of all patents mentioning edge computing use cases, followed by autonomous driving and other applications.
GlobalData Practice Head of Innovation Products Sourabh Nyalkalkar said: “The rise in generative AI adoption has led to fierce competition for superior AI accelerator chips. While large players like Google, Nvidia, and Intel dominate the market, there are clear signs of increasing disruption by startups in this space. Additionally, companies like Tesla and Apple are also developing their in-house capabilities, as evident from the innovation leadership map in Technology Foresights."
Meta Platforms, advancing its large language model Llama for generative AI solutions, introduced its own AI accelerator chip last month. Meanwhile, Samsung secured a significant contract worth $750 million (€688.99 million) from Naverto supply edge AI accelerator chips. Both companies are recognised among the leaders in AI accelerator chips by Technology Foresights.
The US currently leads in AI accelerator chip innovation, closely followed by China and South Korea. Among Chinese firms, Huawei is at the forefront with its Ascend series of AI accelerator chips, followed by Alibaba, Baidu, and others.
In recent years, 20 per cent of all patents in AI accelerator chip technology were filed by startups, signalling growing disruption in the market. Hailo, the Israeli startup specialising in AI accelerator chips for edge devices, recently raised $120 million (€110.09 million). Cortica, leading in developing AI processors for autonomous driving applications, and Femtosense, known for creating specialised AI chips for small, energy-efficient electronic devices, stand out among startups for their innovation strength.
Incumbents are responding to the changing market dynamics. OpenAI, the leading generative AI company, is reportedly considering potential acquisition targets to reduce its dependence on Nvidia. Intel and AMD have bolstered their innovation portfolios through the acquisitions of Habana Labs in 2019 and Nod.ai in 2023, respectively.
Nyalkalkar said that with nearly $4 billion (€3.67 billion) raised by specialised AI chip startups between 2021 and 2023, and substantial scale-up efforts by large players to meet growing demands, the growth opportunity for AI accelerator chips appears robust.
“With more than 90 companies developing innovations in this space identified by Technology Foresights, industry stakeholders should closely monitor the landscape for potential mergers and acquisition opportunities in the near future," he said.