WaferLLM project demonstrates major speed and efficiency gains for large language models, potentially ushering in a new era for real-time AI across sectors.
Researchers at the University of Edinburgh, in collaboration with Europe’s largest cluster of advanced wafer-scale processors at the EPCC National Supercomputing Centre, have announced a major step forward in accelerating artificial intelligence (AI) model performance. The research, led by Dr Luo Mai from the university’s School of Informatics, introduces a new software solution—WaferLLM—designed specifically for novel wafer-scale computer chips that could fundamentally change how large language models (LLMs) operate.
Wafer-scale chips, are substantially larger than conventional GPU chips. Roughly the diameter of a dinner plate, these chips integrate hundreds of thousands of computation cores and vast on-chip memory onto a single silicon substrate. The result is a highly parallel environment where vast amounts of data can be moved and processed internally much faster than the slower inter-chip communications that bottleneck current GPU clusters.
However, this hardware’s full promise remained out of reach until the development of dedicated software. Today’s LLMs typically rely on GPUs and general-purpose software stacks. “Wafer-scale computing has shown remarkable potential, but software has been the key barrier to putting it to work,” explained Dr Luo Mai.
The new WaferLLM system specifically addresses these challenges, orchestrating complex parallel computations and memory transfers that take advantage of the wafer-scale design. In critical benchmark tests at EPCC, WaferLLM delivered up to a tenfold reduction in query latency compared to clusters of 16 of the most advanced GPUs when running popular LLMs like Meta’s LLaMA and Alibaba’s Qwen. This acceleration means LLMs can now respond to queries in real time—measured in milliseconds—opening the door to new applications across science, finance, healthcare, and beyond.
The system also proved up to twice as energy efficient as equivalent GPU solutions when operating at scale—a key consideration as global demand for AI computation grows and data centre energy consumption comes under scrutiny.
According to industry estimates, the AI chip market is primed for rapid expansion, projected to grow from $25 billion in 2024 to over $150 billion by 2030 as demand for high-efficiency AI infrastructure rises. Technologies like WaferLLM—open-sourced for broader adoption—promise to help future-proof AI deployments by slashing operational costs and power requirements, while also removing limitations on model size and complexity that have historically slowed AI progress.
“With WaferLLM, we show that the right software design can unlock that potential, delivering real gains in speed and energy efficiency for large language models. This is a step toward a new generation of AI infrastructure – one that can support real-time intelligence in science, healthcare, education, and everyday life,” said Dr Luo Mai, lead researcher on the project.
