According to a report released in the energy publication Joule next year, Google’s electricity consumption could potentially surge by approximately 29 billion kilowatt-hours annually if it opted to integrate conceptual AI into every search, a figure roughly equivalent to Iceland’s total annual population consumption.
The looming challenge accompanying AI adoption is succinctly encapsulated in this statistic: the substantial energy demand for training and maintaining these models is only set to escalate.
However, companies are beginning to recognize the opportunity to tackle AI’s energy dilemma. Oriole Networks, a spinout from University College London (UCL), recently secured a £10 million seed funding round from investors including XTX Ventures, the venture arm of algorithmic trading firm XTX, Clean Growth Fund, Dorilton Ventures, and UCL Technology Fund.
Revolutionizing System Infrastructure
The training of AI models heavily relies on graphics processing units (GPUs) such as those produced by Nvidia. Large language models (LLMs), the underlying technology powering chatbots like ChatGPT, necessitate the use of tools like large language models (LLMs) for training. This process involves aggregating thousands of GPUs together, with XTX boasting a microprocessor comprising 20,000 GPUs.
Traditionally, GPUs are interconnected using Ethernet cables, a more efficient alternative to WiFi connections.
While GPUs have significantly evolved over the past decade, the infrastructure linking them has not kept pace. James Regan, CEO of Oriole—established just last month—notes that this infrastructure has become the bottleneck.
Wireless networks have the potential to reduce the computational burden on a cluster when handling a substantial data load. However, they are also energy-intensive, constituting about 20% of the total energy consumption of an AI cluster (with the remainder stemming from actual computing processes).
Oriole’s Innovative Solution
Oriole’s groundbreaking solution stems from the research of Professor George Zervas at UCL, a pioneer in computer network enhancement for over two decades. Zervas, also the CTO of Oriole, has devised a method for GPU interconnection using optical fibers and light beams. Regan asserts that this technology could potentially boost data transfer rates between GPUs by up to 100 times.
Regan highlights that optical networks consume significantly less energy compared to conventional wireless setups. Oriole claims that at the laboratory level, their technology has slashed energy consumption to a mere 2-3% of a standard system.
Several companies, including Google with its Project Apollo, are exploring optical networking solutions. Regan lauds Google’s efforts while underscoring that Oriole’s technology might offer a faster and more cost-effective alternative by leveraging optical systems to replace traditional networking components.
Commercialization Strategy
Oriole has licensed Zervas’ pioneering technology from UCL, covering the physical design of the optical system and the underlying system architecture. The company aims to collaborate with existing network infrastructure manufacturers for hardware production. Rather than selling complete supercomputers, Oriole plans to market its networking system, allowing customers to integrate their computing resources with Oriole’s technology in lieu of Ethernet cables.
Upon completing product development, Oriole will transition to offering its software to clients. Regan anticipates this transition may take a few years, emphasizing the need for thorough testing and refinement beyond the laboratory environment.
