SerpApi.Org
China’s researchers have built an optical interconnect system that dramatically speeds up distributed AI inference while using a fraction of the computing resources required by conventional GPU-based systems. The prototype achieved more than a 100-fold increase in inference speed while relying on only about one-ninth of the compute power of a commercial GPU.
The system, developed by researchers at Peking University, connects multiple computing chips through an on-chip all-optical network instead of conventional electrical links. The approach is designed to reduce delays and improve data movement between chips, one of the growing bottlenecks in scaling AI workloads.
At the center of the platform is a 400 Gbps silicon photonic transceiver that converts electrical signals into optical signals and back again. It works alongside a custom 16×16 optical switch chip that routes data between computing nodes, creating a scalable communication network with an aggregate switching bandwidth of up to 6.4 Tbps.
The researchers say the design shifts the focus from simply adding more computing hardware to improving how chips communicate, allowing multiple processors to work together more efficiently during AI inference.
Light replaces bottlenecks
A key feature of the optical switch is its total loss of less than 5 dB, including coupling loss. According to the team, this allows high-speed, error-free transmission without requiring external optical gain compensation. The switch also maintains error-free performance across multiple communication paths and supports a spectral response exceeding 100 nm, making it suitable for future bandwidth expansion through wavelength-division multiplexing.
To demonstrate the architecture, the researchers deployed a five-layer convolutional neural network for image denoising. Each layer was assigned to a separate computing unit, while the optical switch connected the processors into a pipeline.
Instead of repeatedly storing intermediate data in memory before sending it to the next processor, the system transmitted feature maps directly through the optical network. This reduced delays associated with memory transfers and kept the computing units working continuously.
Compared with a commercial GPU running the same image-denoising task, the optical system delivered more than a hundred times faster inference while using only about one-ninth of the computational resources.
Scaling AI differently
The researchers believe the work highlights a different way to improve AI performance as models continue to grow.
“Specific objectives can be realised under limited computational resources when algorithms, processor micro-architectures and chip-level interconnections are co-designed,” the authors wrote.
“This fabric can also alleviate unsustainable energy usage in data centres and optimise latency or consumption in edge-computing scenarios,” they added.
The team says advances in co-packaged optics, silicon photonic transceivers and faster AI chip interfaces could help turn on-chip optical supernodes into a practical foundation for future distributed computing systems. Such systems could provide the high bandwidth, low latency, and energy efficiency needed to support next-generation AI workloads without relying solely on larger clusters of increasingly power-hungry processors.
The study was published in the journal National Science Review.
