IBM Optical Breakthrough Promises Speedier Data Transfer, More Efficient AI Training
The innovation is "bringing the power of fiber optics onto the chip itself," said Mukesh Khare, general manager of IBM Semiconductors
Modern data centers are largely interconnected by fiber optic cables. Fiber optics are durable and allow for the secure transmission of large amounts of data over long distances. Of course, copper remains prominent in the underpinnings of today’s communications. While it can’t approach the speed and power of fiber optics, copper persists due to its affordability, availability and electrical conductivity.
Even though copper serves its purposes—racks in data centers still predominantly run communications on copper-based electrical wires—IBM researchers looking deeper into the machinery see its limitations. A bottleneck has emerged, a gap centering on computer chips, including the graphical processing units (GPUs) used in AI and machine learning. While the chips themselves grow ever-faster—for instance, IBM’s 2-nanometer node chip contains more than 50 billion transistors and was initially projected to quadruple cellphone battery life—the capabilities for these chips to communicate with one another has lagged.
“The chips still communicate electronically using copper wire, and we all know that our best communications technology is fiber optics. That’s why fiber optics is used everywhere else for long-distance communication,” Mukesh Khare, general manager of IBM Semiconductors and vice president of Hybrid Cloud at IBM Research, said during a December press briefing announcing a new optics breakthrough.
Next-Generation Co-Packaged Optics
IBM Research has been looking into this growing gap, and last month, its Chiplet and Advanced Packaging Team announced its findings: new processes for chip assembly and packaging, known in shorthand as co-packaged optics (CPO). These advances are seen as a breakthrough that will usher in the next generation of CPO technology.
“This innovation,” Khare adds, “is basically bringing the power of fiber optics onto the chip itself.”
Specifically, CPO carries the promise to transform the ways in which high-bandwidth data is transmitted between chips, circuit boards and servers. This advanced version of CPO is spurred by the first successful, publicly announced use of polymer optical waveguide (PWG) technology. PWG could allow for as much as a 6x increase in the number of optical fibers that can be at the edge of a silicon photonic chip, a concept known as “beachfront density,” Khare noted.
Thus, the potential exists for vastly speedier communications—as much as 80X beyond current capabilities—by opening up the bandwidth to allow for chips within data centers to communicate from far greater distances.
As Khare explains, “If you have 10,000 accelerators in a data center, you need to be able to talk to accelerators which are a few meters apart, not just a few inches apart from each other. So now this enables us to go as long as a 100-meter distance with such a high bandwidth.”
Greater Efficiency for AI
Alongside faster communications, these CPO advancements carry the potential to greatly reduce energy consumption. This is particularly significant with the continued growth of AI and the proliferation of large language models (LLMs).
Khare noted that the energy consumed to train gen GPT-4 equates to the energy it would take to provide temperature control for more than 52,000 U.S. homes over an entire century. And that’s just one LLM. Since the emergence of generative AI and LLMs, worldwide demand for computing resources has soared, doubling every six months, Khare noted. Previously, computing demand was doubling every 20 months.
“Now you can see the benefit of CPO,” Khare said. “Now, how does it convert when we train these large models in large data centers with thousands of accelerators, thousands of GPUs? One could actually train a large language model up to five times faster with this new CPO process compared to what is possible today with electronic communications.”
So, when will this technology become commercially available? IBM’s John Knickerbocker, a distinguished engineer and technical assistant to Khare, simply said things are on the right track. He explained that this developmental version of CPO has demonstrated its compatibility with IBM’s existing capabilities, including board attachment and slip chip technology. Strenuous testing and piloting conducted at the IBM Bromont manufacturing site has also yielded positive results.
“We’re very excited to say it’s ready from a technology perspective to be scaled into manufacturing,” Knickerbocker said.