Happy Sunday, if you’ve been following along with my data center cooling solution and fiber optics series (Pt.1 on the supply chain maze), you know I’m into the hidden, but very profitable, layers that make all this happen. Most chatter about AI fixates on GPUs and CPUs, compute power, and inference vs training, but then the second most important component, after GPU, in AI infrastructure: networking. Crickets.
I don’t think most people grasp how crucial networking tech is to the AI revolution. It’s the glue holding massive data centers together, shuttling exabytes of data at blistering speeds without bottlenecks. And geopolitically it’s just as complex, if not more so, than chips. Trade wars, export bans, and antitrust probes are hitting networking harder than chips in some ways. Yet, for some reason, it’s not getting the spotlight.
In the second part of my series, I want to write about my favorite story to contextualize some of this networking technology: NVIDIA’s acquisition of Mellanox in 2019, partly because it is one of the tech stories that involved Israel, US, and China. It’s also a story of foresight, timing, and tech that quietly supercharged everything but is really, really not talked about enough.
[ Here is an interactive timeline I built for your reference]
In the sun-baked hills of Yokneam, Israel, a modest town north of Tel Aviv near Haifa, a group of engineers gathered in the spring of 1999, fueled by little more than army rations of ingenuity and a shared disdain for sluggish data flows. Eyal Waldman,with a resume dotted by stints at Intel and the chip startup Galileo Technology, decided to change the clogged future for data flow. The dot-com era was exploding, servers were multiplying, but the network connecting them? Archaic. "We needed something mellifluous," Waldman would later recall in interviews, coining the company's name from a poetic nod to smooth, harmonious flow - Mellanox Technologies was born.
The founding team origins trace back to the late 1980s and early 1990s, when Waldman, Michael Kagan, and Shai Cohen first crossed paths at Intel Israel. They set up shop in a nondescript building. Mellanox aimed to invent InfiniBand, a revolutionary interconnect fabric that promised ultra-low latency and high bandwidth, ditching the clunky Ethernet of the time for something purpose-built for high-performance computing (HPC).
At the beginning, Mellanox operated as a "fabless" company - designing chips in-house but farming out the actual manufacturing to heavyweights like TSMC to keep costs down and focus on innovation. They sank their early efforts into creating chips that supported a clever trick called Remote Direct Memory Access, or RDMA: a way for data to jump straight from one system's memory to another's without bothering the central processor. This became the beating heart of InfiniBand, their flagship creation, and it transformed supercomputing by letting machines "talk" faster than ever.
And here's where fiber optics entered the picture. Unlike bulky copper cables that lose signal strength over short distances, Mellanox's early adapters and switches tapped into fiber: thin strands of glass carrying data as pulses of light. They used multi-mode fibers for shorter hops inside data centers and single-mode for longer stretches, hitting speeds up to 25 gigabits per second right out of the gate. This meant sprawling server farms could expand without the usual slowdowns, cutting interference and enabling the kind of massive setups that power today's AI dreams.
By the mid-2000s, after a low-key IPO in 2007 that pegged their value at around $500 million, Mellanox had carved out a stronghold in supercomputing corners. Their fiber-fueled InfiniBand let GPUs hand off data seamlessly, powering everything from scientific simulations to early enterprise clouds. Not content to stay niche, they branched out to Ethernet, the everyday networking standard everyone already used, rolling out ConnectX network cards and SwitchX chips that could juggle both InfiniBand's precision and Ethernet's accessibility. Their LinkX transceivers wove in glass fibers to push Ethernet speeds beyond 100 gigabits, trimming delays in the sprawling clouds run by giants like Microsoft and Google. Before NVIDIA, Mellanox was this scrappy powerhouse, laser-focused on engineering excellence and rock-solid software that won over the big hyperscalers. They zeroed in on high-performance computing tools that turned out to be tailor-made for the AI surge ahead.
Then came NVIDIA. The courtship started in the mid-2010s through partnerships. NVIDIA's GPUs paired beautifully with Mellanox's RDMA over fiber for AI workloads. By 2018, as AI demands surged, NVIDIA saw the gap: GPUs alone weren't enough; they needed networking. The $6.9 billion deal (At the time, Mellanox’s market capitalization was around $5.9B as of the prior Friday’s close), announced in March 2019, was, in my opinion, one of the best acquisition bargains in tech and a standout example of clean integration (despite early doubt by outsiders).
The acquisition drew immediate regulatory scrutiny from watchdogs across the globe. The US Federal Trade Commission, the European Union, and the UK Competition and Markets Authority all poked around, wary of a monopoly in the making. But the real challenges unfolded in Beijing, where China's State Administration for Market Regulation (SAMR) held the deal hostage amid the escalating US-China trade war. Delays stretched for over a year, with whispers of national security concerns.
The tension peaked as the trade war intensified, with US restrictions on chip exports to China ramping up in 2019. Analysts fretted over a veto that could kill the deal outright, but in April 2020, amid the chaos of a global pandemic, China conditionally approved it : NVIDIA had to ensure fair access to Mellanox's tech for Chinese competitors, no discriminatory bundling, and full interoperability.
Post-acquisition (closed April 2020 amid those regulatory hurdles), Mellanox turbocharged NVIDIA's AI expansion. InfiniBand for niche training clusters, Ethernet for broad inference clouds, all amplified by fiber optics in CPO (Co-Packaged Optics which I want to talk about in a bit) for 2025's 800G+ speeds. But the China saga lingered, erupting again in December 2024 with an antitrust probe alleging violations of those very conditions, and as of August 2025, the investigation remains ongoing.
Anyway, the acquisition closed in April 2020, just months before the world went into lockdown and, more crucially, and two years shy of the ChatGPT kicked off the AI explosion. In hindsight, was it luck? Sure, a bit, NVIDIA couldn't have predicted the exact timing of generative AI's breakout. The acquisition, valued at $6.9 billion, at that time seemed like a solid but unspectacular bet on data center plumbing: bolstering NVIDIA's GPUs with Mellanox's fiber-optic-fueled interconnects for smoother, faster data flows. But it was also the killer foresight of Jensen betting big on "accelerated computing" for data centers, and Mellanox plugged a gaping hole in that vision. Without it, scaling AI models across massive GPU clusters would've been much more clunkier. The deal turned a $7 billion bet into a cornerstone of NVIDIA's $3 trillion valuation by 2025. If anything, it underscores how AI's "tidal wave" amplified Mellanox's tech just when the world needed it most.
But again, what made networking, fiber optics and Mellanox so valuable? Mellanox is master of the interconnect game, building the high-speed highways that let data zip between servers, storage, and GPUs without traffic jams. And fiber optics is the express lane in all this. Mellanox's gear leaned heavily on fiber optics for those blistering speeds over longer distances. Sure, every 19 year old now yaps about CUDA as NVIDIA's unbreakable moat, the software ecosystem that locks devs into their GPUs. But RDMA over fiber is the huge enabler for InfiniBand in supercomputing and networking is the hardware moat nobody talks about enough.
Diving deeper on the technical side, Mellanox's networking technologies have three big pillars, each supercharged by fiber optics for AI's demands:
To give some quick context: InfiniBand and Ethernet are both networking technologies used to connect servers, storage, and compute elements in data centers, but they serve different purposes. Ethernet is the standard, versatile protocol you've probably used at home or in offices, it's cost-effective, widely compatible, and great for general-purpose networking like web traffic or cloud services. InfiniBand, on the other hand, is a more specialized protocol optimized for environments needing extremely low latency and high throughput, like scientific simulations or AI model training.
This was Mellanox's original core technology: a proprietary networking fabric designed for ultra-low-latency data transfer in high-performance computing (HPC). A key feature is RDMA (Remote Direct Memory Access), which allows data to move directly between the memories of different systems without involving the CPU, reducing overhead and speeding things up. Fiber optics plays a central role here: InfiniBand switches, such as NVIDIA's Quantum-X series, rely on multi-mode or single-mode fiber cables to achieve speeds of 800Gbps or more over distances exceeding 100 meters, minimizing signal degradation in large setups.
Before the acquisition, Mellanox primarily targeted HPC applications. After 2020, NVIDIA adapted it for AI training scenarios, where consistent, predictable performance is critical in tightly integrated clusters. Early concerns were valid: InfiniBand's structured topology can face issues with AI inference's variable workloads, causing potential congestion in massive cloud environments. NVIDIA has mitigated some of this in 2025 through updates like SHARP (Scalable Hierarchical Aggregation and Reduction Protocol), which has reduced failure rates by about 50%. Overall, InfiniBand remains suited for "niche" high-precision tasks like training, while Ethernet handles more general AI applications.
Mellanox extended their expertise by adapting RDMA to Ethernet through RoCE (RDMA over Converged Ethernet), combining InfiniBand's efficiency with Ethernet's lower cost and broader compatibility. Here, RDMA works the same way: direct data transfers bypassing the CPU, but over standard Ethernet infrastructure. ConnectX network interface cards (NICs) from Mellanox incorporate transceivers that support 100Gbps+ Ethernet connections over fiber, allowing RoCE in cloud setups without needing a complete switch to InfiniBand. This makes it useful for AI's variable traffic patterns, where data bursts are common.
After the acquisition, NVIDIA resolved initial scaling limitations in earlier models like ConnectX-7 (such as handling packet loss in busy networks) with the ConnectX-8 in 2025, improving reliability for inference tasks in large-scale AI deployments. NVIDIA deploys RoCE in hybrid environments, where Ethernet's flexibility reduces dependency on proprietary systems and helps avoid lock-in, making it ideal for cost-focused cloud operations.
These are programmable network cards from Mellanox, with BlueField data processing units (DPUs) handling tasks like networking, storage management, and security separately from the main CPUs or GPUs. This frees up system resources for core AI work. High-speed SerDes (serializer/deserializer) technology integrates fiber optics for ports running at 200Gbps or higher, ensuring fast data handling.
Before the acquisition, Mellanox was a leader in this area; afterward, NVIDIA combined it with their software for better AI integration. In 2025, advancements like CPO (Co-Packaged Optics, which I want to talk about in details later) have cut power use and costs by around 3x, helping with overall scaling in data centers. NVIDIA uses SmartNICs to optimize mixed workloads, offloading routine tasks to keep GPUs focused on compute-heavy AI processes.
That is a lot. Thank you for reading this for. Now how does this complement NVIDIA? A way to think about this is CUDA's the dev moat, but networking's the scale moat. GPUs crunch numbers, but without Mellanox's fiber-fueled RDMA/RoCE, training LLMs across 72-rack clusters (like NVIDIA's GB200 NVL72) would bottleneck. Everyone fixates on CUDA's software lock-in, but networking's hardware glue, amplified by fiber optics, lets NVIDIA sell full-stack AI factories. Mellanox brought the blueprint, turning NVIDIA into the interconnected AI. Key ways this has powered performance:
Scaling AI Clusters: Mellanox's InfiniBand and NVLink provide ultra-low latency interconnects, allowing GPUs to communicate seamlessly. This is essential for AI training, where data must shuttle exabytes without delays. In practice, combining GPUs with Mellanox networking yields 20-25% computation time improvements and up to 50% faster model runs compared to GPUs alone.
Inference Efficiency: For AI inference, RoCE (RDMA over Converged Ethernet) from Mellanox blends speed with cost-effectiveness, reducing power and latency. This has boosted inference performance, with up to 30% cost savings.
Revenue and Growth Metrics: The acquisition, initially $6.9B, has exploded in value. NVIDIA's networking unit (largely Mellanox-derived) hit $4.9B in Q2 FY2026 revenue, up 64% sequentially and 56% YoY, surpassing gaming ($12.9B annually in 2024) and contributing 13% of $39.1B data center revenue. This represents a 5x return on investment, with InfiniBand demand alone growing 5x YoY.
TCO and Efficiency Gains: Mellanox tech lowers total cost of ownership (TCO) by 15-30% in some setups, with up to 30% cost savings over competitors. For every $1 spent on NVIDIA infrastructure (including Mellanox networking), clients see operational efficiencies, with NVLink enabling 130TB/s per rack.
Anyway, what I'm getting at is, if we're all hyping GPUs as the rockstars of AI, networking's hardware glue is just as crucial for scaling those massive clusters. Speaking of which, next up: Let's unpack CPO (Co-Packaged Optics) and LPO (Linear Pluggable Optics), the two forces driving the market and technology now.