By JIM THEODORAS
Still in its infancy, 100 Gigabit Ethernet (100GbE) already appears to have entered a midlife crisis. Critics have been vocal about 100GbE's shortcomings: It's too expensive… The modules are too big… There's no serial client port… The components are backordered until eternity.…
Industry has responded quickly to address these criticisms. Let's look closer at the milestone technology, its strengths and weaknesses, and the 100GbE tune-up already underway.
Shortcomings
No sooner had 100GbE networking arrived to the relief of carriers seeking a leap forward in the scalability of their infrastructures than the nascent technology's detractors began to complain about:
- Price: The rule of thumb is that each new bandwidth milestone (typically a 10-fold advancement) should cost 2.5X the previous. With 100GbE stag-nating at 6X to 12X the cost of a 10GbE port, customer dissatisfaction around price is considerable.
- Size: Vendors rallied around the CFP module form factor relatively early on in the lifecycle of 100GbE, which enabled everyone to quickly commence work. However, given that CFP was based upon existing technologies for demonstration purposes, the modules turned out quite big. To customers accustomed to svelte 10G SFP+ ports, the idea that a client port would be cinderblock-sized seemed beyond absurd.
- Lack of a serial client port: Given 40/100GbE was the first Ethernet protocol designed simultaneously for both telecom and datacom applications and encompassed no fewer than 10 PHY definitions, it came as a shock that something might be missing in the coverage. Sure enough, while IEEE defined 100GbE PHYs from 10 m to 40 km and the Optical Internetworking Forum (OIF) extends to 2000 km, the humble serial singlemode fiber client port appears to have been somehow forgotten. The bread and butter of Ethernet connections is the simple patch cable. IEEE 802.3ba offers 100-m multimode and 10-km singlemode options, leaving a sizable gap in coverage. The 100-m multimode fiber PHY is the wrong fiber type without enough link budget for most inner-office uses; the 10-km singlemode PHY is overkill.
- Component shortages: Despite public statements otherwise, 100GbE is not really shipping–at least not of a scale comparable to 10GbE. 100GbE is real and works well as intended. And, despite price challenges, willing buyers are plentiful. But component shortages continue to thwart the supply chain. 100GbE is possible only through a clever combination of esoteric technologies, many struggling to reach volume production.
Typically, a period of disillusionment follows when products based upon a standard first hit market. While those who labored years to introduce a standards-based product are proud as new parents of a first child, others not emotionally evolved quickly identify deficiencies and suggest how things should've been done differently.
This isn't to say that the criticisms are wrong–just that it's not uncommon for shortcomings to be identified following an innovation's appearance. In this light, there's a strong argument to be made that 100GbE is running par for the normal course of milestone technologies.
Widely distributed pain
The early criticisms are evidence of the huge, urgent challenge that is being tackled by 100GbE.
The pain of meeting that challenge is widely shared. Current and future module form factors that 100GbE users demand will challenge optical vendors. The addition of a high-speed gearbox to the protocol stack will challenge serializer-deserializer (SerDes) vendors. High-speed analog-to-digital converters will challenge chip vendors. Massive processing power will challenge chip vendors and mathematicians. Interferometers will challenge waveguide vendors. Eight optical processing channels will challenge integrated-optics vendors. Wide, fast data buses will challenge host-board designers. The list goes on.
It is this very technical difficulty and the current shortages that indicate the right long-term path has been taken with 100GbE. If designers had taken an easier route with off-the-shelf technologies, there would be little hope of meeting the long-term price/bit needed to be viable in the marketplace. As 100GbE transmission requires an amount of processing power that is unprecedented in Ethernet history, delivery is dependent on future silicon geometry reductions and optical integration–neither of which can be rushed. Both must follow natural technology-development curves. Once achieved, the resulting metrics (power, size, and price) will be more palatable.
The good news is that unprecedented industry teamwork characterizes the ongoing 100GbE tune-up. Standards-body and industry-consortia activity has reached fever pitch. This collaborative spirit is helpful because the huge investment demanded must be evenly distributed, like the aforementioned pain.
Moving forward
Let's look at how the industry is addressing 100GbE's early shortcomings.
Price: It is somewhat disingenuous to compare a coherent, first-generation 100GbE ultra-long-haul network-side port to a third-generation 10GbE SFP+ client-side port. Prices fall with time, volume, and technology advances. Over time, more vendors get into the game, expanding customer choice. And it's amazing what healthy competition can do to margins.
As volumes increase, products can transition from host companies' prototype labs to contract manufacturers' production lines. As integrated-circuit geometries shrink, price points should fall. Specifically, major fabs' transition to 45/28-nm geometries, silicon-on-insulator substrates, high-dielectric constant materials, and metal gate technologies will greatly benefit all silicon ASICs and ASSPs in the 100GbE food chain.
Size: The main problem with the CFP is size. One could reasonably argue that it appears CFP designers wrapped sheet metal around a 10-port 10GbE line card, relabeled it, and called it a day.
However, a more appropriate comparison would be early largeform-factor, 300-pin modules–about the same size as a CFP, but not Zpluggable. All bandwidth steps cut their teeth on unwieldly packages before maturing into more appropriate sizes. Also, the same multi-source agreement (MSA) that developed the CFP has publically detailed potential second- (CFP2) and third-generation (CFP4) offerings (see Figure 1). CFP2 is a logical progression, moving from a 10-wide CAUI data bus to a four-wide CAUI-4 and supporting the same PHY definitions. CFP4 attempts to be a third-generation client-port data bus, using the upcoming CPPI-4 data bus.
The main source of contention with CFP4 plans is that other data protocols (InfiniBand, Fibre Channel, etc.) have already adopted a pre-existing form factor, the QSFP+, for the same purpose (see "New QSFP+ transceiver designs go the distance" on page 21 of this issue). Will CFP4 benefits justify the industry split, or is this a case of Ethernet choosing CFP4 over QSFP+ just to be different? Proponents of CFP4 claim additional jitter and power budgets warrant change, while QSFP+ proponents disagree, claiming differences to be minor and saying failure to leverage the huge industry investment in QSFP would be unwise.
Lack of serial client port: Actually, 100GbE has a serial client port: the 100GbE-LR4 PHY definition. The main concern with this PHY appears to be cost–high for a 10-km link and even more disproportionate as you recede to a 10-m patch cable.
One driver of the higher cost is the need for four color-stabilized lasers, along with the associated wavelength multiplexer/demultiplexer. Existing alternatives in the standard are based upon ribbon cable, which is fine for card-to-card interconnect but requires special considerations for routes that go through plenums/conduits (and you can forget about using patch panels or many couplers).
A cheaper/better solution that uses the now-ubiquitous duplex fiber-optic patch cable is needed. Curiously, the current favorite alternative is a 10-wavelength PHY. If four channels are too expensive, how could 10 be better? The answer lies in the electronics, as proponents of the new PHY argue that no gearbox circuitry is needed and all lanes stay at 10 Gbps, rather than climb to 25 Gbps. (Supporters have formed the 10x10 MSA.) Finally, some industry thought leaders are assembling a call for interest (CFI) for a lower-cost, 2-km serial 100GbE singlemode PHY.
Component shortages: No fundamental physical limitation makes 100GbE components unobtainable. Rather, today's shortages are simply market dynamics at work. As volumes rise, the number of vendors climbs, and technology advances, then parts will become more readily available.
Where hidden value lies
There is hidden value in 100GbE that goes beyond simple price/bit comparisons. The coherent detection used in 100GbE's OIF guise brings with it extra link budget, as well as modulation and wavelength flexibility.
At the network level, WDM expand-ed bandwidth transmission–but also hindered growth by creating more equipment and configuration rules. 100GbE's emergence means a return to more flexible networks with fewer configuration rules.
There is a magic that develops when 100GbE's coherent detection, gridless reconfigurable optical add/drop multiplexers (ROADMs), and Raman amplifiers are combined (see Figure 2). Coherent detection gives sufficient link budget to lower dispersion compensation. Raman amplification reduces the EDFA optical gain required. Combined, coherent detection and Raman amplification can greatly reduce the amount of EDFAs needed, and get rid of them entirely in some cases. Fewer EDFAs means less reliance on gain-slope compensation and, in the case of EDFA-less networks, eliminates being tied to EDFA wavelength bands. Gridless ROADMs enable more flexible channel allocation. When combined with coherent detection, they signal bandwidth allocation-on-demand, since 10G, 40G, and 100G channels can co-exist on the same fiber without wasteful guard bands.
Combining these technologies suddenly renders network architectures powerful and flexible–
yet simple, too, with fewer EDFAs and no gain-slope compensators, dispersion-compensation spools, wavelength grids, or complex configuration rules to contend with. Many networks become simple glass fiber between router nodes–as before WDM's advent.
This is where 100GbE's true value lies, and the benefit is more than worth a few growing pains along the way.
The next big thing
What's next? It might seem premature to ask, but–given the ever-increasing time and difficulty required to reach the next Ethernet speed milestone–the sooner, the better. Moreover, the best way to reduce costs associated with an existing Ethernet data rate is to start on the next generation, as all lower rates benefit.
The second generation of 100GbE is already in sight (narrow parallel data buses with more attention to metrics), and the third generation is on the horizon (serial data buses and small client-only ports). The very technologies that enable the third generation of a major speed step form the foundation of the next. There is disagreement in the industry as to whether that speed step will be 400GbE or 1 terabit Ethernet (TbE), but, for now, let's just call it TbE.
High-speed, data-bus technologies that enable quad- or single-lane 100GbE will be paralleled to form TbE's wider busses. The same coherent detection that came to 100GbE's rescue will be used to generate sufficient gain for reasonable link distances. Advances in digital signal processing leveraged for QPSK encoding and decoding will be extended to more sophisticated modulation techniques already developed in other fields of communications. Integrated optics will be expanded to more channels with more components integrated. Gridless ROADMs will allow TbE to take up various amounts of spectral width on the fiber, depending upon reach needed.
As TbE technology creeps from labs to hero experiments, field trials and, eventually, nascent products, one can be sure that detractors will quickly appear to point out what should've been done differently.
JIM THEODORAS is senior director of technical marketing at ADVA Optical Networking. He is also president of the Ethernet Alliance.