The New Datacom Imperative: Next-Generation Optical Ethernet and Multi-Fiber Connector Inspection

While structured cabling using multi-fiber connectors such as MPO/MTP® have been in use in enterprise data centers for many years, the prevalence of this connector type continues to steeply increase. This is due to the confluence of commercial dynamics (including the apparently insatiable consumer demand for broadband data services) and technical dynamics (including the need for parallel full-duplex lanes of transceiver optics for performance purposes). Concurrently, the relentless drive for high optical network operating efficiency, and minimal lost productivity, is leading to an expanding desire, particularly by hyperscale network operators, for 100% microscopic inspection of their infrastructure connectivity.

AFL estimates there are more than 10 million MPO/MTP connectors in use across the world today, with more than 1 million forecasted to be fielded in 2019. An MPO connector market compound annual growth rate (CAGR) of at least 10% is expected to sustain for the next five years.

As all communications engineers and technicians are aware, nearly all link failures occur at points of connection, and very rarely across unbroken network spans. At the same time, the signal modulation technology that underlies the new short-reach 200G/400G optical Ethernet standard reduces the available link budget in high-bandwidth datacom applications – making the operational need for immaculate connector endfaces all the more imperative.

Next-Generation Optical Ethernet Transport Standards (200G/400G) in the Data Center

Data centers typically rapidly adopt the latest networking technologies as their operators seek a competitive performance edge. The new IEEE 802.3bs 200G/400G Ethernet standard specifies the use of PAM4 modulation, a departure from the older non-return to zero (NRZ) modulation method. PAM4 provides higher spectral efficiency, but because it encodes two bits (four states) into the same carrier signal dynamic range as NRZ (which encodes one bit or two states), PAM4 requires about 9.6 dB more link optical signal-to-noise ratio than NRZ to maintain the same symbol error rate statistics (see Figure 1).

Figure 1. Example PAM4 eye diagram showing the four encoding states. (Photo courtesy of Keysight Technologies)

While this physical layer transmission technology change may initially seem unrelated to optical connector cleanliness, there is a distinct and important link. In 10G, 40G, and 100G systems using NRZ modulation, a contaminated connector endface may cause optical losses that can be largely ignored on the short-reach cabling common in data centers. However, in 200G and 400G systems using PAM4 modulation, the same level of endface contamination will erode a greatly reduced link budget margin, driving optical network technologists to demand pristine connector endfaces. Best-practice connector cleaning and inspection procedures will become essential to maintain the highest levels of performance and reliability.

A simple real-world analogy might paint a picture here. To a family car driving along at 30 miles per hour, road debris is a mere annoyance. To a sports car racing along at 120 miles per hour, the same road debris is a big risk that may even cause a fatal crash.

The light-carrying core of a single-mode MPO fiber has a diameter of 9 microns or an endface surface area (πr²)of about 64 square microns. A 2-micron-diameter speck of dust has a surface area of about 3 square microns, or about 5% of the endface surface area. A 5% reduction in laser power is about -0.2 dB. In an environment in which link budgets are narrowing and transmit laser power is a significant contributor to overall data center power consumption (there are tens of thousands of semiconductor lasers in a typical modern data center), it is quite easy to understand the opex-driven desire of network technologists for completely clean transport optics.

Moving into 2020, hyperscale data centers are expected to become even larger, leading to structured MPO cabling that is naturally longer in some spans. Unlike the signal losses (attenuation) due to cable-reach physics (about 0.4 dB per kilometer at 1550 nm on single-mode fiber), connector contamination losses can be identified using proper microscopic inspection techniques and fully mitigated using proper endface cleaning methods.

Data Center Cable Infrastructure – An Increasingly Valuable Asset

It is possible to quickly model the asset value of a 400G link for a hypothetical broadband internet service provider. The major players in this global market are chasing the goal of providing the majority of their residential and business subscribers with 1-Gbps downstream service by 2020. Today in the United States, a typical consumer pays about $100 per month or $1200 per year for fiber-to-the-home internet service. With no statistical oversubscription, a 400G link serves 400 customers, and thus places the asset value of each 400G MPO terminated cable (eight fibers at 100 Gbps per fiber full duplex) at $480,000 per year. With a conservative 2:1 oversubscription ratio, this rises to nearly $1 million per year. These rough economics underscore the importance of proper maintenance to avoid network downtime, including multi-fiber connector inspection and cleaning as needed.

In 2002, NTT-AT published a finding that up to 80% of failures in optical networks are caused by contaminated connector endfaces (https://sticklers.microcare.com/resources/faqs/why-clean-fiber-optic-connectors/). And in 2016, the Ponemon Institute/Emerson Network Power reported that the average cost of a data center outage is about $740,000 (https://www.emerson.com/en-us/news/corporate/network-power-study). These numbers provide strong quantitative motivation for data center operators to routinely inspect and clean their multi-fiber connectors.

The Rise of Fast MPO Inspection Tools

Endface inspection of multi-fiber connectors at turnup and during normal maintenance operations can easily identify connector contamination. But until recently, the standard method of MPO connector inspection involved the use of awkward and expensive mechanical scanning stages attached to the front-end snout of an inspection microscope probe. This labor-intensive method yields good results, but might take up to a minute to collect IEC 61300-3-35 auto-analysis pass/fail results for each fiber. The high opex associated with mechanically scanning tens of thousands of MPO connectors in hyperscale data centers has until recently made the goal of 100% endface inspection unrealistic.