Coherent routing: Enabling scalable and efficient IP-optical integration

New developments in IP and optics are reigniting the industry’s interest in the possibility of IP-optical integration. Router interfaces supporting 400 Gigabit Ethernet (400GbE) protocol are becoming the new currency for high-bandwidth packet connections, and 800GbE interfaces will soon be available in the latest generation of routing platforms.

In parallel, the latest fifth generation of coherent optics is allowing two tracks of application optimization. First, performance-optimized coherent optics operating at 90 Gbaud provide the next leapfrog in capacity-reach capabilities, and second, power and profile optimized 400G pluggable coherent transceivers enable these to be deployed directly in router ports, with future generations expected to scale to 800G speeds.

Pluggable 400G coherent optics offer an attractive option for integrating WDM capabilities in router by eliminating the short-reach interconnect optics needed to connect routers to transponders and reduces overall space and power requirements at a node. This allows continued network scaling, simplification of architectures, and lower total cost of ownership (TCO).

Successful IP-optical integration requires more than plugging coherent optics into router ports, however. Equally important is the need to optimize the overall network architecture across both IP and optical layers for a wide range of network use cases. One must also consider the important role of optical line systems that connect routers to each other and the multi-layer management and control systems needed for network operationalization.

This article will cover how this wholistic approach to end-to-end IP-optical integration and optimization, also called “coherent routing,” will unlock the resulting benefits for network operators.

Coherent routing applications and implementation

When considering where and how to best leverage the benefits of coherent routing, network operators should determine their typical use cases, which may include:

• Simple point-to-point connections between routers, in campus networks, access links, and for data center interconnect (DCI).
• Metro aggregation networks that collect hub-and-spoke traffic from access nodes to service hubs or central offices (COs) across fiber rings having many nodes.
• Metro core networks that connect these service hubs to each other and carry traffic between COs, internet exchanges (IXs), and co-location facilities.
• Regional and long-haul core backbones that connect cities, disparate data centers, and internet peering sites across countries and continents.

Optimizing cost, capacity and reach across these diverse applications requires the ability to select from a range of optics options (Figure 1). Router-pluggable coherent optics are well suited to point-to-point applications like DCI across campus, metro, and regional distances. 400ZR pluggable router optics are ideal for single-span links up to 120 km, while the extended reach of the 400ZR+ optics enable reach to be extended to several hundreds of kilometers.

The reach of 400ZR+ optics also enables IP-optical integration in metro aggregation and core networks. These are complemented by 400G multihaul optics that are router- or transponder-pluggable and expand performance to enable a greater number of reconfigurable optical add/drop multiplexer (ROADM) pass-throughs, which are typical in metro and regional core networks.

For other IP-optical network applications, network operators will continue to leverage high-performance 90 Gbaud coherent transport optics, which extend optical reach to thousands of kilometers. They can be used either to provide connectivity across long fiber spans or to aggregate multiple lower-speed router ports, operating at 100GbE for example, onto a single high-speed wavelength. Thus IP-optical networks can continue to use coherent transport when needed and leverage the opportunity for coherent routing where it adds value (Figure 2).

Optimize router and link topology

Another important aspect of a successful and efficient coherent routing implementation is to optimize the overall router and link topology to maximize scaling, ensure network resiliency, and minimize TCO. This optimization involves using routers where they are needed, while minimizing their use where they are not.

A key function of a router is to enable statistical multiplexing of packet traffic along the data path. This happens, for example, when low-capacity (e.g., 10GbE, 40GbE, and 100GbE) access ports are concentrated onto high-capacity 400GbE egress ports. This IP aggregation is essential for optimizing available port and link capacity, while addressing requirements such as traffic burstiness, busy hours, failures, redundancy, and load balancing.

Once enough IP traffic has been aggregated to efficiently fill a 400GbE port, the additional statistical multiplexing gains from subsequent router hops diminishes, while the chance of congestion and resulting packet delays and losses increases.

An efficient IP transport topology would connect efficiently filled 400G links as a single-hop connection between source and destination routers and minimize intermediate router hops when these are not needed. After all, each additional router hop adds latency; consumes valuable resources such as fabric capacity, interface ports, space, and power; and very importantly, requires 400G pluggable optics at the ingress and egress simply to transit the packet traffic.

For applications such as DCI or access links that simply require a point-to-point connection between routers, all of this is easily managed: routers connect in a single hop with no intermediate transits, using 400ZR/ZR+ optics, connected over a point-to-point optical line system that can multiplex WDM channels over a shared fiber, with optical amplifiers used to compensate for fiber loss.

In more complex network topologies, IP traffic doesn’t simply start and end between two adjacent nodes, but instead may need to traverse many intermediate sites across complex fiber paths to get to its destination. In that case, the ideal IP link topology for an efficiently aggregated 400G port would be a cost-effective single-hop router connection between the source and destination nodes in the network that bypasses intermediate locations.

This is where the “optical” in IP-optical integration comes in. By utilizing a ROADM-enabled optical layer, network operators can create virtual fiber paths between any two router nodes over complex fiber topologies. These virtual paths allow many routers to share a common fiber network, such as a ring or interconnected fiber mesh topology, while simultaneously enabling single-hop leaf-spine IP topologies over dedicated wavelength connections. In essence, ROADMs virtualize the physical network by introducing a logical transport underlay of single-hop router connections, which can be dynamically provisioned, reconfigured, and optimized to implement the most efficient IP link topology (Figure 3).

While using point-to-point WDM line systems in DCI applications may be sufficient, and optimal in more complex, networked applications, this approach can result in inefficient and inflexible IP network topologies and many unnecessary multi-hop router connections. An efficient coherent routing strategy inherently supports both point-to-point line systems for the former and ROADM capabilities for the latter, to ensure that IP topology is optimized in conjunction with the optical layer, rather than being limited by it.

Operations and management

So far, we’ve explored the technical and architectural paths for successful IP-optical integration using coherent routing. However, real-world implementation should also fundamentally address the operational and management aspects so critical to success.

The disaggregation of transponder and optical line systems, and integration of the photonic service plane into the router, removes the rigid demarcation between the IP routing and optical transport layers and makes multilayer coordination of operations a necessity. Combined with migration to using open, model-driven multivendor APIs, this evolution enables network operators to reimagine their operational best practices as they operationalize IP-optical integration in their networks.

An automated operational approach that transcends and encompasses both the IP and optical layers is predicated on streamlining the management of services using an intent-based approach that frees the operator from having to micro-manage the network. By abstracting the intricacies of network configurations using a minimal set of high-level parameters, an intent-based approach can greatly simplify the configuration and commissioning of router-based coherent optics such as 400ZR/ZR+ by triggering automated workflows with policy enforcement to establish the desired network state.

Intent-based configuration workflows can be used to discover and identify pluggable coherent optics equipped in routers; configure optical interface parameters such as power, modulation, and wavelength; monitor performance indicators such as forward error correction (FEC) and bit error rate (BER); and coordinate end-to-end IP-optical parameters such as channel selection, wavelength routes, physical diversity, and restoration options, for example.

These multilayer management and control tools can also help operational teams with different roles and responsibilities work more effectively and flexibly across both horizontally and vertically oriented organizational models (Figure 4). Some operations teams can focus on a specific network layer, while others may focus across domains to coordinate tasks across multiple network layers. A multilayer automation platform greatly improves operational efficiency and flexibility by combining network resource control, service enablement, analytics, assurance, and workflow orchestration into a comprehensive tool suite critical to successful end-to-end operationalization.

In implementing a successful IP-optical approach that is more than the sum of its parts, operators need a complete set of hardware and software building blocks that include a range of pluggable coherent optics, routing platforms optimized for 400G coherent pluggable transceivers, multifunction optical line systems, and multilayer, end-to-end management. When combined and deployed in synergy, the combination of these building blocks into a complete coherent routing architecture provides flexible and efficient options for addressing a wide range of network use cases. This strategy enables operators to evolve, expand, and upgrade when needed, ensuring they will realize the expected benefits of IP-optical integration.

SERGE MELLE is director, IP-optical product marketing, at Nokia.