Affordable, high-speed Ethernet solutions to bridge the enterprise and the metro
Alternatives replace dedicated metro channels and overlayed ATM, frame relay, or IP data services, with efficient multiplexed network schemes, enabling lower cost services.
By JEAN-MARC PATENAUDE and JIM FINCH
Broadcom Corp.
Today's telecom service providers face a serious cost/revenue dilemma--network costs are growing faster than legacy voice revenues, while data traffic is rising at an explosive rate. At the same time, there is an increasing demand for higher speed links between enterprise LAN backbones and the service provider's WAN network. To address this dilemma, service providers must achieve the seemingly impossible task of cutting equipment and operating costs, while simultaneously increasing revenues through deployment of new broadband services and improved billing strategies, creating more affordable high-speed solutions for customers.
The biggest challenge lies in the metropolitan area network, through which enterprise networks and LANs access the transport network. While network technologies have rapidly advanced toward Gigabit Ethernet in the LAN and 10 Gbits/sec, (SONET OC-192) in the WAN core, the access network in the metro continues to lag, creating a traffic bottleneck.
Based on today's circumstances in the enterprise, core transport network, and metro access network, viable alternatives can create an affordable bridge between the high-speed Ethernet technologies evolving from the enterprise, and the high-speed WAN technologies emerging within the core optical network.
Local, transport and access networks
Local area network (LAN)
Since its development 25 years ago, Ethernet has evolved as the predominant network technology in enterprise and local area networks, and is adopted by nearly every corporate network. Ethernet is a proven packet-switched technology optimized for data and features a combination of low implementation and support costs, and relative simplicity of installation and maintenance.
Over the years, Ethernet speeds have increased from an initial commercial data transmission rate of 10 Mbits/sec to 100 Mbits/sec, 1 Gbit/sec, and more recently, 10 Gbits/sec. Today, Gigabit Ethernet is gaining widespread acceptance in the LAN, largely driven by rapidly increasing volume of email containing large attachments, Internet traffic, network attached storage, e-commerce and soon, streaming multimedia applications.
The spread of faster Ethernet technologies has been made feasible by a continued decline in price per megabit of traffic, resulting from technological advances and manufacturing efficiencies. The first wave of 10-Gigabit Ethernet (10-GbE) solutions are already shipping in volume and 10-GbE has now become the standard for this next generation technology. These standards are nearing completion, and are targeted for final ratification this year.
Long haul and metro transport networks,
Today's optical transport network provides the data highways necessary to transfer information between geographical locations, and consists of optical pipes of a given size established between two points. The North American network is predominantly SONET-based, with the long haul transport core consisting primarily of OC-192 SONET, transmitting data at 9.958 Gbits/sec, and DWDM technology to carry high traffic volume. In recent years, both new and traditional long haul carriers have made significant investments in the optical transport core to develop current high-speed capabilities.
Figure 1 provides an overview of metropolitan and access networks. SONET OC-48 or DWDM rings are typically used by the Local Exchange Carriers (LECs) to transport traffic within metro regions to interconnect central offices, points-of-presence (PoPs), and carrier hotels using SONET as the transport protocol.
The access network,
Between enterprise networks and the core transport network lies access networks of the Regional Bell Operating Companies (RBOCs) and incumbent local exchange carriers (ILECs), as well as PoPs for Internet services provided by Internet service providers and applications service providers, (see Figure 1). Most connections available in the legacy access network are leased private lines, generally 1.5-Mbits/sec DS-1 and 45-Mbits/sec DS-3 lines, mapped onto OC-3 (155 Mbits/sec), OC-12 (622 Mbits/sec), and more recently, OC-48 SONET rings. These are dedicated channels between the customer and the service provider, for a specified amount of bandwidth. They contribute to the metro bottleneck because of their relatively slow speed.
To further complicate the access network picture, various overlay technologies including ATM, frame relay, and IP, are used to deliver multiple levels of data services over the SONET ring infrastructure, generally using dedicated circuit-switched channels available from the LECs.
Problems in the metro or access network, however, are not solely due to lack of bandwidth. There has been significant investment over the past decade by RBOCs and ILECs to replace legacy copper connections with optical fiber in access networks. As a result, there is now an installed base of SONET optical fiber rings in many business parks and metropolitan areas.
Despite investment in fiber, service providers are often not producing anticipated revenues from bandwidth services, in part because of inefficient use of resources. Yet, high costs prohibit business customers from adding more or faster bandwidth to keep pace with increasing demand, and they often experience long waits to add access lines, increase bandwidth, or install digital subscriber line (DSL) lines, although this has improved somewhat recently.
Efficient, low-cost connectivity for data
The DS-1 connections that are now widely used for data access were originally designed to carry voice traffic, and are inefficient when handling data. The nature of data transmission, characterized by short peaks often followed by little or no traffic, makes dedicated DS-1 channels highly inefficient for data traffic. A statistical multiplexing scheme that intersperses data from multiple users could significantly increase overall bandwidth consumption, without creating noticeable delays to users. Unlike voice traffic, where latency can interfere with the conversation, a delay of a few seconds for email traffic is generally acceptable. Statistical multiplexing allows service providers to oversubscribe the bandwidth, for instance selling 2 Gbits/sec of bandwidth from a 622- Mbits/sec ring, since not all users will fully utilize their allocated bandwidth simultaneously (see Figure 2).
More efficient services are needed in the metro to map data services onto existing optical fiber to improve utilization and reduce cost-per-megabit of access bandwidth. The challenge is to develop new solutions that use the existing infrastructure to provide lower cost, higher speed, and more efficient transmission for data traffic, enabling carriers to reduce costs and generate additional revenue streams.
Three examples illustrate cost, revenue and bandwidth issues:
In some instances, the challenge is not lack of bandwidth, but poor use of bandwidth in the access ring. A business park with an OC-3 optical ring, capable of transporting 155 Mbits/sec of combined data and voice traffic, may have only two customers accessing service, one with two DS-1 lines, and the other with four DS-1 lines. In this case, only 9 Mbits/sec of the available 155 Mbits/sec is used, for a utilization rate of approximately six percent. It would be beneficial for the service provider to offer lower priced data services to attract additional customers, and make use of the access ring bandwidth, provided that such lower priced data services can be deployed profitably. This may be an extreme example, but the situation is not uncommon.
In another instance, available DS-1 and DS-3 channels may be nearly fully subscribed, causing an apparent need for additional bandwidth in the access and metro networks. However, each of these access channels is not fully used at all times. If other subscribers could access the available bandwidth via statistical multiplexing, overall utilization rates increase significantly.
In another common situation, a company orders a DS-3 line through its local carrier to link two geographically separate Ethernet LAN sites. The customer must then convert Ethernet traffic to a DS-3 format before handing it off to the service provider. The service provider then grooms the service for the transport network by mapping the DS-3 channel into SONET, transports traffic between the two locations over the SONET network, and converts back to DS-3 at the far end access point. Finally, the customer converts DS-3 into Ethernet with his own router.
This solution does not support the higher efficiencies of statistical multiplexing, and involves several protocol conversions that create additional costs, including more expensive routers with a WAN-capable port on the customer premises. The dedicated circuit bandwidth allocation results in a higher price to a user, even though the user would have been satisfied with a statistically multiplexed service. Finally, the two LAN sites are not linked with complete transparency due to the DS-3 conversions, requiring further LAN administration overhead by the customer.
Metro link alternatives
By providing lower cost bandwidth services, service providers will be able to attract additional customers, improving their overall revenue structure. Ethernet is a very promising alternative to replace leased lines and their overlayed ATM, frame relay, or IP data services, with more efficient statistically multiplexed transport mechanisms. This enables a lower incremental cost to deploy services to subscribers. There are now several companies (ILECs, System OEMs, and others) actively engaged to define service models and network management infrastructure required for this new application based on Ethernet. To do this, Ethernet must be extended to include the traffic shaping, billing, and security attributes necessary to make it a viable data service for the metro. This extended Ethernet can be implemented in a mesh network of Ethernet switches and routers that implicitly support statistical multiplexing.
Another approach is to create data-optimized implementations of SONET to better manage available bandwidth, such as IEEE 802.17 resilient packet ring (RPR), and other statistical multiplexing SONET implementations proposed by multiple vendors. This option can be used to connect router farms to points of presence for service providers, and may be applicable in other access network applications.
Ethernet
The inherent advantages of Ethernet include overall lower cost and ease of installation and maintenance, proven by 25 years of widespread use. The cost per gigabit for LAN ports or interfaces has dropped much faster than prices in SONET-based WANs, making Gigabit Ethernet, and the future 10-GbE, much more economically feasible than any rival technology. This is due in part to the less stringent jitter requirements of Ethernet compared to SONET-based alternatives, and to volume manufacturing. Further cost reduction is possible by using CMOS components that support high levels of integration, lower heat dissipation, and enable more channels with the same packing density.
The proposed 10-GbE specification addresses many of today's network challenges, while maintaining interoperability with existing technologies. The new standard enables Ethernet packets to travel across SONET links efficiently, providing an inexpensive ubiquitous high-speed link between existing Ethernet networks, SONET, and DWDM networks.
Use of 10-GbE in metro links connecting Ethernet traffic further increases efficiency by reducing the need for protocol conversion, since more than 95% of all IP traffic originates in Ethernet networks. In the example of two Ethernet LANs linked via the transport network, use of a 10-GbE link, could reduce cost, eliminate the overhead for protocol conversion, and enable transparent communication between LANs (see Figure 1). Service providers creating new, "greenfield" networks do not need to consider a legacy infrastructure. These may be the Competitive Local Exchange Carriers, but could also be an ILEC deploying a new network in a new metro market.
The proposed standard extends the distance for single-mode fiber implementations from 5 km for Gigabit Ethernet to 40 km for 10-GbE. This gives companies the option of moving data centers to more cost-effective locations and to support multiple campus locations within a 40-km range. These 10-GbE backbones also provide more cost-effective switch-to-switch, and switch-to-service connects, and support Gigabit Ethernet service to workstations, eventually extending support to the desktop.
To support the requirements of metro or WAN access platforms, Ethernet must provide service classification (high priority voice vs. lower priority email); policing and rate limiting; bandwidth provisioning; billing support; and multiple levels of security. Virtual LAN (VLAN) technology is another way to classify service, using VLAN tags to indicate the priority level of a given packet.
An additional requirement for Ethernet is fault tolerance to approach network reliability levels offered by SONET networks, which use a ring topology to provide alternative paths in case of failure with a 50-millisecond restoration time. Ethernet switches and routers must have enough intelligence and knowledge of network topology in the mesh to reroute traffic to an alternate path in case of failure.
RPR and SONET Derivatives
Another alternative to handle the requirements of the metro network is to develop SONET-based derivatives that maintain SONET advantages, while reducing its cost. Several related statistical multiplexing alternatives, based on packet rings, have been proposed by leading network equipment vendors. The emerging RPR standard is underway to develop a combined, standards-based solution for statistical multiplexing over a ring-based network. RPR and proprietary approaches use packet ring technology, rather than SONET TDM, and are compatible with the available optical infrastructure. (see Figure 1). They also maintain the high degree of fault tolerance and rapid restoration time offered by SONET, while improving efficiency through the use of both rings (main and protect), compared with traditional SONET, where one ring must be reserved for network fault protection. In addition, they offer a high level of management capability for traffic prioritization and shaping, provide high levels of security, and the ability to bill for different quality of service levels.
As a result, RPR and related packet ring solutions, offer an alternative to traditional TDM-based SONET for metro applications, either in selected applications such as connecting router farms to POPs, or in areas where ring bandwidth is already highly used.
Adoption of alternatives
Emerging solutions based on Gigabit Ethernet/10-GbE combinations or RPR offer new alternatives to service providers looking to increase the efficiency of access rings. The ideal solution for a specific metro market depends on many factors, including the level of current SONET access network use, local regulations, service provider management policies, the dominant data service in the network, and other economic parameters not necessarily related to the technology. Although the ideal solution will vary depending on conditions in the network, utilization, type of traffic, and related factors, the most effective solution in terms of total deployment, and network management costs, will prevail.
Jean-Marc Patenaude is director of strategic marketing at Broadcom Corp. (www.broadcom.com), headquartered in Irvine, CA. Jim Finch is director of WAN switch marketing for Broadcom.
