"The network is the computer" is a phrase and philosophy most often attributed to Scott McNealy, chairman and chief executive of Sun Microsystems Inc. This phrase has been brought to life in the new Enron Intelligent Network (EIN) infrastructure. It is no accident that Sun Microsystems Inc. is one of the four technology allies behind the EIN's "Pure IP" national fiber network. The other two are Cisco Systems Inc., which is providing the routing intelligence and expertise, and CIENA Corp., which is providing the dense wavelength-division multiplexing (DWDM) photonic layer.
When the EIN is fully turned up, it will be capable of transmitting 1.44 Tbits/sec on each fiber route, a capacity equal to almost 1 million North American T1s (1.554 Mbits/sec) or 720,000 European E1s (2.048 Mbits/sec) . The initial rollout of the network, which is available now between several major U.S. cities, is presently capable of transmitting 7.5 Gbits/sec.
It would not have been possible to implement this network cost-effectively without DWDM technology. Using the Ciena MultiWave Sentry system, Enron has a total capacity of 96 OC-48 Synchronous Optical Network (SONET) channels on a single pair of fiber. The use of DWDM results in a significant savings in fiber, but the most impressive savings are in equipment cost, deployment, real estate, and power. With DWDM, a single set of optronic equipment is used instead of the 96 SONET sets that would otherwise be required in every terminal node and signal regeneration station.
The network was designed to enable Internet service providers (ISPs), competitive local-exchange carriers (CLECs), digital subscriber loop (DSL) providers, and other carriers that provide IP-based services to have the high-capacity, long-distance, broadband capabilities that they need to meet the growing bandwidth requirements of high-speed DSL, cable modem, and wireless access customers. While the network currently provides its very wide-bandwidth and extremely low-delay connections only within North America, it will soon be deployed into Central and South America, vastly increasing not only capacity, but the geographic reach of the network.
The EIN architecture is most accurately characterized as "IP over glass," which means that Internet protocol bits are placed directly on fiber for high-speed communication to their ultimate destination, thereby eliminating intermediate protocols, simplifying the architecture, lowering costs, increasing reliability, and reducing errors. The "IP over glass," or "IP over fiber," architecture has been proposed in academic circles for a long time and has even been demonstrated on a limited basis. The EIN, however, is the first large-scale commercial backbone network to use this streamlined technical approach.
In a traditional network architecture, IP bits entering the network can be sent across either a private-line SONET circuit or an Asynchronous Transfer Mode (ATM) switching fabric that sometimes uses SONET as well. SONET circuits provide an excellent guarantee of service quality, but the cost is very high due to their dedicated nature. ATM, on the other hand, provides an excellent guarantee of quality of service, along with the ability to support variable bit-rate data. But with an "ATM cell tax" cost of from 10.5% to greater than 60% of additional overhead plus ATM equipment and personnel, the true cost of ATM, even in a bandwidth-rich network, is generally thought to be too high.This protocol overhead is, in fact, one of the main reasons for Enron's decision to deploy a pure IP network as opposed to a traditional carrier architecture such as atm/sonet. The EIN is designed to accommodate only IP traffic. Thus, its architecture is a simple one: IP bits come in through a Cisco 12000 series Gigabit Switch Router (GSR), they are transported across an OC-48 circuit connection provided by CIENA's DWDM system, and then IP bits go out through another Cisco router. No ATM, no frame relay, and no SONET (see Fig. 1).
The Cisco GSR is optimized for single-protocol, IP-only performance. It receives the IP bits from the customer IP connection, makes a hardware-based routing decision, and begins sending IP bits to their destinations. Customer IP connections are presently defined at the 45-Mbit/sec (DS-3), the 155-Mbit/sec (OC-3c), and 622-Mbit/sec (OC-12c) levels. Enron Communications may or may not provide a customer-side router, depending upon the situation.
One of the most frequently asked questions about the EIN is how non-IP connections are accomplished. There are two ways. One is to encapsulate non-IP protocols in IP packets. For example, the ubiquitous IBM Systems Networking Architecture protocol can be encapsulated within IP using edge devices capable of implementing IBM's own Data Link Switching or the Internet Engineering Task Force's (IETF's) RFC 1490 encapsulation. The encapsulation is performed at the edges of the network, and IP packets are delivered to the EIN for transport. A second way to accommodate traffic that is not native IP is to interface to the EIN via the EIN applications gateways, the first of which will provide video connectivity.
After leaving the GSR, the IP bits move over 2.5-Gbit/sec Optical Networking Interfaces, which are OC-48c-like interface connections between the GSR and the CIENA MultiWave Sentry 4000. From the CIENA system, signals are optically multiplexed into one of 40 available optical wavelengths for transmission to a terminating DWDM system. At the receiving end, of course, functions are performed in reverse order so that the customer's IP bits may be sent to their destination. All Layer 2 and 1 overhead has been removed, and bits are transferred directly to a timed optical circuit operating at Layer 0, the photonic layer.
The simpler IP-only approach allows customers to determine the best architecture for distribution- and access-tier operation (see Fig.2). The company's carrier customer might, in fact, choose to use an ATM technology for a multiservice distribution tier, but strip off the ATM before providing pure IP to the EIN transport. On the other hand, IP-service-specific carriers can go IP all the way from the customer premises to the EIN edge, through the EIN core, and back out the other side. In fact, in either case, a variety of traditional or emerging access technologies may be employed-from the older, more traditional Integrated Services Digital Network and frame relay to ATM residential broadband access to cable modems and DSL devices.
As the world shifts away from traditional circuit-switched time-division-multiplexed telephony, the EIN IP backbone also provides the flexibility that carrier customers require because the IP-only aspect of the backbone allows customers to implement the specific type and manufacturer of gateway they require. As long as the output of the gateway or translation device is IP, there is no interface difficulty and the customer can translate the number and range of traditional signaling protocol elements or special calling features they want. Carrier customers using this backbone are also free to use any standard or proprietary scheme for voice encoding, conferencing, video encoding, or encryption due to its standardized, utility nature. This allows the Enron Communications backbone to supply raw bandwidth for transport only or to be a geographically diverse part of a carrier customer virtual private network solution.
A common concern is how a pure IP network will provide reliability without the rapid restoration of the SONET layer. The EIN ensures reliability in two ways. The first is to simply take advantage of the rerouting inherent in the IP layer, which is what is commonly done in ISP networks. These IP rerouting techniques are enhanced by the use of the Cisco tag switching protocol, an implementation of the proposed IETF multi-protocol label-switching standard. Using tag switching, it is possible to pre-engineer restoration paths for any network link or element failure, eliminating the time-consuming requirement to compute the best new path and providing for instantaneous restoration.
An additional protection is provided, however, in the photonic layer. A by-product of the increased capacity is an increase in network reliability. Conventional wisdom tells us that by adding new devices to a network, such as the new DWDM terminal equipment, overall reliability is lowered due to the probability that device failures will not overlap in time and will therefore produce a less reliable overall connection. The CIENA DWDM equipment provides the functions of SONET (protection, management, and multiplexing) via an architecture that scales to high bandwidths more easily and cost-effectively. The CIENA optical-networking architecture also allows much more flexibility in protection implementation. Each node contains two transceivers for each wavelength accessed. The signals from these two transceivers propagate in opposite directions around the ring, providing diversely routed traffic protection. This architecture also provides unidirectional path-switched ring (UPSR)-style protection switching as an option (see Fig. 3).
Note that these protection options are selectable on a per-wavelength basis. In a multiservice network application where SONET and ATM equipment were also used, some wavelengths could operate as a UPSR with protection performed by the terminal, some could operate as a bidirectional line-switched ring with protection provided by the terminal, and others can operate as a UPSR with protection provided by the DWDM node. Multiple protection applications can even be implemented on different wavelengths in the same node.
The EIN provides rapid and cost-effective transport, but that is not the entire story. The underlying pure IP capability was developed with another goal in mind: making it possible to provide the desktop-to-desktop applications that customers demand and carriers are increasingly trying to provide in order to become "turn-key" providers. As customer demands increase, the definition of "network" begins to change. No longer is it enough to simply provide movement of bits from one location to another; additional services must be provided as well.
Almost all of the growth in demand for bandwidth comes from computer-to-computer communication. Some of this demand is driven directly by human beings, such as by a user of an Internet Web browser. But much of it comes from computer systems performing background tasks for the eventual benefit of human users, such as running ERP back-office applications, document synchronization for desktop collaboration sessions, or other such services.
A significant challenge for enterprise users is that they have been responsible for bridging the gap between the utility and ubiquity of the Internet and the reliability and security of the traditional telephony transmission networks. As a first step to bridging that gap, the EIN provides IP-based transmission that is more closely aligned with the enterprise IT mission of delivering IP services to end-user desktops.
The second step is the embedding of large-scale computing servers from Sun Microsystems as part of the EIN services platform. Many of the functions provided by a carrier network can be enhanced by the use of distributed computing techniques for storing and forwarding information. Computation and memory can be successfully traded for additional bandwidth.
Take a streaming media solution, for example. Let's say that 10,000 end users in different locations all need to watch a video presentation critical to their business, that this video presentation originates on a local area network in a distant city, and that the video stream is encoded at 500 kbits/sec to preserve a near-broadcast quality. Today, a traditional network connecting them all would have to provide 5 Gbits/sec of capacity between the two cities to deliver the service. However, by using the computing capabilities of the EIN, the long-distance portion can be reduced to only slightly more than 500 kbits/sec, with the presentation re-originated locally for delivery to all of the end users.
This use of computing systems to augment very high network bandwidth to maximize performance for end users of the service is unique in the industry.
Enron Communications chose the "IP over glass" architecture because it eliminates all of the bandwidth-stealing overhead and complexity of traditional multiservice transport systems, thus providing more bandwidth for applications, simplified and higher-reliability transport, and a simpler, easier-to-manage network focused on delivering maximum impact for IP applications.
Stan Hanks is vice president, research and technology, at Enron Communications Inc. (Portland, OR).